Difference between revisions of "VMA"
Jump to navigation
Jump to search
| (10 intermediate revisions by 6 users not shown) | |||
| Line 1: | Line 1: | ||
| − | == | + | == VMA backup format == |
| − | Since the 2.3 release PVE has a new format for its very powerful vm backup feature: .vma, replacing the old one, the common .tar format. | + | Since the 2.3 release PVE has a new format for its very powerful vm backup feature: .vma, replacing the old one, the common .tar format. As with the old .tar, also .vma files can be compressed in .lzo or .gz format. |
| − | + | You can read more about Backup and Restore in http://pve.proxmox.com/wiki/Backup_and_Restore | |
| − | + | == The reasons for the switch from tar == | |
| − | + | PVE supports various storage models (some of them still in experimental flavor): lvm, sheepdog, ceph, local, nfs, iscsi, and can allow for usage of raw, qcow, vmdk, images and so on. | |
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
see, amongst the other wiki pages: | see, amongst the other wiki pages: | ||
| − | * | + | *https://pve.proxmox.com/wiki/Storage |
| − | * | + | *https://pve.proxmox.com/pve-docs/chapter-pveceph.html |
| − | |||
The main reason for developing a brand new format for vm backup, was to being able to use just one format, and efficiently, in particular when it comes to snapshots usage, which can also be done in a number of ways depending on the storage model, which showed drawbacks with the plain old standard .tar format, and .vma try to address and efficiently allow one single backup behavior. | The main reason for developing a brand new format for vm backup, was to being able to use just one format, and efficiently, in particular when it comes to snapshots usage, which can also be done in a number of ways depending on the storage model, which showed drawbacks with the plain old standard .tar format, and .vma try to address and efficiently allow one single backup behavior. | ||
| − | There is a very detailed explanation of those reasons | + | There is a very detailed explanation of those reasons available |
| + | [https://git.proxmox.com/?p=pve-qemu.git;a=blob;f=backup.txt here]. | ||
| + | Here is a short summary: | ||
| − | *Most VM backup solutions use some kind of snapshot to get a consistent VM view at a specific point in time (eg: LVM snapshots, qcow2 snapshot, qemu | + | *Most VM backup solutions use some kind of snapshot to get a consistent VM view at a specific point in time (eg: LVM snapshots, qcow2 snapshot, qemu livebackup), but they can involve considerable overhead, in different ways. |
| − | + | *Some storage types/formats supports internal snapshots using some kind of reference counting (rados, sheepdog, dm-thin, qcow2). It would be possible to use that for backups, but for now we want to be storage-independent. | |
| − | + | *There is the need of a way to be more efficient, and avoid any unnecessary step, but to make that work, the backup archive need to be able to store image data 'out of order', although this will not work with traditional archive formats like tar. | |
| − | |||
| − | |||
| − | |||
| − | *There is the need of a way to be more efficient, and avoid any unnecessary step, but to make that work, the backup archive need to be able to store image | ||
| − | |||
| − | data 'out of order', although this will not work with traditional archive formats like tar. | ||
*The new method/format allows for very good performance, works on any storage type and image format, doe not need temporary storage, it is simple archive format, which is able to store sparse files efficiently. | *The new method/format allows for very good performance, works on any storage type and image format, doe not need temporary storage, it is simple archive format, which is able to store sparse files efficiently. | ||
| Line 41: | Line 30: | ||
So it's easy to understand that the aim is to get a good, simple, efficient and consistent backup behavior, more specifically suited to the vm backup, unlike the old traditional file/folder tools! | So it's easy to understand that the aim is to get a good, simple, efficient and consistent backup behavior, more specifically suited to the vm backup, unlike the old traditional file/folder tools! | ||
| − | + | == The VMA format specification == | |
| − | The format details can be found [https://git.proxmox.com/?p=pve-qemu | + | The format details can be found [https://git.proxmox.com/?p=pve-qemu.git;a=blob;f=vma_spec.txt here]. |
| − | + | == Command line utility == | |
~# vma | ~# vma | ||
| Line 55: | Line 44: | ||
vma verify <filename> [-v] | vma verify <filename> [-v] | ||
| − | + | == Things to be aware of == | |
| + | |||
| + | Please note that as the data is written in a out of order fashion, tools like rdiff will not be able to compute | ||
| + | a delta between VMA files properly, crating a very large diff. | ||
| + | https://forum.proxmox.com/threads/vma-archive-restore-outside-of-proxmox.14226/#post-76387 | ||
| − | + | if you need to extract a VMA file outside a proxmox environment, follow these steps: | |
| + | see (all) this thread: http://forum.proxmox.com/threads/14226-VMA-archive-restore-outside-of-Proxmox | ||
| − | + | [[Category: HOWTO]] | |
Revision as of 12:32, 4 April 2019
VMA backup format
Since the 2.3 release PVE has a new format for its very powerful vm backup feature: .vma, replacing the old one, the common .tar format. As with the old .tar, also .vma files can be compressed in .lzo or .gz format.
You can read more about Backup and Restore in http://pve.proxmox.com/wiki/Backup_and_Restore
The reasons for the switch from tar
PVE supports various storage models (some of them still in experimental flavor): lvm, sheepdog, ceph, local, nfs, iscsi, and can allow for usage of raw, qcow, vmdk, images and so on.
see, amongst the other wiki pages:
The main reason for developing a brand new format for vm backup, was to being able to use just one format, and efficiently, in particular when it comes to snapshots usage, which can also be done in a number of ways depending on the storage model, which showed drawbacks with the plain old standard .tar format, and .vma try to address and efficiently allow one single backup behavior.
There is a very detailed explanation of those reasons available here. Here is a short summary:
- Most VM backup solutions use some kind of snapshot to get a consistent VM view at a specific point in time (eg: LVM snapshots, qcow2 snapshot, qemu livebackup), but they can involve considerable overhead, in different ways.
- Some storage types/formats supports internal snapshots using some kind of reference counting (rados, sheepdog, dm-thin, qcow2). It would be possible to use that for backups, but for now we want to be storage-independent.
- There is the need of a way to be more efficient, and avoid any unnecessary step, but to make that work, the backup archive need to be able to store image data 'out of order', although this will not work with traditional archive formats like tar.
- The new method/format allows for very good performance, works on any storage type and image format, doe not need temporary storage, it is simple archive format, which is able to store sparse files efficiently.
So it's easy to understand that the aim is to get a good, simple, efficient and consistent backup behavior, more specifically suited to the vm backup, unlike the old traditional file/folder tools!
The VMA format specification
The format details can be found here.
Command line utility
~# vma usage: vma command [command options] vma list <filename> vma create <filename> [-c config] <archive> pathname ... vma extract <filename> [-v] [-r <fifo>] <targetdir> vma verify <filename> [-v]
Things to be aware of
Please note that as the data is written in a out of order fashion, tools like rdiff will not be able to compute a delta between VMA files properly, crating a very large diff. https://forum.proxmox.com/threads/vma-archive-restore-outside-of-proxmox.14226/#post-76387
if you need to extract a VMA file outside a proxmox environment, follow these steps: see (all) this thread: http://forum.proxmox.com/threads/14226-VMA-archive-restore-outside-of-Proxmox