Passthrough Physical Disk to Virtual Machine (VM): Difference between revisions

From Proxmox VE
Jump to navigation Jump to search
No edit summary
 
(19 intermediate revisions by 5 users not shown)
Line 1: Line 1:
By adding the raw physical device to the Virtual machine, you can test installers and other disk repair tools that work with disk controllers like <code>ddrescue</code>, Clonezilla or Ubuntu Rescue Remix.




* before adding physical disk to host make note of vendor,serial  so that you'll know which disk to share in /dev/disk/by-id/
'''NOTE''': This guide is meant for QEMU/KVM based Virtual Machines, '''not''' for Container. For the latter see https://forum.proxmox.com/threads/container-with-physical-disk.42280/#post-203292


* lshw on pve host after adding,   
As the disk is attached to the physical and virtual host, this will also prevent Virtual Machine live migration. A second side effect is host system IO wait, when running ddrescue, other VM's running on the host can stutter.
 
= Attach Pass Through Disk =
== Identify Disk ==
Before adding a physical disk to host make note of vendor, serial so that you'll know which disk to share in /dev/disk/by-id/
 
=== lshw ===
lshw is not installed by default on Proxmox VE (see <code>lsblk</code> for that below), you can install it by executing <code>apt install lshw</code>
<pre>
<pre>
lshw -class disk -class storage
...
           *-disk
           *-disk
                 description: ATA Disk
                 description: ATA Disk
Line 16: Line 27:
                 size: 2794GiB (3TB)
                 size: 2794GiB (3TB)
                 configuration: ansiversion=5 sectorsize=4096
                 configuration: ansiversion=5 sectorsize=4096
...
</pre>
</pre>


*device names like /dev/sdc should never be used as those can change. instead use /dev/disk/by-id . check by listing all of that directory then look for the disk added by matching serial number from lshw and the physical disk :
Note that device names like <code>/dev/sdc</code> should never be used, as this can change between reboots.
Use the stable <code>/dev/disk/by-id</code> paths instead.
Check by listing all of that directory then look for the disk added by matching serial number from lshw and the physical disk:
<pre>
<pre>
ls -l /dev/disk/by-id/ata-ST3000DM001-1CH166_Z1F41BLC
ls -l /dev/disk/by-id/ata-ST3000DM001-1CH166_Z1F41BLC
lrwxrwxrwx 1 root root 9 Jan 21 10:10 /dev/disk/by-id/ata-ST3000DM001-1CH166_Z1F41BLC -> ../../sda
lrwxrwxrwx 1 root root 9 Jan 21 10:10 /dev/disk/by-id/ata-ST3000DM001-1CH166_Z1F41BLC -> ../../sda
</pre>
or try
<pre>
ls -l /dev/disk/by-id | grep Z1F41BLC
</pre>
</pre>


=== List disk by-id with lsblk ===
The <code>lsblk</code> is pre-installed, you can print and map the serial and WWN identifiers of attached disks using the following two commands:
lsblk -o +MODEL,SERIAL,WWN
ls -l /dev/disk/by-id/
You can also use an extended one liner to get the path directly:


*add to kvm :
<pre>
<pre>
qm set  592  -virtio2 /dev/disk/by-id/ata-ST3000DM001-1CH166_Z1F41BLC
lsblk |awk 'NR==1{print $0" DEVICE-ID(S)"}NR>1{dev=$1;printf $0" ";system("find /dev/disk/by-id -lname \"*"dev"\" -printf \" %p\"");print "";}'|grep -v -E 'part|lvm'
</pre>


update VM 592: -virtio2 /dev/disk/by-id/ata-ST3000DM001-1CH166_Z1F41BLC
 
<pre>
NAME                        MAJ:MIN RM  SIZE RO TYPE MOUNTPOINT DEVICE-ID(S)
sda                            8:0    0  7.3T  0 disk  /dev/disk/by-id/wwn-0x5000c500c35cd719 /dev/disk/by-id/ata-ST8000DM004-2CX188_ZCT1DNY1
sdb                            8:16  1    29G  0 disk  /dev/disk/by-id/usb-Generic_STORAGE_DEVICE-0:0
sdc                            8:32  0 931.5G  0 disk  /dev/disk/by-id/usb-JMicron_Generic_0123456789ABCDEF-0:0
sdd                            8:48  0  1.8T  0 disk  /dev/disk/by-id/wwn-0x5000c500661eeebd /dev/disk/by-id/ata-ST2000DX001-1CM164_Z1E783H2
</pre>
 
 
[https://unix.stackexchange.com/questions/387855/make-lsblk-list-devices-by-id make-lsblk-list-devices-by-id]
 
=== Short List ===
<pre>
find /dev/disk/by-id/ -type l|xargs -I{} ls -l {}|grep -v -E '[0-9]$' |sort -k11|cut -d' ' -f9,10,11,12
</pre>
</pre>


* check
<pre>
/dev/disk/by-id/ata-ST8000DM004-2CX188_ZCT1DNY1 -> ../../sda
/dev/disk/by-id/wwn-0x5000c500c35cd719 -> ../../sda
/dev/disk/by-id/usb-Generic_STORAGE_DEVICE-0:0 -> ../../sdb
/dev/disk/by-id/usb-JMicron_Generic_0123456789ABCDEF-0:0 -> ../../sdc
/dev/disk/by-id/ata-ST2000DX001-1CM164_Z1E783H2 -> ../../sdd
/dev/disk/by-id/wwn-0x5000c500661eeebd -> ../../sdd
</pre>
 
== Update Configuration ==
=== Hot-Plug/Add physical device as new virtual SCSI disk ===
<pre>
qm set 592 -scsi2 /dev/disk/by-id/ata-ST3000DM001-1CH166_Z1F41BLC
</pre>
<pre>
update VM 592: -scsi2 /dev/disk/by-id/ata-ST3000DM001-1CH166_Z1F41BLC
</pre>
 
=== Hot-Unplug/Remove virtual disk ===
<pre>
qm unlink 592 --idlist scsi2
</pre>
<pre>
update VM 592: -delete scsi2
</pre>
 
== Check Configuration File ==
<pre>
<pre>
grep Z1F41BLC /etc/pve/qemu-server/592.conf
grep Z1F41BLC /etc/pve/qemu-server/592.conf
</pre>
<pre>
scsi2: /dev/disk/by-id/ata-ST3000DM001-1CH166_Z1F41BLC,size=2930266584K
</pre>
== Stop and Restart KVM Virtual Machine ==
You may need to configure the guest operating system now that the disk is available.


virtio2: /dev/disk/by-id/ata-ST3000DM001-1CH166_Z1F41BLC,size=2930266584K
= Tutorial =
</pre>
[https://dannyda.com/2020/08/26/how-to-passthrough-hdd-ssd-physical-disks-to-vm-on-proxmox-vepve/ Example with screenshots]


*stop and restart ... then configure disk from within the kvm.
= Disk Recovery Tools =
# Ubuntu Rescue Remix - [http://www.geekyprojects.com/storage/how-to-recover-data-even-when-hard-drive-is-damaged/ how to use Ubuntu Rescue Remix and Ddrescue]
# ddrescue
# gnu ddrescue
# Clonezilla
# TestDisk
# PhotoRec
# Recuva
# Foremost
# Parted Magic
# SpinRite - Low Cost Commercial - Smartctl tutoral for Proxmox VE planned

Latest revision as of 23:32, 6 December 2022

By adding the raw physical device to the Virtual machine, you can test installers and other disk repair tools that work with disk controllers like ddrescue, Clonezilla or Ubuntu Rescue Remix.


NOTE: This guide is meant for QEMU/KVM based Virtual Machines, not for Container. For the latter see https://forum.proxmox.com/threads/container-with-physical-disk.42280/#post-203292

As the disk is attached to the physical and virtual host, this will also prevent Virtual Machine live migration. A second side effect is host system IO wait, when running ddrescue, other VM's running on the host can stutter.

Attach Pass Through Disk

Identify Disk

Before adding a physical disk to host make note of vendor, serial so that you'll know which disk to share in /dev/disk/by-id/

lshw

lshw is not installed by default on Proxmox VE (see lsblk for that below), you can install it by executing apt install lshw

lshw -class disk -class storage

...
           *-disk
                description: ATA Disk
                product: ST3000DM001-1CH1
                vendor: Seagate
                physical id: 0.0.0
                bus info: scsi@3:0.0.0
                logical name: /dev/sda
                version: CC27
                serial: Z1F41BLC
                size: 2794GiB (3TB)
                configuration: ansiversion=5 sectorsize=4096
...

Note that device names like /dev/sdc should never be used, as this can change between reboots. Use the stable /dev/disk/by-id paths instead. Check by listing all of that directory then look for the disk added by matching serial number from lshw and the physical disk:

ls -l /dev/disk/by-id/ata-ST3000DM001-1CH166_Z1F41BLC
lrwxrwxrwx 1 root root 9 Jan 21 10:10 /dev/disk/by-id/ata-ST3000DM001-1CH166_Z1F41BLC -> ../../sda

or try

ls -l /dev/disk/by-id | grep Z1F41BLC

List disk by-id with lsblk

The lsblk is pre-installed, you can print and map the serial and WWN identifiers of attached disks using the following two commands:

lsblk -o +MODEL,SERIAL,WWN
ls -l /dev/disk/by-id/

You can also use an extended one liner to get the path directly:

lsblk |awk 'NR==1{print $0" DEVICE-ID(S)"}NR>1{dev=$1;printf $0" ";system("find /dev/disk/by-id -lname \"*"dev"\" -printf \" %p\"");print "";}'|grep -v -E 'part|lvm'


NAME                         MAJ:MIN RM   SIZE RO TYPE MOUNTPOINT DEVICE-ID(S)
sda                            8:0    0   7.3T  0 disk   /dev/disk/by-id/wwn-0x5000c500c35cd719 /dev/disk/by-id/ata-ST8000DM004-2CX188_ZCT1DNY1
sdb                            8:16   1    29G  0 disk   /dev/disk/by-id/usb-Generic_STORAGE_DEVICE-0:0
sdc                            8:32   0 931.5G  0 disk   /dev/disk/by-id/usb-JMicron_Generic_0123456789ABCDEF-0:0
sdd                            8:48   0   1.8T  0 disk   /dev/disk/by-id/wwn-0x5000c500661eeebd /dev/disk/by-id/ata-ST2000DX001-1CM164_Z1E783H2


make-lsblk-list-devices-by-id

Short List

find /dev/disk/by-id/ -type l|xargs -I{} ls -l {}|grep -v -E '[0-9]$' |sort -k11|cut -d' ' -f9,10,11,12
/dev/disk/by-id/ata-ST8000DM004-2CX188_ZCT1DNY1 -> ../../sda
/dev/disk/by-id/wwn-0x5000c500c35cd719 -> ../../sda
/dev/disk/by-id/usb-Generic_STORAGE_DEVICE-0:0 -> ../../sdb
/dev/disk/by-id/usb-JMicron_Generic_0123456789ABCDEF-0:0 -> ../../sdc
/dev/disk/by-id/ata-ST2000DX001-1CM164_Z1E783H2 -> ../../sdd
/dev/disk/by-id/wwn-0x5000c500661eeebd -> ../../sdd

Update Configuration

Hot-Plug/Add physical device as new virtual SCSI disk

qm set 592 -scsi2 /dev/disk/by-id/ata-ST3000DM001-1CH166_Z1F41BLC
update VM 592: -scsi2 /dev/disk/by-id/ata-ST3000DM001-1CH166_Z1F41BLC

Hot-Unplug/Remove virtual disk

qm unlink 592 --idlist scsi2
update VM 592: -delete scsi2

Check Configuration File

grep Z1F41BLC /etc/pve/qemu-server/592.conf
scsi2: /dev/disk/by-id/ata-ST3000DM001-1CH166_Z1F41BLC,size=2930266584K

Stop and Restart KVM Virtual Machine

You may need to configure the guest operating system now that the disk is available.

Tutorial

Example with screenshots

Disk Recovery Tools

  1. Ubuntu Rescue Remix - how to use Ubuntu Rescue Remix and Ddrescue
  2. ddrescue
  3. gnu ddrescue
  4. Clonezilla
  5. TestDisk
  6. PhotoRec
  7. Recuva
  8. Foremost
  9. Parted Magic
  10. SpinRite - Low Cost Commercial - Smartctl tutoral for Proxmox VE planned