Officially ratified at the end of 2018, NVMe over TCP (Non-Volatile Memory Express over TCP) is a still very little-known protocol that allows communication between NVMe-based storage devices via TCP/IP networks. Sorry if it’s not much!
The idea behind NVMe over TCP is to extend the capabilities of NVMe beyond PCIe connectionsallowing storage drives to be accessed across networks TCP/IP standard. The advantage is, obviously, that of being able transfer data at the highest possible speed while reducing latency and improving performance.
NVMe over TCP allows you to leverage existing networks, such as Ethernet and WiFi networks, to access NVMe storage drives, eliminating the need for specialized hardware infrastructure such as switches or compatible network cards NVMe-oF (NVMe over Fabrics). The latter devices, among other things, require fiber optic cables as a means of data transport.
Clone disks and transfer data at maximum speed with NVMe over TCP
It is a software engineer, Vasudeva Kamath, who tells how he managed to clone an NVMe drive 512 GB on a new notebook, connected to the local network, without physically disconnecting and reconnecting any storage media.
Exposing the unit NVMe PCIe SSD to clone through NVMe over TCP, it becomes very simple to transfer data from an old to a new disk using the LAN. Especially if you can count on Multigigabit Ethernet. If this were not the case, transport the data However, via the network it is still very feasible, albeit benefiting from lower performance.
Configure the NVMe storage drive for access over a TCP/IP network
As Kamath explains, the trick is to use a Live Linux distribution (the engineer used GRML) then issue a series of commands from the terminal window on the source system (the one from which you want copy the data). We explain in detail the commands you find in the paragraph “Exporting Disk over NVMe TCP“:
1. modprobe nvemt-tcp: Loads the `nvmet-tcp` kernel module, which is responsible for handling NVMe over TCP communication.
2. cd /sys/kernel/config/nvmet: Moves to the directory containing the NVMe over TCP and NVMe-oF module settings.
3. mkdir ports/0: Creates an NVMe port (port 0) to accept NVMe client connections.
4. cd ports/0: Moves to the virtual directory corresponding to the newly created communication port.
5. echo "ipv4" > addr_adrfam: Set the address type to IPv4 for the NVMe port.
6. echo 0.0.0.0 > addr_traaddr: Set the transport address to 0.0.0.0, which means the port accepts connections from any IP address.
7. echo 4420 > addr_trsvcid: Set the service port (service port) to 4420, which is the default port for NVMe over TCP.
8. echo tcp > addr_trtype: Set the transport type to TCP.
9. cd /sys/kernel/config/nvmet/subsystems: Moves to the directory containing the NVMe subsystem file.
10. mkdir testnqn: Creates a new NVMe subsystem called testnqn.
11. echo 1 > testnqn/allow_any_host: Allows any host to connect to this subsystem without authentication.
12. mkdir testnqn/namespaces/1: Create a namespace (namespace) in the subsystem testnqn.
13. cd testnqn: Moves to the subsystem directory testnqn.
14. echo "/dev/nvme0n1" > namespaces/1/device_path: Specifies the path of the NVMe device (/dev/nvme0n1) which will be exported as namespace 1.
15. echo 1 > namespaces/1/enable: Enable namespace 1 within the subsystem testnqn.
16. ln -s "../../subsystems/testnqn" /sys/kernel/config/nvmet/ports/0/subsystems/testnqn: Creates a symbolic link from the port directory to the subsystem testnqnallowing the NVMe port to serve the specified subsystem.
Establish connection with the storage device from which to copy data
Still using the same live Linux distribution on the target device, you can run the following commands to browse for the source storage media and then establish the connection via NVMe over TCP:
nvme discover -t tcp -a <ip> -s 4420
nvme connectl-all -t tcp -a <> -s 4420
nvme list
The indication <ip> must be replaced withsystem IP address; the last command shows the device connected to the new notebook, to which you want to transfer the data.
Clone the disk and resize the partition
Per copy the data from one storage drive to another via NVMe over TCP, you can clone the disk with the dd utility. The syntax to use is the following:
dd if=/dev/nvme2n1 of=/dev/nvme0n1 status=progress bs=40M
if=/dev/nvme2n1: Specify the input file (if stands for “input file“) from which the data will be read. Inside the command,/dev/nvme2n1represents the source storage device from which the data will be read.of=/dev/nvme0n1: Specify the file di output (of stands for “output file“):/dev/nvme0n1represents the destination storage device to which the data will be written.status=progress: This option enables the display of a progress message indicating the status of the copy operation. It shows information such as how many bytes have been copied so far and at what speed.bs=40M: Specify the block size reading/writing. In this case,40Mindicates that the command will use data blocks of size 40 MB to read and write data. Setting an appropriate block size can optimize data copy performance, especially when working with large amounts of data.
As a final step, Kamath simply has resized the partition created on the destination disk so as to occupy the 1 Terabyte space available on the destination drive.
Opening image credit: Copilot Designer.