NVMe calls targets “subsystems”. Good for them.

Create the subsystem:

  /> subsystems/ create nqn.2014-08.rocks.narf.southpark

Create a port and set its properties:

  /> ports/ create 1
  /> ports/1/ set addr trtype=rdma
  /> ports/1/ set addr adrfam=ipv4
  /> ports/1/ set addr traddr=172.20.64.13
  /> ports/1/ set addr trsvcid=4420
  /> ports/1/subsystems create nqn.2014-08.rocks.narf.southpark 

NVMe uses “namespaces” (which are numbers) instead of LUNs. Good for them.

Create, set, and enable namespace:

  /> subsystems/nqn.2014-08.rocks.narf.southpark/namespaces create 1
  /> subsystems/nqn.2014-08.rocks.narf.southpark/namespaces/1 set device path=/dev/nvme0n1
  /> subsystems/nqn.2014-08.rocks.narf.southpark/namespaces/1 enable

You can use a file as a backstore. The syntax is not at all obvious. Why “group=device”? Whatever:

  />subsystems/nqn.2014-08.rocks.narf.hostname/namespaces/1 set group=device path=/path/to/some/host.img

I haven't checked if this works for partitions or SATA drives, but I think it should?

I haven't figured out ramdisks yet. There isn't a built-in system like in LIO.

Create ACLs:

  /> hosts/ create nqn.2014-08.rocks.narf.sadness
  /> subsystems/nqn.2014-08.rocks.narf.southpark/ set attr allow_any_host=0
  Parameter allow_any_host is now '0'.
  /> subsystems/nqn.2014-08.rocks.narf.southpark/allowed_hosts create nqn.2014-08.rocks.narf.sadness 

NVMe calls initiators “hosts”. Good for them.

Load the module (should be able to list this in /etc/modules)

  # modprobe nvme-rdma

Discovery:

  # nvme discover -t rdma -a 172.20.64.13 -s 4420

Log in:

  # nvme connect -t rdma -n nqn.2014-08.rocks.narf.southpark -a 172.20.64.13 -s 4420

Rescan:

  # nvme ns-rescan /dev/nvme1

• If a namespace you expect to appear isn't appearing, check the target to see if you forget to enable it

Disconnect a subsystem:

  # nvme disconnect -d /dev/nvme1

Gotta admit, that feels a lot nicer than logging out of iscsiadm.