nndocs:nvme-of
Differences
This shows you the differences between two versions of the page.
| Next revision | Previous revision | ||
| nndocs:nvme-of [2024/02/23 20:18] – created (doesn't work yet) naptastic | nndocs:nvme-of [2024/09/08 22:00] (current) – Move NQN info to sandbox naptastic | ||
|---|---|---|---|
| Line 1: | Line 1: | ||
| TODO Please put storage traffic on its own IB subnet and limit things sanely. | TODO Please put storage traffic on its own IB subnet and limit things sanely. | ||
| - | |||
| - | NQNs: nqn.2014-08.rocks.narf.hostname is the whole thing. No :01 or any of that jazz, at least not yet. | ||
| The goal is to run everything over RDMA (InfiniBand specifically) and maybe have TCP as a fallback. (If TCP, then I probably want to try VMA... which I don't want to do.) | The goal is to run everything over RDMA (InfiniBand specifically) and maybe have TCP as a fallback. (If TCP, then I probably want to try VMA... which I don't want to do.) | ||
| - | =====Target===== | + | ====Target==== |
| NVMe calls targets " | NVMe calls targets " | ||
| - | Create the subsystem, port and properties: | + | Create the subsystem: |
| + | /> subsystems/ create nqn.2014-08.rocks.narf.southpark | ||
| - | /> subsystems/ create nqn.2014-08.rocks.narf.southpark | + | Create a port and set its properties: |
| - | /> hosts/ create nqn.2014-08.rocks.narf.sadness | + | |
| /> ports/ create 1 | /> ports/ create 1 | ||
| /> ports/1/ set addr trtype=rdma | /> ports/1/ set addr trtype=rdma | ||
| - | Parameter trtype is now ' | ||
| /> ports/1/ set addr adrfam=ipv4 | /> ports/1/ set addr adrfam=ipv4 | ||
| - | Parameter adrfam is now ' | ||
| /> ports/1/ set addr traddr=172.20.64.13 | /> ports/1/ set addr traddr=172.20.64.13 | ||
| - | Parameter traddr is now ' | ||
| /> ports/1/ set addr trsvcid=4420 | /> ports/1/ set addr trsvcid=4420 | ||
| - | Parameter trsvcid is now ' | ||
| /> ports/ | /> ports/ | ||
| NVMe uses " | NVMe uses " | ||
| - | Create and share namespace: | + | Create, set, and enable |
| /> subsystems/ | /> subsystems/ | ||
| /> subsystems/ | /> subsystems/ | ||
| - | Parameter path is now '/ | ||
| /> subsystems/ | /> subsystems/ | ||
| - | | + | |
| + | You can use a file as a backstore. | ||
| + | /> | ||
| + | |||
| + | I haven' | ||
| + | |||
| + | I haven' | ||
| Create ACLs: | Create ACLs: | ||
| + | /> hosts/ create nqn.2014-08.rocks.narf.sadness | ||
| /> subsystems/ | /> subsystems/ | ||
| Parameter allow_any_host is now ' | Parameter allow_any_host is now ' | ||
| /> subsystems/ | /> subsystems/ | ||
| - | =====Initiator===== | + | ====Initiator==== |
| NVMe calls initiators " | NVMe calls initiators " | ||
| - | [root]@[shark][13: | + | Load the module (should be able to list this in /etc/modules) |
| - | [root]@[shark][08: | + | |
| - | Failed to write to /dev/nvme-fabrics: | + | |
| - | | + | |
| - | + | ||
| - | And on southpark: | + | |
| - | + | ||
| - | [956501.535293] nvme nvme1: Connect rejected: status 8 (invalid service ID). | + | |
| - | [956501.535302] nvme nvme1: | + | |
| - | + | ||
| - | Ok, let's try this from sadness. | + | |
| - | Target configuration (nvmetcli) : | + | |
| - | + | ||
| - | /hosts> cd / | + | |
| - | /> hosts/ create nqn.2014-08.rocks.narf.sadness | + | |
| - | /> subsystems/ | + | |
| - | This doesn't work from inside nvmetcli. The newline character grunks up the whole thing. | + | Discovery: |
| + | # nvme discover -t rdma -a 172.20.64.13 -s 4420 | ||
| - | | + | Log in: |
| + | | ||
| - | On sadness: | + | Rescan: |
| + | # nvme ns-rescan /dev/nvme1 | ||
| - | root@sadness: | + | * If a namespace you expect to appear isn't appearing, check the target |
| - | Failed | + | |
| - | from dmesg on sadness: | + | Disconnect a subsystem: |
| + | # nvme disconnect -d /dev/nvme1 | ||
| - | [88648.940995] nvme nvme1: rdma connection establishment failed (-104) | + | Gotta admit, that feels a lot nicer than logging out of iscsiadm. |
nndocs/nvme-of.1708719533.txt.gz · Last modified: 2024/02/23 20:18 by naptastic