Differences

This shows you the differences between two versions of the page.

--- nndocs:srp [2025/02/24 19:08] – massively improve, though it still doesn't work naptastic
+++ nndocs:srp [2025/04/15 18:42] (current) – naptastic
@@ Line 12: / Line 12: @@
 Now strip the first 4 bytes off (they change anyway) and remove the :'s
-  /srpt> ib.fe800000000000005849560e53b70b01/acls create ib.fe800000000000005849560e59150301
+  fe800000000000005849560e59150301
-  Created Node ACL for ib.fe800000000000005849560e59150301
+The top 64 bits of the ACL might need to be 0 or they might be the same as the GUID; I don't know the initiator ID actually gets generated and I'm not doing any more digging into it.
+  /srpt> ib.fe800000000000005849560e53b70b01/acls create ib.00000000000000005849560e59150301
+  Created Node ACL for ib.00000000000000005849560e59150301
   Created mapped LUN 0.
+A Linux SRP target is always visible from all InfiniBand partitions. That limits its usefulness to me.
 ====Dependencies====
@@ Line 20: / Line 26: @@
   apt install srptools
-===Login fails===
+Find targets to connect to:
-shark (initiator):
-   [74794.509035] scsi host11: ib_srp: REJ received
-   [74794.509038] scsi host11: ib_srp: SRP LOGIN from fe80:0000:0000:0000:5849:560e:5915:0301 to fe80:0000:0000:0000:5849:560e:53b7:0b09 REJECTED, reason 0x00010001
-   [74794.509048] scsi host11: ib_srp: Connection 0/12 to fe80:0000:0000:0000:5849:560e:53b7:0b09 failed
-southpark (target):
+    # srp_daemon -o -v -c -p 1
-   [4483481.918835] ib_srpt Received SRP_LOGIN_REQ with i_port_id 5849:560e:53b7:0b09:5849:560e:5915:0301, t_port_id 5849:560e:53b7:0b01:5849:560e:53b7:0b01 and it_iu_len 8260 on port 1 (guid=fe80:0000:0000:0000:5849:560e:53b7:0b09); pkey 0xb068
+  * -o means "run once" otherwise dmesg on all your hosts will get polluted with SRP login noise.
-   [4483481.945664] ib_srpt rejected SRP_LOGIN_REQ because target port ibp33s0f1_1 has not yet been enabled
+  * -v means "say what you're doing"
-   [4483481.959142] ib_srpt Rejecting login with reason 0x10001
+  * -c means "emit target information in a format we can use later"
+  * -p 1 means "only scan on HCA port 1" so obviously change this if you are initiating from port 2...
-"port has not yet been enabled" according to the source code, checks the HCA port itself if its enabled. I think the value isn't being curried correctly somewhere, because:
+Lazy benchmarking seems good:
-  [root]@[southpark][/sys/kernel/config/target/srpt/0xfe800000000000005849560e53b70b01/tpgt_1]# cat enable
+  [root]@[shark][~]# dd if=/dev/sdb of=/dev/null bs=4M
++0 records in
++0 records out
+  17179869184 bytes (17 GB, 16 GiB) copied, 5.38771 s, 3.2 GB/s
+  [root]@[shark][~]# dd if=/dev/zero of=/dev/sdb bs=4M
+  dd: error writing '/dev/sdb': No space left on device
++0 records in
++0 records out
+  17179869184 bytes (17 GB, 16 GiB) copied, 13.7431 s, 1.3 GB/s
+...so I'm using some for swap. We'll see how long it takes me to induce a kernel panic. ;)