Differences

This shows you the differences between two versions of the page.

--- nndocs:infiniband [2024/12/24 20:07] – improve storage partition naptastic
+++ nndocs:infiniband [2025/11/08 03:09] (current) – [Storage] linkify SRP naptastic
@@ Line 78: / Line 78: @@
 Here's a block for my ATA over Ethernet experiments. Subject to change. IP addresses are necessary for setting up VXLAN tunnels. Checking if IPv6 tunnels perform differently from IPv4 tunnels is on the to-do list. I suspect they perform better. Needs testing.
-  storage=0xb128, ipoib, rate=12, mtu=5, scope=2, defmember=full: ALL=full
+  storage=0xb128, ipoib, rate=12, mtu=5, scope=2, defmember=full:
       mgid=ff12:401b::ffff:ffff   # IPv4 Broadcast address
       mgid=ff12:401b::1           # IPv4 All Hosts group
@@ Line 205: / Line 205: @@
   * See [[iSCSI]] for information about enabling and using iSER.
   * See [[NVMe-oF]] for using NVMe over InfiniBand.
-  * I'm curious about the performance of SRP. I have gotten it to work before but it's not working now. Also not a huge priority.
+  * [[SRP]] is fast and, in most other ways, sucks.
 ====Networking====
 ===VXLAN===
-VXLAN is not the only way to get an Ethernet device on Infiniband, but as far as I can tell it's the only decent one. Neither ConnectX-3 nor Connect-IB has VXLAN offload support.
+VXLAN is not the only way to get an Ethernet device on Infiniband, but as far as I can tell it's the only decent one. None of my hardware has VXLAN offload support.
   * VXLAN id can be anything from 0-16777215 inclusive. I make it match the network number.
@@ Line 218: / Line 218: @@
     export vxlan=64
-    export vxlan_name=ibnet0 # this doesn't get used by anything or stored anywhere
     export local=172.20.64.9
     export group=225.172.20.64
@@ Line 230: / Line 229: @@
 //Et viola//, Ethernet on top of InfiniBand.
-If/when I get hardware capable of VXLAN offload, the dstport might have to change.
+Note: The Infiniband MTU is 4k; IPoIB takes 4 bytes and VXLAN takes 50, leaving a max MTU of 4042.
 ====Multimedia====
 Yeah, someday I want to throw video frames around. There's an RFC or ISO standard for that, IIRC. There's also lgproxy, which is RDMA-aware.
@@ Line 240: / Line 238: @@
   * 5849560e59150301 - shark Connect-IB
   * 5849560e53b70b01 - southpark Connect-IB
-  * - duckling has a Connect-IB and I don't know its GUID
+  * 5849560e53660101 - duckling Connect-IB
   * 7cfe900300a0a080 - uninstalled Connect-IB
   * (there are several more uninstalled Connect-IB cards)