Differences

This shows you the differences between two versions of the page.

--- nndocs:fibre-channel [2025/01/01 18:21] – [Questions] wwpn:wwnn naptastic
+++ nndocs:fibre-channel [2025/01/12 01:59] (current) – [Speed] naptastic
@@ Line 1: / Line 1: @@
 =====Fibre Channel=====
+All the lab hardware is Qlogic 2562, 2564, or 2662. There isn't a switch, so everything is point-to-point.
-===16G===
+Put this in /etc/modprobe.d/qla2xxx.conf:
-southpark
-  * 51402ec000f265ac
-  * 51402ec000f265ae
-shark
+  options qla2xxx qlini_mode=exclusive
-  * 51402ec000f1ef48
-  * 51402ec000f1ef4a
-===8G===
+This allows dual-port cards to be targets (T) and initiators (I) simultaneously. To ensure targets don't get plugged into targets and initiators into initiators, I've adopted these conventions:
-duckling
+  * dual-port cards put the initiator on the first port and the target on the second port. (IT)
-  * 21000024ff5db100
+  * quad-port cards go the other way: target / initiator / target / initiator.
-  * 21000024ff5db101
-formerly shark, now uninstalled
+Remember the ports on the 16G cards are backwards when you're figuring out what plugs into what.
-  * 21000024ff5db098
-  * 21000024ff5db099
-formerly southpark, now uninstalled
+I want a switch.
-  * 21000024ff668780
-  * 21000024ff668781
-  * 21000024ff668782
-  * 21000024ff668783
-uninstalled
+====QConvergeConsole CLI====
-  * 21000024ff6686f8
+Download from Marvell or find it in ~/Library/hardware/qlogic-2xxx. Installing the .deb on Debian 12 created a problem for me where it couldn't find libncli.so.
-  * 21000024ff6686f9
-  * 21000024ff6686fa
-  * 21000024ff6686fb
-uninstalled (labeled "BOOT TURNED OFF")
+  LD_LIBRARY_PATH=/usr/lib64 /opt/QLogic_Corporation/QConvergeConsoleCLI/qaucli
-  * 21000024ff522dac
-  * 21000024ff522dad
-uninstalled
+...or install the RPM instead.
-  * 21000024ff5875fe
-  * 21000024ff5875ff
-uninstalled
+  * If you see "IO_PAGE_FAULT" in dmesg every 5 seconds, add "iommu=pt" to the kernel command line. It's probably specific to EPYC CPUs.
-  * 21000024ff31eb4c
+  * The 26xx cards are perfectly happy to boot MBR drives in BIOS mode.
-  * 21000024ff31eb4d
+  * The 26xx cards do not have dual personality. They are storage-only.
-uninstalled
+====Speed====
-  * 21000024ff3af5ea
+G Fibre Channel goes about 7.9 Gbps. 16G goes 14. Both protocols can approach line rate to a ramdisk without having to try very hard.
-  * 21000024ff3af5eb
-uninstalled
+FC gear is interoperable across three generations. In testing, everything was compatible and worked as expected except for 16G optics in 8G ports: those negotiated 4G connections instead of 8G.
-  * 21000024ff5f1830
-  * 21000024ff5f1831
-=====Questions=====
+The difference in power and heat from different generations of optics is not measurable. 16G optics don't require heatsinks.
-  - How fast can the 16GFC gear go?
-    * Line rate to a ramdisk without even trying hard.
-  - Does NPIV work? (Not on the first try.)
-    * It fails the same way the 8GFC gear did... if we get it working here, can we get it working there?
-    * it's wwpn:wwnn. The port number comes first.
-  - Can it use old optics and work the same way at lower speeds?
-    * Yes.
-  - Does it use less power that way?
-  - Can they speak NVMe?
-  - Can I boot NVMe?
-  - Does it make sense to export ZFS via NVMe instead of iSCSI?
-=====Answers=====
+====NPIV====
-===QConvergeConsole CLI===
+NPIV can't be done without a switch. See 2025-01-01-fc-vports.txt for how to do it if I ever get a switch.
-Download from Marvell or find it in ~/Library/hardware/qlogic-2xxx. Installing the .deb on Debian 12 created a problem for me where it couldn't find libncli.so.
-  LD_LIBRARY_PATH=/usr/lib64 /opt/QLogic_Corporation/QConvergeConsoleCLI/qaucli
+===NVMe===
+G gear can't speak NVMe. I didn't try it with the 16G gear yet because I couldn't stand the noise of the fans.
-...or install the RPM instead.
+====16G WWNs====
+  * uninstalled
+    - 51402ec000f265ac
+    - 51402ec000f265ae
+  * uninstalled
+    - 51402ec000f1ef48
+    - 51402ec000f1ef4a
-  * If you see "IO_PAGE_FAULT" in dmesg every 5 seconds, add "iommu=pt" to the kernel command line. It's probably specific to EPYC CPUs.
+====8G WWNs====
-  * The 26xx cards are perfectly happy to boot MBR drives in BIOS mode.
+  * shark
-  * The 26xx cards do not have dual personality. They are storage-only.
+    - 21000024ff5db098
-  * The ports are BACKWARDS!!! RAGE
+    - 21000024ff5db099
+  * duckling
+    - 21000024ff5db100
+    - 21000024ff5db101
+  * southpark
+    - 21000024ff668780
+    - 21000024ff668781
+    - 21000024ff668782
+    - 21000024ff668783
+  * uninstalled
+    - 21000024ff6686f8
+    - 21000024ff6686f9
+    - 21000024ff6686fa
+    - 21000024ff6686fb
+  * uninstalled
+    - 21000024ff522dac
+    - 21000024ff522dad
+  * uninstalled
+    - 21000024ff5875fe
+    - 21000024ff5875ff
+  * uninstalled
+    - 21000024ff31eb4c
+    - 21000024ff31eb4d
+  * uninstalled
+    - 21000024ff3af5ea
+    - 21000024ff3af5eb
+  * uninstalled
+    - 21000024ff5f1830
+    - 21000024ff5f1831