Differences

This shows you the differences between two versions of the page.

--- nndocs:sandbox [2024/12/26 18:36] – [IQNs] update naptastic
+++ nndocs:sandbox [2025/11/06 00:13] (current) – update hard drive locations naptastic
@@ Line 11: / Line 11: @@
 | Food | cherry |
-====IQNs====
+====IQN and NQN====
-We do iSCSI Qualified Names by the book, but as simply as possible. A couple of examples:
+We do iSCSI and NVMe Qualified Names by the book, but as simply as possible. A couple of examples:
     iqn.2014-08.rocks.narf.southpark:duckling
-    iqn.2004-12.com.naptastic.quirk:68
+    nqn.2004-12.com.naptastic.quirk:68
-  * 'iqn' is always 'iqn'
+  * 'iqn' for iSCSI or 'nqn' for NVMe-oF
-  * YYYY-MM is the month in which the **domain** was registered.
+  * YYYY-MM is the month in which the domain was registered.
   * the hostname is in backwards DNS order
-  * :initiator is so I can maintain 1:1 relationships between targets and initiators
+  * :initiator or :namespace
-I haven't been using :initiator and I'm not 100% sure that it's going to do what I expect. The last time I tried having multiple targets on the same physical network port, iscsiadm got pretty confused. I'm planning my next experiment, but I want to tear down **all** the iSCSI and try rebuilding it again "correctly":
+From the perspective of a host, every storage bus has a number. Bus 0 is the local PCI bus. Bus 1 is for the host acting as an initiator. Higher numbers are for a host acting as a target. Connections go like this:
-  * every initiator gets a dedicated target;
+  iqn.2004-12.com.naptastic.rocky:01 -> iqn.2014-08.rocks.narf.southpark:68
-  * every initiator/target pair gets a dedicated IB partition; and
-  * storage IB partitions are IPv6-only.
-(I also want to try this with SRP.)
+Since rocky's host number is 68, any other host wishing to share storage with rocky should do so on their own target bus number 68. The network is responsible to connect rocky's bus 01 with southpark's bus 68.
-Ideally, /etc/iscsi/initiatorname.iscsi gets the right value as part of system provisioning, and the target only ever has to use the host's correct name. There's a strong warning about changing IninitatorName in that file; if you're changing it to the correct value (i.e., the value in the target's ACL) then it's fine.
-====NQNs====
-NQNs: nqn.2014-08.rocks.narf.hostname is the whole thing. No :01 or any of that jazz, at least not yet.
 ====Interfaces====
@@ Line 68: / Line 61: @@
       hardware ethernet 00:10:18:f8:2e:14;
       hardware ethernet 00:10:18:f8:2e:16;
+====IPv6====
+IPv6 local unique addresses have four parts:
+  - 8 bits of fixed prefix (fd)
+  - 40 bits of pseudorandom prefix
+  - 16 bits of subnet
+  - 64 bits of GUID
+My top 48 bits are going to be fd20:4e41:5020. Subnet bits should match their Infiniband subnet. Host bits come from the hardware address.
 ====IP Addresses====
@@ Line 80: / Line 82: @@
 ====Networks====
-  * 0-3/22: "green": all emo* and eth* devices should get bridged to this network.
+I'm not currently using InfiniBand subnets for anything. They weren't helpful in partitioning SRP or iSCSI traffic.
+VLAN and VNI can match the third octet. 0 is not valid, so use the largest number in the subnet.
+^ Network ^ IPv4 Range ^ IPv6 Range ^ IB Subnet ^ VLAN ^ VXLAN IP ^ VXLAN ID ^
+| green | 172.20.0/22 | fd20:4e41:5020:ffff::/64. | ffff | 3 | 225.172.20.64 | 3 |
+| mgmt | 172.20.4/24 | - | ffff | 4 | 225.172.20.64 | 4 |
+  * 0-3/22: "green": Most eth* and en* devices should get bridged to this network.
     * 0: .1 is the network controller (router, DHCP, DNS); .2+ is for assigned hosts
     * 1: .0-255 contains IPMI interfaces and nothing else right now
@@ Line 90: / Line 100: @@
       * .96-103: shark
       * .104-111: southpark (cudapark, )
-  * Unused: 4-7 (/22) (if security cameras ever get a network, it should come from this range.)
+  * 4: "mgmt": IPMI and other "out-of-band" management
+  * Unused: 5 (/24)
+  * Unused: 6-7 (/23) (if security cameras ever get a network, it should come from this range.)
   * Future Multipath: 8-15 (/21 in /24's)
   * Unused: 16-31 (/20)
   * Unused: 32-63 (/19)
-  * RDMA networks (/19): IP over InfiniBand and Ethernet can't be bridged, so they need to be separate subnets. I //could// bridge ib0+ib1 or mlx0+mlx1 but that takes away flexibility so I'm not going to.
+  * RDMA networks (/19): IP over InfiniBand and Ethernet can't be bridged, so they need to be separate subnets. I'm not sure if ib0/ib1 is a useful division.
     * 64: ib0
     * 65: ib1
-    * 66: mlx0
+    * 66-67 (/23): Reserved
-    * 67: mlx1
     * 68-79 (/22): Reserved
     * 80-95 (/21): Reserved
@@ Line 121: / Line 132: @@
 there might be other san* VLANs defined. I don't remember.
-I am no longer able to manage the TP-link switch. It's fixed configuration. I'm not using VLANs so I don't know how it will behave with them.
+I am no longer able to manage the TP-link switch. It's fixed configuration. I don't know how it would handle traffic with VLAN tags.
 ====Host Numbers====
@@ Line 146: / Line 157: @@
 ----
+=====Where Data Lives=====
+===Storage===
+  * Production volumes:
+    * Samsung 850 512GB NVMe
+      * installed in southpark on the motherboard, I think?
+    * Western Digital Black 2TB NVMe
+      * installed in shark on the motherboard
+    * Two Sabrent 1TB NVMe drive
+      * installed in southpark on the x16 riser card
+      * Partitioned unfortunately; I'd like to move everything off them.
+    * janet: 4TB Samsung QVO.
+    * Crucial 120GB SATA SSD boot volume
+      * southpark's boot volume
+    * 512GB Crucial SATA SSD
+      * duckling boot, swap, and local snapshots
+    * black: Three 4TB drives in a RAID-5 holding backups; currently in shark.
+  * Not installed:
+    * 2TB Vulcan Z SATA SSD (probably for shark as a backup)
+===So Where Does Shit Live?===
+  * VM Images:
+  * TV shows, movies, software, stuff like that: janet
+  * Home directories:
+  * OS volumes: In their respective hosts
+I am still struggling with the fact that things really have to boot off local storage. Fibre Channel without a switch isn't worth it and a switch isn't worth it.
+----
 =====Big Idea Dumping Ground=====
+===SSH tunnel===
+  ssh -D 1337 -q -C -N david@do.naptastic.com
+====SoC on a PCI card====
+Mostly this is to enable other projects. It needs at least these features:
+  * reasonable amount of CPU and DRAM
+  * PCIe 3.0 x1 or better
+  * Internal ports:
+    - m.2 slot
+    - Explicitly copy rpi's 40-pin header
+    - LP-DIMM
+  * External ports:
+    - HDMI
+    - gigabit Ethernet
+    - (2) USB 3.1
+The card presents itself to its host as one or more devices. It can pass through its USB host adapter, network adapter, and m.2 slot, and act as a GPU. It supports SR-IOV. You can boot off it. With the right hardware support, it should be able to stay awake while the host powers off or reboots.
 ====PCIe networking====
 These products:
@@ Line 165: / Line 222: @@
 Everything except retimer cards needs an API and UI for managing things.
 ====Better Audio Stack====
 Want to:
@@ Line 176: / Line 234: @@
 ====Better DAW====
 Core concept is a piece of virtual tape that's infinitely long, infinitely wide, and has an infinite number of tape heads that can all be accessed remotely. A recording session is a server you log into, send recorded audio, receive mixed audio, and update a database of what should play when and with what settings.
+====RDMA to a Commodore 64====
+The 6502 can't drive the C64's memory anywhere near full-speed, but the ROM port has a DMA pin, so a DMA-capable cartridge can do impossibly cool things. I'd like to find a good demo playback engine that already exists, and adapt it to play back data being streamed in over a network.
 ====Better Mixer====
 FIXME