Differences

This shows you the differences between two versions of the page.

--- nndocs:initial [2024/02/15 16:32] – add infiniband info naptastic
+++ nndocs:initial [2024/12/05 18:16] (current) – stub out thing naptastic
@@ Line 1: / Line 1: @@
 ===== Naptastic Network Playbook =====
+====Partition drives====
-== Pre-install ==
 Partitioning: The most recent rebuild was shark, for which I took Debian's default EFI setup for the 2TB OS drive. I only changed / to BTRFS instead of ext4. Debian installs itself to a subvolume named @root and makes that subvolume the default, so it's ready for snapshotting backups.
-    apt -y install curl vim whois
+Add /mnt/snapshots in fstab.
-Make sure to enable contrib, non-free, non-free-firmware in /etc/apt/sources.list.
+====Move ~ to a BTRFS subvolume====
+Separate snapshotting the OS from snapshotting your files.
-Comment out svcrdma in /etc/rdma/modules/rdma.conf or nfs-kernel-server won't start. I'm not sure what's **actually** needed to make NFS/RDMA work.
+====SSH key for root====
+  mkdir /root/.ssh
+  chmod 700 /root/.ssh
+  curl https://keys.naptastic.com/david/naptastic.pub >> /root/.ssh/authorized_keys
-==== All systems ====
+====Reconfigure SSH====
-  * network interface names. Can I automate this? (Of course I can.)
+Configure ssh for a high-numbered port, and not to allow password-based logins. Restart sshd. Verify that 'ssh root@localhost' fails. (You didn't forward your agent, did you?).
-  * Profile stuff: Bash, Vim, ? Can I automate this? (Of course I can.)
-  * Network stuff. Too much goddamn customization.
-  * disable journald; configure logging for everything.
-    * Make a list of things that need to log
-    * How exactly do I want logging set up?
-=== All systems with Mellanox adapters ===
+====Backups====
-  * mst* installer
+Create /mnt/snapshots. Create an entry in /etc/fstab the same as for / but with "subvol=/":
-  * copy production opensm configuration in case you have take over as SM
-  * (what needs to change here for VMs using virtual functions?)
-If your host is going to do audio stuff, follow the LAD playbook. Which also badly need an update.
+  # / was on /dev/nvme0n1p2 during installation
+  UUID=some-long-string /               btrfs   noatime,nodiratime,subvol=@rootfs 0       0
+  UUID=some-long-string /mnt/snapshots  btrfs   noatime,nodiratime,subvol=/ 0       0
-If your host is going to host web stuff, follow the LAMP playbook but keep in mind it's badly out-of-date. Be prepared to search, read docs, look for a better way.
+====Install Shorewall====
-===== Getting Started =====
+  * customize interfaces, rules, policy...?
+  * /etc/default/shorewall
+  * /etc/shorewall/shorewall.conf
+  * systemctl enable
+  * reboot a bunch of times because it's not passing traffic for no reason
-Perform a network install of Debian.
+====Pick the best mirror====
+If the repo is installed or mounted locally:
+  deb file:///mnt/debian bookworm main contrib non-free non-free-firmware
+  deb-src file:///mnt/debian bookworm main contrib non-free
-run the naptastic installer if it didn't run as part of the installer:
+If you have to get it via HTTP:
+  deb http://mirror.narf.rocks/debian/ bookworm main contrib non-free non-free-firmware
+  deb-src http://mirror.narf.rocks/debian/ bookworm main contrib non-free non-free-firmware
-  curl https://naptastic.com/initial.sh | /bin/bash
+If you have to use public mirrors (RIP):
+  deb http://deb.debian.org/debian/ bookworm main contrib non-free non-free-firmware
+  deb-src http://deb.debian.org/debian/ bookworm main contrib non-free non-free-firmware
-  # This is not necessary unless the installer failed to do it.
+====Remove crap====
-  mkdir /root/.ssh
+  apt -y remove update-notifier pulseaudio-module-bluetooth bluez blueman bluez-cups bluez-obexd modemmanager rtkit
-  chmod 700 /root/.ssh
-  curl https://keys.naptastic.com/david/naptastic.pub >> /root/.ssh/authorized_keys
-Configure ssh for a high-numbered port, and not to allow password-based logins. Restart sshd. Verify that 'ssh root@localhost' fails. (You didn't forward your agent, did you?).
+====Install extra packages====
+(This is set up so you can triple-click each block you need, copy/paste or middle-paste into your terminal, then press enter at the end and install everything in one transaction.)
-==== Networking ====
+  apt -y install curl vim whois
-  ln -s /dev/null /etc/systemd/network/99-default.link
+Hardware hosts add:
+   smartmontools mdadm qemu-kvm gparted
-Here is what a link file looks like for an Ethernet device:
+Desktops:
+   hexchat synaptic terminator fonts-lohit-knda fonts-knda keepassx evolution virt-manager network-manager-gnome
-    $ cat /etc/systemd/network/20-igb0.link
+Audio workstations:
-    #
+   qjackctl alsa-tools-gui eq10q jalv jamin lilv-utils
-    # Remember to `update-initramfs -u` after changing this file!
-    #
-    [Match]
-    MACAddress=b4:2e:99:38:a9:66
-    [Link]
-    Name=emo0
-    MTUBytes=9000
-Also update the interface names in /etc/network/interfaces. My current convention is to name devices with just a prefix and a number. "eth" is out because of kernel naming, so
+If you plan to compile your own kernel:
-  * emoX for ports on the motherboard
+   bison flex libssl-dev ncurses-dev libelf-dev
-  * enX for gigabit stand-up cards
-  * mlxX for Mellanox devices in Ethernet mode (It could happen!)
-==== InfiniBand ====
+====Networking====
+===Device Names===
+  ln -s /dev/null /etc/systemd/network/99-default.link
-Just make a page for this please.
+Here is what a link file looks like for an Ethernet device:
+  $ cat /etc/systemd/network/20-igb0.link
+  #
+  # Remember to `update-initramfs -u` after changing this file!
+  #
+  [Match]
+  MACAddress=b4:2e:99:38:a9:66
+  [Link]
+  Name=emo0
+  MTUBytes=9000
-    apt -y install ibutils infiniband-diags libvma mstflint rdma-core qperf
+  * emoX for ports on the motherboard
+  * enX for (multi-)gigabit stand-up cards
+  * ibX for InfiniBand IPoIB devices (these do not need .link files though)
+  * mlxX for Mellanox devices in Ethernet mode
-  * opensm config--the defaults are awful
+===Port Conventions===
-  * partitions--the default config is awful
+X is 0-indexed. Port 0 on a stand-up card is the farthest from the motherboard. On a motherboard, it's the left-most port if there's more than one port. I configure ConnectX-3 cards to be InfiniBand on port 0 and Ethernet on port 1, so a system ends up with interfaces named ib0 and mlx1.
-    * Subnet speeds listed below.
-  * the iSCSI page should mention iSER
-  * NFS-RDMA needs info
-    * **DO NOT** restart nfsd if there are active RDMA mounts. They will not recover.
-    * export needs 'insecure' option
-    * echo 'rdma 20049' > /proc/fs/nfsd/portlist # does not persist across reboots or restarts
-    * edit /etc/nfs.conf, set rdma=yes (I THINK) and rdma-port=20049 to make it persist
-Here are the speeds for partitions.conf:
+===InfiniBand===
+  * [[infiniband]]
+  * mst* installer
+  * copy production opensm configuration in case you have take over as SM
+  * (what needs to change here for VMs using virtual functions?)
-    #   2  = 2.5   GBit/s SDR
+Comment out svcrdma in /etc/rdma/modules/rdma.conf or nfs-kernel-server won't start. I'm not sure what's **actually** needed to make NFS/RDMA work.
-    #   3  = 10    GBit/s SDR x4, QDR
-    #   4  = 30    GBit/s SDR x12
-    #   5  = 5     GBit/s DDR
-    #   6  = 20    GBit/s DDR x4
-    #   7  = 40    GBit/s QDR x4
-    #   8  = 60    GBit/s DDR x12
-    #   9  = 80    GBit/s QDR x8 (?)
-    #   10 = 120   GBit/s QDR x12
-    #   11 = 14    Gbit/s FDR
-    #   12 = 56    Gbit/s FDR x4
-    #   there are more, faster, in a weird order. Grep opensm's header files.
-===== Options =====
+===Ethernet configuration===
+Is a total mess right now. I don't know what I'm doing or how I want to do it.
-maybe you feel like it, maybe you don't. idk.
+====Hugepages====
+Useful for databases, PHP, Factorio, and probably other things! Add something like this to /etc/sysctl.conf:
-== Disable TTY screen blanking (Only necessary on Debian Jessie and older) ==
+  vm.nr_hugepages=512
-Add this to ''/etc/rc.local'' above the ''exit 0'' line:
+To make a non-persistent change,
-  sh -c 'setterm -blank 0 -powersave off -powerdown 0 < /dev/console > /dev/console 2>&1'
+  sysctl -w vm.nr_hugepages=512
-== For all systems ==
+Make sure your locked memory limit is equal to or greater than the amount of RAM you're reserving for hugepages.
-  apt -y remove update-notifier pulseaudio-module-bluetooth bluez blueman bluez-cups bluez-obexd modemmanager rtkit
+====Shell profile====
+  * Profile stuff: Bash, Vim, ? Can I automate this? (Of course I can.)
-== For hardware boxes ==
+====Logging====
+  * disable journald; configure logging for everything.
+  * Make a list of things that need to log
-  apt -y install smartmontools qemu-kvm gparted
+====Email====
+lol, not yet
-== If you are using a graphical desktop environment ==
+====Disable TTY screen blanking====
+I don't know when this is necessary anymore. Add this to ''/etc/rc.local'' above the ''exit 0'' line:
-(this needs an update: not sure about fonts packages. Also some network-manager-*-gnome packages might be useful depending on what kind of networking you're doing.)
+  sh -c 'setterm -blank 0 -powersave off -powerdown 0 < /dev/console > /dev/console 2>&1'
-  apt -y install hexchat synaptic terminator fonts-lohit-knda fonts-knda keepassx evolution virt-manager network-manager-gnome
+====Audio workstations====
-== If you plan to do audio work ==
-  apt -y install qjackctl alsa-tools-gui eq10q jalv jamin lilv-utils
   cat >> /etc/pulse/daemon.conf
   default-sample-format = s24le
@@ Line 126: / Line 130: @@
   ^D
-== If you plan to compile your own kernel ==
+More on [[lad]].
-  apt -y install bison flex libssl-dev ncurses-dev libelf-dev