====== Setup ======

Some notes on setting up a new Proxmox server:

  * Install on ZFS root (set advanced options ''ashift'' to either 9 for 512-byte sector ssds, or 12 for 4k sectors)
    * (Check what sector sizes an SSD supports with ''nvme id-ns -H /dev/nvme0n1 | grep "Relative Performance"''
  * Run community [[https://community-scripts.github.io/ProxmoxVE/scripts?id=post-pve-install|post-pve-install]] to enable open-source repos, update and disable nag screen
  * Join node into cluster (datacenter->clusters grab join info from existing cluster, join cluster on new node)

===== Hardware-specific quirks =====

  * NUC with e1000e interface drops offline after a few hours/days: [[https://community-scripts.github.io/ProxmoxVE/scripts?id=nic-offloading-fix|nic-offloading-fix]]


===== Using trusted SSL certificates =====

https://pve.proxmox.com/wiki/HTTPSCertificateConfiguration

==== ACME setup for DNS RFC2136 ====

  * Copy the tsig key to ''/usr/local/share/nsupdate.key''
  * Navigate to ACME plugins and create a new one:
    * ID: FreeIPA
    * Type: RFC2136
    * Key: /usr/local/share/nsupdate.key
    * Server: ares.jellybean.sihnon.net
  * Create the ACME account via webui
  * Create the certificate request using DNS type and FreeIPA plugin
  * Order the new certificate

===== Using IPtables to redirect the web interface to port 443 =====

https://forum.proxmox.com/threads/proxmox-3-0-web-ui-on-port-443-instead-of-8006.13964/

===== Network Interfaces =====

  * Edit ''vmbr0'', remove physical nic and IP address
  * Create OVSBridge ''ovsbr0''
  * Create OVSIntPort ''mgmt'' attached to ''ovsbr0'' and add the management IP back onto this
  * Create OVSBond ''bond0'' attached to ''ovsbr0'' and add all physical nics to this, with mode ''balance-tcp''
  * Apply changes
  * Update switch configuration to enable LACP
  * Confirm connectivity
  * Datacenter->SDN->Apply to setup vlans

===== OpenVSwitch =====

<code bash>
apt-get install openvswitch-switch
</code>

===== SDN =====

  * Install dependencies: <code bash>
apt install frr-pythontools
systemctl enable --now frr.service
</code>
  * ''Datacenter->SDN->Zones''
    * Create ''lan'' zone of type ''vlan'' bound to ''ovsbr0''
  * ''Datacenter->SDN-VNets''
    * Create one vnet for each vlan, setting the zone to ''lan'' and entering the vlan id in ''tag'' field.
    * When creating VMs, select this vnet to assign to the vlan without needing to re-enter the vlan id.

===== VM Tweaks =====

==== Unique disk serial numbers ====

TrueNAS SCALE demands unique serial numbers on each VM disk for stable imports. Proxmox doesn't have UI for this, but it can be done by editing the VM config files.

  * Generate a unique serial number with: <code bash>
uuidgen | sed -E "s|-||g" | cut -c 1-20
</code>
  * Edit the VM definition at ''/etc/pve/qemu-server/${VMID}.conf'' to add ''serial=XXX'' to the end of the ''scsi'' line(s)