installing a new machine at NYI

This information is correct as of November 2020.

Metadata

Add the machine to https://rackmonkey.freebsd.org/. Document the physical connections and any peculiarities of the new machine in the notes field. Also note the MAC addresses so the machine can be found again if it inadvertently moves to a different switch port for whatever reason.

After committing to admin-svn, wait for the update emails to show up and resolve any alerts.

Power

On admin.nyi, console to rpduXXXX/lpduXXXX and name the power outlets.

Network

Modern machines will have IPMI rather than serial ports.

For a fresh installation, put both IPMI and network in the install vlan. Also remember to name the port.

On admin.nyi, ssh to rsw, and set things up. 

set interfaces ge-N/0/XX description "hostname ipmi"
set interfaces ge-N/0/YY description "hostname port 0"
set interfaces ge-N/0/XX unit 0 family ethernet-switching vlan members install
set interfaces ge-N/0/YY unit 0 family ethernet-switching vlan members install
commit

Now grep /var/log/daemon for the IPMI MAC address on admin.nyi to find the DHCP lease in the install vlan. Then you can get to the console with ipmitool -U ADMIN -P ADMIN -H 10.0.7.XX sol activate.

Initial install

Follow the instructions in /etc/motd of the pxeroot environment. The root password is in secrets/nyi.asc in the admin-svn repository.

Before rebooting the machine, jail into the install environment and set up production network information in /etc/rc.conf. You may also need to add something like comconsole_port="0x2f8" to /boot/loader.conf.

When you're happy the machine has a chance of showing up on the network following a reboot, exit the jail and reboot the server.

Move to production network

While the machine is booting, move the network interface (not IPMI!) to the production vlan on the switch: 

delete interfaces ge-N/0/YY unit 0 family ethernet-switching vlan members install
set interfaces ge-N/0/YY unit 0 family ethernet-switching vlan members XXXXX
commit

When the machine comes up, verify that the network works by pinging freebsd.org on the console.

Finish installation

Follow instructions in /etc/motd for the new install.

You'll probably want to set up swap too. 

gmirror label swap0 ada0p2 ada1p2
gmirror label swap1 ada2p2 ada3p2

And adjust /etc/fstab accordingly.

Do a sysrc local_unbound_enable="YES". The clusteradm scripts will configure it.

After /etc/motd is clear of warnings, reboot the machine to make sure everything still works.

Move IPMI to production

At this stage, the IPMI port is still in the install VLAN and you're talking to it with ipmitool on admin.nyi.

ssh to the new machine from somewhere else and become root and disconnect your ipmitool on admin.nyi (with ~.).

On the new machine, over ssh: 

pkg install ipmitool
ipmitool lan set 1 ipsrc static
ipmitool lan set 1 ipaddr 10.0.0.XXX
ipmitool lan set 1 netmask 255.255.255.0
ipmitool lan set 1 defgw ipaddr 10.0.0.1

Now move the IPMI port to the netmgmt vlan on rsw: 

delete interfaces ge-N/0/XX unit 0 family ethernet-switching vlan members install
set interfaces ge-N/0/XX unit 0 family ethernet-switching vlan members netmgmt
commit

If you've done everything right, after about a minute, you should be able to console to the new machine from admin.nyi.

old information from before November 2020

(PhilipPaeps note: this information is very old but may still have value when debugging) (MarkLinimon note: WIP; needs some information from clusteradm/YahooNetinstall)

all machines

machines which will pxeboot

machines which will boot from hard drive

clusteradm/NYINetinstall (last edited 2020-11-21T06:26:58+0000 by PhilipPaeps)