To make a new style FreeBSD pkg and svn mirror, we have come up with the following design.
We would like two machines for FreeBSD gateways that allow the project to configure firewalls and routing for the project. These two machines should have connections for outside and inside networks. We would like to have a small transit network between our firewalls and the upstream router, and a second small network (routed via our firewalls) to host the servers. This is likely best done with VLAN configurations. We would like a third VLAN private between the two firewalls for firewall state syncing.
This should be provisioned like a hosted customer network would be, where the customer provides and controls their own firewall.
We would like three machines for FreeBSD admin, svn and pkg services behind this network.
We need a minimum of IPv4 /28 (/27 preferred) and an IPv6 /64 (/48 preferred) for the server subnet. We need an IPv4 /29 and an IPv6 /125 address space for the transport network. We also require some sort of out-of-band console and power access to servers (either serial port or IPMI is fine).
Suggested Minimum Hardware Specifications:
- igw0/1,admin - 2 CPU cores, 2x1Gbe network, 4G ram, 2xSATA HD 500GB space (mirror)
- pkg - 2 CPU cores, 1x1GbE network, 32G ram, 4xSATA HD (4TB) for 12TB space(raidz1)
- svn - 4 CPU cores, 1x1GbE network, 48G ram, 4xSATA HD (300 Gb minimum) for 1TB space (raidz1)
The svn mirror depends on on being able to cache the back end svn repository in ram to maintain a reasonable serving speed. At the time of writing, the two big back end repositories are 7GB for base, and 18GB for ports. Disk speed affects how quickly the cache will "warm up" after a reboot but having enough ram is far more important on an ongoing basis.
The package mirror needs less RAM and doesn't require as fast disk I/O, but needs lots of storage. The pkg mirror is a downstream slave of a master with 14TB of usable space so we're looking for about that ballpark of usable space as a minimum. The configuration doesn't matter too much, it could just as easily be 8x2TB or 16x1TB if that was easier.
Ideally each of these servers would also have at least one hot spare disk available.
To begin configuration, we would like to have one machine installed with FreeBSD and then an account configured for SSH access. We will then take over the machine via the root password and proceed to install and configure all the machines via a netboot environment.
------------------------------------------------------------------------------- | | | Vendor Router | | xxx.xxx.xxx.xxx XXXX:XXXX:XXXX:XXXX::XXXX | | | | | [ xxx.xxx.xxx.xxx/29 XXXX:XXXX:XXXX:XXXX::/125 ] | | [ transit network vlan ] | | | | | igw0-ext.XXX igw-ext.XXX igw1-ext.XXX | | (xxx.xxx.xxx.2) xxx.xxx.xxx.1 (xxx.xxx.xxxx.3) | | (XXXX:XXXX:XXXX:XXXX::2) XXXX:XXXX:XXXX:XXXX::1 (XXXX:XXXX:XXXX:XXXX::3) | | FreeBSD ____________ Firewall ____________ FreeBSD | | Firewall 1 CARP addresses Firewall 2 | | (yyy.yyy.yyy.2) yyy.yyy.yyy.1 (yyy.yyy.yyy.3) | | (yyyy:yyyy:yyyy:yyyy::2) fe80::1 (yyyy:yyyy:yyyy:yyyy::3) | | igw0.SSS igw.SSS igw1.SSS | | 10.0.XXX.1/24 pf sync(vlan0003) 10.0.XXX.2/24 | | | | | [ Internal network vlan ] | | [ yyy.yyy.yyy.yyy/28 yyyy:yyyy:yyyy:yyyy::/64 ] | | | | | +----------------------+----------------------+ | | Admin/Netboot host Package Mirror SVN/www mirror | | yyy.yyy.yyy.4 yyy.yyy.yyy.5 yyy.yyy.yyy.6 | | (yyyy:yyyy:yyyy:yyyy::4) (yyyy:yyyy:yyyy:yyyy::5) (yyyy:yyyy:yyyy:yyyy::6) | |_____________________________________________________________________________|
After the basics are set up, we configure the additional jailed services on the backend machines, assign internal addresses to them and selectively open firewall access.
At this time, we typically have two jails for pkg management - one for control and syncing, the other for serving. One for svn. Potentially several for www mirroring. Potentially two for ftp mirror.