To make a new style FreeBSD pkg and svn mirror, we have come up with the following design.

We would like two machines for FreeBSD gateways that allow the project to configure firewalls and routing for the project. These two machines should have connections for outside and inside networks. We would like to have a small transit network between our firewalls and the upstream router, and a second small network (routed via our firewalls) to host the servers. This is likely best done with VLAN configurations. We would like a third VLAN private between the two firewalls for firewall state syncing.

This should be provisioned like a hosted customer network would be, where the customer provides and controls their own firewall.

We would like three machines for FreeBSD admin, svn and pkg services behind this network.

We need a minimum of IPv4 /28 (/27 preferred) and an IPv6 /64 (/48 preferred) for the server subnet. We need an IPv4 /29 and an IPv6 /125 address space for the transport network. We also require some sort of out-of-band console and power access to servers (either serial port or IPMI is fine).

Suggested Minimum Hardware Specifications:

The svn mirror depends on on being able to cache the back end svn repository in ram to maintain a reasonable serving speed. At the time of writing, the two big back end repositories are 7GB for base, and 18GB for ports. Disk speed affects how quickly the cache will "warm up" after a reboot but having enough ram is far more important on an ongoing basis.

The package mirror needs less RAM and doesn't require as fast disk I/O, but needs lots of storage. The pkg mirror is a downstream slave of a master with 14TB of usable space so we're looking for about that ballpark of usable space as a minimum. The configuration doesn't matter too much, it could just as easily be 8x2TB or 16x1TB if that was easier.

Ideally each of these servers would also have at least one hot spare disk available.

To begin configuration, we would like to have one machine installed with FreeBSD and then an account configured for SSH access. We will then take over the machine via the root password and proceed to install and configure all the machines via a netboot environment.

|                                                                             |
|                               Vendor Router                                 |
|             XXXX:XXXX:XXXX:XXXX::XXXX              |
|                                      |                                      |
|                 [  XXXX:XXXX:XXXX:XXXX::/125 ]           |
|                          [ transit network vlan ]                           |
|                                      |                                      |
|      igw0-ext.XXX               igw-ext.XXX             igw1-ext.XXX        |
|      (            (     |
|        FreeBSD   ____________    Firewall   ____________  FreeBSD           |
|       Firewall 1              CARP addresses             Firewall 2         |
|     (yyy.yyy.yyy.2)            yyy.yyy.yyy.1          (yyy.yyy.yyy.3)       |
| (yyyy:yyyy:yyyy:yyyy::2)         fe80::1          (yyyy:yyyy:yyyy:yyyy::3)  |
|        igw0.SSS                  igw.SSS                  igw1.SSS          |
|     10.0.XXX.1/24           pf sync(vlan0003)         10.0.XXX.2/24         |
|                                      |                                      |
|                          [ Internal network vlan ]                          |
|                [ yyy.yyy.yyy.yyy/28  yyyy:yyyy:yyyy:yyyy::/64 ]             |
|                                      |                                      |
|               +----------------------+----------------------+               |
|          Admin/Netboot host    Package Mirror        SVN/www mirror         |
|          yyy.yyy.yyy.4          yyy.yyy.yyy.5         yyy.yyy.yyy.6         |
|  (yyyy:yyyy:yyyy:yyyy::4) (yyyy:yyyy:yyyy:yyyy::5) (yyyy:yyyy:yyyy:yyyy::6) |

After the basics are set up, we configure the additional jailed services on the backend machines, assign internal addresses to them and selectively open firewall access.

At this time, we typically have two jails for pkg management - one for control and syncing, the other for serving. One for svn. Potentially several for www mirroring. Potentially two for ftp mirror.

Teams/clusteradm/generic-mirror-layout (last edited 2014-07-10 13:57:14 by GavinAtkinson)