Ten64
The Ten64 networking platform is a product by Traverse Technologies and they kindly provided me with a full system. It's a NXP LS1088a based platform featuring DPAA2 (FreeBSD driver development) support with 8xGE + 2xSFP+, USB3, multiple mPCIe slots, .. See their excellent documentation for all the details.
insert nice photos of the machine here
Traverse Technologies recently published a FreeBSD preview for Ten64 image.
I am just jotting down my notes regarding FreeBSD support here.
Current status (20220921; slightly outdated probably but as of February 2024)
FDT support for MDIO for GE0-7 (copper ports) was developed for DmitrySalychev's DPAA2 driver DPAA2: FDT MDIO support #16 pull request
- FreeBSD is lacking (generic) SFP+ FDT support currently; to be added another day.
Support for the PCA9540 I2C MUX for the SFP+s was done: Review D35701 commit to main and commit to stable/13.
LS1088a clockgen support was developed FreeBSD review D35617 commit to main commit to stable/13 and helps:
- SD card probes properly. I tested with MMCCAM and reading the SD card was fine.
Mathew McBride's ds1307 extensions for the Epson RX-8035 is out-of-tree work in progress. Compiling the device ds1307 into the kernel or loading the module from loader seems to make things happy.
If loaded as module from multi-user we do not get the time set even if manually triggering a sysctl debug.clock_do_io=1. Reason for this is that vfs_moutroot() is responsible to have the clock set and calls CLOCK_GETTIME() indirectly so that log files or other timestamps do not appear to go backwards (if no RTC is found it'll use the vfs root node date?). Look for the inittodr() call.
- There seems to be still an issue restoring hours; I lost 8 of them over night ... to be investigated. Matt might have solved this.
Review D35837 for RTC. commit to main MFC to stable/13
- SSD seems to work fine
- USB3 and PCIe (M.2) put something in each and was detected flawlessly.
- We need more I2C mux bits for SFP and Admin LED from what I can see. Should be quick to do.
- First one done (see PCA9540 above)
Second one (ti,tca9539) here Review D35702 commit to main commit to stable/13 gives you Admin led support (and other bits); need to compile into the kernel or load from loader it seems ... try this: sh -c 'while : ; do echo "0" > /dev/led/ten64:admin && sleep 1 && echo "1" >> /dev/led/ten64:admin && sleep 3; done' will turn the LED off for 1s and then on for 3s. Look at led(4) for more ideas like morse ...
Optional and only nice to have would be proper VSC8514 mii support. Recognised since this commit
fsl,dpaa2-console for FDT got committed with this commit
I need to go and see what else then. soctherm timeouts probably? That's WIP now. Review D35759 limits the number of sites to the values found in device-tree for the SoC as a first step. Next is to deal with the differences of the SoCs, Review D35764. commit 1 to main and commit 2 to main, commit to stable/13, commit to stable/13.
hw.temperature.core-cluster: 58.0C hw.temperature.soc: 57.0C root@ten64-01:~ # sysctl hw.temperature hw.temperature.core-cluster: 55.0C hw.temperature.soc: 55.0C
uart0: console (115131,n,8,1) - where does this odd value come from?
Here's a list of device compat strings, not disabled, with no driver attached:
1 arm,mmu-500 (no driver attached) -- iommu 8 arm,sp805-wdt (no driver attached) 1 fsl,dpaa2-ptp (no driver attached) 1 fsl,ls1088a-ftm-alarm (no driver attached) 1 fsl,ls1088a-rcpm (no driver attached) 1 fsl,sec-v5.0 (no driver attached) 1 gpio-keys (no driver attached)
Fan control: EMC2301 support but I have not idea how on FreeBSD we would link the temperature thermal-zones information an fan control etc. all together. powered?
More of the I2C devices.
How to switch between MDIO and FIXED for interfaces
If you want to make use of link-state and media information and be able to set it for the 8 GE ports you want to follow the documentation and do the following in the U-Boot console once:
env delete gbemode saveenv reset
You do not want to do the same for sfpmode just yet until FreeBSD grows FDT/SFP support.
How to netboot
I am not sure what the proper method is but I ended up adjusting the U-Boot menu and added a run setup_distroboot_efi && before the run dhcp which makes sure all the bits needed for DPAA2 are loaded.
SFP GPIOs
While I do have trouble with the i2c implementation on FreeBSD (might be personal), while DmitrySalychev had patches for sfp support early 2023 I wrote a short script to name and check the GPIOs:
# dev.gpio.4.%pnpinfo: name=gpio@76 compat=ti,tca9539 # dev.gpio.4.%desc: TCA6416 I/O expander # gpioctl -f /dev/gpioc4 # PIN 0 - 7 GC="gpioctl -f /dev/gpioc4" # # gpioctl -f /dev/gpioc4 -l # pin 00: 0 gpio_P00<IN> # pin 01: 1 gpio_P01<IN> # pin 02: 0 gpio_P02<IN> # present # pin 03: 0 gpio_P03<IN> # pin 04: 0 gpio_P04<IN> # pin 05: 1 gpio_P05<IN> # pin 06: 0 gpio_P06<IN> # present # pin 07: 0 gpio_P07<IN> # pin 08: 0 gpio_P10<IN> # pin 09: 0 gpio_P11<IN> # pin 10: 0 gpio_P12<IN> # pin 11: 0 gpio_P13<IN> # pin 12: 0 gpio_P14<OUT> # pin 13: 0 gpio_P15<OUT> # pin 14: 1 gpio_P16<IN> # pin 15: 0 gpio_P17<IN> check_name() { local val spfs zeros ones val=$1 sfps="$2" zeros="$3" ones="$4" case "${val}" in 0) s="${sfps}: ${zeros}" ;; 1) s="${sfps}: ${ones}" ;; *) s="${sfps}: (invalid)" ;; esac } case "$1" in SET_NAMES) ${GC} -n 0 "Lower SFP+ TXFAULT" ${GC} -n 1 "Lower SFP+ TXDISABLE" ${GC} -n 2 "Lower SFP+ PRESENT" ${GC} -n 3 "Lower SFP+ LOS" ${GC} -n 4 "Upper SFP+ TXFAULT" ${GC} -n 5 "Upper SFP+ TXDISABLE" ${GC} -n 6 "Upper SFP+ PRESENT" ${GC} -n 7 "Upper SFP+ LOS" ;; SET_IN_OUT) ${GC} -c 0 IN PU ${GC} -c 1 OUT PU ${GC} -c 2 IN ${GC} -c 3 IN ${GC} -c 4 IN PU ${GC} -c 5 OUT PU ${GC} -c 6 IN ${GC} -c 7 IN ;; SET_ON) # Turn both SFP+ on ${GC} 1 1 ${GC} 5 1 ;; SHOW_STATUS) for i in `jot 8 0`; do v=`${GC} ${i}` s="" case ${i} in 0) check_name ${v} "Lower SFP+" "no transmitter fault" "transmitter Fault" ;; 1) check_name ${v} "Lower SFP+" "transmitter disabled" "transmitter enabled" ;; 2) check_name ${v} "Lower SFP+" "present" "not present" ;; 3) check_name ${v} "Lower SFP+" "lost signal" "has sginal" ;; 4) check_name ${v} "Upper SFP+" "no transmitter fault" "transmitter Fault" ;; 5) check_name ${v} "Upper SFP+" "transmitter disabled" "transmitter enabled" ;; 6) check_name ${v} "Upper SFP+" "present" "not present" ;; 7) check_name ${v} "Upper SFP+" "lost signal" "has sginal" ;; *) s="(unknown)" ;; esac printf "%d\t%d\t%s\n" ${i} ${v} "${s}" done ;; *) printf "USAGE: invalid command '$1': [SET_NAMES|SET_IN_OUT|SET_ON|SHOW_STATUS]\n" >&2 exit 1 ;; esac # end