FreeBSD Developer Summit: Advancing ZFS Working Group
Discussing advancing ZFS on FreeBSD
- Integrating ZFS features early and often
- Making FreeBSD the goto place for the latest ZFS features
OpenCrypto (ZFS native encryption, need AES-CCM)
- XUIO (Zero Copy, and sendfile, for ZFS)
- Fault Management
- Storage management tools
- Compressed ARC
- RAID-Z performance improvements
- Faster scrub/resilver
- Channel Programs
- Device Removal
- Redacted Send/Recv
- TRIM support
- vdev metadata classes
- multi-mount support
- zpool checkpoint
- persistent L2ARC
Zero Copy / Sendfile / XUIO
ZFS currently does not play nicely with sendfile(2). The idea with sendfile(2) is to share pages between the buffer cache and the mbuf, to prevent copying. The call results in a range of bytes from an open file descriptor being faulted into the buffer cache, and then those pages being shared all the way through to the network stack until the bytes are transmitted.
With ZFS, since it does not use the OS buffer cache, data is copied from the ZFS ARC to the buffer cache, where it is then shared with the mbuf etc. Solaris has a solution to solve this issue, and it might make sense to adapt our sendfile mechanism to work in a similar way. [Solaris PSARC: Copy Reduction Interface](https://email@example.com/msg17203.html) This creates two new VFS operations, reqzcbuf which asks the filesystem to prepare a UIO for zero copy operations. In the case of ZFS, this involves allocating a new array to track the ARC headers of each buffer that is loaned out from the ZFS ARC.
The member xu_zc_priv of the extended uio structure for zero-copy is a private handle that may be used by the provider to track its buffer headers or any other private information that is useful to map the loaned iovec entries to its internal buffers. The xu_zc_priv member is private to the provider and should not be changed or interpreted in anyway by the callers.
Then the retzcbuf VFS operation is used to return that buffer to the filesystem and it can then free the private member and do the necessary housekeeping. Implementing this same VFS operation for UFS and other filesystems may be a better approach to the future of sendfile(2).
OpenCrypto does not have an implementation of AES-CCM, which will be required in the future for OpenZFS's native storage encryption.
OpenZFS would appreciate any additional review of the native encryption patch by security and crypto experts.
The Linux approach was to port the entire IllumOS Crypto Framework, this is an option but I would expect it will take longer to integrate into FreeBSD, and won't as easily be able to take advantage of accelerators like QuickAssist.
OpenZFS has some patches to vectorize some calculations, like the RAID-Z parity, and some of the checksums. Need someone to integrate these into FreeBSD. PR218203 AVX2 accelerate fletcher4 for ZFS
Uses the SES (SCSI Enclosure Services) protocol to query and interact with backplanes to monitor disks and allow setting the locate and fault LEDs. This does not work in the case of direct-attach backplanes, and AHCI devices. In some of these cases there is still a fault/locate LED that could be used. Could mps(4) and mpr(4) be modified to create a virtual ses device similar to what AHCI does.
Native implementation of the LSI/Avago/Broadcom control tools, to view and manage the disk controller. Also allows updating the firmware. Currently does not support toggling the fault/locate LEDs. Just need to figure out how sas2ircu does it, and reimplement it.