• soundsystem

---

Improving the sound system

We need some nice stuff here. Maybe tables which show what works and what doesn't, stuff we should do, problems, ... if you know something, send a message to multimedia@FreeBSD.org

Infrastructure

Generic

Resources which may be helpful to identify stuff we want to adopt, extend or improve upon:

• http://www.microsoft.com/whdc/device/audio/default.mspx: Microsoft Universal Audio Architecture

• http://developer.apple.com/documentation/MusicAudio/: Apple Audio API documentation

• http://www.alsa-project.org/: Advanced Linux Sound Architecture (ALSA)

• http://www.alsa-project.org/alsa/ftp/manuals/: some chipset manuals, the main site also has some links to chipset manuals provided by vendors

MIDI

No one is working on this. We don't even have a list what works and what doesn't.

SGI IRIX has a long-established MIDI API, which to the belive of the submitter of this information has been used in a variety of professional applications. It might be worth studying that API for ideas/gotchas.

Searching for "MIDI" at http://techpubs.sgi.com/ in the IRIX 6.5 category will bring up quite a bit of documentation, including man pages for the API.

• Generic MIDI specification: http://www.midi.org/

• USB MIDI specification: http://www.usb.org/developers/devclass_docs

Hans Petter Selasky has created a software MIDI library that uses raw MIDI interfaces. All the timing stuff is done in userland. It is available from SVN:

svn --username anonsvn --password anonsvn \

• checkout svn://svn.turbocat.net/i4b/libumidi20

svn --username anonsvn --password anonsvn \

• checkout svn://svn.turbocat.net/i4b/playmidi

Also Hans Petter Selasky has created a raw USB MIDI driver, which is part of the USB project at http://perforce.freebsd.org. The driver is included in the "uaudio" driver, but will most likely be separated into a "umidi" driver in the future.

NetBSD may have some improvements which we could port.

Docs

• The FreeBSD Architrecture Handbook contains a little bit.

• There's a doxygen generated PDF available.

• 4Front Technologies has a good userland API documentation.

• Some notes about locking in the sound system by Ryan, taken while working on his Google Summer of Code 2006 project: RyanBeasley/Notes/snd_locking

• IOCTL reference: RyanBeasley/ioctlref

Ideas

• emit events to some kind of pipe (like all the stuff in devd), e.g., an event can be emitted when a plug is inserted or removed (event-ideas are welcome!); depending upon support in the driver; devd can maybe do something then; maybe as kqueue stuff... needs to be determined.
• loopback device for midi and audio (make sound played through one channel available for recording on another channel):
  • loopback before/after fomat conversation
  • recording with format conversation (e.g., 16bit 48kHz on play, but 8bit 22,5kHz on record)

TODO

This list is lacking a lot of stuff which should be done, and the existing entries would benefit from some more detail...

• There are temporary sysctls (marked with an underscore before the name) which we need to get rid of by having another way of handling those features from userland. A non privilegued user has to be able to change this stuff (an admin is allowed to set some restrictions, e.g., max number of vchans via sysctl) with a tool.

• Add FBSDID() where it is missing.

• Add some regression tests.

Drivers

Development

There's a template (should move to share/examples). Interesting man pages for developing a driver: device, driver, devclass

Basic overview of a FreeBSD driver (not only for sound)

A DRIVER_MODULE() macro defines a driver. It specifies the driver name and the name of the driver's parent driver. It also specifies a pointer to a driver_t structure that specifies the size of the private per-unit structure (a softc), the driver requires and has a pointer to a device_method_t array of structures. The elements of this array are defined by DEVMETHOD() macros. Each DEVMETHOD() specifies a global interface name and the associated private driver routine that provides the method implementing the interface.

Key DEVMETHOD() methods are device_probe() and device_attach. The device_probe() routine is called during intial system boot and device enumeration. It simply determines if it can handle a given device (device_probe routines vote on how well they can handle the device, the winner gets the device). The device_attach routine is called to start supporting operations on the device with the driver. It often initializes and activates a driver interupt routine by calling kernel routine bus_setup_intr(), and both creates the device name in /dev and the cdevsw table by calling kernel routine make_dev(). The cdevsw structure contains pointers to the driver's private open, close, read, write, and ioctl routines (and a few others). It also contains the device text name and major number. The kernel API routines use these driver routines for the corresponding operations. Each device in a FreeBSD system is actually anchored by a specinfo structure, which has a pointer to a cdevsw.

Driver routines access hardware I/O ports and other system resources using an architecture-neutral interface that supports operations such as bus_alloc_resource, bus_space_read() and bus_space_write(). This approach was adopted from NetBSD. A resource is a structure that specifies system resources that belong to a driver. Operations and resources are specified using bus tags and bus handles. A bus tag is a number that uniquely identifies a bus. A bus handle is a number that uniquely identifies something with respect to that bus, for instance, an I/O location.

Emu10kx

This driver now has the possibility to do multichannel recording. This is not for official use, it is only to be able to play around and develop this further until the generic parts of the soundsystem gain the possibility to handle multichannel streams.

Optional multichannel recording (32 channels on Live!, 64 channels on Audigy)

All channels are 16bit/48000Hz/mono, format is fixed. Half of them are copied from sound output, another half can be used to record any data from DSP. What should be recorded is hardcoded in DSP code. In this version it records dummy data, but can be used to record all DSP inputs, for example.

Because there are no support of more-than-stereo sound streams multichannell stream is presented as one 32(64)*48000 Hz 16bit mono stream.

Channel map

• SB Live! (4.0/5.1):

Audigy / Audigy 2 / Audigy 2 Value / Audigy 4:

Envy24

Envy24-based cards are the only consumer soundcards to far, that can offer true quality. They are 'bit-perfect' (no resampling) and use high-quality I2S codecs. That is why most of them -- even not professional grade ones -- have quality and features of professional boards.

Konstantin Dimitrov is the only person that is working on the Envy24 support under FreeBSD and his free time for that is very limited, so it will be extremely slow and when you look at http://envy24.svobodno.com, in the table, you will see how much work should be done (and the table is far from being full). For example there is no code at all for DITs (Digital Interface Transceivers) and if someone write module for DITs, like 'spicds' module, it can be reused by many audio drivers, for example Creative. At least on their new boards they use the same DIT ICs like the ones on Envy24-based boards, so the work exceeds far beyond dealing with Envy24 chips. Any help by owners of Envy24 based cards is more than welcome.

From an user point of view the behavior of snd_envy24(4) and snd_envy24ht(4) (and snd_spicds(4)) is stable enough, however they are highly experimental drivers and need a serious clean-up.

Currently Konstantin is working on adding support for all chips from the VT172x family and more hardware designs. Compared to extending the existing features, this is a huge amount of work. Every Envy24-based card has an unique hardware design. For this reason adding support for an Envy24-based card requires as a first step to reverse engineer the card hardware design. This step takes a huge part of the time to support a card.

The Envy24HT (VT1724) code initially intended for use with the Envy24HT-based card "Terratec Aureon 7.1 Space" actually proved itself to work with all other VT172x chips -- Envy24PT (VT1720), Envy24MT (VT1720T), Envy24HT-S (VT1721), Envy24GT (VT1722) and Envy24DT (VT1723 aka Tremor) -- with minor or no Envy24-related changes at all. This will affect the future of snd_envy24ht(4) and it may be renamed to "snd_vt172x(4)".

Support for cards that use I2C-to-GPIO expanders wired between the control line of the codec and the Envy24 chip is upcomming. This will affect snd_envy24(4). At the moment snd_envy24(4) supports only 3 cards. The work Konstantin is doing ATM may result in support for all VT1712-based cards made by Terratec and M-Audio. Probably then snd_envy24(4) will support about 10 cards then.

snd_envy24ht(4) already has support for more than 10 cards, including all cards made by Terratec and M-Audio that are based on VT172x. Support for cards that are made by the third major manufacturer of Envy24-based cards -- ESI and their consumer division Audiotrak -- is lacking because their cards are hard to find outside Korea and Japan. Konstantin thinks they made the best Envy24-based cards out there. So if you are willing to spend such a card, feel free to him/us.

The next update to snd_envy24ht(4) will maybe support the "Philips PSC724 Ultimate Edge", "Hercules Fortissimo4" and the very new "Audiotrak Prodigy 7.1 HiFi". They all are VT172x-based cards which use the WM8766+WM8776 chips and share the same hardware design.

All of this work to support all chips from the VT172x family and all possible hardware designs takes a huge amount of work and only covers the analog playback. Other features like recording, support for all hardware mixers, MIDI and all other features the envy24 chips provide are not implemented. Adding one of those features after each other -- at least for those chips where Konstantin collected the datasheets -- should be easy. It is not that time consuming, but Konstantin can not spend time for them. Anyone interested in some of those features is encouraged to have a look at the datasheets and to send some patches.

Intel HDA

There's a HDA driver in -current now. A lot of people reported success with their hardware. Some had no luck with it. More reports are welcome.

• HDA specification: http://www.intel.com/standards/hdaudio/

Open issues

• SPDIF not working - disabled for now
• Multichannel/surround not working - The driver tries to output the sound to all possible path. If you have speakers attach to all of them, chances are it all works, but not in a true sense of multichannel/surround. There are few more works left to do on the upper layer of the sound driver to make it work properly.
• Recording is broken on some hardware.
• Pluging in headset does not mute speakers

Success/failure reports

NatSemi Geode GX1 (Cx5530 chipset)

• http://63.249.85.132/gx_audio/index.html

A little bit old. Someone with current and corresponding hardware should have a look at it and test if it works there.

USB audio and MIDI driver

• included in FreeBSD, but needs improvement
• extended one available in perforce as part of a new USB stack (see above in the MIDI section)

• official specs: http://www.usb.org/developers/devclass_docs

Driver Table

• Explanation:

  • Driver module: The name of the driver

  • Manual page: Does a manual page exist?

  • Depends on: What drivers do this driver depend on?

  • In NOTES: Is this driver in src/sys/conf/NOTES?

  • For ARCH: What architectures should we generate hardware notes for, according to src/release/doc/share/misc/dev.archlist.txt?

  • Notes: Additional notes