System-wide speakup

Is it possible to play audio 'globally' or system-wide in Linux? This article is the notes and ramblings of someone who tries to do that.

Background

The Linux kernel includes the Speakup screen reader. Unlike most desktop screen readers, Speakup will aid in reading the Linux virtual TTYs that you can switch to with Ctrl+Alt+F1 through Ctrl+Alt+F6 on most systems.

This functionality is vitally important for troubleshooting in a Linux system and running command line applications, which is unfortunately one of the more accessible experiences on Linux.

Speakup alone is not enough to implement a screen reader: It still needs some speech synthesizer to turn text and control signals in to audio. Once upon a time hardware synthesizers connected over serial were used to do this task, but as computers became more capable software synthesizers became viable.

Speakup supports software synthesizers by exposing a character device in /dev/softsynth. A connector such as connects to this device and implements the synthesizer protocol and handles playback of speech. In practice the software synthesizer is espeakup which plays speech using eSpeak NG.

The problem

- speakup is system-wide, not multi-user. so it must always have access to the audio device. the problem is that this isn't possible in linux.

- espeakup runs as root boot up and log in as root- works fine

- log in as yourself- no audio, pulseaudio claims the sound card. or tries to and gives up because root is playing audio, leaving you without sound

- run espeakup as your own user- you can read what root does, but switching to a root tty now goes silent

- run pulseaudio systemwide is not recommended

- pipewire has most of the same problems

Linux audio history

- ALSA dmix

- exacerbated by sound servers locking hardware

- exacerbated by sound servers giving up seats

- there's no system-wide audio

TODO: when did sound servers lock hardware?

TODO: when did sound servers give up hardware for seats?

system and pulseaudio coordinate will snatch the audio whenever you change to a seat assigned to your user. maybe you could modify these to not cede the audio outside a specific seat (in this case seat = virtual tty you switch in linux with ctrl alt f1 through f7)?

here's some more nightmare stuff i realized: seats only give permission to device nodes. applications still have to gracefully hand off the hardware during a seat switch. pulse does this just by shutting up

pulseaudio doesn't gracefully give up audio, only alsa -> pulseaudio

- pulseaudio race

Attempt 1: Sharing Speakup

- have a proxy that sits between speakup and espeak

- send messages to espeakup instances based on current active UID

- during a switch between instances, wait for the current instance to finish talking OR the stop talking control is sent. then start feeding the new instance data

- have a shim that blocks pulseaudio from starting until it has permission, but also don't consume the buffer

i do not like how i'm basically reinventing flow control but poorly

ok so it turns out i was WRONG: you can't share speakup protocol between multiple synths! the protocol is stateful! ie if you tell it to change voice and switch synth the voice change won't be applied. YAY

Attempt 2: Sharing synth data

so this makes the only viable solution to send PCM data from a root espeakup instance.

on top of that this also means i have to modify espeakup to handle some flow control AND output to a buffer instead of the sound card