Building an Offline-First Multilingual ePub & Audio Reader on Local Infrastructure
I have a confession. My Calibre library has 6,332 books in it, and for years I have been renting access to a bunch of reading apps instead of just reading them. Each one wanted a subscription, a login, or my reading habits. Then I wanted a voice to read my books aloud, and the cloud options wanted to bill me per character. That was the moment I decided to build my own reader.
Why Not Just Use Cloud Voices?
Cloud text to speech sounds fantastic in a two minute demo. Then you do the math on a 400 page novel, and you realize you would also be shipping the entire text of your book to somebody else's server. No thanks.
Here is the fun part. Your phone already carries great neural voices. Apple, Android, and Windows all ship them. The browser can reach them through the Web Speech API, so I decided the reader would do all the talking on the device with no API keys and no monthly bill.
There is a sneaky catch, though. Some browsers quietly send speech requests off to a cloud service when you are not looking. So the reader checks every voice and only uses the ones where voice.localService is true. If it is not local, it does not get to speak.
The Plumbing
The server side turned out to be pleasantly boring. A small FastAPI app runs in a Docker container on one of my homelab machines. It mounts the Calibre folder from my NAS read-only and reads the metadata.db catalog directly, so searching all 6,332 titles feels instant. HAProxy handles the HTTPS in front, and I can open the reader from anywhere on my network and on my phone.
I limited the voice picker to English and Mandarin Chinese, since that is what my family actually listens to, and I made dark slate the default because I read at night.
Where My Phone Fought Back
The first real test was walking around with my phone in my pocket. The screen locked, and about thirty seconds later the voice just stopped. Mobile browsers are ruthless about putting background tabs to sleep to save battery.
The fix was a two part trick. I ask for a screen wake lock with navigator.wakeLock, and I also play a tiny silent audio loop in an audio element. The silent loop convinces the phone that real audio is still playing, so it keeps the pipeline awake. It feels like a hack, and it works.
Then came my favorite bug. In some books, a sentence with bold text or a footnote link got chopped into pieces, and the reader would say the same fragment again and again like a broken record. The inline tags were splitting my text into duplicate chunks. I rebuilt the sentence splitting with Intl.Segmenter and grouped the pieces back together, with a highlight that flows nicely across the tags.
Reading on My Own Terms
Now I open the reader on my phone in dark mode, pick a local voice, and the words light up one sentence at a time while I cook or walk. It costs nothing each month, it has no lag, and my books never leave my house. Honestly, it is one of the most satisfying things I have built this year.
If you want to build your own library stack, I wrote a step by step guide on By Design Won't Fix, covering Calibre-Web and Audiobookshelf in Docker backed by a NAS share: Deploying Audiobookshelf and Calibre-Web in Docker Backed by a Central NAS Share. What is the biggest book collection you have hiding on a hard drive?
Later,
SteveO