This sounds very close to how I started out with a similar project, named Rhasspy: https://github.com/synesthesiam/rhasspy-hassio-addon https://github.com/synes

This sounds very close to how I started out with a similar project, named Rhasspy: https://github.com/synesthesiam/rhasspy-hassio-addon https://github.com/synesthesiam/rhasspy-hassio-addon I ended up using a set of JSGF grammars (one per intent) to generate a statistical language model for use with pocketsphinx. Rhasspy also features a web-based interface for creating custom words -- I have a mapping from Sphinx phonemes to eSpeak phonemes so you can iterate over a pronunciation until it sounds right. As you mentioned, the wake/hotword stuff with Sphinx isn't terribly robust. I've been Docker-izing Mycroft Precise (https://github.com/MycroftAI/mycroft-precise https://github.com/MycroftAI/mycroft-precise) to address this.