Great article, thank you for sharing. About the machine learning model; forgive me if any of these questions were covered in the article: 1) Why classify as

Great article, thank you for sharing. About the machine learning model; forgive me if any of these questions were covered in the article: 1) Why classify as three categories (music, talk, ads) versus just two (ads, content)? Seems like that might help simplify things, since I assume all we care about is whether something is an ad or not. 2) So based on the description you're feeding 4 seconds worth of buffer into the model for classification, every second; the model being a stateless RNN. I'm curious, did you try just using a (1D) CNN instead? That would allow you to use a more robust architecture, versus RNNs which tend to be finicky. (And RNN doesn't seem to provide benefit, since you aren't using its state, other than potentially being a smaller model). 3) AFAIK there are loss functions which penalize false positives (in this case, penalizing incorrectly tagging content as ads). Was this experimented with? 4) This one is more of a curious idea: 4 seconds worth of buffer might lack enough context, which we can see play out in some of the failings alluded to in "Future improvements". So I'm curious... Suppose, if your architecture isn't set up this way already, that you've got a layer before your final layer that's just a little bit bigger than the number of categories. Say, 32 features. (A lot of RNN architectures are built with an output matrix that's HiddenState => Outputs (n=3). So I'm suggesting HiddenState => Embedding [n=32] => Outputs [n=3]). Train that architecture like before. Now, take that trained model and chop off the output layer so that its output is those 32 element vectors. We can now build a secondary, context model on top of it. It takes as input those 32 element vectors (one vector per second) and outputs the same classifications (ad/not-ad). But since its input is much smaller, you can train an RNN with a much, much deeper BPTT. Hopefully this model can not only be more intelligent by using context, but also take the place of your hysteresis. Since the underlying model is fixed, you can pre-compute its outputs on your dataset, so there's absolutely no cost while training the higher order model (and the pre-computed dataset will be really tiny; 32 elements every second would be only 100mb for your 10 day corpus.) Now, this is somewhat dependent on your corpus being of the nature "here's a long section of audio where the ad is tagged precisely". i.e. it can just be a mixed bag of "this whole audio segment is ad/not-ad". If you don't have that kind of data you'd need to create it (assuming my crazy idea is worth the effort). Hopefully you wouldn't need much. This higher order model shouldn't need to be very complex, which means it doesn't need a big corpus. Or you could try to use a CTC loss function, in which case you just need a dataset that's large chunks of audio and vague labels like "this length of audio has 1 ad somewhere in it." By the way, this isn't intended as advice; I'm by no means a domain expert here. I'm really just thinking out loud. And since none of the other comments at the time of this writing are discussing anything other than the morality of this software, I thought I'd take the time to inject some more technical discussion. > hip-hop music, easily mispredicted as advertisements Well given the endemic use of name dropping and product placement in popular hip-hop, I'm not sure that's a misprediction ... :P Joking aside, one thing that might be useful is to find a stream with positively no ads (e.g. Spotify Premium, Apple Music, etc). Play that through a trained classifier. If the classifier ever detects an ad, add that audio sample to your corpus.