3 ms·
It’s just a scratch that needed itching, but I wrote a command-line utility for translating “SRT” format subtitles into other languages. I hit some interesting
by jiggawatts 3y ago
It’s just a scratch that needed itching, but I wrote a command-line utility for translating “SRT” format subtitles into other languages.
I hit some interesting challenges, overcoming which was a valuable set of lessons learnt:
1. GPT4 Turbo slowed down to molasses in some Azure regions recently. Microsoft is not admitting this and is telling people to use GPT3.5 instead. The lesson learned is that using a regional API exposes you to slowdowns and queuing caused by local spikes in demand, such as “back to school” or end of year exams.
2. JSON mode won’t robustly stick to higher level schemas. It’s close enough, but parsing and retries are required.
3. The 128K context in GPT4 is only for the input tokens! The output is limited to 4K.
4. Most Asian languages use as many as one token per character. Translating 1 KB of English can blow through the 4 KB token limit all too easily.
5. You can ask GPT to “continue”, but then you have to detect if you received a partial or a complete JSON response, and stitch things together yourself… and validate across message boundaries.
6. The whole process above is so slow that it hits timeouts all over the place. Microsoft didn’t bother to adjust any of their default Azure SDK timeouts for HTTP calls. You have to do this yourself. It’s easy, just figure which of the three different documented methods are still valid. (Answer: none are.)
7. You’ll need a persistent cache. Just trust me on this.
I simply hashed the input and used that as a file name to store responses that passed the checks.
8. A subtitle file is about 30–100 KB so it needs many small blocks. This makes the AI lose the context. So it’s important to have several passes so it can double check and stitch things together. This is very hard with automatic parsing of outputs.
9. Last but not least: the default mode of Azure is to turn the content policy up to “puritan priest censoring books”. Movies contain swearing, violence, and sex. The delicate mind of the machine can’t handle this, and it will refuse to do as it is asked. You have to dial it down to get it to do anything. There is no “zero censorship” setting. Microsoft says that I can’t feed text to an API that I can watch on Netflix with graphic visuals.
10. The missus says that the AI-translated subtitles are “perfect”, which is a big step up from some fan translated subtitles that have many small errors. Success!
I wrote this as a C# PowerShell module because that makes it easy to integrate the utility as a part of a pipeline. E.g.: I can feed it a directory listing and it’ll translate all of the subtitles.
The performance issues meant I had to process 8x chunks in parallel. Conveniently I already had code lying around to do this in PowerShell with callbacks to the main thread to report progress, etc…