9 ms·
I'd heard over the years that working with isochronous systems was difficult. I'd done a number of real-time systems before, and written OS schedulers and NTP-l
by kabdib 12y ago
I'd heard over the years that working with isochronous systems was difficult. I'd done a number of real-time systems before, and written OS schedulers and NTP-like systems and so forth. A little audio work should be a walk in the park, right? A little manly-man programming from the wrist and we move on to real problems. So I walked into an audio project thinking that "Oh, this latency and synchronization stuff, how bad could it really be?"
Bad.
Nailing down buffer-bloat and sources of latency and jitter in an isochronous system took several months, the last few weeks of which were 80-100 hour weeks just before ship. Several times we thought we'd fixed the issues, only to find that our tests had been inadequate, or that some new component broke what we had built.
I remember nearly being in tears when I finally realized what the clock root of the audio system actually was, and that it wasn't what people had been using. From there everything fell into place.
Don't pull the number of buffers you have out of thin air ("Oh, six is enough, maybe twelve, don't want to run out in my layer of the system, after all...") Don't assume your local underflow or overflow recovery strategy actually works in the whole system. Don't assume your underlying "real time" hypervisor won't screw you by halting the whole god damned system for 20ms while it mucks around with TLBs and physical pages and then resumes you saying, "Have fun putting all the pieces of your pipeline back together, toodle-oo!" Put debug taps and light-weight logging everywhere and attach them to tests. And know what your clock root is, or you will be sunk without a trace.
Isoch is hard.
- weaksauce 12y agoA real postmortem of this would be a fascinating read.
- kabdib 12y agoWell, if something bad happens I might have to work there again, so . . . not for a few years, probably :-)
- walterbell 12y ago>when I finally realized what the clock root of the audio system actually was How did you finally figure this out -- any recommended references?
- gone35 12y ago"A little manly-man programming from the wrist and we move on to real problems." See... it's the little things like this. I am pretty sure that was not your intention, but please do know that turns of phrases like that hurt a little, and exclude a little. To see what I mean, s/man/jew/ or s/man/white/ or some other category and see how it reads.
- engi_nerd 12y agoI did the substitution in my head and the statement sounds silly. Just like "manly-man" sounds silly. Is this genuinely offensive and exclusive though?
- Frondo 12y agoTo call it "offensive" may be a bit strong (though I'd say it's offensive to those of us who enjoy good use of language ;) ), but exclusive? Sure. It links superior programming and masculinity. You're sitting in a conference room full of women who are developers, and you start saying, "So let's do some manly-man programming and get this done," what are these women supposed to think?
- joenathan 12y agoOne would hope they were mature enough to understand an idiom, and not so delicate that they would need to be handled with kid gloves.
- Frondo 12y agoReally? So if your dev team consisted of a mix of men and women (not even counting if anyone identified as neither or somewhere in between), you'd still say "Let's put in some manly-man programming time" to them? What if your dev team consisted of just one employee, a woman? Would you still tell her to put in her manly-man programming time? I'm having trouble seeing it. "Jane, it's crunch time. Can you man up and put in some manly-man programming time this weekend?"
- cbd1984 12y agoNone of this is correct.
- cbd1984 12y agoDown votes don't make me wrong.
- cbd1984 12y agoFlagged for sexism.