3 ms·
Some comments on the issues mentioned at the beginning of the article as I had the same questions in https://ossia.io https://ossia.io: - DAWs have a lot of to
by jcelerier 3y ago
Some comments on the issues mentioned at the beginning of the article as I had the same questions in https://ossia.io https://ossia.io:
- DAWs have a lot of toolbars and panels (browser panel, timeline panel, piano roll panel, fx rack panel, mixer panel, audio editor panel, automation editor panel, settings panel, etc.).
for this Qt always performed brillantly for me. I went from QDockWidget to QSplitter-based layout though.
- Some widgets like decibel meters and other visualizers are constantly being animated, meaning the GUI library needs to efficiently redraw the screen every frame.
all the CPU-based toolkits will only redraw what changed though
- In addition, visualizers can be expensive to render on the CPU (especially spectrograms/spectrometers). Ideally you should use custom shaders to render them on the GPU.
fair, but then you may pay the cost of a GPU -> CPU transfer which isn't always free
- Clips on the timeline are notoriously expensive to render. There needs to be some way to cache the contents of clips into a texture (Either directly or by making use of the GUI library's "damage tracking" which I'll get into later.)
Audio clips are the biggest culprit, because rendering waveforms requires the CPU to first do a linear search through the source material for peak values, and then render the waveform pixel-by-pixel (or even better use custom shaders to send commands to the GPU).
AFAIK a lot of DAWs perform a full scan when the file is loaded and save the result in a database or in a file next to it (e.g. Reaper's .reapeaks, Ableton's .asd...) so that you don't need to perform a complete rescan.
In ossia.io I use three different algorithms / display methods depending on the zoom level used: at "far away" zoom it uses the minmax of the audio slices, at intermediary zoom it draws lines and when getting closer, it starts drawing individual samples.
https://github.com/ossia/score/blob/master/src/plugins/score-plugin-media/Media/Sound/WaveformComputer.cpp https://github.com/ossia/score/blob/master/src/plugins/score...
- Automation clips can contain a bunch of bezier curves, which are slow to render.
Convert those to line segments with an approximation setting that looks good enough and it'll be ten times faster (keep the bezier for your data model of course).
https://github.com/ossia/score/blob/master/src/plugins/score-plugin-curve/Curve/Segment/Power/PowerSegment.cpp#L57 https://github.com/ossia/score/blob/master/src/plugins/score...
https://github.com/ossia/score/blob/master/src/plugins/score-plugin-curve/Curve/Segment/CurveSegmentView.cpp#L157 https://github.com/ossia/score/blob/master/src/plugins/score...
- Piano roll clips can contain lots of little rectangles in order to display a "minimap" of the MIDI notes inside of it.
oh damn yes, I spent so much time on this and it still needs so much optimizing... if someone wants to give a shot at it :D
https://github.com/ossia/score/blob/master/src/plugins/score-plugin-midi/Midi/MidiView.cpp https://github.com/ossia/score/blob/master/src/plugins/score...
https://github.com/ossia/score/blob/master/src/plugins/score-plugin-midi/Midi/MidiNoteView.cpp https://github.com/ossia/score/blob/master/src/plugins/score...
- On top of all this, clips can contain text labels which can also be expensive to render.
Yep, made myself a few "cached text" Qt items over time as the builtin cache wasn't satisfactory
https://github.com/ossia/score/blob/master/src/lib/score/graphics/TextItem.hpp https://github.com/ossia/score/blob/master/src/lib/score/gra...
- The fact that a timeline is zoom-able also makes it harder to cache the rendering of clips. If the timeline changed its zoom level, all visible clips pretty much have to redraw all of their contents.
yep
- Piano rolls can also be expensive to render if there is a bunch of MIDI notes, especially if there are text labels on the notes.
yep
- If the user clicks on a folder in a sample browser containing hundreds or even thousands of files, allocating a label widget for each file in the browser list will be very expensive. Something like the list factory in GTK is needed here.
yep, Qt's also able to cache this. Though for instance for Qt's QFileSystemModel I carry a small patch to disable any kind of sorting when there's more than a few hundred thousand files (which happens for large media libraries): https://github.com/jcelerier/qtbase/commit/9909c3c7902cf7a2b52d24209f0d7511d4956f7c https://github.com/jcelerier/qtbase/commit/9909c3c7902cf7a2b... and also worked a bit with upstream Qt to get it to improve (for instance it was regenerating regexes ALL THE TIME when filtering for specific file extensions)
- We want to reserve as much CPU as possible for the actual audio processing. Ideally the GUI shouldn't take up more than one or two CPU threads.
- On some platforms, we also need to make sure there's actually enough CPU left for 3rd-party plugins to render their GUIs.
yep