2 ms·

With X11 you don't even need a programmable keyboard for this, just make your own xkb layout. These support multiple layers just fine.