I've started searching for something similar, and it looks like such devices are now referred to as DAPs (digital audio players), and the market is very small,

I've started searching for something similar, and it looks like such devices are now referred to as DAPs (digital audio players), and the market is very small, mostly served by devices targeting audiophiles. I'm considering getting a used iPod touch (2019) and then locking it down to just the music player app(s) using parental controls.