You are looking for what we call an "algorithmic music language", where the focus is on manipulating symbolic music representations rather than digital audio.

You are looking for what we call an "algorithmic music language", where the focus is on manipulating symbolic music representations rather than digital audio. OpusMondi is one of these. Others include my project (Scheme for Max, mentioned elsewhere at top level of the thread), Common Music, OpenMusic, etc. Visual options include Max and Pd, which have an entire message/event layer that does not have to be used for digital audio at all. All of these can output to standard synthesizers if the digital sound part is not your interest.