It's hidden weirdly.... go to "Keyboard" prefs pane and "Keyboard" tab (initial one shown when you bring it up. At "Touch Bar shows".... change that from "Ap

It's hidden weirdly.... go to "Keyboard" prefs pane and "Keyboard" tab (initial one shown when you bring it up. At "Touch Bar shows".... change that from "App Controls with Control Strip" to "Expanded Control Strip". Then you will see the individual buttons. You can then further "Customize Control Strip..." and you will now see more buttons. But you will also loose "App Control". Of course, why can't those specific buttons be a customization option there with "App Control with Control Strip" forcing you to choose between volume up/down and "App Control"? Because Apple.