It depends what you want to do with the data. If you want to send your sRGB-profile Y'CbCr JPEG image directly to an sRGB display then you are done after applyi

It depends what you want to do with the data. If you want to send your sRGB-profile Y'CbCr JPEG image directly to an sRGB display then you are done after applying a matrix. If you want to composite multiple images, do other intermediate processing, or display the image on an arbitrary non-sRGB display, you probably want to convert to a linear space along the way.