Almost completely unscientific answer: I have dry skin and can vouch for the fact that your body tends to "find a balance" if you stop using soap, and I would

Almost completely unscientific answer: I have dry skin and can vouch for the fact that your body tends to "find a balance" if you stop using soap, and I would imagine that it would be similar for those with oily skin (it could be that you are creating a lot of oil because the soap is drying out your skin. On the other hand, you may find that you just end up even oilier. The only way to truly find out would be to test it yourself. You may just want to try a less drying soap so your body doesn't need to generate as much oil.