There is a lot of it depends. On an empty stomach 20 minutes does seem a bit long. However you're also conditioned to the feeling you expect coffee to provide,

There is a lot of it depends. On an empty stomach 20 minutes does seem a bit long. However you're also conditioned to the feeling you expect coffee to provide, so the initial jolt of energy may have almost nothing to do with the actual caffeine and more to do with the thought of the caffeine.