If you measure velocity then the points are additive, how do you reconcile 3 point tasks doable in a day and 5 point tasks not doable in a reasonable amount of

If you measure velocity then the points are additive, how do you reconcile 3 point tasks doable in a day and 5 point tasks not doable in a reasonable amount of time? Isn't 5 points less than 2 days then? For the record, in my team we don't use story points or measure velocity. I just had a frustrating discussion with a peer team that didn't plan an important task because "its 5 points and does not fit into a sprint".