You have a unique definition of "handle" that doesn't seem to include "your OS's kernel packet filter keeps working to pre-filter these packets based on an L4 u

You have a unique definition of "handle" that doesn't seem to include "your OS's kernel packet filter keeps working to pre-filter these packets based on an L4 understanding of them before handing them to userspace, or after being handed them by userspace." Which, if your machine is acting as something like a router/NAT/firewall, is kind of... the entire point of the box being there in the communication path.