It also makes sense because the kernel has no way of knowing. When you exec a binary, all that the kernel sees is an ELF image with a blob of bytes in the .text

It also makes sense because the kernel has no way of knowing. When you exec a binary, all that the kernel sees is an ELF image with a blob of bytes in the .text section. It doesn't know which part(s) of those blob of bytes is a signal handler. Even if it did know, the kernel has no way of verifying that the function is only ever invoked by the signal (userspace could manually call the signal handler function).