Epoxy the wires you absolutely need into place, fill the rest of the ports with epoxy, and put the computer in a locked metal box, with only the power button re

Epoxy the wires you absolutely need into place, fill the rest of the ports with epoxy, and put the computer in a locked metal box, with only the power button revealed. If someone is so committed to getting access that they would try to strip the wires apart to connect a USB drive through them, then search everyone for USB drives before they enter and check the state of the wires (and metal box) before they're allowed to leave. Or just throw some software at it so that a very, very loud alarm goes off if a new drive is mounted or detected, search people for USB devices before they enter, and keep regular backups (if you make it impossible to get data out undetected, the only thing you could do with software on a USB drive is corrupt or delete data).