You need to have a memory disclosure bug (they're very common). You don't even need to be able to read all of memory, you just need to know the start offset of

You need to have a memory disclosure bug (they're very common). You don't even need to be able to read all of memory, you just need to know the start offset of where the library is loaded (if the library is from a distribution you can download it and figure out the ROP offsets using the library -- though apparently OpenBSD has defences against that too).