The biggest downside I see is that it would no longer be possible to host a simple website, without also having to publicly expose the vastly increased attack s

The biggest downside I see is that it would no longer be possible to host a simple website, without also having to publicly expose the vastly increased attack surface that the OpenSSL code base brings to the table.