4 ms·

There is a lesson here about the way we go about sharing memory between threads that is implicit and fundamentally broken. Adding extra threading in order to im

by gizmo 6y ago

There is a lesson here about the way we go about sharing memory between threads that is implicit and fundamentally broken. Adding extra threading in order to improve performance is clearly the wrong tradeoff when you're in an environment where threads can introduce bugs like these. When it takes 10 senior engineers to figure out why one user session gets data from another session you've already lost. Shared memory environments are very unsafe and we should be working towards a memory model where data that belongs to different organizations can never get mixed. This isn't impossible, but it requires new programming languages that allow us to express which memory areas can't get mixed up. We've got to start designing our backend infrastructure differently such that this entire class of memory errors can't ever occur.

lstamour 6y ago
In many languages and web frameworks, the assumption is that the application server will do this for you. For example, by issuing a new env variable with each request such that requests can't easily share memory with other requests, though as noted that might not stop background threads from retaining such memory. In other cases, you might assume containers would be enough separation, or maybe different databases (sharding, for example). One approach I've considered but not yet implemented is to take end-user JWT cookies and use them to authenticate to a database such that a user can only ever see the rows they should be allowed access to, and all other SQL queries would fail. But no matter how you implement a security control, it's possible to make mistakes and allow others to see data you didn't intend to share with them. Even Rust, famous for its explicit memory ownership model, could still have similar bugs, they just might be a bit more obvious to spot in some session connection pool implemented incorrectly, for example, or maybe exposed via a compiler optimization gone wrong. Code review will still likely be one of the primary defenses of this kind of bug, though hopefully better tooling and language support to detect race conditions or shared ownership warnings will help over time?
DougWebb 6y ago
Replace Rails and similar server-side frameworks with CGI scripts, and you're done. Every request is a new process with its own memory space. Yes, performance is an issue with that old approach. Around the turn of the century I was responsible for a Perl web application that ran as a CGI script, and to speed it up I wrote my own Perl httpd server that had my web application code pre-loaded. Then for each request I forked a new process, which carried over all of the pre-loaded code and already-running perl.exe, so there was very little startup time (compared to a normal CGI script startup.) I was careful to take full advantage of the operating system's Copy-on-write memory semantics, so that the only memory that had to be allocated for the new process was Perl's runtime stack. All of the memory containing the application code was shared across processes because it never got written to. This web application is still running today, though I left the company in 2010. Back then, it was handling about six million requests per day, about 75% of which was during US daytime working hours. So, around 160 requests/second, spread across five or six servers that were getting old in 2010. The modern frameworks have taken the same concept a step further, and replaced the process fork with a thread pool. That's faster, but it allows memory leaking in a way process forking does not. You have to go out of your way to share memory between processes, but with threads it's easy to do accidentally.
- hu3 6y ago
  Workerman [1] uses a similar approach to put PHP pretty high on TechEmpower [2]. Specially now with PHP8's JIT. [1] https://github.com/walkor/Workerman https://github.com/walkor/Workerman [2] https://www.techempower.com/benchmarks https://www.techempower.com/benchmarks
Cloudef 6y ago
It's called separate processes or unshare after new thread.

by gizmo 6y ago

lstamour 6y ago

In many languages and web frameworks, the assumption is that the application server will do this for you. For example, by issuing a new env variable with each request such that requests can't easily share memory with other requests, though as noted that might not stop background threads from retaining such memory. In other cases, you might assume containers would be enough separation, or maybe different databases (sharding, for example). One approach I've considered but not yet implemented is to take end-user JWT cookies and use them to authenticate to a database such that a user can only ever see the rows they should be allowed access to, and all other SQL queries would fail. But no matter how you implement a security control, it's possible to make mistakes and allow others to see data you didn't intend to share with them. Even Rust, famous for its explicit memory ownership model, could still have similar bugs, they just might be a bit more obvious to spot in some session connection pool implemented incorrectly, for example, or maybe exposed via a compiler optimization gone wrong. Code review will still likely be one of the primary defenses of this kind of bug, though hopefully better tooling and language support to detect race conditions or shared ownership warnings will help over time?

DougWebb 6y ago

Replace Rails and similar server-side frameworks with CGI scripts, and you're done. Every request is a new process with its own memory space. Yes, performance is an issue with that old approach. Around the turn of the century I was responsible for a Perl web application that ran as a CGI script, and to speed it up I wrote my own Perl httpd server that had my web application code pre-loaded. Then for each request I forked a new process, which carried over all of the pre-loaded code and already-running perl.exe, so there was very little startup time (compared to a normal CGI script startup.) I was careful to take full advantage of the operating system's Copy-on-write memory semantics, so that the only memory that had to be allocated for the new process was Perl's runtime stack. All of the memory containing the application code was shared across processes because it never got written to. This web application is still running today, though I left the company in 2010. Back then, it was handling about six million requests per day, about 75% of which was during US daytime working hours. So, around 160 requests/second, spread across five or six servers that were getting old in 2010. The modern frameworks have taken the same concept a step further, and replaced the process fork with a thread pool. That's faster, but it allows memory leaking in a way process forking does not. You have to go out of your way to share memory between processes, but with threads it's easy to do accidentally.

hu3 6y ago
Workerman [1] uses a similar approach to put PHP pretty high on TechEmpower [2]. Specially now with PHP8's JIT. [1] https://github.com/walkor/Workerman https://github.com/walkor/Workerman [2] https://www.techempower.com/benchmarks https://www.techempower.com/benchmarks

Cloudef 6y ago

It's called separate processes or unshare after new thread.