3 ms·
You're absolutely right! The biggest things I learnt from that were: - overprovision resources for big demos by a significant amount - test your demo before y
by strken 6y ago
You're absolutely right! The biggest things I learnt from that were:
- overprovision resources for big demos by a significant amount
- test your demo before you give it
- deploying pets (servers you ssh into and lovingly configure) instead of cattle (instance abc123 that's created automatically) means you can't scale them as easily
- if you have concerns about a big demo, particularly ones that could be fixed by spending $10 on EC2 instances, you should fix them yourself instead of emailing the tech lead and assuming he'll handle it
- spinning up processes with phusion passenger is awful[0] and we should have been using something threaded like puma
That being said, the same footguns apply at various points as you scale, because Ruby's GVL still exists, and because even though threads are lighter weight than processes, they're still relatively expensive to switch between and to maintain in memory compared to e.g. Go/Elixir lightweight threads or evented IO.
Unfortunately (or perhaps fortunately) the problems with threaded IO start applying well above 20 concurrent users, so I don't have any fun stories about failed demos involving Puma instead of Passenger.
[0] Specifically, Rails processes take up a lot of memory, none of it is shared, context switching between processes is pretty expensive, and so you're limited to a smaller number of processes that can become IO-bound very easily
- ketamine__ 6y agoPassenger can be configured to restart processes after they reach a memory threshold. Another optimization is using jemalloc. Going to try this soon with Sidekiq workers.
- jashmatthews 6y ago> Threads are lighter weight than processes they're still relatively expensive to switch between and to maintain in memory ... context switching between processes is pretty expensive This is a really common misconception. Both switching between threads and processes is _extremely_ fast compared to what web applications do. An exceptionally fast web response is generally <1ms. Linux thread context switching takes ~2 microseconds. Forking a new processes takes ~20 microseconds.
- strken 6y agoRelatively expensive was probably not phrased well, but a goroutine context switch is ~200 nanoseconds and instantiating a new one takes ~1 microsecond, from memory and a quick google.
- jashmatthews 6y agoDoing nothing 10x as fast is not all that useful. In a real app it's unlikely the difference in context switching being 0.2% of your CPU time or 0.02% of your CPU time matters. It's also not a fundamental restriction on threads vs Goroutines either. switchto is very nearly as fast a Goroutine context switching https://www.phoronix.com/scan.php?page=news_item&px=Google-User-Thread-Futex-Swap https://www.phoronix.com/scan.php?page=news_item&px=Google-U...
- deleted 6y ago[deleted]
- otabdeveloper4 6y ago> threads are lighter weight than processes, they're still relatively expensive to switch between and to maintain in memory compared to e.g. Go/Elixir lightweight threads or evented IO Not true. Context switching pthreads is orders of magnitude faster than any Go/Elixir concurrency mechanism. So-called "lightweight" threads exist to fix the concurrency problem with garbage collection in scripting languages. Without garbage collection pthreads are fast enough for any problem.
- ramchip 6y agoLightweight threads in Erlang/Elixir were created as a unit of fault tolerance. A lightweight thread can suddenly exit when it hits an error, and the VM mechanisms give you guarantees that any resource it owned will be freed (memory, sockets, registered name, ETS tables, etc.) Using signals and monitors you can extend this to any other type of resources, for instance DB connections in an SQL adapter library. That's the original advantage over OS threads.
- otabdeveloper4 6y agoNo, you don't need cooperative threads for this. Proper exception handling fixes this problem with OS threads for you.