3 ms·

Hitting 300 SimpleDB Requests Per Second on a Small EC2 Instance

14 points by pbnaidu 18y ago

axod 18y ago
"The architecture uses two thread pools: one to run queries and one to get record values. Applications must carefully tune the number of threads in each pool so the queries to overwhelm the gets. Using a query thread pool with 2 threads and a get thread pool with 32 threads it was possible to perform 300 TPS on a small EC2 instances. " This seems horrible design to me. Surely it must be possible to make it asynchronous and event driven - getting rid of the need for threads?
- st3fan 18y ago
  What is the problem with threads? I keep hearing that they are so difficult to work with but yet people ar successfully building very high performance Java applications that take advantage of many CPU cores. Simply by using the proven patterns and concurrency abstractions available to them.
  - axod 18y ago
    Having more threads than you have CPUs is just sloppy inefficient overcomplicated programming :/ If you have control of all the pieces, and don't need to interface to things that block, there's no reason to use more threads than cpus IMHO.
    - Tamerlin 18y ago
      There are times when it makes sense to use more threads than CPU's though; one good (and common) example is web application servers. Applications that use resources like databases that are I/O rather than compute bound can be significantly more efficient with additional threads beyond the number of CPU's available. Of course there's a limit; overkill doesn't do anyone any good.
    - st3fan 18y ago
      """If you have control of all the pieces, and don't need to interface to things that block, there's no reason to use more threads than cpus IMHO""" I only partially agree with this. It certainly makes no sense to have many threads running if you're talking about very CPU intensive application. We (Polar Rose) do very intense image processing (Java combined with native C++ code) and never have more than NCPU worker threads running at the same time. We do have a whole bunch of threads running around that for network services like JMS, JMX, etc. But you can basically ignore the overhead of that. It's like having idle UNIX processes. Or are you also saying that having more then NCPU processes running is sloppy? S.
    - andrewf 18y ago
      Avoiding >NCPU threads is basically doing your context switching explicitly in code, rather than letting your OS and/or VM handle it. http://mailinator.blogspot.com/2008/02/kill-myth-please-nio-is-not-faster-than.html http://mailinator.blogspot.com/2008/02/kill-myth-please-nio-... Seems it isn't necessarily more efficient in many cases.
      - axod 18y ago
        It's certainly more tasteful IMHO, which should count for a lot. Spamming the CPU/OS with thousands of threads is just ugly. How can it not be? Networking should be event driven. It's like having a thread for each pixel on the display or each button...
        st3fan 18y ago
        You speak in emotional terms like 'Tasteful' and 'Ugly'. Which makes a technical discussion about threading really difficult.
        axod 18y ago
        Well in technical terms, using threads can never be as efficient as not using threads. It can be equal, but not better. A lot of these things are a matter of taste though. If you think it's a thing of beauty to create 5,000 threads, then good luck to you :)
st3fan 18y ago
"""This seems horrible design to me. """ Have you actually read the code? I just did and there is barely any thread-related code in the example. It's mostly code to push and pop stuff from a queue. Personally I would turn this into a little Java/AWS framework whose API is based on a high level queue object to abstract away the communication with SQS. Probably simply modelled after a java.util.concurrent.BlockingQueue and configurable through Spring so that it becomes a nice reusable component. Then it gets totally simple and interacting with those (remote) queues will not be any different than with one kept in memory for example. Since it is all thread safe it will be a no-brainer to create a simple workflow where you take stuff from one queue and put it in another. I seriously don't understand why people are so afraid of threads. All you need is a good and well tested abstraction for common patterns. http://www.javaconcurrencyinpractice.com http://www.javaconcurrencyinpractice.com is probably a good start for Java people with the threads-are-evil mindset. S.
- Tamerlin 18y ago
  I agree. Then again, most of the people that I've met who share the threads are evil mindset write bad code and can't manage requirements, like a certain project manager I had who was full of "trivial" last-minute requirements... I suspect that there's a relationship there. :)
  - st3fan 18y ago
    Most people I meet that have the threads-are-evil attitude are Python or Ruby coders who have never experienced a rich runtime where stable concurrency is actually possible :-)
- ardit33 18y ago
  doing concurrency right, is not easy. It takes an exeperienced developer to get it right (i.e good abstraction techniques, recovering from failure and interrupts, etc), so a lot of people are uncomfortable handling multi-threading, mainly b/c it adds yet another layer of what can go wrong.
sah 18y ago
300 requests per second is pretty slow for database queries, isn't it?

14 points by pbnaidu 18y ago

axod 18y ago

"The architecture uses two thread pools: one to run queries and one to get record values. Applications must carefully tune the number of threads in each pool so the queries to overwhelm the gets. Using a query thread pool with 2 threads and a get thread pool with 32 threads it was possible to perform 300 TPS on a small EC2 instances. " This seems horrible design to me. Surely it must be possible to make it asynchronous and event driven - getting rid of the need for threads?

st3fan 18y ago
What is the problem with threads? I keep hearing that they are so difficult to work with but yet people ar successfully building very high performance Java applications that take advantage of many CPU cores. Simply by using the proven patterns and concurrency abstractions available to them.
- axod 18y ago
  Having more threads than you have CPUs is just sloppy inefficient overcomplicated programming :/ If you have control of all the pieces, and don't need to interface to things that block, there's no reason to use more threads than cpus IMHO.
  - Tamerlin 18y ago
    There are times when it makes sense to use more threads than CPU's though; one good (and common) example is web application servers. Applications that use resources like databases that are I/O rather than compute bound can be significantly more efficient with additional threads beyond the number of CPU's available. Of course there's a limit; overkill doesn't do anyone any good.
  - st3fan 18y ago
    """If you have control of all the pieces, and don't need to interface to things that block, there's no reason to use more threads than cpus IMHO""" I only partially agree with this. It certainly makes no sense to have many threads running if you're talking about very CPU intensive application. We (Polar Rose) do very intense image processing (Java combined with native C++ code) and never have more than NCPU worker threads running at the same time. We do have a whole bunch of threads running around that for network services like JMS, JMX, etc. But you can basically ignore the overhead of that. It's like having idle UNIX processes. Or are you also saying that having more then NCPU processes running is sloppy? S.
  - andrewf 18y ago
    Avoiding >NCPU threads is basically doing your context switching explicitly in code, rather than letting your OS and/or VM handle it. http://mailinator.blogspot.com/2008/02/kill-myth-please-nio-is-not-faster-than.html http://mailinator.blogspot.com/2008/02/kill-myth-please-nio-... Seems it isn't necessarily more efficient in many cases.
    - axod 18y ago
      It's certainly more tasteful IMHO, which should count for a lot. Spamming the CPU/OS with thousands of threads is just ugly. How can it not be? Networking should be event driven. It's like having a thread for each pixel on the display or each button...
      - st3fan 18y ago
        You speak in emotional terms like 'Tasteful' and 'Ugly'. Which makes a technical discussion about threading really difficult.
        axod 18y ago
        Well in technical terms, using threads can never be as efficient as not using threads. It can be equal, but not better. A lot of these things are a matter of taste though. If you think it's a thing of beauty to create 5,000 threads, then good luck to you :)

st3fan 18y ago

"""This seems horrible design to me. """ Have you actually read the code? I just did and there is barely any thread-related code in the example. It's mostly code to push and pop stuff from a queue. Personally I would turn this into a little Java/AWS framework whose API is based on a high level queue object to abstract away the communication with SQS. Probably simply modelled after a java.util.concurrent.BlockingQueue and configurable through Spring so that it becomes a nice reusable component. Then it gets totally simple and interacting with those (remote) queues will not be any different than with one kept in memory for example. Since it is all thread safe it will be a no-brainer to create a simple workflow where you take stuff from one queue and put it in another. I seriously don't understand why people are so afraid of threads. All you need is a good and well tested abstraction for common patterns. http://www.javaconcurrencyinpractice.com http://www.javaconcurrencyinpractice.com is probably a good start for Java people with the threads-are-evil mindset. S.

Tamerlin 18y ago
I agree. Then again, most of the people that I've met who share the threads are evil mindset write bad code and can't manage requirements, like a certain project manager I had who was full of "trivial" last-minute requirements... I suspect that there's a relationship there. :)
- st3fan 18y ago
  Most people I meet that have the threads-are-evil attitude are Python or Ruby coders who have never experienced a rich runtime where stable concurrency is actually possible :-)
ardit33 18y ago
doing concurrency right, is not easy. It takes an exeperienced developer to get it right (i.e good abstraction techniques, recovering from failure and interrupts, etc), so a lot of people are uncomfortable handling multi-threading, mainly b/c it adds yet another layer of what can go wrong.

sah 18y ago

300 requests per second is pretty slow for database queries, isn't it?