4 ms·

Did you evaluate EventMachine? If yes would you please share your findings.

by baghali 15y ago

pimeys 15y ago
I've used fibers-enabled EventMachine in one part of our application. It's pretty nice, because you don't have to do callbacks and the Ruby fibers are pretty nice actually. We're using it for handling jobs from Resque, which reads and writes from our database and do a ping to a 3rd party server. But. The amount of bugs in the em-enabled libraries is just amazing. For example em-activerecord just didn't work so well under huge load (dropping SQL-connections), so I had to write my own mini ORM for the workers. Also Resque blocks by default, so of course doing a non-blocking and non-forking version was a priority. The worst thing here are the error messages and backtraces. There are none (DeadFiberException for failing assert) and with hacks you can get out a bit more information if the job crashes. Now thinking this again, it would be a good idea to write it again with Node.js. Just because it's still more mature compared to EventMachine, and all of it's libraries are reactor core friendly by default.
- mattgreenrocks 15y ago
  It's a shame that fibered Ruby feels like such a ghetto. I'd really like to see it be a viable concurrency choice, especially for apps that call a lot of third party services.
- toisanji 15y ago
  I'm using em in production also and I am seeing quite few bugs too. I wanted to ask you some eventmachine related questions, how can I get ahold of you, I didn't see your info in your profile.
  - pimeys 15y ago
    My nick combined with Google's free email service will give you my contact.
- sandGorgon 15y ago
  >For example em-activerecord just didn't work so well under huge load (dropping SQL-connections), so I had to write my own mini ORM for the workers. That's interesting - could you elaborate (here or a blog post) on your findings. what was different between your ORM and em-activerecord that made it more performant. The thing is - my first thought would have been to leave the ORM alone and focus on the DB side (like more sophisticated connection pooling/pgbouncer, etc. ). Which is why I'm interested in what went wrong in the ORM that made it screw up when used in a non-blocking kind of a setup.
  - pimeys 15y ago
    > That's interesting - could you elaborate (here or a blog post) on your findings. what was different between your ORM and em-activerecord that made it more performant. I will do a blog post about this when I finally have some time (christmas holidays, maybe). Em-activerecord worked very nice first, but when I hit it with thousands of concurrent jobs, some workers just dropped dead saying MySQL couldn't answer. This was annoying, because the workers didn't really fail in Resque, just failed to do their job and if this was in production it would've cost us thousands. So, my own ORM is just a database superclass with em-connection-pool, configuration, openstruct and couple of class and instance methods (insert, update, find, query). And now this thing is really really fast, using lot less of sql connections (delayed job had 300 connections, now we'll need only ~40 connections) and scales really well. The thing here is, that when I'm not using Rails at all, I don't know why I should use evented Ruby instead of Node.
    - clojurerocks 15y ago
      This is why theres people who say you shouldnt use an orm at all. I have to admit intially i liked the django orm however after a while i found that i prefered or wanted to make the called directly myself. There was just oo much mystery going on behind the scenes that i couldnt find out about. You can probably use both node and ruby. From what ive seen sites that are using node use it more for the backend to serve ajax then for serving the web pages. Although i have seen few site with very ajax based interfaces that apparently use just node. It really depends on your ui i guess. Good luck with whatever you choose though!

Did you evaluate EventMachine? If yes would you please share your findings.

by baghali 15y ago

Did you evaluate EventMachine? If yes would you please share your findings.

I've used fibers-enabled EventMachine in one part of our application. It's pretty nice, because you don't have to do callbacks and the Ruby fibers are pretty nice actually. We're using it for handling jobs from Resque, which reads and writes from our database and do a ping to a 3rd party server. But. The amount of bugs in the em-enabled libraries is just amazing. For example em-activerecord just didn't work so well under huge load (dropping SQL-connections), so I had to write my own mini ORM for the workers. Also Resque blocks by default, so of course doing a non-blocking and non-forking version was a priority. The worst thing here are the error messages and backtraces. There are none (DeadFiberException for failing assert) and with hacks you can get out a bit more information if the job crashes. Now thinking this again, it would be a good idea to write it again with Node.js. Just because it's still more mature compared to EventMachine, and all of it's libraries are reactor core friendly by default.

mattgreenrocks 15y ago
It's a shame that fibered Ruby feels like such a ghetto. I'd really like to see it be a viable concurrency choice, especially for apps that call a lot of third party services.
toisanji 15y ago
I'm using em in production also and I am seeing quite few bugs too. I wanted to ask you some eventmachine related questions, how can I get ahold of you, I didn't see your info in your profile.
- pimeys 15y ago
  My nick combined with Google's free email service will give you my contact.
sandGorgon 15y ago
>For example em-activerecord just didn't work so well under huge load (dropping SQL-connections), so I had to write my own mini ORM for the workers. That's interesting - could you elaborate (here or a blog post) on your findings. what was different between your ORM and em-activerecord that made it more performant. The thing is - my first thought would have been to leave the ORM alone and focus on the DB side (like more sophisticated connection pooling/pgbouncer, etc. ). Which is why I'm interested in what went wrong in the ORM that made it screw up when used in a non-blocking kind of a setup.
- pimeys 15y ago
  > That's interesting - could you elaborate (here or a blog post) on your findings. what was different between your ORM and em-activerecord that made it more performant. I will do a blog post about this when I finally have some time (christmas holidays, maybe). Em-activerecord worked very nice first, but when I hit it with thousands of concurrent jobs, some workers just dropped dead saying MySQL couldn't answer. This was annoying, because the workers didn't really fail in Resque, just failed to do their job and if this was in production it would've cost us thousands. So, my own ORM is just a database superclass with em-connection-pool, configuration, openstruct and couple of class and instance methods (insert, update, find, query). And now this thing is really really fast, using lot less of sql connections (delayed job had 300 connections, now we'll need only ~40 connections) and scales really well. The thing here is, that when I'm not using Rails at all, I don't know why I should use evented Ruby instead of Node.
  - clojurerocks 15y ago
    This is why theres people who say you shouldnt use an orm at all. I have to admit intially i liked the django orm however after a while i found that i prefered or wanted to make the called directly myself. There was just oo much mystery going on behind the scenes that i couldnt find out about. You can probably use both node and ruby. From what ive seen sites that are using node use it more for the backend to serve ajax then for serving the web pages. Although i have seen few site with very ajax based interfaces that apparently use just node. It really depends on your ui i guess. Good luck with whatever you choose though!