4 ms·

I think the problem with solutions like these it usually comes down throwing down money at more development (Erlang is hard) or throwing down money on more serv

I think the problem with solutions like these it usually comes down throwing down money at more development (Erlang is hard) or throwing down money on more servers (delayed_job, Amazon SQS, etc.). I usually prefer money on servers since it's faster to develop for my team. But I guess for the functionally gifted, this is a great solution.

defunkt 17y ago
We were constrained by network our file system and could not add more machines. We needed an RPC solution to shard the data and move away from our network file system. delayed_job, Amazon SQS, etc are not RPC systems. You can't block waiting on a response like you can with messaging systems. For instance, delayed_job makes an RPC call that asks "how much disk space is this repository using?" It waits until it gets a response from an Ernie handler then continues with its background job.
deleted 17y ago
[deleted]
oomkiller 17y ago
I wouldn't say Erlang is a "hard" language, especially once you understand it. Sure, there aren't a whole hell of a lot of people that are proficient in it, but I think that's just because it's a little obscure. To someone with a little experience with functional programming, Erlang should make a lot of sense.
- silentbicycle 17y ago
  Agreed. I'm learning it now, and while there are some new ideas in the concurrency model, it doesn't strike me as hard to learn or use the way (say) Haskell and C++ are. There's a fairly small number of core elements, and the language design as a whole strongly encourages you to make things loudly crash when anything is wrong, rather than passing on corrupt data. That helps immensely with debugging... Also, I found that a lot of concepts from Prolog carry over nicely. Erlang was initially implemented in Prolog, and that's where it gets pattern matching and immutable variables from. (Along with several stylistic / syntactic quirks.)
charlesju 17y ago
Yeah I'm sure everyone has their own opinions on how to solve this problem (scaling in general), this is just my approach. I'm sure your approach is just as, or more, valid.

defunkt 17y ago

We were constrained by network our file system and could not add more machines. We needed an RPC solution to shard the data and move away from our network file system. delayed_job, Amazon SQS, etc are not RPC systems. You can't block waiting on a response like you can with messaging systems. For instance, delayed_job makes an RPC call that asks "how much disk space is this repository using?" It waits until it gets a response from an Ernie handler then continues with its background job.

deleted 17y ago

[deleted]

oomkiller 17y ago

I wouldn't say Erlang is a "hard" language, especially once you understand it. Sure, there aren't a whole hell of a lot of people that are proficient in it, but I think that's just because it's a little obscure. To someone with a little experience with functional programming, Erlang should make a lot of sense.

silentbicycle 17y ago
Agreed. I'm learning it now, and while there are some new ideas in the concurrency model, it doesn't strike me as hard to learn or use the way (say) Haskell and C++ are. There's a fairly small number of core elements, and the language design as a whole strongly encourages you to make things loudly crash when anything is wrong, rather than passing on corrupt data. That helps immensely with debugging... Also, I found that a lot of concepts from Prolog carry over nicely. Erlang was initially implemented in Prolog, and that's where it gets pattern matching and immutable variables from. (Along with several stylistic / syntactic quirks.)

charlesju 17y ago

Yeah I'm sure everyone has their own opinions on how to solve this problem (scaling in general), this is just my approach. I'm sure your approach is just as, or more, valid.