5 ms·

Honestly this sounds like overcomplicating a system just for the sake of it. Drivetribe can be implemented reliably in Django or Rails without any of these fanc

Honestly this sounds like overcomplicating a system just for the sake of it. Drivetribe can be implemented reliably in Django or Rails without any of these fancier "architectures". But then considering that the product itself is not innovative, perhaps they needed something to sell to their naive recruits.

sewen 10y ago
The main point is that this new architecture has many interesting advantages over the standard Rails & database approach. By logging all the interaction events and computing application state via a stream processor, you get all the benefit's of a log-centric architecture. Building different services as "views" over the history of interactions, forking the stream to test new applications, replaying data to test different recommender models, ramping up models on historic data and falling in with real-time data for continuous updates. That all becomes the same thing. And for many use cases, you get better performance on less hardware because you have a simpler concurrency model.
- deleted 10y ago
  [deleted]
ccleve 10y ago
My thought exactly. The article doesn't give any numbers on how many customers, messages, "tribes", queries per sec, or anything. This market just doesn't sound very big, and it's probable that a plain old CRUD app would handle it just fine.
otterley 10y ago
That's possible. But it's also possible that the product could be huge and complicated later on, in which case they have laid a very strong and capable groundwork, and we should be applauding their foresight and avoiding the creation of technical debt.
ariskk 10y ago
Hi. I am the author of the article. Thank you for spending time to read this. The combined reach of the co-founders is very large, thus being able to provably handle scale was an essential prerequisite. Additionally, the requirements of the platform extend way beyond a simple content server. Content performance is tracked in real-time and this is fed to multiple ranking and recommendation models. Those frequently change, thus we need a way to retroactively process our data. Flexibility is key when trying to build an intelligent platform. Thus, we decided to early-on invest time in the ability to quickly iterate and experiment on algorithms, in real time over live data. You are right that the API fleet could be implemented using the aforementioned technologies; We use Scala and thus decided to use Akka HTTP instead. The challenging part is how you manage state behind that.
- newman314 10y ago
  Can you share some details around numbers and volume? "Very large" does not really convey why going down this route makes sense.
  - ariskk 10y ago
    Unfortunately, I am not allowed to. The problem with this is that beforehand you cannot predict the volumes. 1K requests per second? 10K per second? Maybe 50K per second on special occasions? It is difficult to tell, especially when high profile personalities are involved. PS: we do have lots of load
- aub3bhat 10y ago
  Don't get me wrong, I am not denying that CQRS/stream processing style approach is not useful for any application. Rather it is unsuitable for this particular problem. In my experience all these features sound nice on paper. But you quickly run into practical issues that are far easier when you know approximate information about the state. E.g. Developing a model? you might just want a subset/batch data. Doing BI/Analytics? are you going to continuously tax your server to recompute? The argument about recommender systems is also honestly flimsy, having built and applied such systems to live traffic at very large scale (more than hundreds of millions of users). There is only a small advantage from being able to quickly reconfigure flows. In most cases you have a single baseline model which you compare against for a small fraction of the traffic. The real complexity/gains in recommender systems lie in choice of algorithm/hyper-parameters/features, not on continuous multi armed bandits with 1000 different models applied simultaneously while waiting an infinite amount of time to produce any statistically meaningful answer. In fact for a website like this one, recommender systems can only provide so much advantage. There are actually several really good specialized use cases, e.g. Google secmon-tools uses a system like this one. [1] https://web.stanford.edu/class/cs259d/lectures/Session11.pdf https://web.stanford.edu/class/cs259d/lectures/Session11.pdf
rojoca 10y ago
Honestly, so what? Something got built that wasn't the x-millionth rails app/clone and it made it into production. That's a good thing. Your comment seems to advocate never try anything new.

sewen 10y ago

The main point is that this new architecture has many interesting advantages over the standard Rails & database approach. By logging all the interaction events and computing application state via a stream processor, you get all the benefit's of a log-centric architecture. Building different services as "views" over the history of interactions, forking the stream to test new applications, replaying data to test different recommender models, ramping up models on historic data and falling in with real-time data for continuous updates. That all becomes the same thing. And for many use cases, you get better performance on less hardware because you have a simpler concurrency model.

deleted 10y ago
[deleted]

ccleve 10y ago

My thought exactly. The article doesn't give any numbers on how many customers, messages, "tribes", queries per sec, or anything. This market just doesn't sound very big, and it's probable that a plain old CRUD app would handle it just fine.

otterley 10y ago

That's possible. But it's also possible that the product could be huge and complicated later on, in which case they have laid a very strong and capable groundwork, and we should be applauding their foresight and avoiding the creation of technical debt.

ariskk 10y ago

Hi. I am the author of the article. Thank you for spending time to read this. The combined reach of the co-founders is very large, thus being able to provably handle scale was an essential prerequisite. Additionally, the requirements of the platform extend way beyond a simple content server. Content performance is tracked in real-time and this is fed to multiple ranking and recommendation models. Those frequently change, thus we need a way to retroactively process our data. Flexibility is key when trying to build an intelligent platform. Thus, we decided to early-on invest time in the ability to quickly iterate and experiment on algorithms, in real time over live data. You are right that the API fleet could be implemented using the aforementioned technologies; We use Scala and thus decided to use Akka HTTP instead. The challenging part is how you manage state behind that.

newman314 10y ago
Can you share some details around numbers and volume? "Very large" does not really convey why going down this route makes sense.
- ariskk 10y ago
  Unfortunately, I am not allowed to. The problem with this is that beforehand you cannot predict the volumes. 1K requests per second? 10K per second? Maybe 50K per second on special occasions? It is difficult to tell, especially when high profile personalities are involved. PS: we do have lots of load
aub3bhat 10y ago
Don't get me wrong, I am not denying that CQRS/stream processing style approach is not useful for any application. Rather it is unsuitable for this particular problem. In my experience all these features sound nice on paper. But you quickly run into practical issues that are far easier when you know approximate information about the state. E.g. Developing a model? you might just want a subset/batch data. Doing BI/Analytics? are you going to continuously tax your server to recompute? The argument about recommender systems is also honestly flimsy, having built and applied such systems to live traffic at very large scale (more than hundreds of millions of users). There is only a small advantage from being able to quickly reconfigure flows. In most cases you have a single baseline model which you compare against for a small fraction of the traffic. The real complexity/gains in recommender systems lie in choice of algorithm/hyper-parameters/features, not on continuous multi armed bandits with 1000 different models applied simultaneously while waiting an infinite amount of time to produce any statistically meaningful answer. In fact for a website like this one, recommender systems can only provide so much advantage. There are actually several really good specialized use cases, e.g. Google secmon-tools uses a system like this one. [1] https://web.stanford.edu/class/cs259d/lectures/Session11.pdf https://web.stanford.edu/class/cs259d/lectures/Session11.pdf

rojoca 10y ago

Honestly, so what? Something got built that wasn't the x-millionth rails app/clone and it made it into production. That's a good thing. Your comment seems to advocate never try anything new.