4 ms·

Curious, What are all these companies building that makes scale an issue? I see only CRUD products. At what point does scale become an issue? 1000 concurrent us

Curious, What are all these companies building that makes scale an issue? I see only CRUD products. At what point does scale become an issue? 1000 concurrent users, 10,000, 100,000?

whatnotests 10y ago
State? Keeping a single source of truth which changes every 1/100th second while millions of users are doing something to change that state?
- toomuchtodo 10y ago
  Right, but companies like Sabre [1] solved this in the 60s. If its gotten progressively easier over ~55 years, why should we be wowed now that anyone can spin up a travel aggregator that scales (just as I'm not impressed that Simple/BankSimple exists when they ride on top of a real bank, Bancorp; I'm actually deeply disappointed as one of the first Simple customers that ended up leaving because it took 5 years for them to implement joint checking accounts. At a bank). Amazon, airlines, Ticketmaster, all online companies that have to maintain "truth" about shared inventory and its pricing up to the second. If you're breaking ground, awesome, you're doing something truly revolutionary. Would you be wowed if I built a Shopify clone off of Stripe and Squarespace? Or an app and site that performed ridesharing while simply talking to Uber or Tesla's backend? Probably not. Ahh! There! Perfect example. I eagerly await the video, with baited breath, of a presentation from the team at Tesla rolling out autonomous driving using Nvidia's deep learning chipset. But if you build another Kayak, Hipmunk, etc, I do give you credit for grinding away on it if its a successful business. Grinding away on a business day after day for years is fucking hard. I'd just argue its not revolutionary or breaking new ground. [1] https://en.wikipedia.org/wiki/Sabre_Corporation https://en.wikipedia.org/wiki/Sabre_Corporation Disclaimer: Satisfied HotelTonight app user.
  - cloudjacker 10y ago
    > I'd just argue its not revolutionary or breaking new ground Which is totally fine and not a prerequisite for a business or tech business
    - toomuchtodo 10y ago
      My reply was directed at the comment I replied to that maintaining state at the level and complexity described is a solved problem. Sorry I wasn't less ambiguous.
  - nightski 10y ago
    - majormajor 10y ago
      > If it was a solved problem then you would think it would be common knowledge among practicing engineers. There might even be a textbook available which contained these insights. This sounds pretty similar to how they describe it, actually: "Elasticsearch came to our attention…" [... another thing ...] "Pretty much on the lines of what FriendFeed did..." So basically they reimplemented what other people had done and written about. So (especially at this level of detail) not particularly useful compared to the existing resources they talk about having seen themselves. Later they talk about a big win being "introducing caching." This is all pretty well known stuff, at least in certain circles. And yes, other companies do things in different ways - scaling a single-source-of-truth schema'd database vs going schemaless and horizontal is a source of active debate - but they do talk about these ways. (Though, something I've also seen personally: if you staff your startup with fresh 20-something grads, you're gonna have to rediscover it... but that's another discussion?)
      - deleted 10y ago
        [deleted]
  - alexginzburg 10y ago
    Overbooking in the hotel industry is a normal practice. Cancellation rate is pretty high. Hotel Tonight deals with the last minute inventory, overbooking here would result in unsatisfied customers. I worked for an online hotel booking site in the past. We had integrations with multiple GDS and channel managers, SynXis (Sabre) was one of them. SynXis was not the fastest or most accurate when it came to inventory. SynXis had some ridiculous throughput limitation to query for availability, luckily we did not have to rely on that API. I can't speak about original Sabre technology since I haven't used it myself.
- rokhayakebe 10y ago
  Example?
- StreamBright 10y ago
  I think this can be managed by careful data modelling and smart caching easily.
- ghuntley 10y ago
  Compared to five years ago, not really an impressive accomplishment thanks to advancements such as https://dotnet.github.io/orleans/ https://dotnet.github.io/orleans/
jeffasinger 10y ago
Where I work, the hardest scaling problem we face really boils down to lots of users contending for the same resources. It's very possible that HotelTonight sees the same issues around room availability.
- rokhayakebe 10y ago
  Example?
iamleppert 10y ago
The short answer is nothing. They create their own problems of scalability by their own poor engineering. Who still uses rails for an api/iphone app these days? As far as MySQL being the bottleneck, I'm kind of wondering why that is. I'd imagine 99% of their workload is both cacheable and read only, and thus infinitely scalable. Anyway, I've used Hotel Tonight a few times and most of my experiences with the actual product have been poor. The price I could have got through the hotel directly in a few cases would have been better, and the other time they promised a King bed only to be rebooked at the hotel to a smaller room and double bed. It seems their business model is largely around selling unused inventory at hotels for as high a price they can get, without really caring about the customer. You'll notice they have an absolutely no money back policy. In my case I was forced to take the lesser room which was actually cheaper had I paid for it at the hotel. Since that experience I deleted their app.
- will_hughes 10y ago
  No affiliation with Hotel tonight, but depending on who they're using for providing the inventory they might not be permitted to cache the data across multiple users. Hotels, like Airlines, love their hideously complicated rules around pricing - particularly for market segmentation.

Curious, What are all these companies building that makes scale an issue? I see only CRUD products. At what point does scale become an issue? 1000 concurrent users, 10,000, 100,000?

whatnotests 10y ago

State? Keeping a single source of truth which changes every 1/100th second while millions of users are doing something to change that state?

toomuchtodo 10y ago
Right, but companies like Sabre [1] solved this in the 60s. If its gotten progressively easier over ~55 years, why should we be wowed now that anyone can spin up a travel aggregator that scales (just as I'm not impressed that Simple/BankSimple exists when they ride on top of a real bank, Bancorp; I'm actually deeply disappointed as one of the first Simple customers that ended up leaving because it took 5 years for them to implement joint checking accounts. At a bank). Amazon, airlines, Ticketmaster, all online companies that have to maintain "truth" about shared inventory and its pricing up to the second. If you're breaking ground, awesome, you're doing something truly revolutionary. Would you be wowed if I built a Shopify clone off of Stripe and Squarespace? Or an app and site that performed ridesharing while simply talking to Uber or Tesla's backend? Probably not. Ahh! There! Perfect example. I eagerly await the video, with baited breath, of a presentation from the team at Tesla rolling out autonomous driving using Nvidia's deep learning chipset. But if you build another Kayak, Hipmunk, etc, I do give you credit for grinding away on it if its a successful business. Grinding away on a business day after day for years is fucking hard. I'd just argue its not revolutionary or breaking new ground. [1] https://en.wikipedia.org/wiki/Sabre_Corporation https://en.wikipedia.org/wiki/Sabre_Corporation Disclaimer: Satisfied HotelTonight app user.
- cloudjacker 10y ago
  > I'd just argue its not revolutionary or breaking new ground Which is totally fine and not a prerequisite for a business or tech business
  - toomuchtodo 10y ago
    My reply was directed at the comment I replied to that maintaining state at the level and complexity described is a solved problem. Sorry I wasn't less ambiguous.
- nightski 10y ago
  - majormajor 10y ago
    > If it was a solved problem then you would think it would be common knowledge among practicing engineers. There might even be a textbook available which contained these insights. This sounds pretty similar to how they describe it, actually: "Elasticsearch came to our attention…" [... another thing ...] "Pretty much on the lines of what FriendFeed did..." So basically they reimplemented what other people had done and written about. So (especially at this level of detail) not particularly useful compared to the existing resources they talk about having seen themselves. Later they talk about a big win being "introducing caching." This is all pretty well known stuff, at least in certain circles. And yes, other companies do things in different ways - scaling a single-source-of-truth schema'd database vs going schemaless and horizontal is a source of active debate - but they do talk about these ways. (Though, something I've also seen personally: if you staff your startup with fresh 20-something grads, you're gonna have to rediscover it... but that's another discussion?)
    - deleted 10y ago
      [deleted]
- alexginzburg 10y ago
  Overbooking in the hotel industry is a normal practice. Cancellation rate is pretty high. Hotel Tonight deals with the last minute inventory, overbooking here would result in unsatisfied customers. I worked for an online hotel booking site in the past. We had integrations with multiple GDS and channel managers, SynXis (Sabre) was one of them. SynXis was not the fastest or most accurate when it came to inventory. SynXis had some ridiculous throughput limitation to query for availability, luckily we did not have to rely on that API. I can't speak about original Sabre technology since I haven't used it myself.
rokhayakebe 10y ago
Example?
StreamBright 10y ago
I think this can be managed by careful data modelling and smart caching easily.
ghuntley 10y ago
Compared to five years ago, not really an impressive accomplishment thanks to advancements such as https://dotnet.github.io/orleans/ https://dotnet.github.io/orleans/

jeffasinger 10y ago

Where I work, the hardest scaling problem we face really boils down to lots of users contending for the same resources. It's very possible that HotelTonight sees the same issues around room availability.

rokhayakebe 10y ago
Example?

iamleppert 10y ago

The short answer is nothing. They create their own problems of scalability by their own poor engineering. Who still uses rails for an api/iphone app these days? As far as MySQL being the bottleneck, I'm kind of wondering why that is. I'd imagine 99% of their workload is both cacheable and read only, and thus infinitely scalable. Anyway, I've used Hotel Tonight a few times and most of my experiences with the actual product have been poor. The price I could have got through the hotel directly in a few cases would have been better, and the other time they promised a King bed only to be rebooked at the hotel to a smaller room and double bed. It seems their business model is largely around selling unused inventory at hotels for as high a price they can get, without really caring about the customer. You'll notice they have an absolutely no money back policy. In my case I was forced to take the lesser room which was actually cheaper had I paid for it at the hotel. Since that experience I deleted their app.

will_hughes 10y ago
No affiliation with Hotel tonight, but depending on who they're using for providing the inventory they might not be permitted to cache the data across multiple users. Hotels, like Airlines, love their hideously complicated rules around pricing - particularly for market segmentation.