5 ms·

Ask HN: How does Booking.com guarantee consistency of reservations on scale?

I have been wondering consistency guarantees at scale, specially with a massive number of concurrent users. It seems that Booking.com mainly uses MySQL for most product features and use-cases and I am wondering how do they ensure that a listing is not booked by multiple users? Is the reservation system a single service with it's own storage and they run serializable transactions on it? Or do they utilize other means of distributed transactions, such as: distributed locking, 2PC, 3PC or Sagas?

PaulHoule 3y ago
It’s fundamentally a distributed problem because it involves a hotel. If the worst case scenario the hotel could be totally screwed up (IT error, bankruptcy, nature disaster, …) and you might not be able to stay there. Booking.com has to have something in the contract they serve you under that says in some cases they will put you up in another hotel, offer a refund or something but maybe in others they don’t. Even in the short term though it is the real reservation at the hotel that matters, not the state of their system, so there has to be some kind of concurrency control for the system as a whole.
_m8fo 3y ago
As far as I know they don’t have guarantees. They simply make good on refunding if there’s a downstream issue (e.g. hotel is overbooked).
logicalmonster 3y ago
If I had to make a wild guess, on top of a lot of ways of handling this technically, there's probably a set of business-policies they follow in the event of some problem like a room being double-booked, or a lodging otherwise becoming unavailable after the reservation is made. > In the (hopefully rare) event of a problem, they probably try and keep people happy through a standard playbook of things like refunds, giving future coupons or discounts, and trying and arranging alternative lodging. > Even if their databases could somehow guarantee to be perfectly consistent and no random booking problems occur, they still need those standard policies in place to deal with random hotel problems like flooding or other problems.
- iudqnolq 3y ago
  They have a pleasant, competent round the clock support team empowered to actually fix issues even for cheap customers, in my experience.
contingencies 3y ago
Business background: The unit of the hotel industry is the "roomnight". Globally, let's assume an average of 100 hotels per city and 10,000 cities and 100 rooms per hotel. While any of those numbers could be argued to be off by an order of magnitude they aren't so far from the upper bound reality as a central booking platform. So you are only booking a maximum of 100M roomnights per day, or 4.1M roomnights per hour, or 70K roomnights per minute, or 1200 roomnights per second. The SQLite FAQ states that it can do 50,000 or more inserts per second on an average computer. https://www.sqlite.org/faq.html#q19 https://www.sqlite.org/faq.html#q19 Since 50K/1.2K = 41, we could say roughly 41x100M roominghts per day could be handled on such an average system. So 4.1 billion roomights per day, at which point we are booking over half of the entire global population of humans, every night. Hand-waving generalities aside, I think we can conclude there is no technical problem. Largely, namespacing guarantees uniqueness without the need for a shared ledger to facilitate greater parallelism. Business-wise, if you are worried about contention, the locking is achieved within hotel reservation systems at two levels. One level is the channel, and another level is the hotel itself. The hotel itself holds the ultimate record of note with respect to forward room allocations, re-allocation powers, deferment and cancellation powers, etc. They are the arbiter of truth. However, because hassling staff or systems at the hotel are not always optimal distribution solutions, large channels such as major booking sites will often be hard or soft-allocated groups of rooms in bulk. They are then able to provide their own confident sub-assignment of these rooms before communicating the ultimate guest details to the hotel. Conceptually, this functions similarly to exchange-based trades in crypto, broker-based book allocations in the conventional financial services world, and so on. The ultimate client-facing business process is generally handled with a soft booking at agreed rates, and then a hard confirmation of the receipt and agreement to booking by the hotel receiving the notification of such from the reservation channel. Different hotels offer different interfaces for this, but not so long ago a huge amount was still done by fax. It wouldn't surprise me if a great deal are still done manually by humans using email. Larger operations use APIs, but all fall back to each hotel's reservations department who have the power to issue contracts, change rates, and override agreements. A web-booking consumer would typically pay and receive a confirmation of payment immediately, and for hotels not offering real time reservation confirmation APIs receive a deferred confirmation via email. FWIW IIRC booking.com was one of the higher-vis perl holdouts globally, at least 10 years ago. Source: Around 2007-2009 I developed from scratch a hotel reservation system operating in six human languages which grew to over 3,000 partner hotels. This is not so big these days but at the time was the same size as multiple NASDAQ listed operations.
Veliladon 3y ago
Booking.com doesn't handle the reservations themselves, just facilitates you interacting with the hotel's reservation system. Booking.com probably plugs into the Central Reservation System for the hotel at some point whether it be at the chain level, the hotel level, or through something like the GDS (Global Distribution System). When booking.com starts a transaction it first asks for availability from the hotel, presents that availability to the user, then the user chooses if they want to accept any of the availability. Once the user chooses the reservation it's blocked off directly in the Central Reservation System at the hotel so it can't be double booked. If the rooms fill up between the availability being presented and the user deciding the CRS won't accept the reservation then booking.com then has to handle that use case with the user. So Booking.com itself probably doesn't really have any "floating" reservations where the user is told it's booked and the hotel doesn't know about it. It's all pretty much done in real-time.
- dasil003 3y ago
  Your opening is mostly right, but your conclusion is wrong. Booking.com optimizes for availability for a variety of reasons. The most important is that they are fighting for market share in real time for every room night. Hotels tend to overbook anyway since there are always last minute cancellations and no shows, so Booking.com gets a commitment to sell a certain number of rooms and then they sell that number regardless of whether they can immediately confirm it through connectivity. This is important because the distributed network of connectivity systems is not reliable, both due to the laws of distributed systems, but even more poignantly because the margins for the middlemen and small operators don’t allow for the same level of operational excellence as you would expect from a name brand tech company.
  - carlosjobim 3y ago
    Your description is unfortunately not correct. Booking.com does not work like that, they will connect with the booking system of the hotel in real time through XML and put the reservation directly into your system when the client makes the reservation. Likewise, they will ping you to know if you have any available rooms, before presenting to the customer. Allotment of rooms hasn't really been a thing for at least a decade, although there are still some old fashioned resellers that might manage to scam clueless small hotel owners into working with allotments, but booking.com doesn't do that. Even for a tiny hotel there are cheap and extremely well made booking systems that can connect with booking.com, AirBnB, Expedia, as well as a bunch of other resellers in real-time, keeping everything synchronized and avoiding overbooking. Of course, you can opt to not have any booking system of your own and put your listing on booking, and then yes booking.com will sell rooms without checking anything - since there's nothing they can check against. If you arrange things like that, you'd have to manually remove rooms from bookings system when you make a sale yourself. But that's self-inflicted harm. I used to run a hotel a long time ago, and a couple of the large regional resellers tried to scam me into giving them allotments. I said "Sure, will you pay for your allotted rooms wether you sell them or not?", they said "Of course not!" - and that was the end of it.
crazygringo 3y ago
> specially with a massive number of concurrent users I think that's where you're wrong... first of all, the number of hotel booking transactions is nowhere near the number of interactions on a social media site/app, for example. It's not massive at all -- it's something any modern database server can handle easily, with replication for reads as required. And secondly, even if it were massive, it's not massive per-property which is the only thing that matters in terms of transactions here. Database sharding in this case is incredibly straightforward, compared to many other types of websites. And in both cases, just use standard database transactions. Now, in reality, as other commenters have attested, it's actually an optimistic communications process with each property that has nothing to do with databases at all, and overbooking can absolutely happen when both Booking.hom and Hotels.com book the same last room before the hotel has confirmed one or the other. And then it gets sorted out manually, by the hotel, because the hotel is aware of it.
- trilobyte 3y ago
  > "And secondly, even if it were massive, it's not massive per-property which is the only thing that matters in terms of transactions here. Database sharding in this case is incredibly straightforward, compared to many other types of websites. And in both cases, just use standard database transactions." This is such an important point that a lot of software devs don't get. If you think about the shape of the data, you can often find ways to naturally partition the data so that you are dealing with much smaller scale.
- hedora 3y ago
  Not only is it an optimistic communications process, but the charging of your credit card isn’t even atomic. I once showed up to a hotel hours after my card had been charged successfully. The backend credit card of the reservation company declined the charge, so the hotel room had been booked, but the hotel hadn’t been paid. The hotel manager sat on hold for 10 minutes, and the website fixed it.
throwaway290 3y ago
I have run into consistency issues on booking where it'd show you a room and then say it's not available when you try to book. Go back, refresh, there's a room, book -> error again. But to answer your question, it's probably more about hotels advertising truthful number of rooms than booking's system, and I believe booking makes sure hotels don't overbook by penalizing them.
- lou_cypher 3y ago
  I work for a global hotel company. No, booking does not penalises them. The internal system are fine but Booking is a third party, so quirks are to be expected at times
anonzzzies 3y ago
They don’t ensure that technically. I have had countless times things were double booked. They ensure it humanly by refunds and finding alternatives.
anonzzzies 3y ago
They don’t ensure that technically. I have had countless times things were double booked. They ensure it humanly by refunds and finding alternatives. Their tech model is very simple (I know quite a lot of people there), but it works well.
- dannyw 3y ago
  Yep, I’ve had a couple issues over the years too. Their support wasn’t always great, but I was not left stranded at the end of the day.
thepra 3y ago
Unfortunately last time I used booking.com to book a flight I didn't get my passport registered on the ticket, and it's quite unclear if the fault is of the airline booking or the aforementioned one :/
campanella 3y ago
Hijacking this thread to ask which data structure is best for storing hotel reservations?
- yashkumarverma 3y ago
  I hope you understand that we're way beyond data structures here now
  - campanella 3y ago
    Haha, I hope some day you'll learn that data structure determines access patterns which determines architecture :p But this is way beyond your level here now
miraculixx 3y ago
BASE. Here's a secret: most of the world runs on BASE. Even banks.
alebairos 3y ago
Chapter 7 of Alex Xu’s Volume 2 has a nice description of most of the problems. Below, a post that has a good level of details. https://devinz1993.medium.com/book-notes-system-design-interview-an-insiders-guide-volume-2-f5bd94be5704 https://devinz1993.medium.com/book-notes-system-design-inter... Extracted from the post: Chapter 7. Hotel Reservation System Functional requirements: - View hotels and rooms. - Reserve a hotel room. - Add/delete/update a hotel room. - Support overbooking. Non-functional requirements: - Support high concurrency. - Moderate latency (a few seconds for reservation). Recommendations: - Choose a relational database for the read-heavy workload and ACID properties. - Include an idempotency key in the API for making reservations to avoid double booking. - Use the microservice architecture and use remote procedure calls for inter-service communication. - Keep a pre-populated room type inventory (for 2 years ahead) to support reservations by room type (instead of by room id). Concurrency control: - Pessimistic locking is prone to deadlocks and tends to have bad performance. - Optimistic locking is usually faster but performance dramatically drops when contention is heavy. Scaling: - Shard the database by hotel since most queries need to filter by it. - Cache inventory information (aggregates) in Redis to reduce database load and improve read performance. Tip: - Use the same service to manage reservations and the inventory to avoid data consistency issues that require 2PC or Saga.

PaulHoule 3y ago

It’s fundamentally a distributed problem because it involves a hotel. If the worst case scenario the hotel could be totally screwed up (IT error, bankruptcy, nature disaster, …) and you might not be able to stay there. Booking.com has to have something in the contract they serve you under that says in some cases they will put you up in another hotel, offer a refund or something but maybe in others they don’t. Even in the short term though it is the real reservation at the hotel that matters, not the state of their system, so there has to be some kind of concurrency control for the system as a whole.

_m8fo 3y ago

As far as I know they don’t have guarantees. They simply make good on refunding if there’s a downstream issue (e.g. hotel is overbooked).

logicalmonster 3y ago

If I had to make a wild guess, on top of a lot of ways of handling this technically, there's probably a set of business-policies they follow in the event of some problem like a room being double-booked, or a lodging otherwise becoming unavailable after the reservation is made. > In the (hopefully rare) event of a problem, they probably try and keep people happy through a standard playbook of things like refunds, giving future coupons or discounts, and trying and arranging alternative lodging. > Even if their databases could somehow guarantee to be perfectly consistent and no random booking problems occur, they still need those standard policies in place to deal with random hotel problems like flooding or other problems.

iudqnolq 3y ago
They have a pleasant, competent round the clock support team empowered to actually fix issues even for cheap customers, in my experience.

contingencies 3y ago

Business background: The unit of the hotel industry is the "roomnight". Globally, let's assume an average of 100 hotels per city and 10,000 cities and 100 rooms per hotel. While any of those numbers could be argued to be off by an order of magnitude they aren't so far from the upper bound reality as a central booking platform. So you are only booking a maximum of 100M roomnights per day, or 4.1M roomnights per hour, or 70K roomnights per minute, or 1200 roomnights per second. The SQLite FAQ states that it can do 50,000 or more inserts per second on an average computer. https://www.sqlite.org/faq.html#q19 https://www.sqlite.org/faq.html#q19 Since 50K/1.2K = 41, we could say roughly 41x100M roominghts per day could be handled on such an average system. So 4.1 billion roomights per day, at which point we are booking over half of the entire global population of humans, every night. Hand-waving generalities aside, I think we can conclude there is no technical problem. Largely, namespacing guarantees uniqueness without the need for a shared ledger to facilitate greater parallelism. Business-wise, if you are worried about contention, the locking is achieved within hotel reservation systems at two levels. One level is the channel, and another level is the hotel itself. The hotel itself holds the ultimate record of note with respect to forward room allocations, re-allocation powers, deferment and cancellation powers, etc. They are the arbiter of truth. However, because hassling staff or systems at the hotel are not always optimal distribution solutions, large channels such as major booking sites will often be hard or soft-allocated groups of rooms in bulk. They are then able to provide their own confident sub-assignment of these rooms before communicating the ultimate guest details to the hotel. Conceptually, this functions similarly to exchange-based trades in crypto, broker-based book allocations in the conventional financial services world, and so on. The ultimate client-facing business process is generally handled with a soft booking at agreed rates, and then a hard confirmation of the receipt and agreement to booking by the hotel receiving the notification of such from the reservation channel. Different hotels offer different interfaces for this, but not so long ago a huge amount was still done by fax. It wouldn't surprise me if a great deal are still done manually by humans using email. Larger operations use APIs, but all fall back to each hotel's reservations department who have the power to issue contracts, change rates, and override agreements. A web-booking consumer would typically pay and receive a confirmation of payment immediately, and for hotels not offering real time reservation confirmation APIs receive a deferred confirmation via email. FWIW IIRC booking.com was one of the higher-vis perl holdouts globally, at least 10 years ago. Source: Around 2007-2009 I developed from scratch a hotel reservation system operating in six human languages which grew to over 3,000 partner hotels. This is not so big these days but at the time was the same size as multiple NASDAQ listed operations.

Veliladon 3y ago

Booking.com doesn't handle the reservations themselves, just facilitates you interacting with the hotel's reservation system. Booking.com probably plugs into the Central Reservation System for the hotel at some point whether it be at the chain level, the hotel level, or through something like the GDS (Global Distribution System). When booking.com starts a transaction it first asks for availability from the hotel, presents that availability to the user, then the user chooses if they want to accept any of the availability. Once the user chooses the reservation it's blocked off directly in the Central Reservation System at the hotel so it can't be double booked. If the rooms fill up between the availability being presented and the user deciding the CRS won't accept the reservation then booking.com then has to handle that use case with the user. So Booking.com itself probably doesn't really have any "floating" reservations where the user is told it's booked and the hotel doesn't know about it. It's all pretty much done in real-time.

dasil003 3y ago
Your opening is mostly right, but your conclusion is wrong. Booking.com optimizes for availability for a variety of reasons. The most important is that they are fighting for market share in real time for every room night. Hotels tend to overbook anyway since there are always last minute cancellations and no shows, so Booking.com gets a commitment to sell a certain number of rooms and then they sell that number regardless of whether they can immediately confirm it through connectivity. This is important because the distributed network of connectivity systems is not reliable, both due to the laws of distributed systems, but even more poignantly because the margins for the middlemen and small operators don’t allow for the same level of operational excellence as you would expect from a name brand tech company.
- carlosjobim 3y ago
  Your description is unfortunately not correct. Booking.com does not work like that, they will connect with the booking system of the hotel in real time through XML and put the reservation directly into your system when the client makes the reservation. Likewise, they will ping you to know if you have any available rooms, before presenting to the customer. Allotment of rooms hasn't really been a thing for at least a decade, although there are still some old fashioned resellers that might manage to scam clueless small hotel owners into working with allotments, but booking.com doesn't do that. Even for a tiny hotel there are cheap and extremely well made booking systems that can connect with booking.com, AirBnB, Expedia, as well as a bunch of other resellers in real-time, keeping everything synchronized and avoiding overbooking. Of course, you can opt to not have any booking system of your own and put your listing on booking, and then yes booking.com will sell rooms without checking anything - since there's nothing they can check against. If you arrange things like that, you'd have to manually remove rooms from bookings system when you make a sale yourself. But that's self-inflicted harm. I used to run a hotel a long time ago, and a couple of the large regional resellers tried to scam me into giving them allotments. I said "Sure, will you pay for your allotted rooms wether you sell them or not?", they said "Of course not!" - and that was the end of it.

crazygringo 3y ago

> specially with a massive number of concurrent users I think that's where you're wrong... first of all, the number of hotel booking transactions is nowhere near the number of interactions on a social media site/app, for example. It's not massive at all -- it's something any modern database server can handle easily, with replication for reads as required. And secondly, even if it were massive, it's not massive per-property which is the only thing that matters in terms of transactions here. Database sharding in this case is incredibly straightforward, compared to many other types of websites. And in both cases, just use standard database transactions. Now, in reality, as other commenters have attested, it's actually an optimistic communications process with each property that has nothing to do with databases at all, and overbooking can absolutely happen when both Booking.hom and Hotels.com book the same last room before the hotel has confirmed one or the other. And then it gets sorted out manually, by the hotel, because the hotel is aware of it.

trilobyte 3y ago
> "And secondly, even if it were massive, it's not massive per-property which is the only thing that matters in terms of transactions here. Database sharding in this case is incredibly straightforward, compared to many other types of websites. And in both cases, just use standard database transactions." This is such an important point that a lot of software devs don't get. If you think about the shape of the data, you can often find ways to naturally partition the data so that you are dealing with much smaller scale.
hedora 3y ago
Not only is it an optimistic communications process, but the charging of your credit card isn’t even atomic. I once showed up to a hotel hours after my card had been charged successfully. The backend credit card of the reservation company declined the charge, so the hotel room had been booked, but the hotel hadn’t been paid. The hotel manager sat on hold for 10 minutes, and the website fixed it.

throwaway290 3y ago

I have run into consistency issues on booking where it'd show you a room and then say it's not available when you try to book. Go back, refresh, there's a room, book -> error again. But to answer your question, it's probably more about hotels advertising truthful number of rooms than booking's system, and I believe booking makes sure hotels don't overbook by penalizing them.

lou_cypher 3y ago
I work for a global hotel company. No, booking does not penalises them. The internal system are fine but Booking is a third party, so quirks are to be expected at times

anonzzzies 3y ago

They don’t ensure that technically. I have had countless times things were double booked. They ensure it humanly by refunds and finding alternatives.

anonzzzies 3y ago

They don’t ensure that technically. I have had countless times things were double booked. They ensure it humanly by refunds and finding alternatives. Their tech model is very simple (I know quite a lot of people there), but it works well.

dannyw 3y ago
Yep, I’ve had a couple issues over the years too. Their support wasn’t always great, but I was not left stranded at the end of the day.

thepra 3y ago

Unfortunately last time I used booking.com to book a flight I didn't get my passport registered on the ticket, and it's quite unclear if the fault is of the airline booking or the aforementioned one :/

campanella 3y ago

Hijacking this thread to ask which data structure is best for storing hotel reservations?

yashkumarverma 3y ago
I hope you understand that we're way beyond data structures here now
- campanella 3y ago
  Haha, I hope some day you'll learn that data structure determines access patterns which determines architecture :p But this is way beyond your level here now

miraculixx 3y ago

BASE. Here's a secret: most of the world runs on BASE. Even banks.

alebairos 3y ago

Chapter 7 of Alex Xu’s Volume 2 has a nice description of most of the problems. Below, a post that has a good level of details. https://devinz1993.medium.com/book-notes-system-design-interview-an-insiders-guide-volume-2-f5bd94be5704 https://devinz1993.medium.com/book-notes-system-design-inter... Extracted from the post: Chapter 7. Hotel Reservation System Functional requirements: - View hotels and rooms. - Reserve a hotel room. - Add/delete/update a hotel room. - Support overbooking. Non-functional requirements: - Support high concurrency. - Moderate latency (a few seconds for reservation). Recommendations: - Choose a relational database for the read-heavy workload and ACID properties. - Include an idempotency key in the API for making reservations to avoid double booking. - Use the microservice architecture and use remote procedure calls for inter-service communication. - Keep a pre-populated room type inventory (for 2 years ahead) to support reservations by room type (instead of by room id). Concurrency control: - Pessimistic locking is prone to deadlocks and tends to have bad performance. - Optimistic locking is usually faster but performance dramatically drops when contention is heavy. Scaling: - Shard the database by hotel since most queries need to filter by it. - Cache inventory information (aggregates) in Redis to reduce database load and improve read performance. Tip: - Use the same service to manage reservations and the inventory to avoid data consistency issues that require 2PC or Saga.