9 ms·

I'm curious, how does one gain access to flight schedules/fares? Is this something that anyone can get their hands on and create a service (complexity aside), o

by karakal 10y ago

I'm curious, how does one gain access to flight schedules/fares? Is this something that anyone can get their hands on and create a service (complexity aside), or do you need some sort of license that costs thousands of dollars? Does each airline have their own way of exporting this data? Is there a single entity that aggregates from all of them? How does the actual data look like? (Is it a dump every X hours, or something more modern like a stream you can subscribe to?).

deleted 10y ago
[deleted]
skybrian 10y ago
In Google's case it comes from an acquisition [1]. Not all airlines share data. For example, you won't find Southwest fares except on their own website. [1] http://www.itasoftware.com/about/ http://www.itasoftware.com/about/
- toddkazakov 10y ago
  > For example, you won't find Southwest fares except on their own website. Wrong example. I have worked on the Southwest Project when I was at Amadeus.
- clueless123 10y ago
  Southwest has it's own system, but does share their data through Sabre.. Fun Fact.. upto 2005 they still sent us the data hand delivered on a 9 track tape!
NikolaeVarius 10y ago
There is a GDS that all (or most) vendors send data to https://en.wikipedia.org/wiki/Global_Distribution_System https://en.wikipedia.org/wiki/Global_Distribution_System Downstream companies then query this data and show them to you. The competition comes from the complex caching and delivery mechanisms that the companies use to get you the best price. It's not really an area that is ripe for disruption. The margins are extremely low and there are only really two companies that make a majority of the money. Expedia and the Priceline group.
- baby 10y ago
  I think Amadeus is a popular one, matrix itasoftware is another one. I know that for airlines like easyjet you need to visit their website to get fares though. (Or Google would have to build a site crawler)
- wodenokoto 10y ago
  Is momondo under one of those? At least where I live it is the biggest by far and I always thought it was globally in the top.
djhworld 10y ago
I've known "regular" people in the past that have written small scale services for themselves that scrape prices daily from an airlines own booking system and then provide a service so you can track it and see if there are price fluctuations. Not sure on the legality of it though.
squeaky-clean 10y ago
> Does each airline have their own way of exporting this data? Is there a single entity that aggregates from all of them? I work with a lot of smaller airlines, and from my experience I can tell you no. Many of them can't even tell you their actual route list or flight schedule or pricing logic. They typically buy a booking engine solution from just a few major players (Sabre, Amadeus, TravelPort) who do have centralized data, but access to them is slow, expensive (at scale) and restrictive. I've been told by one of the GDS's that our use case doesn't apply for direct GDS access, and we have to go through the airlines' access method. I don't know if these booking engine solutions include access to an API or not, however most airlines have said either they don't have an available API, or don't know how to share access to one. Some of the larger airlines do seem to have their stuff together. But we're just starting to work with them, and because of all the bureaucracy involved it will likely be months before anything happens. The screen scraping method djhworld mentions is a worthwhile approach if you'd personally like to track a few routes. I've done the same to watch prices to NYC (I visit friends often) on a few of the low-cost airlines. It's nice because you can tailor the logic to your preferences. Such as don't bother searching flights that don't include a weekend in the trip; or set my alert threshold 25% higher when the trip involves a holiday or long weekend.
- sluggg 10y ago
  Your information is spot on. I have been working with Sabre daily for a few years now, and I don't want to think about the time I have wasted waiting for a Sabre response... it does go down once a week, which I guess is a perk
- eddmc 10y ago
  What is your specific use case? I used to work managing API's for one of the GDS companies For screen scraping, there are a number of companies that do this already for multiple carriers. I'd suggest making use of one of these rather than reinventing the wheel. Example: http://xmldocs.travelfusion.com http://xmldocs.travelfusion.com
  - squeaky-clean 10y ago
    My project definitely isn't applicable because it requires caching prices, I don't remember why we were declined access for our other teams/products though. I would have to ask them, as it's been a while. About screen scraping, I guess it depends on how quick you are at writing scraping scripts (and if you find it fun). I could probably write one faster than I could register for that API. You're also adding a layer between you and the prices. You don't know how long Travelfusion has cached these prices for, while it's simple to write a script that opens the real booking engine to get correct prices. I typically only run mine when I'm ready to purchase a ticket, but don't know which dates, rather than running it constantly to find deals. And if I were doing anything more than a few routes on a few carriers, I definitely would use a real API instead of reinventing the wheel. In this case I just think it's kinda fun.
dmbaggett 10y ago
Fares and rules come from ATPCO (http://atpco.net http://atpco.net). Flights come from OAG (http://www.oag.com http://www.oag.com). Seat availability requires data directly from the carrier, usually via a live query to the carrier's reservation system. The GDS companies get this data from these sources and then in turn provide (crappy) APIs for customers to use to query it. In general, if you're doing bookings you can get small amounts of the data (a query at a time) from a GDS. Otherwise you're looking at millions of dollars per year. And then you need to write code to parse it and price tickets using it (approximately 1M LoC if you're terse about it -- more like 30M if you're a GDS). (I know all this because I co-founded ITA Software, whose software now powers Google Flights.)
- solaarphunk 10y ago
  Thank you for building ITA Matrix - it is amazing tool and I don't know what I would do without it.
  - dmbaggett 10y ago
    Lots of other people did a lot more of the actual building than I did, but on their behalf, "You're welcome!"
  - CPLX 10y ago
    A strong second to that, literally my favorite web tool in existence.
  - wfunction 10y ago
    > Thank you for building ITA Matrix - it is amazing tool and I don't know what I would do without it. Honest question (I swear I'm not trying to troll): what do you actually do with it that you can't do (or do as well as) with Google Flights or Kayak or some other site like that? I ask because I've used ITA Matrix and never managed to find a cheaper flight than I can through "normal" means... but I rarely fly multi-destination (and basically never do more complicated stuff) so I'm not sure if the use case is beyond mine or if I just don't know how to utilize it in a useful manner.
melvinmt 10y ago
You should first start reading these great slides from Carl de Macken (ITA Software co-founder, the thing that now powers Google Flights) http://www.demarcken.org/carl/papers/ITA-software-travel-complexity/ITA-software-travel-complexity.pdf http://www.demarcken.org/carl/papers/ITA-software-travel-com... …and then decide if you still want to pursue this.
- dmbaggett 10y ago
  Short answer: you don't!
- jaza 10y ago
  Great read! Incredibly thorough modelling of all components of the flight ticketing problem space, and of the computational complexities involved. Sets a high bar as an example of how to thoroughly model any search-related problem.
joaonunesk 10y ago
It is very expensive to get our hands on the technology components that power the core of such applications. [1] If an airline sells seats, bags, and other ancillary services, why aren't they trying to find more sales channels, to sell more? I believe that airlines haven't been pushing for innovation, and rather, have been slowed down by their very own current third party providers, or "core business partners". Travelling technology applications or reservation systems, first appeared a long time ago [2] and because at that time, they were innovations, some airlines signed up for them, to make more sales by globally or regionally distributing all of their flight related information (seat or ancillary availability, fares, code share agreements, etc.). For this reason, we can consider reservation systems as the very first technological airline business partners. Typically, the reservation systems provided innovation on a distribution factor, which is the travel agency distribution factor. Travel agencies could, and have been connecting to reservation systems and making purchases that increase the number of airline sales, resulting in a win-win, for both agencies and airlines. [3] Reservation systems have other goals, such as flight code share agreements, but these are out of scope. [5] Today, these third party business organisations do not want to lose market share, so they make things expensive or maintain the old prices, and open very little doors on the borders of their systems. Therefore, only typical rich individuals or companies can afford to purchase and maintain the costs of direct connections to these providers, which is a win-loss, win for the reservation systems, and loss for airlines. Fortunately, companies such as Google (with QPX), perform a re-engineering of what these big reservation systems players do, and make it much more affordable and simple to implement [4], and with that, not only they improve the understanding effort of the whole business, but also, provide more sales leads to the airlines, and better user and developing experiences. The future should be standardised, airlines should distant themselves from the reservation systems (if these do not innovate), and open the doors of their businesses, by creating booking process APIs, that allow, developers and startups, to create powerful and innovative applications, that bring sales. [1] https://www.tnooz.com/article/why-is-the-travel-industry-so-unfriendly-to-startups/ https://www.tnooz.com/article/why-is-the-travel-industry-so-... [2] https://en.wikipedia.org/wiki/Sabre_(computer_system) https://en.wikipedia.org/wiki/Sabre_(computer_system) [3] https://en.wikipedia.org/wiki/Travel_agency https://en.wikipedia.org/wiki/Travel_agency [4] https://developers.google.com/qpx-express/v1/pricing https://developers.google.com/qpx-express/v1/pricing [5] https://en.wikipedia.org/wiki/Codeshare_agreement https://en.wikipedia.org/wiki/Codeshare_agreement

by karakal 10y ago

deleted 10y ago

[deleted]

skybrian 10y ago

In Google's case it comes from an acquisition [1]. Not all airlines share data. For example, you won't find Southwest fares except on their own website. [1] http://www.itasoftware.com/about/ http://www.itasoftware.com/about/

toddkazakov 10y ago
> For example, you won't find Southwest fares except on their own website. Wrong example. I have worked on the Southwest Project when I was at Amadeus.
clueless123 10y ago
Southwest has it's own system, but does share their data through Sabre.. Fun Fact.. upto 2005 they still sent us the data hand delivered on a 9 track tape!

NikolaeVarius 10y ago

There is a GDS that all (or most) vendors send data to https://en.wikipedia.org/wiki/Global_Distribution_System https://en.wikipedia.org/wiki/Global_Distribution_System Downstream companies then query this data and show them to you. The competition comes from the complex caching and delivery mechanisms that the companies use to get you the best price. It's not really an area that is ripe for disruption. The margins are extremely low and there are only really two companies that make a majority of the money. Expedia and the Priceline group.

baby 10y ago
I think Amadeus is a popular one, matrix itasoftware is another one. I know that for airlines like easyjet you need to visit their website to get fares though. (Or Google would have to build a site crawler)
wodenokoto 10y ago
Is momondo under one of those? At least where I live it is the biggest by far and I always thought it was globally in the top.

djhworld 10y ago

I've known "regular" people in the past that have written small scale services for themselves that scrape prices daily from an airlines own booking system and then provide a service so you can track it and see if there are price fluctuations. Not sure on the legality of it though.

squeaky-clean 10y ago

> Does each airline have their own way of exporting this data? Is there a single entity that aggregates from all of them? I work with a lot of smaller airlines, and from my experience I can tell you no. Many of them can't even tell you their actual route list or flight schedule or pricing logic. They typically buy a booking engine solution from just a few major players (Sabre, Amadeus, TravelPort) who do have centralized data, but access to them is slow, expensive (at scale) and restrictive. I've been told by one of the GDS's that our use case doesn't apply for direct GDS access, and we have to go through the airlines' access method. I don't know if these booking engine solutions include access to an API or not, however most airlines have said either they don't have an available API, or don't know how to share access to one. Some of the larger airlines do seem to have their stuff together. But we're just starting to work with them, and because of all the bureaucracy involved it will likely be months before anything happens. The screen scraping method djhworld mentions is a worthwhile approach if you'd personally like to track a few routes. I've done the same to watch prices to NYC (I visit friends often) on a few of the low-cost airlines. It's nice because you can tailor the logic to your preferences. Such as don't bother searching flights that don't include a weekend in the trip; or set my alert threshold 25% higher when the trip involves a holiday or long weekend.

sluggg 10y ago
Your information is spot on. I have been working with Sabre daily for a few years now, and I don't want to think about the time I have wasted waiting for a Sabre response... it does go down once a week, which I guess is a perk
eddmc 10y ago
What is your specific use case? I used to work managing API's for one of the GDS companies For screen scraping, there are a number of companies that do this already for multiple carriers. I'd suggest making use of one of these rather than reinventing the wheel. Example: http://xmldocs.travelfusion.com http://xmldocs.travelfusion.com
- squeaky-clean 10y ago
  My project definitely isn't applicable because it requires caching prices, I don't remember why we were declined access for our other teams/products though. I would have to ask them, as it's been a while. About screen scraping, I guess it depends on how quick you are at writing scraping scripts (and if you find it fun). I could probably write one faster than I could register for that API. You're also adding a layer between you and the prices. You don't know how long Travelfusion has cached these prices for, while it's simple to write a script that opens the real booking engine to get correct prices. I typically only run mine when I'm ready to purchase a ticket, but don't know which dates, rather than running it constantly to find deals. And if I were doing anything more than a few routes on a few carriers, I definitely would use a real API instead of reinventing the wheel. In this case I just think it's kinda fun.

dmbaggett 10y ago

Fares and rules come from ATPCO (http://atpco.net http://atpco.net). Flights come from OAG (http://www.oag.com http://www.oag.com). Seat availability requires data directly from the carrier, usually via a live query to the carrier's reservation system. The GDS companies get this data from these sources and then in turn provide (crappy) APIs for customers to use to query it. In general, if you're doing bookings you can get small amounts of the data (a query at a time) from a GDS. Otherwise you're looking at millions of dollars per year. And then you need to write code to parse it and price tickets using it (approximately 1M LoC if you're terse about it -- more like 30M if you're a GDS). (I know all this because I co-founded ITA Software, whose software now powers Google Flights.)

solaarphunk 10y ago
Thank you for building ITA Matrix - it is amazing tool and I don't know what I would do without it.
- dmbaggett 10y ago
  Lots of other people did a lot more of the actual building than I did, but on their behalf, "You're welcome!"
- CPLX 10y ago
  A strong second to that, literally my favorite web tool in existence.
- wfunction 10y ago
  > Thank you for building ITA Matrix - it is amazing tool and I don't know what I would do without it. Honest question (I swear I'm not trying to troll): what do you actually do with it that you can't do (or do as well as) with Google Flights or Kayak or some other site like that? I ask because I've used ITA Matrix and never managed to find a cheaper flight than I can through "normal" means... but I rarely fly multi-destination (and basically never do more complicated stuff) so I'm not sure if the use case is beyond mine or if I just don't know how to utilize it in a useful manner.

melvinmt 10y ago

You should first start reading these great slides from Carl de Macken (ITA Software co-founder, the thing that now powers Google Flights) http://www.demarcken.org/carl/papers/ITA-software-travel-complexity/ITA-software-travel-complexity.pdf http://www.demarcken.org/carl/papers/ITA-software-travel-com... …and then decide if you still want to pursue this.

dmbaggett 10y ago
Short answer: you don't!
jaza 10y ago
Great read! Incredibly thorough modelling of all components of the flight ticketing problem space, and of the computational complexities involved. Sets a high bar as an example of how to thoroughly model any search-related problem.

joaonunesk 10y ago

It is very expensive to get our hands on the technology components that power the core of such applications. [1] If an airline sells seats, bags, and other ancillary services, why aren't they trying to find more sales channels, to sell more? I believe that airlines haven't been pushing for innovation, and rather, have been slowed down by their very own current third party providers, or "core business partners". Travelling technology applications or reservation systems, first appeared a long time ago [2] and because at that time, they were innovations, some airlines signed up for them, to make more sales by globally or regionally distributing all of their flight related information (seat or ancillary availability, fares, code share agreements, etc.). For this reason, we can consider reservation systems as the very first technological airline business partners. Typically, the reservation systems provided innovation on a distribution factor, which is the travel agency distribution factor. Travel agencies could, and have been connecting to reservation systems and making purchases that increase the number of airline sales, resulting in a win-win, for both agencies and airlines. [3] Reservation systems have other goals, such as flight code share agreements, but these are out of scope. [5] Today, these third party business organisations do not want to lose market share, so they make things expensive or maintain the old prices, and open very little doors on the borders of their systems. Therefore, only typical rich individuals or companies can afford to purchase and maintain the costs of direct connections to these providers, which is a win-loss, win for the reservation systems, and loss for airlines. Fortunately, companies such as Google (with QPX), perform a re-engineering of what these big reservation systems players do, and make it much more affordable and simple to implement [4], and with that, not only they improve the understanding effort of the whole business, but also, provide more sales leads to the airlines, and better user and developing experiences. The future should be standardised, airlines should distant themselves from the reservation systems (if these do not innovate), and open the doors of their businesses, by creating booking process APIs, that allow, developers and startups, to create powerful and innovative applications, that bring sales. [1] https://www.tnooz.com/article/why-is-the-travel-industry-so-unfriendly-to-startups/ https://www.tnooz.com/article/why-is-the-travel-industry-so-... [2] https://en.wikipedia.org/wiki/Sabre_(computer_system) https://en.wikipedia.org/wiki/Sabre_(computer_system) [3] https://en.wikipedia.org/wiki/Travel_agency https://en.wikipedia.org/wiki/Travel_agency [4] https://developers.google.com/qpx-express/v1/pricing https://developers.google.com/qpx-express/v1/pricing [5] https://en.wikipedia.org/wiki/Codeshare_agreement https://en.wikipedia.org/wiki/Codeshare_agreement