3 ms·

We (me and a friend) did something similar for multi city round trips within Europe (https://tripchemy.com https://tripchemy.com). We created some data structu

We (me and a friend) did something similar for multi city round trips within Europe (https://tripchemy.com https://tripchemy.com). We created some data structures that allow almost instant searching of such routes and we have scrapers running regularly on Ryanair, easyJet, wizzair, and Transavia. You can query the algorithm here: https://algo.tripchemy.com/routes/TSF?year=2020&month=02&day=5&length=7 https://algo.tripchemy.com/routes/TSF?year=2020&month=02&day... It's based on the following Open source project which my friend made back in the day https://github.com/Whazor/Roundtrip https://github.com/Whazor/Roundtrip

samschooler 7y ago
Is there a reason this wouldn’t work in the US (obvi the prices would be higher). But hypothetically could you support North America and or South America?
- giansegato 7y ago
  Skyscanner does support multiple trips, doesn't it?
- sphix0r 7y ago
  The only thing is that we have the crawl US airlines and support an additional currency(USD). So yes, there is no reason why this shouldn't work for North and South America. Do you think there is a market for this in America?
woutr_be 7y ago
I've always wondered how things like this work; how do you actually get all the routes and price information? Do you need to scrape every route on every possible data individually?
- sphix0r 7y ago
  We crawl the fare information ourself from the airlines'websites.
nreece 7y ago
Cool site! Curious to know what do you use for scraping, without getting blocked?
- sphix0r 7y ago
  Thanks a lot! We use scrapy[1]. Our crawler's behavior is very polite so most of them won't block us. Easyjet for example is very strict against crawlers and will block most of them(and us). [1] https://scrapy.org/ https://scrapy.org/
  - krn 7y ago
    > Easyjet for example is very strict against crawlers and will block most of them(and us). It seems that easyJet has outsourced its anti-scraping protection to the company called "Distil Networks"[1]. I wonder how could they block screen scrapers using Chrome on Android with mobile IP addresses. [1] https://www.youtube.com/watch?v=43wh3beHSBo https://www.youtube.com/watch?v=43wh3beHSBo
arthurfm 7y ago
Great website, although I have one minor nitpick. When searching for flights to or from London it's not clear which airport is which out of LTN, LGW and STN. Screenshot: https://i.vgy.me/J1F0j1.png https://i.vgy.me/J1F0j1.png Would it be possible to add the IATA code in brackets (after London) and possibly another item in the dropdown for all London Airports?

samschooler 7y ago

Is there a reason this wouldn’t work in the US (obvi the prices would be higher). But hypothetically could you support North America and or South America?

giansegato 7y ago
Skyscanner does support multiple trips, doesn't it?
sphix0r 7y ago
The only thing is that we have the crawl US airlines and support an additional currency(USD). So yes, there is no reason why this shouldn't work for North and South America. Do you think there is a market for this in America?

woutr_be 7y ago

I've always wondered how things like this work; how do you actually get all the routes and price information? Do you need to scrape every route on every possible data individually?

sphix0r 7y ago
We crawl the fare information ourself from the airlines'websites.

nreece 7y ago

Cool site! Curious to know what do you use for scraping, without getting blocked?

sphix0r 7y ago
Thanks a lot! We use scrapy[1]. Our crawler's behavior is very polite so most of them won't block us. Easyjet for example is very strict against crawlers and will block most of them(and us). [1] https://scrapy.org/ https://scrapy.org/
- krn 7y ago
  > Easyjet for example is very strict against crawlers and will block most of them(and us). It seems that easyJet has outsourced its anti-scraping protection to the company called "Distil Networks"[1]. I wonder how could they block screen scrapers using Chrome on Android with mobile IP addresses. [1] https://www.youtube.com/watch?v=43wh3beHSBo https://www.youtube.com/watch?v=43wh3beHSBo

arthurfm 7y ago

Great website, although I have one minor nitpick. When searching for flights to or from London it's not clear which airport is which out of LTN, LGW and STN. Screenshot: https://i.vgy.me/J1F0j1.png https://i.vgy.me/J1F0j1.png Would it be possible to add the IATA code in brackets (after London) and possibly another item in the dropdown for all London Airports?