4 ms·
The US Bureau of Transport statistics provides historic flight schedule and actual flight performance data in CSV tables: http://www.transtats.bts.gov/DL_Selec
by einpoklum 6y ago
The US Bureau of Transport statistics provides historic flight schedule and actual flight performance data in CSV tables:
http://www.transtats.bts.gov/DL_SelectFields.asp?Table_ID=236&DB_Short_Name=On-Time http://www.transtats.bts.gov/DL_SelectFields.asp?Table_ID=23...
But it's cumbersome to work with.
I am working (on and off) on a DBMS benchmark based on this data. As part of that endeavor, I have a script which:
* Automates downloading the CSVs.
* Creates an appropriate SQL database schema.
* Performs a bit of rudimentary cleaning (e.g. invalid character codes; optional)
* Loads the CSV files into the database.
So that, from the command-line, you could get the flight on-time performance data by merely typing in something like:
/path/to/usdt-ontime-tools/scripts/setup-usdt-ontime-db -r -db-name ontime --first-year 2019 --last-year 2020
it's available within this repository:
https://github.com/eyalroz/usdt-ontime-tools https://github.com/eyalroz/usdt-ontime-tools
the caveat is that, for now, the only DBMS supported directly is MonetDB: https://www.monetdb.org/ https://www.monetdb.org/ , a FOSS analytics-oriented columnar DBMS.
An adaptation of the script for other systems (MySQL/Maria, PostgreSQL) should be straightforward, since the commands are SQL'ish after all. If you're interested in that, open an issue or write me.
- bernardv 6y agoSpent a lot of time with that dataset a couple of years ago, looking at historical flight delays and cancellation rates. If only this type of data was available outside of the US. The data is updated daily I believe and provides a lot of detail and is of pretty good quality.