5 ms·

The Hong Kong Jockey Club, in particular is excellent with regards to the free data they provide. They really do try to publish everything, from track work time

The Hong Kong Jockey Club, in particular is excellent with regards to the free data they provide. They really do try to publish everything, from track work times to horse weights.[1] Other jurisdictions not so much. There are various scrapers available to obtain generic race card information[2] and APIs from some bookmakers/exchanges.[3] A couple of years ago I read an article lamenting the lack of open data in horse racing, which still largely holds true today. Unfortunately, I can't find the article now (it was either on the Paulick Report or Bloodhorse website), but here's a more recent one from May this year calling for more open data.[4] In fact, a question was asked on HN a couple of years ago about an open horse racing database and as far as I know the answers there are still valid.[5] In general, there is data available for US, UK, IRE, Australian racing (the main jurisdictions) but they have to be paid for.[6][7][8][9] [1] https://racing.hkjc.com/racing/english/index.aspx https://racing.hkjc.com/racing/english/index.aspx [2] https://github.com/4A47/rpscrape https://github.com/4A47/rpscrape [3] https://docs.developer.betfair.com/display/1smk3cen4v3lu3yomq5qye0ni/API+Overview https://docs.developer.betfair.com/display/1smk3cen4v3lu3yom... [4] https://www.bloodhorse.com/horse-racing/articles/232487/thoroughbred-idea-foundation-calls-for-affordable-data https://www.bloodhorse.com/horse-racing/articles/232487/thor... [5] https://news.ycombinator.com/item?id=7779117 https://news.ycombinator.com/item?id=7779117 [6] https://www.betwise.co.uk/smartform https://www.betwise.co.uk/smartform [7] https://www.proformracing.com/ https://www.proformracing.com/ [8] https://southcoastdatabase.com.au/ https://southcoastdatabase.com.au/ [9] https://www.equibase.com/ https://www.equibase.com/

primeradical 7y ago
I work in the industry, can confirm, the tactics used to dissuade people for aggregating horse racing data so we can sell $2 PDFs are extremely counter-productive and reflect the age of the industry. Several orgs, including Equibase (US-based, the gate keeper of a good portion of handicapping data) will regularly send cease and desist orders to people who attempt to automate aggregation of data even with free, publicly available content. That's at least half the reason PDFs are used when customers purchase data access, to make aggregation harder (you should see some of the white space, character encoding fuckery they use to throw off aggregators). I suppose some of this often depends the quality of the data as well. Most data entry happens at the track during the race by a human, none of the data collection about races or the horse stats are collected by a computer, it's 95% hand entered. That also goes for pedigree information and other statistics including medications, weights, etc. And 100% of that is usually self-reported. Much of the current handicapping in the industry is everyone trying to protect their personal mountains of data. Tech-minded people would love to provide open, controlled, API services so that people can do what they will with our mountains of data. But "giving it away for free" is a non-starter for the good ole boys at the top..
- JoeAltmaier 7y ago
  Its a business, around a game. I guess they can do whatever they want with it.
- phillc73 7y ago
  You've hit the nail on the head. It's all about vested interests. I was involved for a number of years with a UK based horse racing ratings service (handicapping if in the US). This service used to license their base data from the Press Association[1] and then run algorithms on top to produce the ratings. There's certain things I can't say due to NDAs which are probably still in effect, but the cost of licensing this basic data was in excess of £10k per annum. So, unless you were a serious bettor or were looking to operate a service of some kind, it's beyond the pocket of most individuals. Timeform in the UK also license some of their own proprietory data, via an API[2]. They've published some pricing on their website and you're looking at between £6k - £12k per year. This is just to access data which is available via their website for a subscription fee of £75 per month, but via their API. There's even a specific UK organisation which apparently has the permission from the British Horse Racing Authority to officially licence key racing data. This is who sells the data to bookmakers, form guides, racing newspapers etc. They have a rate card published on their website.[3] Private, pro-punter? £8.5k per year please. It's a bit of a rort really. Most of the data is "freely" available online or in the racing press, but if you want to access it any useable format, either build a scraper (good luck with staying on top of the website changes) or pay a stack to access things programmatically. [1] https://pa.media/racing-betting/horseracing/ https://pa.media/racing-betting/horseracing/ [2] https://www.timeform.com/commercial/products/api https://www.timeform.com/commercial/products/api [3] http://www.racecoursedatacompany.com/ http://www.racecoursedatacompany.com/
- listenallyall 7y ago
  As you stated, the vast majority of racing data is collected, measured and entered by hand, by people who are paid to perform this job. It costs enormous amounts of money to employ all these people to watch every race in meticulous detail and gather all the data required to publish the Daily Racing Form. Why would you expect them NOT to protect this proprietary, valuable information? Almost all tracks publish result charts online for free along with race videos. If you want free, why not compile the data yourself? How long would DRF or Equibase exist if people could access their data for free?

primeradical 7y ago

I work in the industry, can confirm, the tactics used to dissuade people for aggregating horse racing data so we can sell $2 PDFs are extremely counter-productive and reflect the age of the industry. Several orgs, including Equibase (US-based, the gate keeper of a good portion of handicapping data) will regularly send cease and desist orders to people who attempt to automate aggregation of data even with free, publicly available content. That's at least half the reason PDFs are used when customers purchase data access, to make aggregation harder (you should see some of the white space, character encoding fuckery they use to throw off aggregators). I suppose some of this often depends the quality of the data as well. Most data entry happens at the track during the race by a human, none of the data collection about races or the horse stats are collected by a computer, it's 95% hand entered. That also goes for pedigree information and other statistics including medications, weights, etc. And 100% of that is usually self-reported. Much of the current handicapping in the industry is everyone trying to protect their personal mountains of data. Tech-minded people would love to provide open, controlled, API services so that people can do what they will with our mountains of data. But "giving it away for free" is a non-starter for the good ole boys at the top..

JoeAltmaier 7y ago
Its a business, around a game. I guess they can do whatever they want with it.
phillc73 7y ago
You've hit the nail on the head. It's all about vested interests. I was involved for a number of years with a UK based horse racing ratings service (handicapping if in the US). This service used to license their base data from the Press Association[1] and then run algorithms on top to produce the ratings. There's certain things I can't say due to NDAs which are probably still in effect, but the cost of licensing this basic data was in excess of £10k per annum. So, unless you were a serious bettor or were looking to operate a service of some kind, it's beyond the pocket of most individuals. Timeform in the UK also license some of their own proprietory data, via an API[2]. They've published some pricing on their website and you're looking at between £6k - £12k per year. This is just to access data which is available via their website for a subscription fee of £75 per month, but via their API. There's even a specific UK organisation which apparently has the permission from the British Horse Racing Authority to officially licence key racing data. This is who sells the data to bookmakers, form guides, racing newspapers etc. They have a rate card published on their website.[3] Private, pro-punter? £8.5k per year please. It's a bit of a rort really. Most of the data is "freely" available online or in the racing press, but if you want to access it any useable format, either build a scraper (good luck with staying on top of the website changes) or pay a stack to access things programmatically. [1] https://pa.media/racing-betting/horseracing/ https://pa.media/racing-betting/horseracing/ [2] https://www.timeform.com/commercial/products/api https://www.timeform.com/commercial/products/api [3] http://www.racecoursedatacompany.com/ http://www.racecoursedatacompany.com/
listenallyall 7y ago
As you stated, the vast majority of racing data is collected, measured and entered by hand, by people who are paid to perform this job. It costs enormous amounts of money to employ all these people to watch every race in meticulous detail and gather all the data required to publish the Daily Racing Form. Why would you expect them NOT to protect this proprietary, valuable information? Almost all tracks publish result charts online for free along with race videos. If you want free, why not compile the data yourself? How long would DRF or Equibase exist if people could access their data for free?