11 ms·

You Can't Use the Live UK Train Data Without Accepting a Gagging Clause

128 points by mocko 14y ago

meaty 14y ago
Annoyingly this wasn't always the case. Since privatization this has been the case. Prior to that you could get a TOPS terminal/feed for a nominal fee and do what you want with it. Now, it's covering arses and nothing more.
- SideburnsOfDoom 14y ago
  It's not just CYA. It's also profit, corporate reputation management and empire-building.
  - huhsamovar 14y ago
    Ironically, this is precisely the attitude that prevents companies from improving their product. Such a shame.
    - CapitalistCartr 14y ago
      That's not ironic; the goal of a company is not to improve their product; its to SELL it. "Profit, corporate reputation management and empire-building" support selling; product improvement may or may not, depending on the market and the improvement.
    - rhizome 14y ago
      Improvements are a cost-center.
  - MaysonL 14y ago
    In other words, CYA 'cause if people knew exactly how bad your service was, somebody's head might roll.
rurounijones 14y ago
I remember a while ago I wanted to get the location of things like bus-stops. (after the whole "showusabetterway" waste-of-time site the UK gov created. ) Found the information on a site some and found that I could ONLY use it if I was under contract from a UK local council. Most of the info sources listed on that site were bound up in huge amounts of red tape / brick walls Frankly I am not surprised that the trains are in the same situation.
- robinhouston 14y ago
  Some things do get better, at least. The database that includes the locations of bus stops (NaPTAN) is now open data: http://data.gov.uk/dataset/naptan http://data.gov.uk/dataset/naptan
  - rurounijones 14y ago
    Ahhh, NaPTAN, that was what it was called. Thank you, I gave up a long time ago with the project that was going to use this data. Glad that things are getting better at least.
blibble 14y ago
people forget that ATOC exists to protect and further the interests of their members: the privately owned Train Operating Companies -- they don't owe random developers anything. Network Rail on the other had are (basically) state owned, and their extremely comprehensive feeds are now completely open: http://www.networkrail.co.uk/data-feeds/ http://www.networkrail.co.uk/data-feeds/ these feeds even include the raw signalling data (you even get events when trains activate level crossings) -- there's nothing stopping you building your own NRE API. I've been slowly building up a database of signal locations/berths to lat/longs by matching up my physical position (while riding on the train) to the NR feed events, with the idea of being able to produce a much better map of where the trains are in realtime.
- SideburnsOfDoom 14y ago
  > they don't owe random developers anything IMHO ATOC, like everyone else, owe random developers one thing: to deal with them fairly; and not say one thing but do another. To not say that they're open when they're just giving it lip service.
  - johnpapdu 14y ago
    There's no legal obligation to do that, so why should they? Because of morals? Are your morals demonstrably more correct than theirs? Why should your morals have anything to do with their morals? If you want a company to do something, you either need to go through your representative in our democratic process to suggest a change to the law, withdraw your custom, or shut up. Hoping that they follow your arbitrary morals is bizarre.
    - SideburnsOfDoom 14y ago
      So if I understand your argument, it is basically "how can you possibly say that the arbitrary morals of 'fair dealing' is more correct than 'screwing everyone over'? Bizarre. Shut up." Is that right? Or is there more to it than a simple derailing tactic?
coob 14y ago
As someone who had to shutter an app due to legal threats from ATOC, this doesn't surprise me one iota.
JasonFruit 14y ago
I guess I'm not revolutionary enough, but I can't see the problem with this. It makes sense for a private company to protect their reputation, and it makes sense not to provide data to people who run them down in public. They have an agreement to that effect that is provided in advance, and you can opt out be deciding not to participate. I know I'm supposed to feel rage whenever there are conditions placed on the use of data, but I just can't muster it up in this case. Sorry.
- blocking_io 14y ago
  A private company which is funded by companies which are contracted by the government to provide rail service. As a taxpayer subsidised body, they should have a greater degree of accountability.
  - JasonFruit 14y ago
    By that logic, every contractor hired mainly by government would have to behave like a public organization. I don't think that leads to sensible conclusions, and it's redolent of after-the-fact justification.
    - SideburnsOfDoom 14y ago
      > By that logic, every contractor hired mainly by government would have to behave like a public organization. You say that like it's a bad thing. In fact, on major criticism of outsourcing and privatisation is that it becomes way to shelter dubious activities from Freedom-Of-Information requests that would apply if the work was done by a public organization.
    - blocking_io 14y ago
      Not so. NRE's raison d'être is to provide information to the public about train services funded by the public. There is no aspect of it's purpose that is not connected to taxpayer funding.
- SideburnsOfDoom 14y ago
andreyf 14y ago
NRE have a stranglehold on the only comprehensive source of real-time passenger train data in the UK. They developed it to power their own website but have recognised in it a way to exert power over anyone who wants to use real time train data and to stifle criticism. What is referred to by "their website"? Why not just scrape the data?
- blocking_io 14y ago
  Presumably National Rail Enquiries. How can you base a commercial application on scraping a website? That is not a sensible business model.
  - warrenmiller 14y ago
    Google do
darklajid 14y ago
I'm torn. While I certainly don't like a 'gagging clause', I can only look with envy at the data you UK guys can get your hand at. So, maybe it's not perfect, maybe not good enough, but being able to get at that much data in the first place _is_ already quite remarkable imo. I'd love to see a German equivalent, gagging clause or not.
- andrewem 14y ago
  What's the transit data availability situation in Germany? In the US, various public transit agencies do varyingly good jobs of this. Boston's MBTA has lots of data available, both geo, schedule and real-time, and hasn't made any complaints or tried to stop a site which is at least somewhat critical (http://howfuckedisthet.com http://howfuckedisthet.com), and have been stupendously helpful to me as a developer. New York's MTA built their own system using an open transit API for bus real-time data (http://bustime.mta.info/wiki/Developers/Index http://bustime.mta.info/wiki/Developers/Index). Certain of my interactions with them have been less than efficient, let's say.
  - andrewem 14y ago
    I just noticed this page on Transport for London's API docs [1], the summary of which is "Use our data, but don't pretend to be us". +1 for straightforward guidelines like this. For instance, "Please don't... Use our brands - the roundel, unless it's part of a map interface" , but "Please do... Use the words 'London Underground', 'Tube' and 'London bus'". [1] http://www.tfl.gov.uk/businessandpartners/syndication/17102.aspx http://www.tfl.gov.uk/businessandpartners/syndication/17102....
danpalmer 14y ago
This happens with Bus data as well in the UK. Although possibly for different reasons. I work with live bus time data and before getting access I had to sign an NDA because bus companies are fined by the council for poor performance. With access to that data I can see the percentage of buses that are late and could compromise the business of the companies.

128 points by mocko 14y ago

meaty 14y ago

Annoyingly this wasn't always the case. Since privatization this has been the case. Prior to that you could get a TOPS terminal/feed for a nominal fee and do what you want with it. Now, it's covering arses and nothing more.

SideburnsOfDoom 14y ago
It's not just CYA. It's also profit, corporate reputation management and empire-building.
- huhsamovar 14y ago
  Ironically, this is precisely the attitude that prevents companies from improving their product. Such a shame.
  - CapitalistCartr 14y ago
    That's not ironic; the goal of a company is not to improve their product; its to SELL it. "Profit, corporate reputation management and empire-building" support selling; product improvement may or may not, depending on the market and the improvement.
  - rhizome 14y ago
    Improvements are a cost-center.
- MaysonL 14y ago
  In other words, CYA 'cause if people knew exactly how bad your service was, somebody's head might roll.

rurounijones 14y ago

I remember a while ago I wanted to get the location of things like bus-stops. (after the whole "showusabetterway" waste-of-time site the UK gov created. ) Found the information on a site some and found that I could ONLY use it if I was under contract from a UK local council. Most of the info sources listed on that site were bound up in huge amounts of red tape / brick walls Frankly I am not surprised that the trains are in the same situation.

robinhouston 14y ago
Some things do get better, at least. The database that includes the locations of bus stops (NaPTAN) is now open data: http://data.gov.uk/dataset/naptan http://data.gov.uk/dataset/naptan
- rurounijones 14y ago
  Ahhh, NaPTAN, that was what it was called. Thank you, I gave up a long time ago with the project that was going to use this data. Glad that things are getting better at least.

blibble 14y ago

people forget that ATOC exists to protect and further the interests of their members: the privately owned Train Operating Companies -- they don't owe random developers anything. Network Rail on the other had are (basically) state owned, and their extremely comprehensive feeds are now completely open: http://www.networkrail.co.uk/data-feeds/ http://www.networkrail.co.uk/data-feeds/ these feeds even include the raw signalling data (you even get events when trains activate level crossings) -- there's nothing stopping you building your own NRE API. I've been slowly building up a database of signal locations/berths to lat/longs by matching up my physical position (while riding on the train) to the NR feed events, with the idea of being able to produce a much better map of where the trains are in realtime.

SideburnsOfDoom 14y ago
> they don't owe random developers anything IMHO ATOC, like everyone else, owe random developers one thing: to deal with them fairly; and not say one thing but do another. To not say that they're open when they're just giving it lip service.
- johnpapdu 14y ago
  There's no legal obligation to do that, so why should they? Because of morals? Are your morals demonstrably more correct than theirs? Why should your morals have anything to do with their morals? If you want a company to do something, you either need to go through your representative in our democratic process to suggest a change to the law, withdraw your custom, or shut up. Hoping that they follow your arbitrary morals is bizarre.
  - SideburnsOfDoom 14y ago
    So if I understand your argument, it is basically "how can you possibly say that the arbitrary morals of 'fair dealing' is more correct than 'screwing everyone over'? Bizarre. Shut up." Is that right? Or is there more to it than a simple derailing tactic?

coob 14y ago

As someone who had to shutter an app due to legal threats from ATOC, this doesn't surprise me one iota.

JasonFruit 14y ago

I guess I'm not revolutionary enough, but I can't see the problem with this. It makes sense for a private company to protect their reputation, and it makes sense not to provide data to people who run them down in public. They have an agreement to that effect that is provided in advance, and you can opt out be deciding not to participate. I know I'm supposed to feel rage whenever there are conditions placed on the use of data, but I just can't muster it up in this case. Sorry.

blocking_io 14y ago
A private company which is funded by companies which are contracted by the government to provide rail service. As a taxpayer subsidised body, they should have a greater degree of accountability.
- JasonFruit 14y ago
  By that logic, every contractor hired mainly by government would have to behave like a public organization. I don't think that leads to sensible conclusions, and it's redolent of after-the-fact justification.
  - SideburnsOfDoom 14y ago
    > By that logic, every contractor hired mainly by government would have to behave like a public organization. You say that like it's a bad thing. In fact, on major criticism of outsourcing and privatisation is that it becomes way to shelter dubious activities from Freedom-Of-Information requests that would apply if the work was done by a public organization.
  - blocking_io 14y ago
    Not so. NRE's raison d'être is to provide information to the public about train services funded by the public. There is no aspect of it's purpose that is not connected to taxpayer funding.
SideburnsOfDoom 14y ago

andreyf 14y ago

NRE have a stranglehold on the only comprehensive source of real-time passenger train data in the UK. They developed it to power their own website but have recognised in it a way to exert power over anyone who wants to use real time train data and to stifle criticism. What is referred to by "their website"? Why not just scrape the data?

blocking_io 14y ago
Presumably National Rail Enquiries. How can you base a commercial application on scraping a website? That is not a sensible business model.
- warrenmiller 14y ago
  Google do

darklajid 14y ago

I'm torn. While I certainly don't like a 'gagging clause', I can only look with envy at the data you UK guys can get your hand at. So, maybe it's not perfect, maybe not good enough, but being able to get at that much data in the first place _is_ already quite remarkable imo. I'd love to see a German equivalent, gagging clause or not.

andrewem 14y ago
What's the transit data availability situation in Germany? In the US, various public transit agencies do varyingly good jobs of this. Boston's MBTA has lots of data available, both geo, schedule and real-time, and hasn't made any complaints or tried to stop a site which is at least somewhat critical (http://howfuckedisthet.com http://howfuckedisthet.com), and have been stupendously helpful to me as a developer. New York's MTA built their own system using an open transit API for bus real-time data (http://bustime.mta.info/wiki/Developers/Index http://bustime.mta.info/wiki/Developers/Index). Certain of my interactions with them have been less than efficient, let's say.
- andrewem 14y ago
  I just noticed this page on Transport for London's API docs [1], the summary of which is "Use our data, but don't pretend to be us". +1 for straightforward guidelines like this. For instance, "Please don't... Use our brands - the roundel, unless it's part of a map interface" , but "Please do... Use the words 'London Underground', 'Tube' and 'London bus'". [1] http://www.tfl.gov.uk/businessandpartners/syndication/17102.aspx http://www.tfl.gov.uk/businessandpartners/syndication/17102....

danpalmer 14y ago

This happens with Bus data as well in the UK. Although possibly for different reasons. I work with live bus time data and before getting access I had to sign an NDA because bus companies are fined by the council for poor performance. With access to that data I can see the percentage of buses that are late and could compromise the business of the companies.