11 ms·

Traff: A universal format for live traffic information

152 points by KoftaBob 4y ago

necovek 4y ago
Is the server usable to set up a public OpenStreetMap-like traffic server where users can install trivial apps on their phones just like Google does for all the Android users (edit) to submit their own anonymized traffic information?
- capableweb 4y ago
  Seems so? https://traffxml.gitlab.io/app-navit.html https://traffxml.gitlab.io/app-navit.html (also at https://www.navit-project.org/ https://www.navit-project.org/ apparently) https://traffxml.gitlab.io/ https://traffxml.gitlab.io/ seems to be a better landing page than the submission URL which just contains a bunch of loosely organized git repositories.
  - necovek 4y ago
    I am looking for the other direction. What the above describes is Navit receiving traffic information, I want Navit (or ideally, an even simpler app) to submit data to an open source traffic server. The idea is to crowd-collect traffic information just like how Google does it (monitoring speed of those who have their Android phones with Google GPS services on, but out of willing participants): basically transparent to the users (other than them having to install the app and then figure out a way to allow them to run it in the background on recent Android systems :). I'd feel much more comfortable submitting to a server where I've got full control of the data it emits (I can better ensure it retains my privacy by using new route ids often, for instance).
    - capableweb 4y ago
      Maybe this is more what you're looking for? https://www.openstreetmap.org/traces https://www.openstreetmap.org/traces Seems to be public traces the public are submitting willingly. What's missing is a service to integrate that data into the map view/routing, unless this is exactly what Traff integrates as well, unsure.
      - necovek 4y ago
        Nope: those are not anonimized nor real time. Basically, Google knows a road is congested by having dozens of people (with their Android phones and location services turned on) slow down on a road where they would usually go much faster. I want the same thing with better privacy guarantees and that's open source and free to use.
        toomuchtodo 4y ago
        You’re looking for a sensor fabric that can track mobile devices with their signal strength and time of flight, as it’s likely you’ll never get enough critical mass of devices to do this with open source apps. This allows you to distill traffic velocity without device access or onboard device telemetry being shipped. Google can do this because they control the Android and Google Maps ecosystem. Without such scale, you’re at a significant disadvantage.
        necovek 4y ago
        Yes, I understand that. But the same variation in resources holds true for eg. Google Maps vs OpenStreetMaps, yet OSM is still sometimes much better than GMaps. And Google also crowdsources their data even with all the resources at their disposal. Whether there'd be a critical mass depends on a lot of factors, and it's a safe bet that there wouldn't be one. But I would wish us to try :) FWIW, there would huge commercial interests for other parties to provide these, eg all the taxi and delivery services could have their drivers participate, while allowing their app developers to integrate that traffic information into their own route estimations and optimizations.
        necovek 4y ago
        Btw, your suggestions make me think of another interesting approach that local governments could take. Instead of devices reporting themselves, we could have hotspots observing mobile phone signals (eg. integrate them into traffic lights): we'd need at least 3 nearby for triangulation, and they'd have to be everywhere, so it would require government support and desire to provide traffic info for the public.
- 29083011397778 4y ago
  That's actually what this appears to be (though under the RoadEagle name on F-Droid). Looks like this [0] is the backend. [0] https://gitlab.com/traffxml/traff-server https://gitlab.com/traffxml/traff-server
  - necovek 4y ago
    I was a bit unclear: I want to submit anonymized traffic information to an open source traffic server, just like how Google uses Android phones' speeds to guess what roads are congested.
mikece 4y ago
Slightly off topic, but it would be nice if there was an option for a group-level Readme on Gitlab. Seeing a collection of projects doesn't tell me much about what is going on. For info beyond the code repo there is this: https://traffxml.gitlab.io/#about https://traffxml.gitlab.io/#about
- deleted 4y ago
  [deleted]
- mikercampbell 4y ago
  Same. I clicked the link and bounced because I wasn't sure which repo to open to get the basics, and wasn't willing to iterate through.
  - sytse 4y ago
    The view is now sorted by last created by default. Maybe most stars https://gitlab.com/traffxml?sort=stars_desc https://gitlab.com/traffxml?sort=stars_desc (and last created for repos with the same starts) is a better default sort? BTW The repo with the specification seems to be https://gitlab.com/traffxml/traff https://gitlab.com/traffxml/traff and it contains a presentation with more context https://gitlab.com/traffxml/traff/-/blob/master/TraFF%20Presentation.pdf https://gitlab.com/traffxml/traff/-/blob/master/TraFF%20Pres...
- dnsmichi 4y ago
  GitLab team member here. > it would be nice if there was an option for a group-level Readme on Gitlab. Thanks for the idea. The proposal for Group READMEs is discussed in this thread: https://gitlab.com/gitlab-org/gitlab/-/issues/15041#note_1020157227 https://gitlab.com/gitlab-org/gitlab/-/issues/15041#note_102... Suggest adding your preferences for the proposed solutions, thanks.
  - mikece 4y ago
    Why can't there be a group-level setting that allows me to designate any Markdown file in any of the repos in the group as the group-level readme? You could even make a filename such as GroupReadme.md a reserved name that can only be used once in a group. I could also see someone possibly wanting to move that file -- and it's history -- to another repo as a group matures: you start with repos like "API" and "WebUI" but eventually a "Docs" repo could be created as proof of concept solidifies and accumulates personnel and ceremony.
noja 4y ago
Is this at all like RDS-TMC? https://en.wikipedia.org/wiki/Traffic_message_channel#Functionality https://en.wikipedia.org/wiki/Traffic_message_channel#Functi...
- danuker 4y ago
  Yes, they even compare with TMC in the specification: https://gitlab.com/traffxml/traff/raw/master/TraFF%20Specification.pdf?inline=false https://gitlab.com/traffxml/traff/raw/master/TraFF%20Specifi... > The approach chosen is essentially based upon the following principles: > • Use the TMC data model as a starting point
danuker 4y ago
A traffic service could integrate the live data with say, weekly patterns detected in public OpenStreetMap GPS traces, to get better estimates of actual conditions: https://www.openstreetmap.org/traces https://www.openstreetmap.org/traces
cbolat 4y ago
Honestly, why XML? Isn't JSON 100 times better, smaller and easy to use than XML? why choose a legacy format (with known security problems in parsers, compability, and unnecessary use of bandwidth) in 2022?
- alexchamberlain 4y ago
  It looks like the primary implementation is in Java, which is still all about SOAP services and XSDs.
  - rschoultz 4y ago
    We are on Java 19 now. Several that spits out the same sentiment as you seem to be stuck with Java 8 and EE legacy projects from 2012.
- magicalhippo 4y ago
  Tooling around JSON schema is lacking. For example, we have a fairly simple XSD which still requires the latest JSON schema version due to a choice element (IIRC), which Swagger doesn't fully support yet.
  - jillesvangurp 4y ago
    I think the reason is that mostly people don't define schemas for their json to begin with. Back when xml was still commonly used, xml schemas were optional as well. The notion of insisting on adding longish urls to every attribute and namespacing elements and attributes, kind of defeated the purpose of using xml. It made everything harder; including parsing, xsl transformations, xpath expressions, etc.
    - HyperSane 4y ago
      "adding longish urls to every attribute and namespacing elements and attributes" Who thought this was a good idea?
- 4y ago
llui85 4y ago
See: https://traffxml.gitlab.io/ https://traffxml.gitlab.io/ for a better summary.
- dang 4y ago
  Ok, we've changed to that from https://gitlab.com/traffxml https://gitlab.com/traffxml above. Thanks!
karussell 4y ago
If you lack the source data you might get them from official feeds. A few years ago I started to collect them: https://github.com/graphhopper/open-traffic-collection https://github.com/graphhopper/open-traffic-collection
- jolmg 4y ago
  That's really cool, but it seems to be monthly/annual data. I wonder how much longer before there are service providers with satellites providing live hourly traffic data without needing some sort of ground support. Something that can compete with Google's live traffic, which I imagine pulls data from the location of their ubiquitous phones...
uoaei 4y ago
Once, just once, I would like a traffic routing or analysis system to comprehend the concept of lanes. There's no point coloring a segment red if I'm turning right, when all the slow-moving traffic is waiting for a left. Anyone who spends more than a day driving a car in a populous area will recognize this. I noticed that Traff can represent blocked or closed lanes as individual events, but otherwise considers "fast" and "slow" traffic only at the scale of streets and roads. Guess I will be waiting longer, still.

152 points by KoftaBob 4y ago

Is the server usable to set up a public OpenStreetMap-like traffic server where users can install trivial apps on their phones just like Google does for all the Android users (edit) to submit their own anonymized traffic information?

capableweb 4y ago
Seems so? https://traffxml.gitlab.io/app-navit.html https://traffxml.gitlab.io/app-navit.html (also at https://www.navit-project.org/ https://www.navit-project.org/ apparently) https://traffxml.gitlab.io/ https://traffxml.gitlab.io/ seems to be a better landing page than the submission URL which just contains a bunch of loosely organized git repositories.
- necovek 4y ago
  I am looking for the other direction. What the above describes is Navit receiving traffic information, I want Navit (or ideally, an even simpler app) to submit data to an open source traffic server. The idea is to crowd-collect traffic information just like how Google does it (monitoring speed of those who have their Android phones with Google GPS services on, but out of willing participants): basically transparent to the users (other than them having to install the app and then figure out a way to allow them to run it in the background on recent Android systems :). I'd feel much more comfortable submitting to a server where I've got full control of the data it emits (I can better ensure it retains my privacy by using new route ids often, for instance).
  - capableweb 4y ago
    Maybe this is more what you're looking for? https://www.openstreetmap.org/traces https://www.openstreetmap.org/traces Seems to be public traces the public are submitting willingly. What's missing is a service to integrate that data into the map view/routing, unless this is exactly what Traff integrates as well, unsure.
    - necovek 4y ago
      Nope: those are not anonimized nor real time. Basically, Google knows a road is congested by having dozens of people (with their Android phones and location services turned on) slow down on a road where they would usually go much faster. I want the same thing with better privacy guarantees and that's open source and free to use.
      - toomuchtodo 4y ago
        You’re looking for a sensor fabric that can track mobile devices with their signal strength and time of flight, as it’s likely you’ll never get enough critical mass of devices to do this with open source apps. This allows you to distill traffic velocity without device access or onboard device telemetry being shipped. Google can do this because they control the Android and Google Maps ecosystem. Without such scale, you’re at a significant disadvantage.
        necovek 4y ago
        Yes, I understand that. But the same variation in resources holds true for eg. Google Maps vs OpenStreetMaps, yet OSM is still sometimes much better than GMaps. And Google also crowdsources their data even with all the resources at their disposal. Whether there'd be a critical mass depends on a lot of factors, and it's a safe bet that there wouldn't be one. But I would wish us to try :) FWIW, there would huge commercial interests for other parties to provide these, eg all the taxi and delivery services could have their drivers participate, while allowing their app developers to integrate that traffic information into their own route estimations and optimizations.
        necovek 4y ago
        Btw, your suggestions make me think of another interesting approach that local governments could take. Instead of devices reporting themselves, we could have hotspots observing mobile phone signals (eg. integrate them into traffic lights): we'd need at least 3 nearby for triangulation, and they'd have to be everywhere, so it would require government support and desire to provide traffic info for the public.
29083011397778 4y ago
That's actually what this appears to be (though under the RoadEagle name on F-Droid). Looks like this [0] is the backend. [0] https://gitlab.com/traffxml/traff-server https://gitlab.com/traffxml/traff-server
- necovek 4y ago
  I was a bit unclear: I want to submit anonymized traffic information to an open source traffic server, just like how Google uses Android phones' speeds to guess what roads are congested.

mikece 4y ago

Slightly off topic, but it would be nice if there was an option for a group-level Readme on Gitlab. Seeing a collection of projects doesn't tell me much about what is going on. For info beyond the code repo there is this: https://traffxml.gitlab.io/#about https://traffxml.gitlab.io/#about

deleted 4y ago
[deleted]
mikercampbell 4y ago
Same. I clicked the link and bounced because I wasn't sure which repo to open to get the basics, and wasn't willing to iterate through.
- sytse 4y ago
  The view is now sorted by last created by default. Maybe most stars https://gitlab.com/traffxml?sort=stars_desc https://gitlab.com/traffxml?sort=stars_desc (and last created for repos with the same starts) is a better default sort? BTW The repo with the specification seems to be https://gitlab.com/traffxml/traff https://gitlab.com/traffxml/traff and it contains a presentation with more context https://gitlab.com/traffxml/traff/-/blob/master/TraFF%20Presentation.pdf https://gitlab.com/traffxml/traff/-/blob/master/TraFF%20Pres...
dnsmichi 4y ago
GitLab team member here. > it would be nice if there was an option for a group-level Readme on Gitlab. Thanks for the idea. The proposal for Group READMEs is discussed in this thread: https://gitlab.com/gitlab-org/gitlab/-/issues/15041#note_1020157227 https://gitlab.com/gitlab-org/gitlab/-/issues/15041#note_102... Suggest adding your preferences for the proposed solutions, thanks.
- mikece 4y ago
  Why can't there be a group-level setting that allows me to designate any Markdown file in any of the repos in the group as the group-level readme? You could even make a filename such as GroupReadme.md a reserved name that can only be used once in a group. I could also see someone possibly wanting to move that file -- and it's history -- to another repo as a group matures: you start with repos like "API" and "WebUI" but eventually a "Docs" repo could be created as proof of concept solidifies and accumulates personnel and ceremony.

noja 4y ago

Is this at all like RDS-TMC? https://en.wikipedia.org/wiki/Traffic_message_channel#Functionality https://en.wikipedia.org/wiki/Traffic_message_channel#Functi...

danuker 4y ago
Yes, they even compare with TMC in the specification: https://gitlab.com/traffxml/traff/raw/master/TraFF%20Specification.pdf?inline=false https://gitlab.com/traffxml/traff/raw/master/TraFF%20Specifi... > The approach chosen is essentially based upon the following principles: > • Use the TMC data model as a starting point

danuker 4y ago

A traffic service could integrate the live data with say, weekly patterns detected in public OpenStreetMap GPS traces, to get better estimates of actual conditions: https://www.openstreetmap.org/traces https://www.openstreetmap.org/traces

cbolat 4y ago

Honestly, why XML? Isn't JSON 100 times better, smaller and easy to use than XML? why choose a legacy format (with known security problems in parsers, compability, and unnecessary use of bandwidth) in 2022?

alexchamberlain 4y ago
It looks like the primary implementation is in Java, which is still all about SOAP services and XSDs.
- rschoultz 4y ago
  We are on Java 19 now. Several that spits out the same sentiment as you seem to be stuck with Java 8 and EE legacy projects from 2012.
magicalhippo 4y ago
Tooling around JSON schema is lacking. For example, we have a fairly simple XSD which still requires the latest JSON schema version due to a choice element (IIRC), which Swagger doesn't fully support yet.
- jillesvangurp 4y ago
  I think the reason is that mostly people don't define schemas for their json to begin with. Back when xml was still commonly used, xml schemas were optional as well. The notion of insisting on adding longish urls to every attribute and namespacing elements and attributes, kind of defeated the purpose of using xml. It made everything harder; including parsing, xsl transformations, xpath expressions, etc.
  - HyperSane 4y ago
    "adding longish urls to every attribute and namespacing elements and attributes" Who thought this was a good idea?
4y ago

llui85 4y ago

See: https://traffxml.gitlab.io/ https://traffxml.gitlab.io/ for a better summary.

dang 4y ago
Ok, we've changed to that from https://gitlab.com/traffxml https://gitlab.com/traffxml above. Thanks!

karussell 4y ago

If you lack the source data you might get them from official feeds. A few years ago I started to collect them: https://github.com/graphhopper/open-traffic-collection https://github.com/graphhopper/open-traffic-collection

jolmg 4y ago
That's really cool, but it seems to be monthly/annual data. I wonder how much longer before there are service providers with satellites providing live hourly traffic data without needing some sort of ground support. Something that can compete with Google's live traffic, which I imagine pulls data from the location of their ubiquitous phones...

uoaei 4y ago

Once, just once, I would like a traffic routing or analysis system to comprehend the concept of lanes. There's no point coloring a segment red if I'm turning right, when all the slow-moving traffic is waiting for a left. Anyone who spends more than a day driving a car in a populous area will recognize this. I noticed that Traff can represent blocked or closed lanes as individual events, but otherwise considers "fast" and "slow" traffic only at the scale of streets and roads. Guess I will be waiting longer, still.