9 ms·

Show HN: Cantools – an extensive set of CAN bus tools

220 points by eerimoq 7y ago

fullstop 7y ago
If you work with J1939 there is a can-j1939 kernel module which was mainlined in the 5.4 Linux kernel. It handles address negotiation and other time-sensitive matters which would have had to be done in user space previously. https://github.com/linux-can/can-utils/blob/master/can-j1939-kickstart.md https://github.com/linux-can/can-utils/blob/master/can-j1939...
- irishcoffee 7y ago
  Welp this makes my day-job a touch simpler now. Thank you for the heads up!
dmd 7y ago
Ugh, where was this when I needed it in 2013? At $OLDJOB my boss and I wrote pretty much this entire thing, in an unholy mix of bash and Tcl, much more poorly.
- efrecon 7y ago
  Any chance this unholy mix is open sourced?
  - dmd 7y ago
    Sorry, no. Very proprietary medical device. (And, really, you wouldn't want it. Really. shudders)
- phkahler 7y ago
  I cant tell you how many times I've seen this type of thing done at different companies. Many of those projects died. The last one was a LIN tool written in python. My coworker wrote it. He wanted to open source it, so I talked our boss, the local IP guy and eventually the legal department. Legal thought I wanted them to write a license. No, that's not useful, I needed to know which of GPL, MIT, BSD the company would approve. They came with some of the misguided concerns companies tend to have. The other question I could not get answered is who in management needed to OK it. Oh and would the copyright belong to the company or the guy who wrote it. None of my questions could seem to find definitive answers. The guy who wrote the tool quit and it became abandon ware inside the company. I had mentioned that nobody stays around forever and the author would have done some maintenance after he's gone. I said this well before he left as an argument in favor of open sourcing it. Anyway, I hope a lot of people bring this to life so we can all stop reinventing this wheel or paying certain companies to rent theirs. Related: Busmaster, but its Windows only.
  - kapep 7y ago
    > Legal thought I wanted them to write a license. I'm not surprised. I once had a conversation with some business lawyers that were specialized on what is easiest described as "IT law" [1] (mostly intellectual property, licensing, contract law, data privacy) who were working in the field of IT/software. You could talk with them about client-server and p2p systems or even basic encryption and data security, so they definitely knew a bit about technology. They knew about open source software but I was surprised that they had never heard about source licenses nor about the creative common license. I tried to explain the basics to them but they didn't seem to get the concept of a license that could be reused by more than one licenser and were confused why a license would need a name (like 'MIT license'). In their view, licenses were always individual drafted documents with terms that needed to be negotiated with business partners or were imposed on customers. They didn't believe me that you don't create your own custom license for open source projects and thought that I was talking about copy&pasting bad license templates from the internet. When I told them the basic concept of the GPL they thought I was bullshitting them. [1]: https://en.wikipedia.org/wiki/IT_law https://en.wikipedia.org/wiki/IT_law
    - rusk 7y ago
      Whats that saying about a person not being able to learn something that threatens their livlihood?
seren 7y ago
This seems to work with a candump format input, does it works with canalyzer *.asc format ?
- saterHater 7y ago
  Yeah, using -L you can shell redirect to a workable log file. I've done this a few times. They also build the asc2bin conversion utility to convert any logs to the more common *.bin format.
floathub 7y ago
This looks well designed and thought out. Does anyone know if it has been applied to NMEA-2000 (a marine electronics flavor of CANbus)?
- novakinblood 7y ago
  I was thinking the same question. I could only find C++ implementation at https://github.com/ttlappalainen/NMEA2000 https://github.com/ttlappalainen/NMEA2000 which I assume you have found as well.
  - floathub 7y ago
    Yes. And that ttlappalainen project generally inherits from the CANBoat project.
- bnycum 7y ago
  Different project, but maybe this helps. https://github.com/canboat/canboat https://github.com/canboat/canboat
- PinguTS 7y ago
  NMEA2000 is the marine electronics flavored J1939. Everything but the data dictionary is J1939. You can just switch on the J1939 protocol in WireShark.
nocman 7y ago
I would like to suggest that in your "About" suggestion you provide a short description what a "CAN bus" is (with links to https://en.wikipedia.org/wiki/CAN_bus https://en.wikipedia.org/wiki/CAN_bus - or whatever you feel is appropriate ). I'm a very technical person, and I honestly don't think I'd ever heard of it. I spent a decent bit of time looking on the site trying to figure out what it was before resorting to Google. It's not a huge deal, but seems to me like it would be helpful, and worth the effort.
- samreight 7y ago
  If you saw a library for say Image Processing or cattle husbandry for Java/JVM would you insist that the project provide a description of “Java”? I’m going to guess if you’re being honest you’d say no. CAN really is not an obscure thing at all. Also it is ones of those things that if you don’t know what it is, you probably don’t need this. Otherwise the google burden is minimal for the curious.
  - kingosticks 7y ago
    Obscure or not, I think the latter part of this comment is the important part. If you don't know what it is, you are not the target audience and you won't become the target audience by suddenly knowing what a CAN bus is. So no, I'd say it's actually not worth the effort to add an explanation. It may even give real users the wrong idea.
    - reaperducer 7y ago
      If you don't know what it is, you are not the target audience and you won't become the target audience by suddenly knowing what a CAN bus is. So you've never visited a web site, read a bit about a topic you knew nothing about, and then thought, "Wow, that sounds like something I'd like to get into?" There are a lot of people out there who are curious about the world around them and enjoy expanding what they know instead of stagnating in a knowledge silo.
      - 7y ago
m3kw9 7y ago
What’s missing is a can bus sim
- sulZ 7y ago
  I've recently become interested in a can bus sim, actually. If you know of any resources I would appreciate any pointers.
  - ocdtrekkie 7y ago
    There are some basic simulators out there that use single board computers. I've been meaning to try to build one at some point. https://github.com/carloop/simulator https://github.com/carloop/simulator (RasPi) https://github.com/limiter121/esp32-obd2-emulator https://github.com/limiter121/esp32-obd2-emulator (ESP32)
    - sulZ 7y ago
      Thanks. I'll check them out.
- btashton 7y ago
  Perhaps CANdevStudio would be what you are looking for? https://github.com/GENIVI/CANdevStudio https://github.com/GENIVI/CANdevStudio
t34543 7y ago
Beautiful - I’ve been using Intrepid Vehicle Spy for awhile but the cost is hard to justify for a hobbyist.
jmiskovic 7y ago
Thank you for making cantools. I'm using it daily and enjoying how easy and powerful it is. Two other projects also deserve to be mentioned. The opendbc is attempt to make sense of CAN traffic in different cars, they reverse-engineer messages and provide descriptions to each. It's part of comma.ai open source autonomous driving effort. The other project is CANboat which does similar thing for NMEA2000 protocol used on boats. The industry so far is very closed and rejects concept of interoperability. It's nice that somebody is taking steps in right direction.
- eerimoq 7y ago
  I'm glad you find it useful.
- ACS_Solver 7y ago
  > The industry so far is very closed and rejects concept of interoperability. It's nice that somebody is taking steps in right direction. Sadly very true. I've worked in the industry, and while there has been a lot of standardization, there's definitely not a lot of interoperability. Car makers and tier 1 suppliers in my experience are very insistent on keeping their knowledge in-house, distrustful of open source tools and extremely unwilling to share any technical information that could lead to greater interoperability.
jmt_ 7y ago
I'm very interested in beginning to monitor some information on my vehicle. Is the usual workflow involve pulling a dbc file off the vehicle with a CAN bus reader and using software to look into it? Or is it more complicated? Anyone have recommendations as to the reader itself?
- cjohnson318 7y ago
  I don't think you can pull a DBC file off of a vehicle. I think you have to build the DBC file yourself by reverse engineering everything--you'd monitor the CAN network, and then turn the light on and off, sit in the driver set, etc. The DBC file is the car's secret sauce.
  - zro 7y ago
    There's also at least one repo [1] of DBC files that have already been reverse-engineered by the community [1] https://github.com/commaai/opendbc https://github.com/commaai/opendbc
- selpop 7y ago
  You can get away with ELM327 based readers (the cheap bluetooth/Wi-Fi OBDII readers you see on Amazon/eBay) if you want fairly standard data like speed, and some manufacturer specific data in the form of PIDs (https://en.wikipedia.org/wiki/OBD-II_PIDs https://en.wikipedia.org/wiki/OBD-II_PIDs) If you want more involved data, or to send messages, it's mostly sniffing, doing an action, and seeing what changes. `can-utils` has a nice UI for this, which lets you filter messages that don't change easily. (Also, I've actually never seen a `.dbc` file before, but it looks like a map of PIDs, usually you [or the community for your car] have to reverse engineer those with the sniffing method, afaik) - The ELM327 can technically stream all the CAN messages on the bus and do that type of sniffing, but it has a very small buffer and gets completely hosed with chatty cars. I personally use the MCP2515 with a Raspberry PI for more involved car hacking. https://www.raspberrypi.org/forums/viewtopic.php?t=141052 https://www.raspberrypi.org/forums/viewtopic.php?t=141052 The electrical modification mentioned is needed for the most common MCP2515 modules because the PI isn't 5v tolerant, but it's an easy enough fix I have two of them hooked up to a Raspberry PI (one for each of my car's CAN busses, some cars have more), and that with `can-utils` has gotten me pretty far.
zro 7y ago
Thanks for all your hard work! We used this constantly at $oldjob and it saved us a ton of time and effort.
thbr99 7y ago
We have been using proprietary CANoe for well over 6 years. Will evaluate this. But don't know if corporate will like it or not.
- DoingIsLearning 7y ago
  Different use cases in my opinion. As it is described this would more likely meet the use-cases of CANalizer (sniffing, logging, lightweight scripting) but not CANoe (bus simulation, node simulation full on HIL Simulation).
monocasa 7y ago
Throwing out there that Wireshark has CAN parsing ability. For any else of y'all that have been stuck with a legacy non standard CAN protocol running on industrial automation, writing a Lua protocol parser and a little script to convert whatever internal format you've been using to pcap-ng will be an afternoon or two very well spent.
- PinguTS 7y ago
  In industrial automation there are mainly 2 standard protocols, which can obtain easily: DeviceNet and CANopen.
  - monocasa 7y ago
    There's also an ton of shops using literally no standard above the 8bytes+IDbits that CAN controllers spit out and accept.
contingencies 7y ago
Has anyone taken canbus and deployed it on a greenfield non-automotive system? If so, what was the thought process to justify use of this protocol vs. something like ethernet? The selling point seems to be 'save copper' but the associated overheads (tooling, training, reduced throughput[0], larger connectors, multidrop bus debug drawbacks, etc.) seem like they would far outweigh any nominal savings in all but the highest volume production scenarios. [0] https://en.wikipedia.org/wiki/ELM327#Protocols_supported_by_ELM327 https://en.wikipedia.org/wiki/ELM327#Protocols_supported_by_...
- 0_____0 7y ago
  A decent number micros support CAN natively without a PHY, (STM32), it works well over short to medium length link, and is just _vastly simpler_ than ethernet in both protocol and backend implementation. Nothing so complicated TCP/IP stack to deal with, programming for it is more akin to using a serial peripheral on your device.
  - contingencies 7y ago
    I agree it is simpler from an MCU programming perspective.
- crtlaltdel 7y ago
  Yes. It was used as an industrial communications bus to coordinate slave devices from a master controller. Drop-length was a non-issue as the majority of our devices used an in-rail (DIN mounted) interconnect bus. The larger "drops" were often a under 10" and just jumped one DIN rail to the next DIN rail in the cabinet. This was selected for the following reasons: 1. These devices were installed in HEAVY EM environments (think in close proximity to very large electric motors) and the electrical characteristics 2. The system topology physically supported multi-drop 3. We had a good deal of talent with CAN experience 4. Our device was already supporting a bunch of wired and wireless protocols (rs422, rs485, ethernet, btle, wifi, etc) and tbh CAN support was a gimme on the SOM we used as the device's core, so initially it made it into consideration by chance
prostodata 7y ago
Here is another project for working with CAN: https://github.com/hardbyte/python-can https://github.com/hardbyte/python-can I used it (as well as Cantools) for importing various CAN-specific data formats.
rationalthug 7y ago
For those interested in CAN bus for UAVs, there is https://github.com/uavcan https://github.com/uavcan as well.

220 points by eerimoq 7y ago

If you work with J1939 there is a can-j1939 kernel module which was mainlined in the 5.4 Linux kernel. It handles address negotiation and other time-sensitive matters which would have had to be done in user space previously. https://github.com/linux-can/can-utils/blob/master/can-j1939-kickstart.md https://github.com/linux-can/can-utils/blob/master/can-j1939...

irishcoffee 7y ago
Welp this makes my day-job a touch simpler now. Thank you for the heads up!

dmd 7y ago

Ugh, where was this when I needed it in 2013? At $OLDJOB my boss and I wrote pretty much this entire thing, in an unholy mix of bash and Tcl, much more poorly.

efrecon 7y ago
Any chance this unholy mix is open sourced?
- dmd 7y ago
  Sorry, no. Very proprietary medical device. (And, really, you wouldn't want it. Really. shudders)
phkahler 7y ago
I cant tell you how many times I've seen this type of thing done at different companies. Many of those projects died. The last one was a LIN tool written in python. My coworker wrote it. He wanted to open source it, so I talked our boss, the local IP guy and eventually the legal department. Legal thought I wanted them to write a license. No, that's not useful, I needed to know which of GPL, MIT, BSD the company would approve. They came with some of the misguided concerns companies tend to have. The other question I could not get answered is who in management needed to OK it. Oh and would the copyright belong to the company or the guy who wrote it. None of my questions could seem to find definitive answers. The guy who wrote the tool quit and it became abandon ware inside the company. I had mentioned that nobody stays around forever and the author would have done some maintenance after he's gone. I said this well before he left as an argument in favor of open sourcing it. Anyway, I hope a lot of people bring this to life so we can all stop reinventing this wheel or paying certain companies to rent theirs. Related: Busmaster, but its Windows only.
- kapep 7y ago
  > Legal thought I wanted them to write a license. I'm not surprised. I once had a conversation with some business lawyers that were specialized on what is easiest described as "IT law" [1] (mostly intellectual property, licensing, contract law, data privacy) who were working in the field of IT/software. You could talk with them about client-server and p2p systems or even basic encryption and data security, so they definitely knew a bit about technology. They knew about open source software but I was surprised that they had never heard about source licenses nor about the creative common license. I tried to explain the basics to them but they didn't seem to get the concept of a license that could be reused by more than one licenser and were confused why a license would need a name (like 'MIT license'). In their view, licenses were always individual drafted documents with terms that needed to be negotiated with business partners or were imposed on customers. They didn't believe me that you don't create your own custom license for open source projects and thought that I was talking about copy&pasting bad license templates from the internet. When I told them the basic concept of the GPL they thought I was bullshitting them. [1]: https://en.wikipedia.org/wiki/IT_law https://en.wikipedia.org/wiki/IT_law
  - rusk 7y ago
    Whats that saying about a person not being able to learn something that threatens their livlihood?

seren 7y ago

This seems to work with a candump format input, does it works with canalyzer *.asc format ?

saterHater 7y ago
Yeah, using -L you can shell redirect to a workable log file. I've done this a few times. They also build the asc2bin conversion utility to convert any logs to the more common *.bin format.

floathub 7y ago

This looks well designed and thought out. Does anyone know if it has been applied to NMEA-2000 (a marine electronics flavor of CANbus)?

novakinblood 7y ago
I was thinking the same question. I could only find C++ implementation at https://github.com/ttlappalainen/NMEA2000 https://github.com/ttlappalainen/NMEA2000 which I assume you have found as well.
- floathub 7y ago
  Yes. And that ttlappalainen project generally inherits from the CANBoat project.
bnycum 7y ago
Different project, but maybe this helps. https://github.com/canboat/canboat https://github.com/canboat/canboat
PinguTS 7y ago
NMEA2000 is the marine electronics flavored J1939. Everything but the data dictionary is J1939. You can just switch on the J1939 protocol in WireShark.

nocman 7y ago

I would like to suggest that in your "About" suggestion you provide a short description what a "CAN bus" is (with links to https://en.wikipedia.org/wiki/CAN_bus https://en.wikipedia.org/wiki/CAN_bus - or whatever you feel is appropriate ). I'm a very technical person, and I honestly don't think I'd ever heard of it. I spent a decent bit of time looking on the site trying to figure out what it was before resorting to Google. It's not a huge deal, but seems to me like it would be helpful, and worth the effort.

samreight 7y ago
If you saw a library for say Image Processing or cattle husbandry for Java/JVM would you insist that the project provide a description of “Java”? I’m going to guess if you’re being honest you’d say no. CAN really is not an obscure thing at all. Also it is ones of those things that if you don’t know what it is, you probably don’t need this. Otherwise the google burden is minimal for the curious.
- kingosticks 7y ago
  Obscure or not, I think the latter part of this comment is the important part. If you don't know what it is, you are not the target audience and you won't become the target audience by suddenly knowing what a CAN bus is. So no, I'd say it's actually not worth the effort to add an explanation. It may even give real users the wrong idea.
  - reaperducer 7y ago
    If you don't know what it is, you are not the target audience and you won't become the target audience by suddenly knowing what a CAN bus is. So you've never visited a web site, read a bit about a topic you knew nothing about, and then thought, "Wow, that sounds like something I'd like to get into?" There are a lot of people out there who are curious about the world around them and enjoy expanding what they know instead of stagnating in a knowledge silo.
    - 7y ago

m3kw9 7y ago

What’s missing is a can bus sim

sulZ 7y ago
I've recently become interested in a can bus sim, actually. If you know of any resources I would appreciate any pointers.
- ocdtrekkie 7y ago
  There are some basic simulators out there that use single board computers. I've been meaning to try to build one at some point. https://github.com/carloop/simulator https://github.com/carloop/simulator (RasPi) https://github.com/limiter121/esp32-obd2-emulator https://github.com/limiter121/esp32-obd2-emulator (ESP32)
  - sulZ 7y ago
    Thanks. I'll check them out.
btashton 7y ago
Perhaps CANdevStudio would be what you are looking for? https://github.com/GENIVI/CANdevStudio https://github.com/GENIVI/CANdevStudio

t34543 7y ago

Beautiful - I’ve been using Intrepid Vehicle Spy for awhile but the cost is hard to justify for a hobbyist.

jmiskovic 7y ago

Thank you for making cantools. I'm using it daily and enjoying how easy and powerful it is. Two other projects also deserve to be mentioned. The opendbc is attempt to make sense of CAN traffic in different cars, they reverse-engineer messages and provide descriptions to each. It's part of comma.ai open source autonomous driving effort. The other project is CANboat which does similar thing for NMEA2000 protocol used on boats. The industry so far is very closed and rejects concept of interoperability. It's nice that somebody is taking steps in right direction.

eerimoq 7y ago
I'm glad you find it useful.
ACS_Solver 7y ago
> The industry so far is very closed and rejects concept of interoperability. It's nice that somebody is taking steps in right direction. Sadly very true. I've worked in the industry, and while there has been a lot of standardization, there's definitely not a lot of interoperability. Car makers and tier 1 suppliers in my experience are very insistent on keeping their knowledge in-house, distrustful of open source tools and extremely unwilling to share any technical information that could lead to greater interoperability.

jmt_ 7y ago

I'm very interested in beginning to monitor some information on my vehicle. Is the usual workflow involve pulling a dbc file off the vehicle with a CAN bus reader and using software to look into it? Or is it more complicated? Anyone have recommendations as to the reader itself?

cjohnson318 7y ago
I don't think you can pull a DBC file off of a vehicle. I think you have to build the DBC file yourself by reverse engineering everything--you'd monitor the CAN network, and then turn the light on and off, sit in the driver set, etc. The DBC file is the car's secret sauce.
- zro 7y ago
  There's also at least one repo [1] of DBC files that have already been reverse-engineered by the community [1] https://github.com/commaai/opendbc https://github.com/commaai/opendbc
selpop 7y ago
You can get away with ELM327 based readers (the cheap bluetooth/Wi-Fi OBDII readers you see on Amazon/eBay) if you want fairly standard data like speed, and some manufacturer specific data in the form of PIDs (https://en.wikipedia.org/wiki/OBD-II_PIDs https://en.wikipedia.org/wiki/OBD-II_PIDs) If you want more involved data, or to send messages, it's mostly sniffing, doing an action, and seeing what changes. `can-utils` has a nice UI for this, which lets you filter messages that don't change easily. (Also, I've actually never seen a `.dbc` file before, but it looks like a map of PIDs, usually you [or the community for your car] have to reverse engineer those with the sniffing method, afaik) - The ELM327 can technically stream all the CAN messages on the bus and do that type of sniffing, but it has a very small buffer and gets completely hosed with chatty cars. I personally use the MCP2515 with a Raspberry PI for more involved car hacking. https://www.raspberrypi.org/forums/viewtopic.php?t=141052 https://www.raspberrypi.org/forums/viewtopic.php?t=141052 The electrical modification mentioned is needed for the most common MCP2515 modules because the PI isn't 5v tolerant, but it's an easy enough fix I have two of them hooked up to a Raspberry PI (one for each of my car's CAN busses, some cars have more), and that with `can-utils` has gotten me pretty far.

zro 7y ago

Thanks for all your hard work! We used this constantly at $oldjob and it saved us a ton of time and effort.

thbr99 7y ago

We have been using proprietary CANoe for well over 6 years. Will evaluate this. But don't know if corporate will like it or not.

DoingIsLearning 7y ago
Different use cases in my opinion. As it is described this would more likely meet the use-cases of CANalizer (sniffing, logging, lightweight scripting) but not CANoe (bus simulation, node simulation full on HIL Simulation).

monocasa 7y ago

Throwing out there that Wireshark has CAN parsing ability. For any else of y'all that have been stuck with a legacy non standard CAN protocol running on industrial automation, writing a Lua protocol parser and a little script to convert whatever internal format you've been using to pcap-ng will be an afternoon or two very well spent.

PinguTS 7y ago
In industrial automation there are mainly 2 standard protocols, which can obtain easily: DeviceNet and CANopen.
- monocasa 7y ago
  There's also an ton of shops using literally no standard above the 8bytes+IDbits that CAN controllers spit out and accept.

contingencies 7y ago

Has anyone taken canbus and deployed it on a greenfield non-automotive system? If so, what was the thought process to justify use of this protocol vs. something like ethernet? The selling point seems to be 'save copper' but the associated overheads (tooling, training, reduced throughput[0], larger connectors, multidrop bus debug drawbacks, etc.) seem like they would far outweigh any nominal savings in all but the highest volume production scenarios. [0] https://en.wikipedia.org/wiki/ELM327#Protocols_supported_by_ELM327 https://en.wikipedia.org/wiki/ELM327#Protocols_supported_by_...

0_____0 7y ago
A decent number micros support CAN natively without a PHY, (STM32), it works well over short to medium length link, and is just _vastly simpler_ than ethernet in both protocol and backend implementation. Nothing so complicated TCP/IP stack to deal with, programming for it is more akin to using a serial peripheral on your device.
- contingencies 7y ago
  I agree it is simpler from an MCU programming perspective.
crtlaltdel 7y ago
Yes. It was used as an industrial communications bus to coordinate slave devices from a master controller. Drop-length was a non-issue as the majority of our devices used an in-rail (DIN mounted) interconnect bus. The larger "drops" were often a under 10" and just jumped one DIN rail to the next DIN rail in the cabinet. This was selected for the following reasons: 1. These devices were installed in HEAVY EM environments (think in close proximity to very large electric motors) and the electrical characteristics 2. The system topology physically supported multi-drop 3. We had a good deal of talent with CAN experience 4. Our device was already supporting a bunch of wired and wireless protocols (rs422, rs485, ethernet, btle, wifi, etc) and tbh CAN support was a gimme on the SOM we used as the device's core, so initially it made it into consideration by chance

prostodata 7y ago

Here is another project for working with CAN: https://github.com/hardbyte/python-can https://github.com/hardbyte/python-can I used it (as well as Cantools) for importing various CAN-specific data formats.

rationalthug 7y ago

For those interested in CAN bus for UAVs, there is https://github.com/uavcan https://github.com/uavcan as well.