6 ms·

USB Reverse Engineering: Down the Rabbit Hole

283 points by alias1 8y ago

jonhohle 8y ago
What a weekend find! This is a nice piece in a puzzle I'm trying to solve related to a consumer audio device with upgradeable firmware. The platform of the device is used for both a prosumer audio device, a professional device, as well as other consumer devices. On top of that, the prosumer device appears to have different functionality enabled based on the part of the world you buy the device, but my hypothesis is that the hardware is the same, regardless of region, and the firmware just implements a dark pattern to limit functionality (perhaps to avoid additional FCC certifications, support incidents, etc.). Anyway, I've spent a few nights extracting the firmware from other regions, found a great presentation where the company details the hardware and platform, and discovered that the audio capabilities of the hardware for the prosumer device approach that of the professional device (higher sampling frequencies and resolution during recording). My hope was to be able to flash the device with a modified file system that enabled the features, but ran into the firmware appearing to be encrypted, but potentially decrypted on the host before being pushed to the device. I've been decompiling the drivers to see if I could dump keys or the decrypted payloads, but also thought about sniffing the USB I/O and dumping that way. It looks like this is a nice intro to help me on my way.
- alias1 8y ago
  Sounds like a really interesting project! Hopefully you manage to figure it out. Glad I could help start your own rabbithole of a journey! ;p
- digi_owl 8y ago
  Sounds like a real classic market segmentation tactic. I recall reading about a oscilloscope or something that had two models, one monochrome and slightly worse performance, one color and slightly better better. Turns out that if you flashed the firmware of the latter onto the former, the performance improved to match the latter's. Basic thing is that by doing this, the marketing people has calculated that they may reach a larger total customer pool than by only offering the more expensive variant.
  - alias1 8y ago
    I can definitely understand the marketing/psychology appeal of it.. but it also feels like a bit of a rip that I either end up with a) a less good product, or b) paid more for the same thing. That is.. until we figure out the ways to DIY upgrade them ;)
    - digi_owl 8y ago
      And then get slapped with DMCA or something, because "circumvention"?
    - vvanders 8y ago
      I'd be really leery about reflashing a $X,000 scope. The NRE on those are huge so just because it's the same hw doesn't mean you aren't still paying for something when you buy the more expensive version.
    - taneq 8y ago
      > b) paid more for the same thing I find it interesting that, here of all places, people consider two different software applications to be "the same thing" just because they run on the same hardware. I dare say a significant percentage of the development budget for any smart piece of equipment these days goes into the software, not the hardware.
      - red75prime 8y ago
        Putting "#ifdef PROSUMER" or something to this effect into the source code shouldn't be too costly. Disabling existing functionality is easy and it feels like cheating even if it's called "market segmentation".
        gmueckl 8y ago
        This is what developers like us tend to think. But consider this: the development cost of the sodtware does not change much in order to create feature reduced versions of most products. But if you are in a small market where the volume of expected sales for the full version of the product can barely lead to proftability, creating a cheaper version for a higher volume market is a sound business strategy. This would not work with a single undifferentiated product.
- duxup 8y ago
  Reminds me of some popular headphones (I forget the name) that had two models. One was the standard model and one was the pro model. Someone went to repair their headphones and found that the only difference between the two models was some extra foam that was installed to distort the sound bit in the standard model... I wish I could remember who that was.
  - lesbaker 8y ago
    It was Sennheiser headphones: https://news.ycombinator.com/item?id=2214158 https://news.ycombinator.com/item?id=2214158
    - duxup 8y ago
      Thank you!
jerrysievert 8y ago
this is a great writeup, definitely falls into something I've been dreaming of building, but was planning to hack around the usb part of it (using a usb-uart adapter), but now I'm thinking maybe I do it right! should make for a great weekend. unfortunately, it seems that the pre-req article is down, so here's the cached copy: http://webcache.googleusercontent.com/search?q=cache:https://www.linuxvoice.com/drive-it-yourself-usb-car-6/ http://webcache.googleusercontent.com/search?q=cache:https:/...
- alias1 8y ago
  Glad I inspired you to hack it the right way! Thanks for sharing the cached link :)
corndoge 8y ago
I've sunk countless hours looking for a way to emulate USB devices in software under Windows. Two years ago there were many different ways. Today there are zero ways that I know of, apparently all removed within the last two years by an astounding tragic coincidence. To elaborate on this section: http://devalias.net/devalias/2018/05/13/usb-reverse-engineering-down-the-rabbit-hole/#where-next http://devalias.net/devalias/2018/05/13/usb-reverse-engineer... - There are two versions of the USB/IP drivers. The initial version has a bug where disconnection of a device causes a blue screen. This version is signed by ReactOS, and so you can install this version's drivers without needing to jump through any hoops under Windows. Then there's a patched version that does not cause the bluescreen. Great! Except that this version is not signed by ReactOS. Okay, maybe we can get ReactOS to sign it -- nope, they discontinued their driver signing program because of new regulations by Microsoft put in place within the last year (https://www.reactos.org/wiki/Driver_Signing https://www.reactos.org/wiki/Driver_Signing). You can also no longer install unsigned drivers on Windows 10 by putting your machine into test mode. This capability was silently removed by Microsoft within the last year. You can still put your device into test mode, but the driver signing checker will still block installation. This is a damn shame because USB/IP is a great project, and it even has a signed driver but the patched version of the driver is not signed. This is infinitely frustrating. - Microsoft used to have this perfect thing called UDE (https://docs.microsoft.com/en-us/windows-hardware/drivers/usbcon/developing-windows-drivers-for-emulated-usb-host-controllers-and-devices https://docs.microsoft.com/en-us/windows-hardware/drivers/us...). They removed it last year. - Kernel mode drivers for Windows now have to go through this signing process (https://docs.microsoft.com/en-us/windows-hardware/drivers/install/kernel-mode-code-signing-policy--windows-vista-and-later- https://docs.microsoft.com/en-us/windows-hardware/drivers/in...) as of last year. As mentioned before, this is why ReactOS discontinued their driver signing program as it's too difficult to meet these new requirements. I have read every forum post, stack overflow post, Windows technical docs, everything I could possibly find for a way to do stable, pure software emulation of USB devices on Windows. They all evaporated within the last year or two. So frustrating as I really really want to build out something but there's simply no way to do it. At this point I have resorted to attempting to hook every Windows system call related to USB device initialization and spoofing them but have been met with no success thus far. I should note that the reason I want to do this in software is because I want to distribute the end result to a relatively large amount of potential users as GPL'd software. For this reason hardware emulation or patching / downgrading Windows isn't an option sadly. Does anyone know of something I missed?
forapurpose 8y ago
> I won't get deep on describing all of the facts, since that's what Wikipedia is good at Sigh. So many links, so much reading, and not one to the USB spec (that I found by skimming and searching). It's not just this article, but seemingly every 'deep dive does it'. Why do people read everything on the Internet, no matter the source and quality, but skip (what is usually) the best, most important source?
- alias1 8y ago
  I originally wrote this as a way to collate all of the things I was reading as I went from zero knowledge on the subject. I like to document the path I took, and don't really see the point in reimplementing the wheel when someone else has already done a good job of it. That might not align with everyones style.. but can't please everyone.
  - forapurpose 8y ago
    > can't please everyone Absolutely; and pleasing an audience of one (me) isn't a great business move. Thanks for your exceptional effort. > I like to document the path I took I think people learn more from this kind of inductive approach.
- mng2 8y ago
  To be fair, specs and standards generally aren't written as introductions, so they're kind of hard to make sense of when starting from zero. USB, with its infernal descriptors and endpoints, benefits mightily from third-party explanations.
  - forapurpose 8y ago
    > specs and standards generally aren't written as introductions Sometimes, but I often find them to be pretty good, and if I look at third party sources first, I usually wish I'd come to the spec much sooner in the process; the actual spec tends to make most third party sources redundant or plainly erroneous. But of course, not all specs are equal; some are unreadable or poor introductions.
ncmncm 8y ago
I see so much custom hardware. Can somebody tell me why running the drivers of interest in a virtualbox/qemu image, and logging interaction on the host OS, doesn't do a better job than custom hardware?
- _xzxj 8y ago
  I'm not sure why this person listed so much hardware, I've done some USB protocol reversing using a TotalPhase Beagle 480. I think I probably could have done something in software, but the nice thing is the TotalPhase software is a lot like Wireshark -- which I was already familiar with. I wasn't totally familiar with hacking around USB so it made learning how USB worked much easier instead of me just pulling data off the wire and trying to start from scratch.
  - jack12 8y ago
    With the software-only approach (e.g. usbmon), the interface is even more like Wireshark :) https://wiki.wireshark.org/CaptureSetup/USB https://wiki.wireshark.org/CaptureSetup/USB
- rvense 8y ago
  Is that possible? I thought vbox et al enabled USB communication by allowing the guest OS drivers to hit the hardware directly.
  - ncmncm 8y ago
    Maybe you need Linux for the host OS, that not everybody already has. It might be like KVM switches, that nobody using Linux would ever feel like they needed.
- alias1 8y ago
  Depending on your needs, you absolutely can do it in software. The first few steps are about software, and in particular, probably just using Wireshark + USBPcap or usbmon is sufficient for basic cases. The hardware side of things is the area that interests me more though, if you want to get into higher speeds than can be handled in software only, or if you want to start doing weird/interesting things in the 'emulate' side of it (not just capturing (eg. facedancer). Coming from a security perspective, those sorts of tools are invaluable for getting to some of the depths that I may want to reach.

283 points by alias1 8y ago

What a weekend find! This is a nice piece in a puzzle I'm trying to solve related to a consumer audio device with upgradeable firmware. The platform of the device is used for both a prosumer audio device, a professional device, as well as other consumer devices. On top of that, the prosumer device appears to have different functionality enabled based on the part of the world you buy the device, but my hypothesis is that the hardware is the same, regardless of region, and the firmware just implements a dark pattern to limit functionality (perhaps to avoid additional FCC certifications, support incidents, etc.). Anyway, I've spent a few nights extracting the firmware from other regions, found a great presentation where the company details the hardware and platform, and discovered that the audio capabilities of the hardware for the prosumer device approach that of the professional device (higher sampling frequencies and resolution during recording). My hope was to be able to flash the device with a modified file system that enabled the features, but ran into the firmware appearing to be encrypted, but potentially decrypted on the host before being pushed to the device. I've been decompiling the drivers to see if I could dump keys or the decrypted payloads, but also thought about sniffing the USB I/O and dumping that way. It looks like this is a nice intro to help me on my way.

alias1 8y ago
Sounds like a really interesting project! Hopefully you manage to figure it out. Glad I could help start your own rabbithole of a journey! ;p
digi_owl 8y ago
Sounds like a real classic market segmentation tactic. I recall reading about a oscilloscope or something that had two models, one monochrome and slightly worse performance, one color and slightly better better. Turns out that if you flashed the firmware of the latter onto the former, the performance improved to match the latter's. Basic thing is that by doing this, the marketing people has calculated that they may reach a larger total customer pool than by only offering the more expensive variant.
- alias1 8y ago
  I can definitely understand the marketing/psychology appeal of it.. but it also feels like a bit of a rip that I either end up with a) a less good product, or b) paid more for the same thing. That is.. until we figure out the ways to DIY upgrade them ;)
  - digi_owl 8y ago
    And then get slapped with DMCA or something, because "circumvention"?
  - vvanders 8y ago
    I'd be really leery about reflashing a $X,000 scope. The NRE on those are huge so just because it's the same hw doesn't mean you aren't still paying for something when you buy the more expensive version.
  - taneq 8y ago
    > b) paid more for the same thing I find it interesting that, here of all places, people consider two different software applications to be "the same thing" just because they run on the same hardware. I dare say a significant percentage of the development budget for any smart piece of equipment these days goes into the software, not the hardware.
    - red75prime 8y ago
      Putting "#ifdef PROSUMER" or something to this effect into the source code shouldn't be too costly. Disabling existing functionality is easy and it feels like cheating even if it's called "market segmentation".
      - gmueckl 8y ago
        This is what developers like us tend to think. But consider this: the development cost of the sodtware does not change much in order to create feature reduced versions of most products. But if you are in a small market where the volume of expected sales for the full version of the product can barely lead to proftability, creating a cheaper version for a higher volume market is a sound business strategy. This would not work with a single undifferentiated product.
duxup 8y ago
Reminds me of some popular headphones (I forget the name) that had two models. One was the standard model and one was the pro model. Someone went to repair their headphones and found that the only difference between the two models was some extra foam that was installed to distort the sound bit in the standard model... I wish I could remember who that was.
- lesbaker 8y ago
  It was Sennheiser headphones: https://news.ycombinator.com/item?id=2214158 https://news.ycombinator.com/item?id=2214158
  - duxup 8y ago
    Thank you!

jerrysievert 8y ago

this is a great writeup, definitely falls into something I've been dreaming of building, but was planning to hack around the usb part of it (using a usb-uart adapter), but now I'm thinking maybe I do it right! should make for a great weekend. unfortunately, it seems that the pre-req article is down, so here's the cached copy: http://webcache.googleusercontent.com/search?q=cache:https://www.linuxvoice.com/drive-it-yourself-usb-car-6/ http://webcache.googleusercontent.com/search?q=cache:https:/...

alias1 8y ago
Glad I inspired you to hack it the right way! Thanks for sharing the cached link :)

corndoge 8y ago

I've sunk countless hours looking for a way to emulate USB devices in software under Windows. Two years ago there were many different ways. Today there are zero ways that I know of, apparently all removed within the last two years by an astounding tragic coincidence. To elaborate on this section: http://devalias.net/devalias/2018/05/13/usb-reverse-engineering-down-the-rabbit-hole/#where-next http://devalias.net/devalias/2018/05/13/usb-reverse-engineer... - There are two versions of the USB/IP drivers. The initial version has a bug where disconnection of a device causes a blue screen. This version is signed by ReactOS, and so you can install this version's drivers without needing to jump through any hoops under Windows. Then there's a patched version that does not cause the bluescreen. Great! Except that this version is not signed by ReactOS. Okay, maybe we can get ReactOS to sign it -- nope, they discontinued their driver signing program because of new regulations by Microsoft put in place within the last year (https://www.reactos.org/wiki/Driver_Signing https://www.reactos.org/wiki/Driver_Signing). You can also no longer install unsigned drivers on Windows 10 by putting your machine into test mode. This capability was silently removed by Microsoft within the last year. You can still put your device into test mode, but the driver signing checker will still block installation. This is a damn shame because USB/IP is a great project, and it even has a signed driver but the patched version of the driver is not signed. This is infinitely frustrating. - Microsoft used to have this perfect thing called UDE (https://docs.microsoft.com/en-us/windows-hardware/drivers/usbcon/developing-windows-drivers-for-emulated-usb-host-controllers-and-devices https://docs.microsoft.com/en-us/windows-hardware/drivers/us...). They removed it last year. - Kernel mode drivers for Windows now have to go through this signing process (https://docs.microsoft.com/en-us/windows-hardware/drivers/install/kernel-mode-code-signing-policy--windows-vista-and-later- https://docs.microsoft.com/en-us/windows-hardware/drivers/in...) as of last year. As mentioned before, this is why ReactOS discontinued their driver signing program as it's too difficult to meet these new requirements. I have read every forum post, stack overflow post, Windows technical docs, everything I could possibly find for a way to do stable, pure software emulation of USB devices on Windows. They all evaporated within the last year or two. So frustrating as I really really want to build out something but there's simply no way to do it. At this point I have resorted to attempting to hook every Windows system call related to USB device initialization and spoofing them but have been met with no success thus far. I should note that the reason I want to do this in software is because I want to distribute the end result to a relatively large amount of potential users as GPL'd software. For this reason hardware emulation or patching / downgrading Windows isn't an option sadly. Does anyone know of something I missed?

forapurpose 8y ago

> I won't get deep on describing all of the facts, since that's what Wikipedia is good at Sigh. So many links, so much reading, and not one to the USB spec (that I found by skimming and searching). It's not just this article, but seemingly every 'deep dive does it'. Why do people read everything on the Internet, no matter the source and quality, but skip (what is usually) the best, most important source?

alias1 8y ago
I originally wrote this as a way to collate all of the things I was reading as I went from zero knowledge on the subject. I like to document the path I took, and don't really see the point in reimplementing the wheel when someone else has already done a good job of it. That might not align with everyones style.. but can't please everyone.
- forapurpose 8y ago
  > can't please everyone Absolutely; and pleasing an audience of one (me) isn't a great business move. Thanks for your exceptional effort. > I like to document the path I took I think people learn more from this kind of inductive approach.
mng2 8y ago
To be fair, specs and standards generally aren't written as introductions, so they're kind of hard to make sense of when starting from zero. USB, with its infernal descriptors and endpoints, benefits mightily from third-party explanations.
- forapurpose 8y ago
  > specs and standards generally aren't written as introductions Sometimes, but I often find them to be pretty good, and if I look at third party sources first, I usually wish I'd come to the spec much sooner in the process; the actual spec tends to make most third party sources redundant or plainly erroneous. But of course, not all specs are equal; some are unreadable or poor introductions.

ncmncm 8y ago

I see so much custom hardware. Can somebody tell me why running the drivers of interest in a virtualbox/qemu image, and logging interaction on the host OS, doesn't do a better job than custom hardware?

_xzxj 8y ago
I'm not sure why this person listed so much hardware, I've done some USB protocol reversing using a TotalPhase Beagle 480. I think I probably could have done something in software, but the nice thing is the TotalPhase software is a lot like Wireshark -- which I was already familiar with. I wasn't totally familiar with hacking around USB so it made learning how USB worked much easier instead of me just pulling data off the wire and trying to start from scratch.
- jack12 8y ago
  With the software-only approach (e.g. usbmon), the interface is even more like Wireshark :) https://wiki.wireshark.org/CaptureSetup/USB https://wiki.wireshark.org/CaptureSetup/USB
rvense 8y ago
Is that possible? I thought vbox et al enabled USB communication by allowing the guest OS drivers to hit the hardware directly.
- ncmncm 8y ago
  Maybe you need Linux for the host OS, that not everybody already has. It might be like KVM switches, that nobody using Linux would ever feel like they needed.
alias1 8y ago
Depending on your needs, you absolutely can do it in software. The first few steps are about software, and in particular, probably just using Wireshark + USBPcap or usbmon is sufficient for basic cases. The hardware side of things is the area that interests me more though, if you want to get into higher speeds than can be handled in software only, or if you want to start doing weird/interesting things in the 'emulate' side of it (not just capturing (eg. facedancer). Coming from a security perspective, those sorts of tools are invaluable for getting to some of the depths that I may want to reach.