12 ms·

Analyzing the FBI’s Explanation of How They Located Silk Road

319 points by nikcub 12y ago

praptak 12y ago
If I ever set up a hidden service I'll make sure it lives on a box that has no public IP whatsoever. Preferably a physical box.
- throw3 12y ago
  how?
  - nknighthb 12y ago
    The simplest case is trivial. You place the hidden service box (HSB) and the Tor gateway (TG) on private IP space, and deny the HSB all connectivity to anything except the TG. Ideally you actually place the HSB and TG in different network segments so packets from the HSB must pass through a router to get to the TG and vice-versa, ensuring a compromised HSB has no possibility of sniffing any incriminating broadcast traffic to/from the TG.
    - dobbsbob 12y ago
      Then you shill your service with a pseudonym connected to your real name so they find you anyway
  - praptak 12y ago
    All of the stuff that I expect to leak or get hacked lives on box A in a private network. The second host in this network would be the Tor gateway B, hardened, stripped down of any unnecessary stuff. Its only function would be to route traffic to/from A into Tor. Obviously it would have to be connected to the public internet but at least it would have much less software that can be hacked. As an additional security measure, B could live behind a firewall that only let connections to Tor nodes. (yup, it might be a PITA to set up.)
    - nknighthb 12y ago
      Not that big a PITA, I don't think. People already use Tor exit node lists to block activity from Tor exit nodes, you're kind of just looking for the inverse. Shouldn't be hard to create a script to generate e.g. iptables rules that permit connections to Tor public relays. The security benefit is a bit marginal, though. The public relay list is basically world-writable.
      - dublinben 12y ago
        Entire operating systems like Whonix already exist to deliver that kind of security.
        nknighthb 12y ago
        We're really talking about two different kinds of "security" here. There's blocking access to anything but a Tor gateway (including in the case of a compromise at the kernel level), and blocking access by the Tor gateway to anything but Tor relays. But that raises another point I'd neglected. On the Tor gateway, you can block access to the public internet by anything other than the Tor process. The security benefit against maliciousness is iffy, since the most likely route of compromise for a Tor gateway that's been set up with even a little care is the Tor process, but it certainly reduces the chance of accidents.
        dublinben 12y ago
        How do you actually target the Tor gateway machine though? You can attack a Hidden Service, because there must be a webserver that responds to requests. The Tor gateway just passes traffic from the network to your client device.
        nitrogen 12y ago
        How do you actually target the Tor gateway machine though? Not the OP, and I don't know much about Tor's protocol, but at a guess one might start with deliberately malformed layers of the onion, so you route through a Tor entry node, the outer onion layer is removed, then the packet is forwarded to a Tor relay. The Tor relay then unwraps the next layer of the onion, but the data within that layer is deliberately malformed to trigger a bug in the Tor code running on that relay.
- nwf 12y ago
  Unfortunately, the technology you need to do that is not yet finished: https://trac.torproject.org/projects/tor/ticket/9498 https://trac.torproject.org/projects/tor/ticket/9498 ETA: I should have said "one possible technology"; there may be others, but I am pretty sure that an IP-less Tor node requires that you play the Tor Bridge game and stream the Tor protocol over a non-IP link. I have had a prototype of this design running, but have yet to have it to a point I consider robust.
  - nickodell 12y ago
    This looks like it's unrelated.
  - icebraining 12y ago
    praptak didn't say IP-less, but no public IP. If the server has an IP in the 192.168.1.X range, that doesn't tell an attacker much, supposedly.
siliconc0w 12y ago
I think it's more likely by header the FBI meant HTTP response header not IPV4 header. Legally I don't know if we should allow "fuzzing" without a warrant. They've put people in jail for doing less.
- Igglyboo 12y ago
  Weev was put in prison for wayyyy less than what the FBI did.
  - eli 12y ago
    And the judge who released him from prison expressed skepticism about that. Regardless, two wrongs...
    - wcummings 12y ago
      Yes, but that's not why he was released. Those charges are still untested, he was let off the hook due to the venue.
  - meowface 12y ago
    I don't think Weev should have been tried criminally at all, but I kind of disagree with this purely from a technical standpoint. The FBI discovered a flaw that let them see the IP address of a web server. Weev discovered a flaw that let him see private information of AT&T customers. Customer information is generally considered much more sensitive than an IP address.
    - totony 12y ago
      Privacy is relative. In Weev's case, AT&T considered the information of its users to be sensible, private information, while, in SR's care, the IP address is a very sensible and private information, considering that tor was used to hide it.
colinbartlett 12y ago
All of these analyses of how they might have located the server are great. I'm pleased to see folks trying to prove or disprove that the FBI is not trying some parallel construction here. But what worries me is that all of this will have to be eventually dumbed down to an explanation that a jury of average joes can understand. The increasingly technical complexity of evidence in some of these cases worries me that our system doesn't have an adequate way to deal with it.
- dobbsbob 12y ago
  They probably won't argue this in front of a jury since it's a pre-trial motion to challenge the evidence the FBI collected. No idea how a judge will understand this, I guess they bring in experts to testify if the FBI's claim is possible or not.
- lifeisstillgood 12y ago
  Honestly I have no worries about this or any other "technical" issue like financial fraud cases. If a cop can learn enough to catch someone then it is possible to explain that to a nurse or an electrician. Jurors don't switch off because they are learning how to move money between banks or break into servers - they switch off because it takes three months to present a case.
  - EpicEng 12y ago
    > If a cop can learn enough to catch someone then it is possible to explain that to a nurse or an electrician. You think a cop did this? No... the FBI has technical personnel. Just like any corporation, they have different departments to fulfill different needs.
    - lifeisstillgood 12y ago
      Not quite what I meant - no I don't think cops were writing SQLi but they learnt enough to laugh at Little Bobby Tables, just as they can learn enough to follow the money through 17 bank accounts. This is the equivalent of is the FBI allowed to search my trash if the trash is in a trashcan on my property or on the sidewalk, being pulled by a street cleaner or knocked over by a fox. We can make it as complicated as we like but the law needs to draw some lines somewhere and stick to them.
zachrose 12y ago
Is the constitutional issue here that exploiting a security hole (SQL injection, remote execution to dump the $_SERVER variable, what have you) constitutes an illegal search? If so, is their any guidance from case law about the difference between some sort of legal poking and prodding vs illegal hacking?
- eurleif 12y ago
  I wonder if this could be considered a case of exigent circumstances? [0] The exigent circumstances exception can be invoked "to forestall the imminent escape of a suspect, or destruction of evidence". Arguably, that's what this was -- Silk Road had a security flaw that could be fixed at any moment, and the FBI may have felt they needed to act quickly to obtain evidence before the security flaw was fixed. [0] https://en.wikipedia.org/wiki/Exigent_circumstance_in_United_States_law https://en.wikipedia.org/wiki/Exigent_circumstance_in_United...
  - deleted 12y ago
    [deleted]
  - Donzo 12y ago
    Well, they were also committing criminal activity in "plain view" so to speak.
- strathmeyer 12y ago
  In the past simply typing in random URLs has been deemed illegally circumventing a site's security. Like most things it's fine when the government does it but you'll get jailtime if you do it to one of their friends.
- gcb4 12y ago
  it can probably set a precedent that will free all scapegoat script kiddies currently in outsourced prisons.
- downandout 12y ago
  In general in order to break the law you must have a warrant. For example, a cop entering private property without a warrant or probable cause is nothing more than a trespasser. These agents likely violated the CFAA by doing what they did. Because they didn't have a warrant, IMO it probably constitutes an illegal search, but the courts may well have a very different opinion. Regardless of which side loses this argument, this will likely find its way to the appellate courts and possibly the Supreme court, as it seems to be an unanswered question of great importance.
hadoukenio 12y ago
If they obtained a warrant, is fuzzing/hacking Silk Road the equivalent of kicking down a door during a raid?
- ascorbic 12y ago
  They didn't obtain a warrant.
- inportb 12y ago
  Probably... though I have an easier time imagining someone picking a lock or, in the case of fuzzing, using a bump key.
Eleopteryx 12y ago
Slightly off topic: could you please increase the font size on your site? It's actually below 12pt. The blockquotes are more readable than the body of the post.
- DougBTX 12y ago
  It is 17px for me, but most browsers support zooming into a page, that will make text easier to read on almost any site, but could make it more difficult if accidentally zoomed out.
  - wcummings 12y ago
    Every browser has supported this since forever
- nikcub 12y ago
  I only noticed today after somebody sent me a screenshot that i'd been viewing my own website zoomed in at two levels. I updated the stylesheet and bumped the size up earlier but it might not come through without a hard refresh because of crazy caching™
asuffield 12y ago
So we have a pretty good guess of how they did it, and while their affidavit doesn't give a lot of important details, ultimately they were just interacting with a web service in the way it is supposed to be interacted with - by sending it http requests - and the buggy site revealed itself to them because the guy who wrote it wasn't a very good web developer. I can understand why they wouldn't want to give more details than they have to; it makes the prosecution's work harder. But I and many others have been arguing for years that there's nothing illegal about sending messages to a server and looking at the responses, if you aren't trying to damage anything. I can't see any reason why it would be less legal for the FBI to do it.
- jrochkind1 12y ago
  We agree, but one reason we'd want more details about the FBI going to greater lengths to trigger the bug, is that when the court confirmed that was legal for the FBI to do, it could add precedent that it's legal for others to go to those greater lengths too. (legal or just general consensus, which effects judges decisions even when it's not a legal precedent)
- zackbrown 12y ago
  "...they were just interacting with a web service in the way it is supposed to be interacted with - by sending it http requests" That line of reasoning sounds a lot like the Andre 'Weev' Auernheimer case [0], where he gathered AT&T customers' email addresses by interacting with a server simply by 'sending it HTTP requests.' The FBI made its position clear on that case, prosecuting Weev for "conspiracy to gain unauthorized access to computers" and ultimately getting the guy sentenced to three years in prison. The overarching circumstances are clearly different but undeniably parallel. It seems curious to me that the FBI could use these some sort of apparently 'criminal' tactics (by their own precedent) as legal grounds in their case against DPR. [0] http://www.huffingtonpost.com/2013/03/18/andrew-weev-auernheimer_n_2900387.html http://www.huffingtonpost.com/2013/03/18/andrew-weev-auernhe...
  - thrownaway2424 12y ago
    Are you also surprised that the FBI can shoot people dead, while most people would get prosecuted for that?
godgod 12y ago
http://en.wikipedia.org/wiki/Parallel_construction http://en.wikipedia.org/wiki/Parallel_construction
godgod 12y ago
http://en.wikipedia.org/wiki/Parallel_construction http://en.wikipedia.org/wiki/Parallel_construction
BadDev 12y ago
Why is using var_dump debugging bad practice? Or is he insinuating that using var_dump while testing on a live site is bad practice?
- brianzelip 12y ago
  He is saying using `var_dump` on a live server is a bad practice.
- davis 12y ago
  Yeah, I'm pretty certain that he means using var_dump on a live server (especially of $_SERVER) is incredibly dumb.
- wcummings 12y ago
  It's definitely bad practice on a live server, and frankly its a pretty painful way to debug PHP, at least without xdebug.
- girvo 12y ago
  With XDebug and proper breakpoints (or HHVM's debugger) I don't see any need to do var_dump debugging these days. Far more powerful to have an XDebug client like Codebug drop you into a REPL at your breakpoint so you can explore what's gone wrong.
- cookiecaper 12y ago
  I think he infers that both are bad practice, but for the record I don't think there's anything wrong with var_dump debugging. Sure, a real interactive debugger is nicer, but it can be a pain to set up with PHP (I know, having set these up several times). Print-based debugging is fine as long as you don't expose sensitive information to end users.
  - girvo 12y ago
    Difficult to set up? I disagree, Xdebug with connectback is amazingly easy to set up in a Linux VM! On windows, yes, it's hard, but Linux and OS X its very simple nowadays. I should write blog post on how to do it (having done it dozens of times over the past 6 months)
kevinpet 12y ago
I'm suspicious of anyone who seems to have trouble understanding the FBI's claims. It's clear to me that what the FBI is claiming is that some part of the CAPTCHA was linked to the server directly, rather than its TOR address.
- istjohn 12y ago
  From the article: Even in the hypothetical case where – for some unrealistic reason – the Silk Road hidden site was including an image on an external server by referencing its IP address or hostname, the agents would still observe this traffic as having come from Tor. There is no magic way that the traffic from a real IP embedded within the HTML of a hidden service would find its way directly to a client without passing over the Tor network and through Tor nodes. Were this the case, it would be a huge vulnerability in Tor, as it would allow the administrator of a hidden site to uncover visitors by including an element that is served directly to the client over clearnet (thankfully it isn’t and this doesn’t work – try it).
  - thefreeman 12y ago
    I don't think they are presenting it accurately. I think what they are indicating is it was something like this: Login page ... $code_to_embed_captcha ... I believe $code_to_embed_captcha was returning something along the lines of http://REAL_SERVER_IP/captcha.jpg http://REAL_SERVER_IP/captcha.jpg instead of http://ONION_ADDRESS/captcha.jpg http://ONION_ADDRESS/captcha.jpg This wouldn't allow you to identify users, the request for captcha.jpg is still routed through TOR. However it does reveal the true IP of the server.
    - patio11 12y ago
      This is my guess as well. That or the captcha had a query parameter like ?nonce=1234&redirect_url=$HOSTNAME/login
    - makomk 12y ago
      Yeah, except that as he said the Silk Road site was pretty heavily scrutinized and if it was that simple other people would have spotted it.
gcb4 12y ago
1. get warranty to track someone id'ed as recipient of usps drug shipment. 2. exploit client tor missconfig 3. repeat 1-2 until you have a net so wide of tor users that only node out is the service
m3talridl3y 12y ago
Nothing will keep you from getting hacked by a determined government, but using Apache/PHP isn't exactly trying to make their life difficult. But at least they didn't use Ruby on Rails. :P
- jradd 12y ago
  So you are saying Rails might make intruders' lives less difficult? Of course, the _implementation_ is what would be more or less secure, but i'm a little curious what you meant.
meowface 12y ago
While I agree that the FBI's explanation as they gave it is impossible, I see 2 possible explanations for how their explanation would make sense. I don't know why they would glaze over these details, but: 1. They could have meant simply viewing all the HTTP response headers with something like Chrome's dev tools, Firebug, or possibly a proxy like Burp Suite. 2. They may have set up Tor to essentially use no hops and use only their own host as an exit node. That way they would be able to directly observe traffic from a clearweb IP address in a PCAP. Or they could set just a single hop and specified another host they control as an exit node. This seems way overboard just to find an exposure like this, but it's possible that the FBI or NSA have such test environments to probe for other Tor-related vulnerabilities and misconfigurations.
batbomb 12y ago
We just barely have some solid ground protecting phones you have on-person from unwarranted search, and, AFAIK, there's no real laws protecting your rights when you have data residing on third party servers in foreign countries. I'm definitely not a lawyer, but the server had an unknown provenance and it was physically located in Iceland, and that probably makes for some fuzzy ground at best as to whether or not what anything the FBI did to gain some sort of access to the physical server, even if it was made possible by parallel construction, would even be illegal.
dmix 12y ago
Came across a discussion on Twitter pointing out reddit users who apparently discovered IP addresses in error messages printing out in the HTTP response: 26 Mar 2013: http://www.reddit.com/r/SilkRoad/comments/1b1lvy/warning_the_silk_road_revealed_its_public_ip_last/ http://www.reddit.com/r/SilkRoad/comments/1b1lvy/warning_the... 26 Mar 2013: http://www.reddit.com/r/Bitcoin/comments/1b1n7y/warning_to_silk_road_users_sr_is_leaking_their/ http://www.reddit.com/r/Bitcoin/comments/1b1n7y/warning_to_s... 03 May 2013: http://www.reddit.com/r/SilkRoad/comments/1dmznd/should_we_be_worried_showing_on_login_page/ http://www.reddit.com/r/SilkRoad/comments/1dmznd/should_we_b...
- cookiecaper 12y ago
  Wow, I love how those threads are full of people who insist the OP is full of shit and that Silk Road doesn't even have a public IP address. Maybe we'd have fewer incidents like these if keyboard jockeys were a bit more teachable. Even super 1337 h4xx0r5 make mistakes, often obvious ones. Software is complicated. Let this be a lesson to any of us who may dismiss a correction too quickly.
  - pyre 12y ago
    Yes. They all tend to talk in certainties about something that they are only speculating about: > you would have to go out of your way to expose > your public IP via Tor. > However, the hidden service you connect to, > would be the load balancer. This service would > have zero knowledge of its "external" ip > address, as it would be running in a VM > pointing to a port on the loopback device. (because obviously nothing else is possible) > - Server is almost 100% certainly running in a > VM and doesn't have a public IP. > > - The entire harddrive for the server (which > includes all the the information such as > private messages and addresses) is almost 100% > encrypted. > > This is the motherfucking SilkRoad. They've > been operating so successfully for so long > because they're not amateurs. OP is a liar. ... etc ...
    - nrser 12y ago
      the funny part is: the people i know that used the service (who were far down towards 'non' on the technical spectrum) complained regularly about it's terrible design, stability and security, and were under the impression it was administrated by parties that had little idea what they were doing. they theorized it was a big scheme to go Gox* on them once enough coins were present in the system. it's interesting that these people, who have no idea what an ip or load balancer or VM is, were able to pick up on this. i would guess that the h4xx0r5 actually spent little time using the service or had the bit of their own necks being exposed taint their assessment. * yeah, i know no one really knows exactly what happened, but it's fun to poke fun.
s_q_b 12y ago
Look, this is a great chance to see if anything is amiss. We can simply directly test our hypotheses: Start up the Sill Road server with a tor circuit in front of it (isolated from the main network.) Then have the FBI demonstrate how to get the server to cough up it's IP. It's easy to prove who's telling the truth here.
- malka 12y ago
  not really, it could be parallel construction : they found the flaw after having seized the server.
hueving 12y ago
Whoever wrote this article either very poorly understands network traffic or is being intentionally obtuse to create doubt. He states that there is no way that a public IP would show up in a packet capture over a Tor network. This is blatantly false and idiotic to think. HTTP servers, caching proxies, TLS terminators and load balancers have a habit of putting extra HTTP headers in the HTTP response to make the path of a request more clear. It's very possible that under some error condition one of them included an X-FORWARDED-FOR header or something similar that included the IP. This is all application layer data and can contain IPs. Even ignoring headers, they could have triggered an error page that just dumps all PHP server variables, one of which includes local IP addresses. Edit: It's infuriating that this article author doesn't understand that a packet sniffer can see contents of the HTTP protocol.
- michael_storm 12y ago
  He states that there is no way that a public IP would show up in a packet capture over a Tor network. No, he does not. He says that there's no way a public IP would show up at layers 3 or 4 of the OSI model. The author takes the position that the IP leak must have come from layer 7 (quoting from the article): [...] and it is at the application layer that the FBI uncovered the IP address. Depending on how one interprets the FBI's wording, this impossibility contradicts their accounting of how they discovered the IP. Let's look at what the FBI said: Upon examining the individual packets of data being sent back from the website,3 [sic] we noticed that the headers of some of the packets reflected a certain IP address not associated with any known Tor node as the source of the packets. Note that they're talking about headers of the packets, not HTTP headers. This is the FBI, though, so it's possible they confused "packets" with "HTTP requests". Taking their words at face value, though, the FBI seem to say that they got the IP address from the IP headers on packets received from Silk Road, or at least traffic generated by their browser on behalf of Silk Road. That's what the author argues is impossible. Nobody, least of all the author, disputes that Silk Road could have leaked its IP in HTTP headers, or via some other misconfiguration. But if they did, why didn't the FBI just say that? Why claim that they leaked it in the IP layer? Two possibilities come to mind. The first is that the FBI got the IP from the application layer, as everyone believes is possible, and misattributed the leak to the IP layer due to terminology confusion. The second is that they got the IP from another source, attempted to deceptively misattribute the source, and accidentally picked a fake source the leak could not have come from. If the second option is the case, we're looking at an example of parallel construction. Edit: Grammar. Gotta keep up appearances.
godgod 12y ago
Two Words: Parallel Construction.
godgod 12y ago
Two Words: Parallel Construction.
ianstallings 12y ago
The guy was not a good developer and they caught him because of it. It really does appear to be that simple. No parallel reconstruction needed when you're dumping server vars to the public.
coriny 12y ago
Well, I can't say much about the reality of the FBI approach, y'all seems to have that better covered. Instead, I'm led to ask at what point will people stop using "myself" as a formal version of "me"? It isn't, it actually has a completely different usage. In this case: "the SR Server was located by myself and another member of the CY-2". Here 'by myself' means 'on my own', not 'by me', so it directly confuses the meaning of this sentence. To me the obvious meaning is that they both work independently and found the same thing. About the only other time you can use '-self' is in the reflexive, as in "I did this to myself", "you do this to yourself". I will never do something to yourself, and you will never do something to myself. So, I know this is completely out of place, but if I can get one person to not do this in the future, I will make my life a bit happier. Also if I can persuade that small group of nameless people (you know who you are) that "could care less" actually means the opposite of what they think it does; and that "alot" is not a word. I'm all for the English language evolving and expanding, but let's weed out the bad ideas which only increase confusion and bring nothing good. And seriously, "alot"? "Lot" is a three letter word, it's not hard to spell. As for "a" ...
- _archon_ 12y ago
  I understand your confusion, but this is a common usage case for the nominative first person of "me" in formal speech. This is one of a gajillion special case word choice situations. This particular sentence structure (Direct object, past perfect verb, subject) suggests "myself". However, the writer should have put in the other person's name before "myself", as is common practice, leading to confusion. Consider: "the SR Server was located by another member of the CY-2 and myself". I'm not claiming that "me" wouldn't parse, but that using the current rules properly would not cause confusion in the first place. Someone is trying to sound intelligent, while putting their name first so as to emphasize their contribution.
  - coriny 12y ago
    So, I'm definitely not confused :) I do appreciate it's very common in what I'd call pseudo-formal speech, mostly used when trying to impress someone one perceives to be of higher social station. Estate agents do it a lot when talking to clients. And so it does have the "well, lots of people do it" support. However, if you go back to the roots (Latin), "myself" is specifically the reflexive form of the first person. When the first person is the object or the dative, it is "me/to me". So to have correct agreement with the verb, your example should end in "and me". While your example is easy enough to parse, it doesn't actually follow the rules - though maybe the common practice. And if you rearrange the two object nouns (i.e. back to the original) the sentence loses clarity. Using "me" never loses clarity, and the ordering of object nouns shouldn't matter beyond providing emphasis. Basically, that example is ok in Perl, but not in Java ;) And ultimately, using the reflexive form incorrectly never makes a sentence clearer, and it's longer to write and say. Anyway, this is so far off the OP's topic, which is actually much more important & interesting, I do apologise to him/her.
ufmace 12y ago
The thing that struck me as the most odd about this article was the assertion that keeping a hidden service securely hidden must be easy, because the TOR website's Wiki article on it is so short. I think it's clear that there isn't that much proven experience out there in keeping a hidden service secure against a determined and well-funded adversary, and what experience there is probably hasn't made it to the TOR Wiki page for various reasons. Just the last HN article on this generated dozens of comments with various best-practices for securing such a service that clearly weren't followed by Silk Road, and we're mostly amateurs at this.

319 points by nikcub 12y ago

If I ever set up a hidden service I'll make sure it lives on a box that has no public IP whatsoever. Preferably a physical box.

throw3 12y ago
how?
- nknighthb 12y ago
  The simplest case is trivial. You place the hidden service box (HSB) and the Tor gateway (TG) on private IP space, and deny the HSB all connectivity to anything except the TG. Ideally you actually place the HSB and TG in different network segments so packets from the HSB must pass through a router to get to the TG and vice-versa, ensuring a compromised HSB has no possibility of sniffing any incriminating broadcast traffic to/from the TG.
  - dobbsbob 12y ago
    Then you shill your service with a pseudonym connected to your real name so they find you anyway
- praptak 12y ago
  All of the stuff that I expect to leak or get hacked lives on box A in a private network. The second host in this network would be the Tor gateway B, hardened, stripped down of any unnecessary stuff. Its only function would be to route traffic to/from A into Tor. Obviously it would have to be connected to the public internet but at least it would have much less software that can be hacked. As an additional security measure, B could live behind a firewall that only let connections to Tor nodes. (yup, it might be a PITA to set up.)
  - nknighthb 12y ago
    Not that big a PITA, I don't think. People already use Tor exit node lists to block activity from Tor exit nodes, you're kind of just looking for the inverse. Shouldn't be hard to create a script to generate e.g. iptables rules that permit connections to Tor public relays. The security benefit is a bit marginal, though. The public relay list is basically world-writable.
    - dublinben 12y ago
      Entire operating systems like Whonix already exist to deliver that kind of security.
      - nknighthb 12y ago
        We're really talking about two different kinds of "security" here. There's blocking access to anything but a Tor gateway (including in the case of a compromise at the kernel level), and blocking access by the Tor gateway to anything but Tor relays. But that raises another point I'd neglected. On the Tor gateway, you can block access to the public internet by anything other than the Tor process. The security benefit against maliciousness is iffy, since the most likely route of compromise for a Tor gateway that's been set up with even a little care is the Tor process, but it certainly reduces the chance of accidents.
        dublinben 12y ago
        How do you actually target the Tor gateway machine though? You can attack a Hidden Service, because there must be a webserver that responds to requests. The Tor gateway just passes traffic from the network to your client device.
        nitrogen 12y ago
        How do you actually target the Tor gateway machine though? Not the OP, and I don't know much about Tor's protocol, but at a guess one might start with deliberately malformed layers of the onion, so you route through a Tor entry node, the outer onion layer is removed, then the packet is forwarded to a Tor relay. The Tor relay then unwraps the next layer of the onion, but the data within that layer is deliberately malformed to trigger a bug in the Tor code running on that relay.
nwf 12y ago
Unfortunately, the technology you need to do that is not yet finished: https://trac.torproject.org/projects/tor/ticket/9498 https://trac.torproject.org/projects/tor/ticket/9498 ETA: I should have said "one possible technology"; there may be others, but I am pretty sure that an IP-less Tor node requires that you play the Tor Bridge game and stream the Tor protocol over a non-IP link. I have had a prototype of this design running, but have yet to have it to a point I consider robust.
- nickodell 12y ago
  This looks like it's unrelated.
- icebraining 12y ago
  praptak didn't say IP-less, but no public IP. If the server has an IP in the 192.168.1.X range, that doesn't tell an attacker much, supposedly.

siliconc0w 12y ago

I think it's more likely by header the FBI meant HTTP response header not IPV4 header. Legally I don't know if we should allow "fuzzing" without a warrant. They've put people in jail for doing less.

Igglyboo 12y ago
Weev was put in prison for wayyyy less than what the FBI did.
- eli 12y ago
  And the judge who released him from prison expressed skepticism about that. Regardless, two wrongs...
  - wcummings 12y ago
    Yes, but that's not why he was released. Those charges are still untested, he was let off the hook due to the venue.
- meowface 12y ago
  I don't think Weev should have been tried criminally at all, but I kind of disagree with this purely from a technical standpoint. The FBI discovered a flaw that let them see the IP address of a web server. Weev discovered a flaw that let him see private information of AT&T customers. Customer information is generally considered much more sensitive than an IP address.
  - totony 12y ago
    Privacy is relative. In Weev's case, AT&T considered the information of its users to be sensible, private information, while, in SR's care, the IP address is a very sensible and private information, considering that tor was used to hide it.

colinbartlett 12y ago

All of these analyses of how they might have located the server are great. I'm pleased to see folks trying to prove or disprove that the FBI is not trying some parallel construction here. But what worries me is that all of this will have to be eventually dumbed down to an explanation that a jury of average joes can understand. The increasingly technical complexity of evidence in some of these cases worries me that our system doesn't have an adequate way to deal with it.

dobbsbob 12y ago
They probably won't argue this in front of a jury since it's a pre-trial motion to challenge the evidence the FBI collected. No idea how a judge will understand this, I guess they bring in experts to testify if the FBI's claim is possible or not.
lifeisstillgood 12y ago
Honestly I have no worries about this or any other "technical" issue like financial fraud cases. If a cop can learn enough to catch someone then it is possible to explain that to a nurse or an electrician. Jurors don't switch off because they are learning how to move money between banks or break into servers - they switch off because it takes three months to present a case.
- EpicEng 12y ago
  > If a cop can learn enough to catch someone then it is possible to explain that to a nurse or an electrician. You think a cop did this? No... the FBI has technical personnel. Just like any corporation, they have different departments to fulfill different needs.
  - lifeisstillgood 12y ago
    Not quite what I meant - no I don't think cops were writing SQLi but they learnt enough to laugh at Little Bobby Tables, just as they can learn enough to follow the money through 17 bank accounts. This is the equivalent of is the FBI allowed to search my trash if the trash is in a trashcan on my property or on the sidewalk, being pulled by a street cleaner or knocked over by a fox. We can make it as complicated as we like but the law needs to draw some lines somewhere and stick to them.

zachrose 12y ago

Is the constitutional issue here that exploiting a security hole (SQL injection, remote execution to dump the $_SERVER variable, what have you) constitutes an illegal search? If so, is their any guidance from case law about the difference between some sort of legal poking and prodding vs illegal hacking?

eurleif 12y ago
I wonder if this could be considered a case of exigent circumstances? [0] The exigent circumstances exception can be invoked "to forestall the imminent escape of a suspect, or destruction of evidence". Arguably, that's what this was -- Silk Road had a security flaw that could be fixed at any moment, and the FBI may have felt they needed to act quickly to obtain evidence before the security flaw was fixed. [0] https://en.wikipedia.org/wiki/Exigent_circumstance_in_United_States_law https://en.wikipedia.org/wiki/Exigent_circumstance_in_United...
- deleted 12y ago
  [deleted]
- Donzo 12y ago
  Well, they were also committing criminal activity in "plain view" so to speak.
strathmeyer 12y ago
In the past simply typing in random URLs has been deemed illegally circumventing a site's security. Like most things it's fine when the government does it but you'll get jailtime if you do it to one of their friends.
gcb4 12y ago
it can probably set a precedent that will free all scapegoat script kiddies currently in outsourced prisons.
downandout 12y ago
In general in order to break the law you must have a warrant. For example, a cop entering private property without a warrant or probable cause is nothing more than a trespasser. These agents likely violated the CFAA by doing what they did. Because they didn't have a warrant, IMO it probably constitutes an illegal search, but the courts may well have a very different opinion. Regardless of which side loses this argument, this will likely find its way to the appellate courts and possibly the Supreme court, as it seems to be an unanswered question of great importance.

hadoukenio 12y ago

If they obtained a warrant, is fuzzing/hacking Silk Road the equivalent of kicking down a door during a raid?

ascorbic 12y ago
They didn't obtain a warrant.
inportb 12y ago
Probably... though I have an easier time imagining someone picking a lock or, in the case of fuzzing, using a bump key.

Eleopteryx 12y ago

Slightly off topic: could you please increase the font size on your site? It's actually below 12pt. The blockquotes are more readable than the body of the post.

DougBTX 12y ago
It is 17px for me, but most browsers support zooming into a page, that will make text easier to read on almost any site, but could make it more difficult if accidentally zoomed out.
- wcummings 12y ago
  Every browser has supported this since forever
nikcub 12y ago
I only noticed today after somebody sent me a screenshot that i'd been viewing my own website zoomed in at two levels. I updated the stylesheet and bumped the size up earlier but it might not come through without a hard refresh because of crazy caching™

asuffield 12y ago

So we have a pretty good guess of how they did it, and while their affidavit doesn't give a lot of important details, ultimately they were just interacting with a web service in the way it is supposed to be interacted with - by sending it http requests - and the buggy site revealed itself to them because the guy who wrote it wasn't a very good web developer. I can understand why they wouldn't want to give more details than they have to; it makes the prosecution's work harder. But I and many others have been arguing for years that there's nothing illegal about sending messages to a server and looking at the responses, if you aren't trying to damage anything. I can't see any reason why it would be less legal for the FBI to do it.

jrochkind1 12y ago
We agree, but one reason we'd want more details about the FBI going to greater lengths to trigger the bug, is that when the court confirmed that was legal for the FBI to do, it could add precedent that it's legal for others to go to those greater lengths too. (legal or just general consensus, which effects judges decisions even when it's not a legal precedent)
zackbrown 12y ago
"...they were just interacting with a web service in the way it is supposed to be interacted with - by sending it http requests" That line of reasoning sounds a lot like the Andre 'Weev' Auernheimer case [0], where he gathered AT&T customers' email addresses by interacting with a server simply by 'sending it HTTP requests.' The FBI made its position clear on that case, prosecuting Weev for "conspiracy to gain unauthorized access to computers" and ultimately getting the guy sentenced to three years in prison. The overarching circumstances are clearly different but undeniably parallel. It seems curious to me that the FBI could use these some sort of apparently 'criminal' tactics (by their own precedent) as legal grounds in their case against DPR. [0] http://www.huffingtonpost.com/2013/03/18/andrew-weev-auernheimer_n_2900387.html http://www.huffingtonpost.com/2013/03/18/andrew-weev-auernhe...
- thrownaway2424 12y ago
  Are you also surprised that the FBI can shoot people dead, while most people would get prosecuted for that?

godgod 12y ago

http://en.wikipedia.org/wiki/Parallel_construction http://en.wikipedia.org/wiki/Parallel_construction

godgod 12y ago

http://en.wikipedia.org/wiki/Parallel_construction http://en.wikipedia.org/wiki/Parallel_construction

BadDev 12y ago

Why is using var_dump debugging bad practice? Or is he insinuating that using var_dump while testing on a live site is bad practice?

brianzelip 12y ago
He is saying using `var_dump` on a live server is a bad practice.
davis 12y ago
Yeah, I'm pretty certain that he means using var_dump on a live server (especially of $_SERVER) is incredibly dumb.
wcummings 12y ago
It's definitely bad practice on a live server, and frankly its a pretty painful way to debug PHP, at least without xdebug.
girvo 12y ago
With XDebug and proper breakpoints (or HHVM's debugger) I don't see any need to do var_dump debugging these days. Far more powerful to have an XDebug client like Codebug drop you into a REPL at your breakpoint so you can explore what's gone wrong.
cookiecaper 12y ago
I think he infers that both are bad practice, but for the record I don't think there's anything wrong with var_dump debugging. Sure, a real interactive debugger is nicer, but it can be a pain to set up with PHP (I know, having set these up several times). Print-based debugging is fine as long as you don't expose sensitive information to end users.
- girvo 12y ago
  Difficult to set up? I disagree, Xdebug with connectback is amazingly easy to set up in a Linux VM! On windows, yes, it's hard, but Linux and OS X its very simple nowadays. I should write blog post on how to do it (having done it dozens of times over the past 6 months)

kevinpet 12y ago

I'm suspicious of anyone who seems to have trouble understanding the FBI's claims. It's clear to me that what the FBI is claiming is that some part of the CAPTCHA was linked to the server directly, rather than its TOR address.

istjohn 12y ago
From the article: Even in the hypothetical case where – for some unrealistic reason – the Silk Road hidden site was including an image on an external server by referencing its IP address or hostname, the agents would still observe this traffic as having come from Tor. There is no magic way that the traffic from a real IP embedded within the HTML of a hidden service would find its way directly to a client without passing over the Tor network and through Tor nodes. Were this the case, it would be a huge vulnerability in Tor, as it would allow the administrator of a hidden site to uncover visitors by including an element that is served directly to the client over clearnet (thankfully it isn’t and this doesn’t work – try it).
- thefreeman 12y ago
  I don't think they are presenting it accurately. I think what they are indicating is it was something like this: Login page ... $code_to_embed_captcha ... I believe $code_to_embed_captcha was returning something along the lines of http://REAL_SERVER_IP/captcha.jpg http://REAL_SERVER_IP/captcha.jpg instead of http://ONION_ADDRESS/captcha.jpg http://ONION_ADDRESS/captcha.jpg This wouldn't allow you to identify users, the request for captcha.jpg is still routed through TOR. However it does reveal the true IP of the server.
  - patio11 12y ago
    This is my guess as well. That or the captcha had a query parameter like ?nonce=1234&redirect_url=$HOSTNAME/login
  - makomk 12y ago
    Yeah, except that as he said the Silk Road site was pretty heavily scrutinized and if it was that simple other people would have spotted it.

gcb4 12y ago

1. get warranty to track someone id'ed as recipient of usps drug shipment. 2. exploit client tor missconfig 3. repeat 1-2 until you have a net so wide of tor users that only node out is the service

m3talridl3y 12y ago

Nothing will keep you from getting hacked by a determined government, but using Apache/PHP isn't exactly trying to make their life difficult. But at least they didn't use Ruby on Rails. :P

jradd 12y ago
So you are saying Rails might make intruders' lives less difficult? Of course, the _implementation_ is what would be more or less secure, but i'm a little curious what you meant.

meowface 12y ago

While I agree that the FBI's explanation as they gave it is impossible, I see 2 possible explanations for how their explanation would make sense. I don't know why they would glaze over these details, but: 1. They could have meant simply viewing all the HTTP response headers with something like Chrome's dev tools, Firebug, or possibly a proxy like Burp Suite. 2. They may have set up Tor to essentially use no hops and use only their own host as an exit node. That way they would be able to directly observe traffic from a clearweb IP address in a PCAP. Or they could set just a single hop and specified another host they control as an exit node. This seems way overboard just to find an exposure like this, but it's possible that the FBI or NSA have such test environments to probe for other Tor-related vulnerabilities and misconfigurations.

batbomb 12y ago

We just barely have some solid ground protecting phones you have on-person from unwarranted search, and, AFAIK, there's no real laws protecting your rights when you have data residing on third party servers in foreign countries. I'm definitely not a lawyer, but the server had an unknown provenance and it was physically located in Iceland, and that probably makes for some fuzzy ground at best as to whether or not what anything the FBI did to gain some sort of access to the physical server, even if it was made possible by parallel construction, would even be illegal.

dmix 12y ago

Came across a discussion on Twitter pointing out reddit users who apparently discovered IP addresses in error messages printing out in the HTTP response: 26 Mar 2013: http://www.reddit.com/r/SilkRoad/comments/1b1lvy/warning_the_silk_road_revealed_its_public_ip_last/ http://www.reddit.com/r/SilkRoad/comments/1b1lvy/warning_the... 26 Mar 2013: http://www.reddit.com/r/Bitcoin/comments/1b1n7y/warning_to_silk_road_users_sr_is_leaking_their/ http://www.reddit.com/r/Bitcoin/comments/1b1n7y/warning_to_s... 03 May 2013: http://www.reddit.com/r/SilkRoad/comments/1dmznd/should_we_be_worried_showing_on_login_page/ http://www.reddit.com/r/SilkRoad/comments/1dmznd/should_we_b...

cookiecaper 12y ago
Wow, I love how those threads are full of people who insist the OP is full of shit and that Silk Road doesn't even have a public IP address. Maybe we'd have fewer incidents like these if keyboard jockeys were a bit more teachable. Even super 1337 h4xx0r5 make mistakes, often obvious ones. Software is complicated. Let this be a lesson to any of us who may dismiss a correction too quickly.
- pyre 12y ago
  Yes. They all tend to talk in certainties about something that they are only speculating about: > you would have to go out of your way to expose > your public IP via Tor. > However, the hidden service you connect to, > would be the load balancer. This service would > have zero knowledge of its "external" ip > address, as it would be running in a VM > pointing to a port on the loopback device. (because obviously nothing else is possible) > - Server is almost 100% certainly running in a > VM and doesn't have a public IP. > > - The entire harddrive for the server (which > includes all the the information such as > private messages and addresses) is almost 100% > encrypted. > > This is the motherfucking SilkRoad. They've > been operating so successfully for so long > because they're not amateurs. OP is a liar. ... etc ...
  - nrser 12y ago
    the funny part is: the people i know that used the service (who were far down towards 'non' on the technical spectrum) complained regularly about it's terrible design, stability and security, and were under the impression it was administrated by parties that had little idea what they were doing. they theorized it was a big scheme to go Gox* on them once enough coins were present in the system. it's interesting that these people, who have no idea what an ip or load balancer or VM is, were able to pick up on this. i would guess that the h4xx0r5 actually spent little time using the service or had the bit of their own necks being exposed taint their assessment. * yeah, i know no one really knows exactly what happened, but it's fun to poke fun.

s_q_b 12y ago

Look, this is a great chance to see if anything is amiss. We can simply directly test our hypotheses: Start up the Sill Road server with a tor circuit in front of it (isolated from the main network.) Then have the FBI demonstrate how to get the server to cough up it's IP. It's easy to prove who's telling the truth here.

malka 12y ago
not really, it could be parallel construction : they found the flaw after having seized the server.

hueving 12y ago

Whoever wrote this article either very poorly understands network traffic or is being intentionally obtuse to create doubt. He states that there is no way that a public IP would show up in a packet capture over a Tor network. This is blatantly false and idiotic to think. HTTP servers, caching proxies, TLS terminators and load balancers have a habit of putting extra HTTP headers in the HTTP response to make the path of a request more clear. It's very possible that under some error condition one of them included an X-FORWARDED-FOR header or something similar that included the IP. This is all application layer data and can contain IPs. Even ignoring headers, they could have triggered an error page that just dumps all PHP server variables, one of which includes local IP addresses. Edit: It's infuriating that this article author doesn't understand that a packet sniffer can see contents of the HTTP protocol.

michael_storm 12y ago
He states that there is no way that a public IP would show up in a packet capture over a Tor network. No, he does not. He says that there's no way a public IP would show up at layers 3 or 4 of the OSI model. The author takes the position that the IP leak must have come from layer 7 (quoting from the article): [...] and it is at the application layer that the FBI uncovered the IP address. Depending on how one interprets the FBI's wording, this impossibility contradicts their accounting of how they discovered the IP. Let's look at what the FBI said: Upon examining the individual packets of data being sent back from the website,3 [sic] we noticed that the headers of some of the packets reflected a certain IP address not associated with any known Tor node as the source of the packets. Note that they're talking about headers of the packets, not HTTP headers. This is the FBI, though, so it's possible they confused "packets" with "HTTP requests". Taking their words at face value, though, the FBI seem to say that they got the IP address from the IP headers on packets received from Silk Road, or at least traffic generated by their browser on behalf of Silk Road. That's what the author argues is impossible. Nobody, least of all the author, disputes that Silk Road could have leaked its IP in HTTP headers, or via some other misconfiguration. But if they did, why didn't the FBI just say that? Why claim that they leaked it in the IP layer? Two possibilities come to mind. The first is that the FBI got the IP from the application layer, as everyone believes is possible, and misattributed the leak to the IP layer due to terminology confusion. The second is that they got the IP from another source, attempted to deceptively misattribute the source, and accidentally picked a fake source the leak could not have come from. If the second option is the case, we're looking at an example of parallel construction. Edit: Grammar. Gotta keep up appearances.

godgod 12y ago

Two Words: Parallel Construction.

godgod 12y ago

Two Words: Parallel Construction.

ianstallings 12y ago

The guy was not a good developer and they caught him because of it. It really does appear to be that simple. No parallel reconstruction needed when you're dumping server vars to the public.

coriny 12y ago

Well, I can't say much about the reality of the FBI approach, y'all seems to have that better covered. Instead, I'm led to ask at what point will people stop using "myself" as a formal version of "me"? It isn't, it actually has a completely different usage. In this case: "the SR Server was located by myself and another member of the CY-2". Here 'by myself' means 'on my own', not 'by me', so it directly confuses the meaning of this sentence. To me the obvious meaning is that they both work independently and found the same thing. About the only other time you can use '-self' is in the reflexive, as in "I did this to myself", "you do this to yourself". I will never do something to yourself, and you will never do something to myself. So, I know this is completely out of place, but if I can get one person to not do this in the future, I will make my life a bit happier. Also if I can persuade that small group of nameless people (you know who you are) that "could care less" actually means the opposite of what they think it does; and that "alot" is not a word. I'm all for the English language evolving and expanding, but let's weed out the bad ideas which only increase confusion and bring nothing good. And seriously, "alot"? "Lot" is a three letter word, it's not hard to spell. As for "a" ...

_archon_ 12y ago
I understand your confusion, but this is a common usage case for the nominative first person of "me" in formal speech. This is one of a gajillion special case word choice situations. This particular sentence structure (Direct object, past perfect verb, subject) suggests "myself". However, the writer should have put in the other person's name before "myself", as is common practice, leading to confusion. Consider: "the SR Server was located by another member of the CY-2 and myself". I'm not claiming that "me" wouldn't parse, but that using the current rules properly would not cause confusion in the first place. Someone is trying to sound intelligent, while putting their name first so as to emphasize their contribution.
- coriny 12y ago
  So, I'm definitely not confused :) I do appreciate it's very common in what I'd call pseudo-formal speech, mostly used when trying to impress someone one perceives to be of higher social station. Estate agents do it a lot when talking to clients. And so it does have the "well, lots of people do it" support. However, if you go back to the roots (Latin), "myself" is specifically the reflexive form of the first person. When the first person is the object or the dative, it is "me/to me". So to have correct agreement with the verb, your example should end in "and me". While your example is easy enough to parse, it doesn't actually follow the rules - though maybe the common practice. And if you rearrange the two object nouns (i.e. back to the original) the sentence loses clarity. Using "me" never loses clarity, and the ordering of object nouns shouldn't matter beyond providing emphasis. Basically, that example is ok in Perl, but not in Java ;) And ultimately, using the reflexive form incorrectly never makes a sentence clearer, and it's longer to write and say. Anyway, this is so far off the OP's topic, which is actually much more important & interesting, I do apologise to him/her.

ufmace 12y ago

The thing that struck me as the most odd about this article was the assertion that keeping a hidden service securely hidden must be easy, because the TOR website's Wiki article on it is so short. I think it's clear that there isn't that much proven experience out there in keeping a hidden service secure against a determined and well-funded adversary, and what experience there is probably hasn't made it to the TOR Wiki page for various reasons. Just the last HN article on this generated dozens of comments with various best-practices for securing such a service that clearly weren't followed by Silk Road, and we're mostly amateurs at this.