5 ms·

Arizona Maricopa county for whatever reason, refused to produce the network routers, when Subpoenaed for the 3rd party Audit by the Arizona senate. They respon

Arizona Maricopa county for whatever reason, refused to produce the network routers, when Subpoenaed for the 3rd party Audit by the Arizona senate. They responded with this letter. https://www.maricopa.gov/DocumentCenter/View/70435/Final-Signed-Letter-to-Senators https://www.maricopa.gov/DocumentCenter/View/70435/Final-Sig...

dillondoyle 5y ago
Disregarding your viewpoints/our bias, as this is HN, here is a technical question: what, if anything, would a router show at this point in time. Do these routers use store logs that would still be accessible? Are they trying to find some rhetorical bamboo in the switch (trying to be funny though those beliefs definitely are not in my book...)?
- dukeofdoom 5y ago
  I know my cheap home router stores logs. I'm pretty sure a commercial grade router would store logs for sure. Though this is not my background. So maybe someone that's a network admin knows more on what kind of routers they might use and how long the logs are typically stored.
- jcrawfordor 5y ago
  My assumption is that they want to have an outside expert examine the configuration of the routers, which could provide some indication whether or not there was access to the internet, for example. Of course it would be difficult to establish whether or not that configuration had been changed in the meantime, and in my experience a lot of election administrators set up their EMS only on a temporary basis, so I would be unsurprised if supporting equipment like network appliances had been repurposed or returned to a lease provider. That said, the EMS is usually a small half-rack setup and I'd be surprised for there to be a router at all. Often it's a single machine. The vendors usually tell you to either never connect it to outside networks, or to do so only for setup and disconnect it before you start election preparation. Most, usually all, in-election connectivity to the machines is via sneakernet. Any kind of detailed network logging becomes a big undertaking at organizational scale, so it's very unlikely that there are any kind of useful persistent traffic logs on the routers. Any such logs would have been kept on a dedicated system like a netflow collector or SIEM, the routers don't have the storage (or storage write endurance). The fact the senate didn't subpoena for such a system suggests it doesn't exist, which is unsurprising. Mass-retention of network traffic logs is a sufficiently expensive and complex thing that not a lot of organizations do it.
  - dillondoyle 5y ago
    Thank you!! That was my guess on logs. Really cool to learn more! You sound like you work in the industry. I'm sorry for that ;( Recent WaPo article on election workers was scary.
  - missblit 5y ago
    > That said, the EMS is usually a small half-rack setup and I'd be surprised for there to be a router at all There isn't, you can actually see their EMS diagram here: https://recorder.maricopa.gov/justthefacts/pdf/Maricopa%20EMS%20Diagram%20V2.pdf https://recorder.maricopa.gov/justthefacts/pdf/Maricopa%20EM... (Switches yes, but no routers)
  - dukeofdoom 5y ago
    I just read the counties response on their website. https://recorder.maricopa.gov/justthefacts/pdf/Maricopa%20EMS%20Diagram%20V2.pdf https://recorder.maricopa.gov/justthefacts/pdf/Maricopa%20EM... It shows the EMS rack and the ballot scanners in a network configuration. Since this seems to be windows based from a video I've seen. At least the software they were using to adjudicate votes. Source: https://rumble.com/vbqj0b-dominon-voting-machines-walkthrough-on-how-they-scan-and-adjudicate.html https://rumble.com/vbqj0b-dominon-voting-machines-walkthroug... I expect the EMS rack have a network port and not a wifi port. Back when I was using windows, connecting to the internet through the network port required a setup step, for things like DHCP server for the dynamic ip. I'm curious, does windows remember the configurations it has used in the past? And does windows keep a log file of the network connections it makes, and how often are those logs rotated typically?
    - jcrawfordor 5y ago
      I see, they do some scanning and adjudication on the EMS which means it's a bigger setup - not necessarily the EMS itself, but it needs workstations hooked up to it for the tabulators and adjudicators to use. Based on some research, the way Maricopa County runs things ballots rejected by the precinct tabulators (due to ambiguous markings usually) are sent in to the central office for adjudication, which is what the scanners and workstations connected to the EMS are for. Probably this is done using the sorter bin feature of the ImageCast tabulator but I'm not sure. I have never used the Dominion adjudication system, it depends on the election administrator and state law how they handle rejected ballots and the two states I have familiarity with have made a decision to return them to the voter instead of sending them in (usually asking the voter to complete a new ballot). There are pros and cons either way. But you can see they use a Netgear switch, so we don't even know if it's managed but I'd guess it's not because Netgear's managed line just isn't that popular. There's probably nothing of interest that could be extracted from that switch. Yes, as far as I know all of Dominion's management software is Windows-based. There's a potentially huge amount of forensic data available from the Windows machines but whether or not you could demonstrate internet access just depends. It's the kind of thing you could probably only prove in the positive. Windows does not keep network logs per se, just there are lots of other logs that can be used to infer certain network activity. The reality is that network traffic is so high volume and low value that it's unusual for anything to record it in much detail, it'd just eat a lot of storage and I/O to never get looked at.
missblit 5y ago
Maricopa has already confirmed that their EMS didn't contain any routers whatsoever (https://recorder.maricopa.gov/justthefacts/#tabs=1 https://recorder.maricopa.gov/justthefacts/#tabs=1). The subpoena requested "all routers used in connection with the November 3, 2020 general election"; so I guess it could arguably cover routers outside of the EMS. That said: #1 It's unlikely that the AZ senate has the votes to hold the county in contempt at this point (the audit has only gotten less popular over time). #2 They want to avoid handing them over for security / privacy reasons (they've mentioned a few times the sheriff's department has concerns -- nothing concrete but c'mon who among us would be happy to hand over a ton of info about an intranet we're in charge of). To put this all in context: the county took a much more aggressive position against the audit after the audit twitter account accused them of deleting a DB file to hide evidence and the AZ senate was about to drag them into a hearing they planned to broadcast on OAN. In particular the county has done an incredible job of countering misinformation with facts on their twitter account.
- dukeofdoom 5y ago
  Since the key fact in dispute is whether the machines were connected to the internet, wouldn't it follow for an auditor to check all routers through which they could have connected to the internet. Seems like the exact thing a forensic audit should check. Since this is going through the courts, and in front of the judge. The judge will ultimately decide if the county is ignoring orders and impose sanctions. Ultimately, the judge could order a bailiff to seize the routers if they find the county non complaint. Also since the states run their own elections and the Senate had broad oversight rights, the judge so far has ruled against the county at every step. The county is just delaying things really.
  - missblit 5y ago
    The subpoena is not a judicial subpoena at this point but a senate subpoena, that's why the senate would need enough votes to enforce it. This is a bit confusing because there was a judge who decided the senate's first subpoena was valid [1], but that too was still a senate subpoena. They certainly could try to go to court to get an enforceable subpoena, if they do I'll make some popcorn. As for the routers my current position towards the AZ auditors is "put up or shut up". If the EMS really was connected to an external network then they must have some event logs somewhere showing an unexplainable MAC address or something; if they don't then asking for unrelated routers just feels like a fishing expedition. [1] https://www.azcentral.com/story/news/politics/elections/2021/02/26/judge-says-maricopa-county-must-provide-2020-ballots-arizona-senate/6825892002/ https://www.azcentral.com/story/news/politics/elections/2021...
kthejoker2 5y ago
The "for whatever reason" is those routers have never been part of any electoral certification process, aren't part of Trump's own election cybersecurity guidance, aren't considered attack vectors, and of course are completely irrelevant since you can just hand count the paper ballots which is what the county already did. But you know .. "whatever" reason.
jcranmer 5y ago
That's answered in the 5th item in the list, nearly a full page long in response. Basically amounts to: a) the election-tabulating machines were never connected to the internet b) we've already given you (well, the cybersecurity experts you hired) enough information to prove it c) we have had two other audits that agree with us that they were never connected d) actually complying with your order presents severe privacy and security risks e) it also is a very severe operational router, since the senate was essentially asking for physical custody of all county routers, which would presumably include things like the routing to the county website to cross-reference real estate deeds or the like But sure, let's ignore their actual claims as to why they didn't do it and imply that they didn't do it because they're afraid of producing culpability of criminal actions.

dillondoyle 5y ago

Disregarding your viewpoints/our bias, as this is HN, here is a technical question: what, if anything, would a router show at this point in time. Do these routers use store logs that would still be accessible? Are they trying to find some rhetorical bamboo in the switch (trying to be funny though those beliefs definitely are not in my book...)?

dukeofdoom 5y ago
I know my cheap home router stores logs. I'm pretty sure a commercial grade router would store logs for sure. Though this is not my background. So maybe someone that's a network admin knows more on what kind of routers they might use and how long the logs are typically stored.
jcrawfordor 5y ago
My assumption is that they want to have an outside expert examine the configuration of the routers, which could provide some indication whether or not there was access to the internet, for example. Of course it would be difficult to establish whether or not that configuration had been changed in the meantime, and in my experience a lot of election administrators set up their EMS only on a temporary basis, so I would be unsurprised if supporting equipment like network appliances had been repurposed or returned to a lease provider. That said, the EMS is usually a small half-rack setup and I'd be surprised for there to be a router at all. Often it's a single machine. The vendors usually tell you to either never connect it to outside networks, or to do so only for setup and disconnect it before you start election preparation. Most, usually all, in-election connectivity to the machines is via sneakernet. Any kind of detailed network logging becomes a big undertaking at organizational scale, so it's very unlikely that there are any kind of useful persistent traffic logs on the routers. Any such logs would have been kept on a dedicated system like a netflow collector or SIEM, the routers don't have the storage (or storage write endurance). The fact the senate didn't subpoena for such a system suggests it doesn't exist, which is unsurprising. Mass-retention of network traffic logs is a sufficiently expensive and complex thing that not a lot of organizations do it.
- dillondoyle 5y ago
  Thank you!! That was my guess on logs. Really cool to learn more! You sound like you work in the industry. I'm sorry for that ;( Recent WaPo article on election workers was scary.
- missblit 5y ago
  > That said, the EMS is usually a small half-rack setup and I'd be surprised for there to be a router at all There isn't, you can actually see their EMS diagram here: https://recorder.maricopa.gov/justthefacts/pdf/Maricopa%20EMS%20Diagram%20V2.pdf https://recorder.maricopa.gov/justthefacts/pdf/Maricopa%20EM... (Switches yes, but no routers)
- dukeofdoom 5y ago
  I just read the counties response on their website. https://recorder.maricopa.gov/justthefacts/pdf/Maricopa%20EMS%20Diagram%20V2.pdf https://recorder.maricopa.gov/justthefacts/pdf/Maricopa%20EM... It shows the EMS rack and the ballot scanners in a network configuration. Since this seems to be windows based from a video I've seen. At least the software they were using to adjudicate votes. Source: https://rumble.com/vbqj0b-dominon-voting-machines-walkthrough-on-how-they-scan-and-adjudicate.html https://rumble.com/vbqj0b-dominon-voting-machines-walkthroug... I expect the EMS rack have a network port and not a wifi port. Back when I was using windows, connecting to the internet through the network port required a setup step, for things like DHCP server for the dynamic ip. I'm curious, does windows remember the configurations it has used in the past? And does windows keep a log file of the network connections it makes, and how often are those logs rotated typically?
  - jcrawfordor 5y ago
    I see, they do some scanning and adjudication on the EMS which means it's a bigger setup - not necessarily the EMS itself, but it needs workstations hooked up to it for the tabulators and adjudicators to use. Based on some research, the way Maricopa County runs things ballots rejected by the precinct tabulators (due to ambiguous markings usually) are sent in to the central office for adjudication, which is what the scanners and workstations connected to the EMS are for. Probably this is done using the sorter bin feature of the ImageCast tabulator but I'm not sure. I have never used the Dominion adjudication system, it depends on the election administrator and state law how they handle rejected ballots and the two states I have familiarity with have made a decision to return them to the voter instead of sending them in (usually asking the voter to complete a new ballot). There are pros and cons either way. But you can see they use a Netgear switch, so we don't even know if it's managed but I'd guess it's not because Netgear's managed line just isn't that popular. There's probably nothing of interest that could be extracted from that switch. Yes, as far as I know all of Dominion's management software is Windows-based. There's a potentially huge amount of forensic data available from the Windows machines but whether or not you could demonstrate internet access just depends. It's the kind of thing you could probably only prove in the positive. Windows does not keep network logs per se, just there are lots of other logs that can be used to infer certain network activity. The reality is that network traffic is so high volume and low value that it's unusual for anything to record it in much detail, it'd just eat a lot of storage and I/O to never get looked at.

missblit 5y ago

Maricopa has already confirmed that their EMS didn't contain any routers whatsoever (https://recorder.maricopa.gov/justthefacts/#tabs=1 https://recorder.maricopa.gov/justthefacts/#tabs=1). The subpoena requested "all routers used in connection with the November 3, 2020 general election"; so I guess it could arguably cover routers outside of the EMS. That said: #1 It's unlikely that the AZ senate has the votes to hold the county in contempt at this point (the audit has only gotten less popular over time). #2 They want to avoid handing them over for security / privacy reasons (they've mentioned a few times the sheriff's department has concerns -- nothing concrete but c'mon who among us would be happy to hand over a ton of info about an intranet we're in charge of). To put this all in context: the county took a much more aggressive position against the audit after the audit twitter account accused them of deleting a DB file to hide evidence and the AZ senate was about to drag them into a hearing they planned to broadcast on OAN. In particular the county has done an incredible job of countering misinformation with facts on their twitter account.

dukeofdoom 5y ago
Since the key fact in dispute is whether the machines were connected to the internet, wouldn't it follow for an auditor to check all routers through which they could have connected to the internet. Seems like the exact thing a forensic audit should check. Since this is going through the courts, and in front of the judge. The judge will ultimately decide if the county is ignoring orders and impose sanctions. Ultimately, the judge could order a bailiff to seize the routers if they find the county non complaint. Also since the states run their own elections and the Senate had broad oversight rights, the judge so far has ruled against the county at every step. The county is just delaying things really.
- missblit 5y ago
  The subpoena is not a judicial subpoena at this point but a senate subpoena, that's why the senate would need enough votes to enforce it. This is a bit confusing because there was a judge who decided the senate's first subpoena was valid [1], but that too was still a senate subpoena. They certainly could try to go to court to get an enforceable subpoena, if they do I'll make some popcorn. As for the routers my current position towards the AZ auditors is "put up or shut up". If the EMS really was connected to an external network then they must have some event logs somewhere showing an unexplainable MAC address or something; if they don't then asking for unrelated routers just feels like a fishing expedition. [1] https://www.azcentral.com/story/news/politics/elections/2021/02/26/judge-says-maricopa-county-must-provide-2020-ballots-arizona-senate/6825892002/ https://www.azcentral.com/story/news/politics/elections/2021...

kthejoker2 5y ago

The "for whatever reason" is those routers have never been part of any electoral certification process, aren't part of Trump's own election cybersecurity guidance, aren't considered attack vectors, and of course are completely irrelevant since you can just hand count the paper ballots which is what the county already did. But you know .. "whatever" reason.

jcranmer 5y ago

That's answered in the 5th item in the list, nearly a full page long in response. Basically amounts to: a) the election-tabulating machines were never connected to the internet b) we've already given you (well, the cybersecurity experts you hired) enough information to prove it c) we have had two other audits that agree with us that they were never connected d) actually complying with your order presents severe privacy and security risks e) it also is a very severe operational router, since the senate was essentially asking for physical custody of all county routers, which would presumably include things like the routing to the county website to cross-reference real estate deeds or the like But sure, let's ignore their actual claims as to why they didn't do it and imply that they didn't do it because they're afraid of producing culpability of criminal actions.