5 ms·

> Second, the FCC objects to producing the relevant materials from the API proxy server log because to do so requires creating a script, which demands "research

> Second, the FCC objects to producing the relevant materials from the API proxy server log because to do so requires creating a script, which demands "research" rather than simply a "search." Funny how a government agency considers it overly burdensome to write what is likely a 4 line script, so goes through a lawsuit that costs taxpayers likely millions of dollars to avoid it. Unless their infrastructure is well and truly F*ed, this is a 30-60 minute task for a junior server admin. Most likely they already ran it and didn't like what it revealed so it's even less effort.

ikeboy 7y ago
This is a technical legal point. You can be forced to produce information you have, but generally not to create new information.
- vageli 7y ago
  They have the information (note that they did not contest that point), the script would presumably allow them to surface that information more quickly.
- ogre_codes 7y ago
  They are arguing that finding data that's in log files is creating new data (because they have to write a script to parse it) which is specious. Logs are designed to be parsed and searched. You might as well claim that running an SQL script to query data is creating new information.
  - colejohnson66 7y ago
    Well, if we’re following that logic, then anytime data is moved from a disk to ram, we’re “creating new data.” Well, yes, it wasn’t there before, so it’s “new data,” but it’s also a copy of old data.
    - Erwin 7y ago
      There have been some weird legal discussions about whether turning on a computer breaks the copyright depending on who does it: https://en.wikipedia.org/wiki/MAI_Systems_Corp._v._Peak_Computer,_Inc https://en.wikipedia.org/wiki/MAI_Systems_Corp._v._Peak_Comp....
      - jacoblambda 7y ago
        That's actually a fairly interesting case. I haven't read into it too much but it seems that they agree that it isn't necessarily a copyright violation based on who does it. It looks like what they got Peak on was instead the fact that they had unlicensed copies of MAI software at their HQ and were loaning MAI equipment in such a way that violated the license agreements. P.S. Please correct me if I got any of this wrong.
        zerocrates 7y ago
        There are some details in the case that are as you say, about Peak having "unclean hands" in effect, but they're fairly beside the point for the case's broader significance. What it really stands/stood for is the notion that a "copy" sufficient for the operation of copyright law occurs when a program is loaded from disk to RAM, and therefore it's possible to infringe copyright merely by using software that you aren't personally licensed for. MAI basically used this theory to make third-party maintenance of their systems illegal, so Congress responded by amending the Copyright Act, but in an extremely narrow way: there's a carve out specifically for maintenance and repair.
  - kortilla 7y ago
    > Logs are designed to be parsed and searched You’ve never dealt with a bad log setup. Things I’ve seen: - inconsistent format depending on which part of the software was logging (think of some modules using logging lib and others using print()) - multiline logs without any indication they are multiline - logs in some dumb binary format that requires the original software to reverse into legible text There are more, but the point is that logs are not often designed to be parsed and searched. They are normally added by developers to be human readable means of reversing internal state to debug issues. Anything above that is (shockingly) bonus material that doesn’t make it into home-grown enterprise/govt software.
    - ogre_codes 7y ago
      > You’ve never dealt with a bad log setup. Things I’ve seen: I'm not sure how this is relevant. I've seen miles of incompetence in my career, that someone can fudge something up royally doesn't change the fundamental purpose of the thing. The point of log files is to store and make retrievable data about server state and actions over time. That someone somewhere can screw that up doesn't change that. In this particular case, I find it unlikely they rolled some custom bizarro log file format regardless because of the nature of the subject matter: Proxy server log files. It's unlikely they rolled their own proxy server (and they certainly don't imply any particular difficulties like that in court statements) so we're most likely talking about reading standard Apache logs or something similar.
      - kortilla 7y ago
        > The point of log files That’s a point of log files. However, it’s not the point tons of developers have in mind when they add log statements. Developers are primarily concerned with state required to reproduce bugs, not long term standardized storage containing enough information for useful attribution or even all important state transitions. You would be surprised how many developers don’t even bother logging item deletions because it’s such a trivial code path. You’re idea of “the fundamental purpose” is from an operators view. It rarely lines up with the developers’ views and you’re in for a bad time if you think people who don’t see your perspective are “fudging something up royally” with “miles of incompetence”.
  - sl1ck731 7y ago
    That is true but more likely they try to push back against every request that isn't simply zip up files or documents and deposit them. They are using the "defense" that it may be difficult to do, but their aim (avoiding the more sinister possibilities) is to stop precedent for other unwieldy requests.
    - ogre_codes 7y ago
      In my experience, government agencies fulfill requests if it's easy and won't put the agency or administration in a bad light. I suspect this falls more into the latter case than the former, they don't want people to see the logs because they know the bulk of the comments are bogus.
- zelon88 7y ago
  Technically you can open that logfile in Notepad and Ctrl+F the thing. If I was the judge and you pulled that argument someone at your agency would now be using that method instead of "researching" a grep query.
pc86 7y ago
Don't underestimate the absolutely jaw-dropping incompetence from even senior "tech" folks employed by governments. Within the last couple years, I've overheard senior, so-called technical government employees 1) complain about Git and wonder aloud why we weren't using Visual Source Safe; 2) insist that rotating through a list of 12 hard-coded passwords, in code, checked into Git, was totally fine; 3) refuse to believe that automated deployments were possible (not hard, or against norms, but physically impossible); 4) try to explain to another so-called technical gov't employee the difference between CSS and JavaScript, and get it wrong; 5) stand up in the middle of a conversation and walk out the door because it's 2:30 PM and their day is over; 6) even more nonsense you wouldn't believe if I showed you a video of it. I would hope Federal is a little better than State, but I'm not convinced.
- JaimeThompson 7y ago
  To be fair I know of people in private industry who have said some of the exact same things. It isn't limited to just government employees.
  - pc86 7y ago
    If you work for a modern software company (not a service company that sells software), and you honestly suggest Visual Source Safe, you'll get fired. There's just no reasonably intelligent reason to suggest it. You practically have to assault someone in the break room to get fired from a government job.
    - barbecue_sauce 7y ago
      Would you get fired, or would people just assume you were being sarcastic?
      - pc86 7y ago
        > > honestly
rasengan 7y ago
How do we know if the IP and headers supplied will be legitimate? There have been other acts in the past [1] that have been covered in government. [1] https://www.vice.com/en_us/article/4xaqz9/hillary-clinton-tech-guy-asked-reddit-for-email-advice https://www.vice.com/en_us/article/4xaqz9/hillary-clinton-te...
erichurkman 7y ago
And when the Government is on the other side with requests, warrants, and subpoenas, they aggressively push with little regard to cost, complexity, or even feasibility. If we replied to a lawful order with "This demands 'research' instead of simply a 'search'," they would tell us tough-shit.
- SamPatt 7y ago
  And that's when they're being "nice." Otherwise they would send armed men to steal your stuff.
btrettel 7y ago
The FOIA request was probably not denied because the FCC lacks the technical abilities to fulfill the request. In my experience it's likely that no technical people were consulted on it. A FOIA office simply looks for any excuse they can use to deny a request if they want to deny it. The reason doesn't need to make sense or be their true reason. Here's an example from my own experience: I can recall a FOIA request I made for the results of searches in an internal government database in spreadsheet format. To my knowledge this can be exported using existing functionality in the web interface for the internal database. The request was denied, and I appealed. They claimed that the information was publicly available, which is absurd. In the appeal I showed that actually almost none of what I wanted was publicly available and the independent appeals office agreed (after a long delay). Then it became clear that the FOIA office thought the request would be burdensome and they tried to deny it on those grounds. I insisted that no, it wouldn't be burdensome. When it finally went to the person who would actually do the work, apparently it took only a few hours to do, and they declined to charge me anything for it. I assume the majority of that time was writing some useful comments in the headers of the spreadsheets. They didn't need to write the comments, so clearly they weren't pressed for time. In the end, the FOIA office spent far more time trying to deny this request than it would have taken to fulfill it.
- ogre_codes 7y ago
  > A FOIA office simply looks for any excuse they can use to deny a request if they want to deny it. The reason doesn't need to make sense or be their true reason. That's exactly what I was hinting at here. The agency has no interest in the public finding out how much of that feedback was bullshit. > In my experience it's likely that no technical people were consulted on it. From the sounds of it, they did at some point consult someone technical and tried to parse their answer as part of their justification for not executing the FOIA request. The fact that they knew it would take a script to extract the data as opposed to just dropping it into a search tool suggests as much anyhow. Though more likely it would just be a couple `sed` commands, a perl/ Python script, or a few command line tools.
irrational 7y ago
Once upon a time, my company lawyers came to me and asked me if we had certain historical data that I could pull from a database and how long it would take to pull it. It only took me an hour or so, so I just pulled the data, stuck it in an excel file and sent it to them. They were not happy. Apparently they had already told the judge that the data was unduly burdensome to pull and would take too long.
zamalek 7y ago
Realistically fabricating comments/evidence is a much larger script. Once you start lying, you have to keep lying.

ikeboy 7y ago

This is a technical legal point. You can be forced to produce information you have, but generally not to create new information.

vageli 7y ago
They have the information (note that they did not contest that point), the script would presumably allow them to surface that information more quickly.
ogre_codes 7y ago
They are arguing that finding data that's in log files is creating new data (because they have to write a script to parse it) which is specious. Logs are designed to be parsed and searched. You might as well claim that running an SQL script to query data is creating new information.
- colejohnson66 7y ago
  Well, if we’re following that logic, then anytime data is moved from a disk to ram, we’re “creating new data.” Well, yes, it wasn’t there before, so it’s “new data,” but it’s also a copy of old data.
  - Erwin 7y ago
    There have been some weird legal discussions about whether turning on a computer breaks the copyright depending on who does it: https://en.wikipedia.org/wiki/MAI_Systems_Corp._v._Peak_Computer,_Inc https://en.wikipedia.org/wiki/MAI_Systems_Corp._v._Peak_Comp....
    - jacoblambda 7y ago
      That's actually a fairly interesting case. I haven't read into it too much but it seems that they agree that it isn't necessarily a copyright violation based on who does it. It looks like what they got Peak on was instead the fact that they had unlicensed copies of MAI software at their HQ and were loaning MAI equipment in such a way that violated the license agreements. P.S. Please correct me if I got any of this wrong.
      - zerocrates 7y ago
        There are some details in the case that are as you say, about Peak having "unclean hands" in effect, but they're fairly beside the point for the case's broader significance. What it really stands/stood for is the notion that a "copy" sufficient for the operation of copyright law occurs when a program is loaded from disk to RAM, and therefore it's possible to infringe copyright merely by using software that you aren't personally licensed for. MAI basically used this theory to make third-party maintenance of their systems illegal, so Congress responded by amending the Copyright Act, but in an extremely narrow way: there's a carve out specifically for maintenance and repair.
- kortilla 7y ago
  > Logs are designed to be parsed and searched You’ve never dealt with a bad log setup. Things I’ve seen: - inconsistent format depending on which part of the software was logging (think of some modules using logging lib and others using print()) - multiline logs without any indication they are multiline - logs in some dumb binary format that requires the original software to reverse into legible text There are more, but the point is that logs are not often designed to be parsed and searched. They are normally added by developers to be human readable means of reversing internal state to debug issues. Anything above that is (shockingly) bonus material that doesn’t make it into home-grown enterprise/govt software.
  - ogre_codes 7y ago
    > You’ve never dealt with a bad log setup. Things I’ve seen: I'm not sure how this is relevant. I've seen miles of incompetence in my career, that someone can fudge something up royally doesn't change the fundamental purpose of the thing. The point of log files is to store and make retrievable data about server state and actions over time. That someone somewhere can screw that up doesn't change that. In this particular case, I find it unlikely they rolled some custom bizarro log file format regardless because of the nature of the subject matter: Proxy server log files. It's unlikely they rolled their own proxy server (and they certainly don't imply any particular difficulties like that in court statements) so we're most likely talking about reading standard Apache logs or something similar.
    - kortilla 7y ago
      > The point of log files That’s a point of log files. However, it’s not the point tons of developers have in mind when they add log statements. Developers are primarily concerned with state required to reproduce bugs, not long term standardized storage containing enough information for useful attribution or even all important state transitions. You would be surprised how many developers don’t even bother logging item deletions because it’s such a trivial code path. You’re idea of “the fundamental purpose” is from an operators view. It rarely lines up with the developers’ views and you’re in for a bad time if you think people who don’t see your perspective are “fudging something up royally” with “miles of incompetence”.
- sl1ck731 7y ago
  That is true but more likely they try to push back against every request that isn't simply zip up files or documents and deposit them. They are using the "defense" that it may be difficult to do, but their aim (avoiding the more sinister possibilities) is to stop precedent for other unwieldy requests.
  - ogre_codes 7y ago
    In my experience, government agencies fulfill requests if it's easy and won't put the agency or administration in a bad light. I suspect this falls more into the latter case than the former, they don't want people to see the logs because they know the bulk of the comments are bogus.
zelon88 7y ago
Technically you can open that logfile in Notepad and Ctrl+F the thing. If I was the judge and you pulled that argument someone at your agency would now be using that method instead of "researching" a grep query.

pc86 7y ago

Don't underestimate the absolutely jaw-dropping incompetence from even senior "tech" folks employed by governments. Within the last couple years, I've overheard senior, so-called technical government employees 1) complain about Git and wonder aloud why we weren't using Visual Source Safe; 2) insist that rotating through a list of 12 hard-coded passwords, in code, checked into Git, was totally fine; 3) refuse to believe that automated deployments were possible (not hard, or against norms, but physically impossible); 4) try to explain to another so-called technical gov't employee the difference between CSS and JavaScript, and get it wrong; 5) stand up in the middle of a conversation and walk out the door because it's 2:30 PM and their day is over; 6) even more nonsense you wouldn't believe if I showed you a video of it. I would hope Federal is a little better than State, but I'm not convinced.

JaimeThompson 7y ago
To be fair I know of people in private industry who have said some of the exact same things. It isn't limited to just government employees.
- pc86 7y ago
  If you work for a modern software company (not a service company that sells software), and you honestly suggest Visual Source Safe, you'll get fired. There's just no reasonably intelligent reason to suggest it. You practically have to assault someone in the break room to get fired from a government job.
  - barbecue_sauce 7y ago
    Would you get fired, or would people just assume you were being sarcastic?
    - pc86 7y ago
      > > honestly

rasengan 7y ago

How do we know if the IP and headers supplied will be legitimate? There have been other acts in the past [1] that have been covered in government. [1] https://www.vice.com/en_us/article/4xaqz9/hillary-clinton-tech-guy-asked-reddit-for-email-advice https://www.vice.com/en_us/article/4xaqz9/hillary-clinton-te...

erichurkman 7y ago

And when the Government is on the other side with requests, warrants, and subpoenas, they aggressively push with little regard to cost, complexity, or even feasibility. If we replied to a lawful order with "This demands 'research' instead of simply a 'search'," they would tell us tough-shit.

SamPatt 7y ago
And that's when they're being "nice." Otherwise they would send armed men to steal your stuff.

btrettel 7y ago

The FOIA request was probably not denied because the FCC lacks the technical abilities to fulfill the request. In my experience it's likely that no technical people were consulted on it. A FOIA office simply looks for any excuse they can use to deny a request if they want to deny it. The reason doesn't need to make sense or be their true reason. Here's an example from my own experience: I can recall a FOIA request I made for the results of searches in an internal government database in spreadsheet format. To my knowledge this can be exported using existing functionality in the web interface for the internal database. The request was denied, and I appealed. They claimed that the information was publicly available, which is absurd. In the appeal I showed that actually almost none of what I wanted was publicly available and the independent appeals office agreed (after a long delay). Then it became clear that the FOIA office thought the request would be burdensome and they tried to deny it on those grounds. I insisted that no, it wouldn't be burdensome. When it finally went to the person who would actually do the work, apparently it took only a few hours to do, and they declined to charge me anything for it. I assume the majority of that time was writing some useful comments in the headers of the spreadsheets. They didn't need to write the comments, so clearly they weren't pressed for time. In the end, the FOIA office spent far more time trying to deny this request than it would have taken to fulfill it.

ogre_codes 7y ago
> A FOIA office simply looks for any excuse they can use to deny a request if they want to deny it. The reason doesn't need to make sense or be their true reason. That's exactly what I was hinting at here. The agency has no interest in the public finding out how much of that feedback was bullshit. > In my experience it's likely that no technical people were consulted on it. From the sounds of it, they did at some point consult someone technical and tried to parse their answer as part of their justification for not executing the FOIA request. The fact that they knew it would take a script to extract the data as opposed to just dropping it into a search tool suggests as much anyhow. Though more likely it would just be a couple `sed` commands, a perl/ Python script, or a few command line tools.

irrational 7y ago

Once upon a time, my company lawyers came to me and asked me if we had certain historical data that I could pull from a database and how long it would take to pull it. It only took me an hour or so, so I just pulled the data, stuck it in an excel file and sent it to them. They were not happy. Apparently they had already told the judge that the data was unduly burdensome to pull and would take too long.

zamalek 7y ago

Realistically fabricating comments/evidence is a much larger script. Once you start lying, you have to keep lying.