6 ms·

In a different place a long time ago we were approached by a company that wanted a collaboration. They had a very specific project in mind, with requirements al

by amoss 2y ago

In a different place a long time ago we were approached by a company that wanted a collaboration. They had a very specific project in mind, with requirements already set. It was a humanitarian project to "find earthquake survivors". If you read the requirements in a very positive way it was an ethically clean project. But the company is a major defense contractor and it was very unusual for them to be involved in humanitarian work. So it made sense to question the requirements after a less generous reading. Detecting mostly occluded humans in a highly cluttered and destroyed urban environment has other use cases. They got as far as "highlighting the survivors with a laser designator" before I started to ask some very pointed questions and start discussing the ethical considerations of us helping to build a system to kill people. I was asked to leave. After that meeting the potential collaboration was rejected and it never came up again. The company continued shopping around other places to get help building their "earthquake rescue system". Never get distracted. Always question the purpose of work. Consider ethics. Getting another job can be a pain. Living with knowing what you helped to build would be much harder.

duxup 2y ago
I'm a little surprised that a defense contractor would bother with that much awkward subterfuge or even farm out a core part of their product to some random other company. The handful of US based defense contractors I've had contact with were VERY up front about what they develop. I think they more so wanted you to know so everyone was on the same page.
- cl0ckt0wer 2y ago
  They probably already have a complete system. They're just looking for new revenue sources.
- pc86 2y ago
  I'll bet my next paycheck the above scenario never happened, or this company was less "major defense contractor" and more "VC-funded SaaS-app defense contractor wannabe." Lockheed Martin doesn't need to trick anyone into working for them, and that includes software developers. If this was a real thing that actually happened, everyone would need a security clearance anyway.
  - duxup 2y ago
    Long long ago I had contact with Lockheed Martin, their emails had wallpaper of jets, bombs, weapons systems.
  - HumblyTossed 2y ago
    A LOT of major (and minor) defense contractors were always on campus at UCF, when I was there, interviewing/recruiting. They had no shortage of students wanting to get out there and get a job.
- neilv 2y ago
  > The handful of US based defense contractors I've had contact with were VERY up front about what they develop. I think they more so wanted you to know so everyone was on the same page. That matches my couple data points. I did a lot of software engineering consulting work through a Federal contractor. When I was first approached, the director told me some of their customers were military, and how would I feel about that. I said something generally favorable, and that I only had a personal rule that I didn't want to work on weapon systems, nor domestic surveillance. That was fine with them, and we did years of great work, on very positive safety programs. Earlier, in school, I did a final project for a class (a simple "agent" system to help people find online communities of interest in Internet chat). The professor later told me, out of the blue, that a federal contractor wanted to license my little project(!), for surveillance(!), and they'd pay me to advise. At least they were upfront about what they wanted it for. I declined and discouraged them.[1] [1] One reason was that I turned off by surveillance of the Internet, as a pre-Web Internet kid (also raised on anti-Soviet/anti-police-state propaganda) who still thought of the Internet as a better world that we insiders wanted to bring to everyone else. The other reason was my little technical approach would've been a waste of money for that surveillance problem. IIRC, I was polite in declining, and didn't mention the first reason, but was probably impolitic in telling them the latter reason.
  - duxup 2y ago
    It is only a couple data points, but the story given was "major defense contractor" and there really are only a few of those. At least as far as the US goes I've never heard of them awkwardly trying to get people to develop something for them in the manner described. In my experience they're very up front about what they do and aren't short on people willing to work for them. Not to say the story was a lie, I don't know, but it sounds very unusual.
    - neilv 2y ago
      I don't know whether there's a common practice that's followed up and down all the org charts of all the employers. I didn't disbelieve the article's story. It might've just, for example, been someone's pet project, with only budget to hire some co-op students and an inexperienced new-grad coder, to make a demo, to pitch for a real project/contract. Or to proof-of-concept a method, to possibly be properly designed and implemented by a real engineering team. Maybe they didn't think they needed experienced serious engineers with clearance for this compartmentalized exercise. Another, maybe less-likely, possibility was something involving a cleanroom implementation as demonstration it was "obvious to first-year students". Or to find a plausibly alternate cleanroom implementation, by people who hadn't been tainted by knowledge of the encumbered one. Guessing not, but I did see something like this done at least once. I'd bumped into this affable young high school student, who was idling around in common areas of my lab, and was outgoing enough to strike up a conversation with random passerby me, and start telling me he was working on a project. It turned out a professor had tasked him to reinvent this one algorithm that was some alum grad student's thesis, which had been patented, and which the alum had been milking rather hard... and the professor didn't tell the high school student about the prior work. I also once had a current PhD student hired as an intern, to apply very niche expertise towards a speculative new feature ("wouldn't it be great if we could do this cutting-edge thing in the new system"), and they were mostly successful, but the product ended up going a different direction. (The PhD student did OK: they then had understanding of a great, tricky real-world domain example, to guide research. A lot of research uses over-simplified examples of applications that they never had the opportunity to really understand.) I did have a twinge while reading the article, wondering why it seemed like they were about to go into methods used by what sounded to my naive ear like a tracking/targeting component of a military system, but I would guess that they didn't spill any secret sauce beans.
babkayaga 2y ago
do you know what happens if military does not have a system to find and target specific people in a building? they just might need to bring down the whole building. everyone in it might die but at least your conscience is clean.
- bagels 2y ago
  So really, we're all murderers for not helping them to murder more carefully? No.
  - pc86 2y ago
    Even the wording of this question conflates a soldier killing a soldier during a conflict (which is objectively not murder) with the wanton execution of civilians with no regard to collateral or preventable damage, which is a war crime. And completely ignores the existence of a middle ground where a high-value military target is killed, preventative measures are taken to limit civilian casualties, and some civilians are killed despite those measures. That is not good but it's also not murder, and not a war crime. I would love to have an honest, thoughtful discussion about how a war can be prosecuted between two powers with minimal civilian harm, but it's not possible when people aren't even honest about what constitutes murder and what doesn't.
    - axus 2y ago
      Attacking computer security is the only example I can think of for conflict without hurting civilians. Any armed conflict is going to cause additional civilian deaths.
      - buttercraft 2y ago
        Attacking computers that control critical infrastructure could absolutely cause civilian deaths.
    - ellen364 2y ago
      I suspect the idea that “people aren't even honest about what constitutes murder and what doesn't” creates a high barrier to the discussion you’d like to have. From your post, I think you’re starting from the pov that governments have the right to decide what is murder vs an acceptable killing. Some of the people I’ve met who are most interested in these ideas are staunch pacifists with a strong religious motivation (e.g. Quaker or Methodist) and reject the idea that governments can declare any killing to be acceptable. I don’t believe they’re dishonest, for all that it’s a very different starting point.
hammock 2y ago
So what happens now, the next time there is an earthquake and we need to find occluded survivors?
- tpoacher 2y ago
  You send the killbots, but replace the existing machine-guns with lasers.
- pc86 2y ago
  They die, or we send people in who themselves risk death.
- mysterydip 2y ago
  Professional trained search and rescue dog teams, from what my kids learned in an imax film at the science center.
eastern 2y ago
What about questioning the purpose of the taxes you pay?
- pc86 2y ago
  Questioning where your tax money goes is a good thing, and a necessary activity to be an informed citizen. However the next step is usually "I don't like $THING, therefore I shouldn't have any of my tax money go toward paying for it" which does not work at scale. At least in the US, we are not a direct democracy so it's not important or even an desirable to have everyone decide where their money goes based on their own individual morals or ideology. We elect leaders and in a perfect world they're smart enough (lol) to see that taxpayer funds are allocated in a manner that best helps Americans (lol).
- Fargren 2y ago
  It's not often that we get to see whataboutism actually start with an actual "What about...". Less snarkily: even if we agreed that paying taxes makes one complicit on evil acts done with that money, it doesn't mean that person should not avoid being complicit in other evil actions where they can.
imglorp 2y ago
I worked for a company that located phones during E911 calls, at the cell network level, without using GPS, down to a few meters. The most humanitarian mission you can imagine. It was golden. But it turns out, authoritarian states wanted to buy the very same tech to locate undesirable people, record their associates and travel history, geofence them, etc. Also turns out salespeople gonna sell. And that's how you go from an ethical position to another kind. People did quit over that.
jkaptur 2y ago
As far as I can tell, this isn't widely known, but around a decade ago some folks deployed a version of this technology for that exact use case: https://www.jpl.nasa.gov/news/finder-search-and-rescue-technology-helped-save-lives-in-nepal/ https://www.jpl.nasa.gov/news/finder-search-and-rescue-techn... The obvious military applications for this technology are explored in Tom Clancy's novel "Rainbow Six" (1998).

by amoss 2y ago

duxup 2y ago

I'm a little surprised that a defense contractor would bother with that much awkward subterfuge or even farm out a core part of their product to some random other company. The handful of US based defense contractors I've had contact with were VERY up front about what they develop. I think they more so wanted you to know so everyone was on the same page.

cl0ckt0wer 2y ago
They probably already have a complete system. They're just looking for new revenue sources.
pc86 2y ago
I'll bet my next paycheck the above scenario never happened, or this company was less "major defense contractor" and more "VC-funded SaaS-app defense contractor wannabe." Lockheed Martin doesn't need to trick anyone into working for them, and that includes software developers. If this was a real thing that actually happened, everyone would need a security clearance anyway.
- duxup 2y ago
  Long long ago I had contact with Lockheed Martin, their emails had wallpaper of jets, bombs, weapons systems.
- HumblyTossed 2y ago
  A LOT of major (and minor) defense contractors were always on campus at UCF, when I was there, interviewing/recruiting. They had no shortage of students wanting to get out there and get a job.
neilv 2y ago
> The handful of US based defense contractors I've had contact with were VERY up front about what they develop. I think they more so wanted you to know so everyone was on the same page. That matches my couple data points. I did a lot of software engineering consulting work through a Federal contractor. When I was first approached, the director told me some of their customers were military, and how would I feel about that. I said something generally favorable, and that I only had a personal rule that I didn't want to work on weapon systems, nor domestic surveillance. That was fine with them, and we did years of great work, on very positive safety programs. Earlier, in school, I did a final project for a class (a simple "agent" system to help people find online communities of interest in Internet chat). The professor later told me, out of the blue, that a federal contractor wanted to license my little project(!), for surveillance(!), and they'd pay me to advise. At least they were upfront about what they wanted it for. I declined and discouraged them.[1] [1] One reason was that I turned off by surveillance of the Internet, as a pre-Web Internet kid (also raised on anti-Soviet/anti-police-state propaganda) who still thought of the Internet as a better world that we insiders wanted to bring to everyone else. The other reason was my little technical approach would've been a waste of money for that surveillance problem. IIRC, I was polite in declining, and didn't mention the first reason, but was probably impolitic in telling them the latter reason.
- duxup 2y ago
  It is only a couple data points, but the story given was "major defense contractor" and there really are only a few of those. At least as far as the US goes I've never heard of them awkwardly trying to get people to develop something for them in the manner described. In my experience they're very up front about what they do and aren't short on people willing to work for them. Not to say the story was a lie, I don't know, but it sounds very unusual.
  - neilv 2y ago
    I don't know whether there's a common practice that's followed up and down all the org charts of all the employers. I didn't disbelieve the article's story. It might've just, for example, been someone's pet project, with only budget to hire some co-op students and an inexperienced new-grad coder, to make a demo, to pitch for a real project/contract. Or to proof-of-concept a method, to possibly be properly designed and implemented by a real engineering team. Maybe they didn't think they needed experienced serious engineers with clearance for this compartmentalized exercise. Another, maybe less-likely, possibility was something involving a cleanroom implementation as demonstration it was "obvious to first-year students". Or to find a plausibly alternate cleanroom implementation, by people who hadn't been tainted by knowledge of the encumbered one. Guessing not, but I did see something like this done at least once. I'd bumped into this affable young high school student, who was idling around in common areas of my lab, and was outgoing enough to strike up a conversation with random passerby me, and start telling me he was working on a project. It turned out a professor had tasked him to reinvent this one algorithm that was some alum grad student's thesis, which had been patented, and which the alum had been milking rather hard... and the professor didn't tell the high school student about the prior work. I also once had a current PhD student hired as an intern, to apply very niche expertise towards a speculative new feature ("wouldn't it be great if we could do this cutting-edge thing in the new system"), and they were mostly successful, but the product ended up going a different direction. (The PhD student did OK: they then had understanding of a great, tricky real-world domain example, to guide research. A lot of research uses over-simplified examples of applications that they never had the opportunity to really understand.) I did have a twinge while reading the article, wondering why it seemed like they were about to go into methods used by what sounded to my naive ear like a tracking/targeting component of a military system, but I would guess that they didn't spill any secret sauce beans.

babkayaga 2y ago

do you know what happens if military does not have a system to find and target specific people in a building? they just might need to bring down the whole building. everyone in it might die but at least your conscience is clean.

bagels 2y ago
So really, we're all murderers for not helping them to murder more carefully? No.
- pc86 2y ago
  Even the wording of this question conflates a soldier killing a soldier during a conflict (which is objectively not murder) with the wanton execution of civilians with no regard to collateral or preventable damage, which is a war crime. And completely ignores the existence of a middle ground where a high-value military target is killed, preventative measures are taken to limit civilian casualties, and some civilians are killed despite those measures. That is not good but it's also not murder, and not a war crime. I would love to have an honest, thoughtful discussion about how a war can be prosecuted between two powers with minimal civilian harm, but it's not possible when people aren't even honest about what constitutes murder and what doesn't.
  - axus 2y ago
    Attacking computer security is the only example I can think of for conflict without hurting civilians. Any armed conflict is going to cause additional civilian deaths.
    - buttercraft 2y ago
      Attacking computers that control critical infrastructure could absolutely cause civilian deaths.
  - ellen364 2y ago
    I suspect the idea that “people aren't even honest about what constitutes murder and what doesn't” creates a high barrier to the discussion you’d like to have. From your post, I think you’re starting from the pov that governments have the right to decide what is murder vs an acceptable killing. Some of the people I’ve met who are most interested in these ideas are staunch pacifists with a strong religious motivation (e.g. Quaker or Methodist) and reject the idea that governments can declare any killing to be acceptable. I don’t believe they’re dishonest, for all that it’s a very different starting point.

hammock 2y ago

So what happens now, the next time there is an earthquake and we need to find occluded survivors?

tpoacher 2y ago
You send the killbots, but replace the existing machine-guns with lasers.
pc86 2y ago
They die, or we send people in who themselves risk death.
mysterydip 2y ago
Professional trained search and rescue dog teams, from what my kids learned in an imax film at the science center.

eastern 2y ago

What about questioning the purpose of the taxes you pay?

pc86 2y ago
Questioning where your tax money goes is a good thing, and a necessary activity to be an informed citizen. However the next step is usually "I don't like $THING, therefore I shouldn't have any of my tax money go toward paying for it" which does not work at scale. At least in the US, we are not a direct democracy so it's not important or even an desirable to have everyone decide where their money goes based on their own individual morals or ideology. We elect leaders and in a perfect world they're smart enough (lol) to see that taxpayer funds are allocated in a manner that best helps Americans (lol).
Fargren 2y ago
It's not often that we get to see whataboutism actually start with an actual "What about...". Less snarkily: even if we agreed that paying taxes makes one complicit on evil acts done with that money, it doesn't mean that person should not avoid being complicit in other evil actions where they can.

imglorp 2y ago

I worked for a company that located phones during E911 calls, at the cell network level, without using GPS, down to a few meters. The most humanitarian mission you can imagine. It was golden. But it turns out, authoritarian states wanted to buy the very same tech to locate undesirable people, record their associates and travel history, geofence them, etc. Also turns out salespeople gonna sell. And that's how you go from an ethical position to another kind. People did quit over that.

jkaptur 2y ago

As far as I can tell, this isn't widely known, but around a decade ago some folks deployed a version of this technology for that exact use case: https://www.jpl.nasa.gov/news/finder-search-and-rescue-technology-helped-save-lives-in-nepal/ https://www.jpl.nasa.gov/news/finder-search-and-rescue-techn... The obvious military applications for this technology are explored in Tom Clancy's novel "Rainbow Six" (1998).