38 ms·

The Pentagon's Silicon Valley Problem

309 points by NDAjam 3y ago

chiefalchemist 3y ago
> The system knows everything about [the terrorist]: where he went, who his friends are, who his family is, what keeps him busy, what he said and what he published. Using artificial intelligence, the system analyzes behavior, predicts risks, raises alerts. Where does "the terrorist" end and me, you and anyone else just minding our own business get inserted instead? And let's say it's not even the gov doing this but some private company with public data, what's to stop the gov from buying "reports" from that company. 100% legal. That is, no rights being violated, etc. Anyone who says, "I have nothing to hide" is a fool, at best.
- humansareok1 3y ago
  I don't support ubiquitous spying at all but are you hanging out with known ISIS members or members of White Nationalist Militias regularly? Because I'm pretty sure that's where the line begins.
  - 7thaccount 3y ago
    I think they're just saying it's a slippery slope. It starts out with good intentions we all agree on, but then continues to slide and more and more of our freedoms erode as they crank up the boiling pot ever so slowly.
  - AlexandrB 3y ago
    There's a lot of grey here. What does "hanging out" mean? If my weird uncle is (unknown to me) in ISIS does spending thanksgiving with him count as "hanging out"? ISIS is at least pretty specific, but what counts as a White Nationalist Militia? Both of these can be redefined to capture more and more of the population if desired.
    - forgotmyinfo 3y ago
      This is what we have attorneys and judges for. And no, obviously Thanksgiving isn't "hanging out". But going to the same weekly meeting and practicing lynching minorities? Yeah, that's a little more than just mashed potatoes and gravy, isn't it. (These contrived "whatabout" gotchas are exhausting. It is abundantly obvious who is and who is not involved with white nationalist militias.)
      - nurple 3y ago
        Yes, all the legal arguments presented before the FISA court by the lawyers working on behalf of those targeted have been really interesting reads!
        dragonwriter 3y ago
        There aren't legal arguments by the targets in law enforcement search (either physical or wiretap) warrant cases either, that mainly only happens if (as does not always happen) the product of the search is used in criminal prosecution later.
      - sakjur 3y ago
        None of those things seem particularly obvious to me.
      - chasd00 3y ago
        It's really the court of public opinion that has the greatest risk of harm at the day to day level. A non-poc going to the gun range and then posting on social media could cause a "White Nationalist Militia" label to get attached by a jilted coworker and then go viral. That can cause serious harm.
        mindslight 3y ago
        Maybe gun clubs should implement DEI programs. Then those pictures would have some colorful people in the background.
  - A4ET8a8uTh0 3y ago
    If you don't know, who they are and just happen to serve a 'bad guy' ( hate that term ) a burger, should you be in the crosshairs? Because this is where this is heading. If you think I am overselling it, remember that police in US can ask for all users from specific location. To your point, if there is indeed a line, it need to be clearly articulated so that the rules of the game clear.
  - hwbehrens 3y ago
    > I'm pretty sure that's where the line begins Based on what? From my perspective, the easiest way to design such a system would be to create entries for every 'actor' in the system, feed in as much data as you can get your hands on, and then let the weights sort themselves out. So for example, if you're hanging out with ISIS members obviously your weights would be higher, but even if you're a server at Applebees you'd still be in the system somewhere. Doing it the other way necessitates some kind of bright-line division, and any such boundary, once defined, becomes susceptible to exploitation. e.g. I won't hang out with the White Nationalist Militia because that puts me "into the system", but I can hang out with insert radical right-wing group where I can talk to 80% of the same people without being flagged. In practice, I imagine that the gradient of extremism is rather gradual and with blurred boundaries.
    - humansareok1 3y ago
      As another poster mentioned this is literally why we have courts. There is a clear line for obtaining a search warrant for example. Precedents exist.
      - raisin_churn 3y ago
        Are you familiar with FISC? I'd say go familiarize yourself with its case law, but you can't, because it's secret. And it authorizes methods much more powerful and invasive than a simple search warrant. Precedent exists, but nobody outside the national security state actually knows what it is.
        humansareok1 3y ago
        You're maybe proposing another line where no spying is legal at all and we should just submit ourselves to the whims of terrorists or other lunatics? Surely there is actually a line where there are tradeoffs between security and privacy and its probably not 0% security and 100% privacy. Perhaps you think all FISA rulings should be public and any sufficiently savvy malicious actors can just read them to know exactly how to avoid suspicion?
        3y ago
  - wpietri 3y ago
    Maybe that's where the line starts, but does it stay there? As an example, look at how the US's anti-communist fervor led to things like COINTELPRO: https://en.wikipedia.org/wiki/COINTELPRO https://en.wikipedia.org/wiki/COINTELPRO Especially today, I think we have to look at every power we might give to government and ask, "What happens if the worst people get access to this?" Because they're certainly going to try.
    - chiefalchemist 3y ago
      Did we give it, or was it taken? Yeah, maybe we consented to The Patriot Act. But when it was renewed, it was *expanded*. Too late now.
  - chiefalchemist 3y ago
    You've heard of guilt by association, well guilt by co-location isn't that far off. Along the same lines, the rationale is going to be, "We need to track everyone so we can be sure to see all relationships and connections, and connections to connections, and so on. Try to draw the line wherever you want, but they're going to step over it, and never look back.
- A4ET8a8uTh0 3y ago
  I think, in a sense, that part is already over. Entities that encompass both ends of the spectrum exist and any remaining gaps are filled by public/private partnership ( and hailed as a great thing just about anywhere ). The scenario of a private corporate entity wielding that power has already come to fruition if you look at what Google or Facebook has available on its users. I think that is the main reason why I am not as.. restrictive on use of LLMs and AI, because I see it as a form evening out the playing field at least a little bit.
  - nurple 3y ago
    I think one of the things that scares me most about API-accessed LLMs is how powerful they are as data collection tools in their own right. OpenAI, for example, recently updated their terms of use to be more vague about how they work with the gov and I have no doubts that giving the NSA access to conversational feeds is absolutely a requirement to their continued operation as an entity, a la lavabit. In fact, part of me thinks that the Sam/Ilya drama and sam's god complex are at least partially rooted in this, alleged, collaboration. Imagine the questions you could pose to a GPT trained on all the conversations had with users that's been enriched with their biographical data. These conversations are often intimate and curiosity driven in a way that seeking the truth could easily be framed as self-radicalization.
    - A4ET8a8uTh0 3y ago
      That API is one the main reasons companies are not as keen on jumping on the bandwagon. They don't want to have OpenAI to have access to their corporate data. But then, there are options of running models locally.. I think your concerns are valid.
- nurple 3y ago
  The terrorist ends wherever a threat to the state's power exists, it's been shown quite well that they don't care if you're domestic or not. This, IMO, is why "self-radicalized" and "domestic terrorist" were injected into our vernacular, to normalize and justify the need to surveil the general public. The thing is, and like I mentioned in a post awhile back: technically competent actors, the ones bound to cause the most harm, would absolutely be using a bespoke method of covert communication. There's really little value, IMO, to the countrywide dragnet outside of sentiment analysis and control. The military complex wove itself early into the tech industry in ways that they could intentionally side-step laws meant to keep such public/private collusion from happening[0]. The impetus for the founding of the collaboration was a report on the importance in controlling perception in future wars. We saw the same strategy deployed directly against the American people during the election "fortification" where DHS and social media colluded to control perception with little regard for truth[1]. [0] https://medium.com/insurge-intelligence/how-the-cia-made-google-e836451a959e https://medium.com/insurge-intelligence/how-the-cia-made-goo... [1] https://homeland.house.gov/2023/11/06/chairmen-green-bishop-uncover-new-evidence-of-dhs-big-tech-collaborating-to-censor-americans/ https://homeland.house.gov/2023/11/06/chairmen-green-bishop-...
  - chiefalchemist 3y ago
    > The thing is, and like I mentioned in a post awhile back: technically competent actors, the ones bound to cause the most harm, would absolutely be using a bespoke method of covert communication. There's really little value, IMO, to the countrywide dragnet outside of sentiment analysis and control. Agreed. And yet their persistence to surveil continue to expand. They're not trying to watch those who are sure to be hiding. They're watching the rest under the guise of "We gotta get those terrorists."
- deleted 3y ago
  [deleted]
throwaway4good 3y ago
I don't understand the headline "problem" of the article. Or the "How Big Tech is losing the wars of the future". Silicon Valley has always been a part of the US military complex. Maybe there was a period sometime in the 90es where it was irrational exuberance and don't be evil. But now we are surely back under manners.
- selimthegrim 3y ago
  TIL ‘under (heavy) manners’
  - 082349872349872 3y ago
    TIL 'put manners on' (in combination, it would appear that whatever 'manners' may be, they are located distal of the cranium)
tivert 3y ago
> Nevertheless, Hamas’s devastating attack on October 7 caught Shin Bet and the rest of Israel’s multibillion-dollar defense system entirely by surprise. The intelligence disaster was even more striking considering Hamas carried out much of its preparations in plain sight, including practice assaults on mock-ups of the border fence and Israeli settlements—activities that were openly reported. Hamas-led militant groups even posted videos of their training online. Israelis living close to the border observed and publicized these exercises with mounting alarm, but were ignored in favor of intelligence bureaucracies’ analyses and, by extension, the software that had informed them. Israeli conscripts, mostly young women, monitoring developments through the ubiquitous surveillance cameras along the Gaza border, composed and presented a detailed report on Hamas’s preparations to breach the fence and take hostages, only to have their findings dismissed as “an imaginary scenario.” The Israeli intelligence apparatus had for more than a year been in possession of a Hamas document that detailed the group’s plan for an attack. > Well aware of Israel’s intelligence methods, Hamas members fed their enemy the data that they wanted to hear, using informants they knew would report to the Israelis. They signaled that the ruling group inside Gaza was concentrating on improving the local economy by gaining access to the Israeli job market, and that Hamas had been deterred from action by Israel’s overwhelming military might. Such reports confirmed that Israel’s intelligence system had rigid assumptions of Hamas behavior, overlaid with a racial arrogance that considered Palestinians incapable of such a large-scale operation. AI, it turned out, knew everything about the terrorist except what he was thinking. That sounds a lot like a company that's implementing data-driven "best practices" from some expensive management consultants. It truly is the best system, regardless of how bad the results are. It's best by definition.
- deleted 3y ago
  [deleted]
- wpietri 3y ago
  Ooh, very interesting point: > That sounds a lot like a company that's implementing data-driven "best practices" from some expensive management consultants. > > It truly is the best system, regardless of how bad the results are. It's best by definition. Well that rings some bells. It's as if there's a religion where the sacred totem is a graph that goes up and to the right. Some question for the crowd: How do systems like this insulate themselves from failure? Before something goes wrong, what prevents seeing the problem? And after something goes wrong, what are the words and behaviors used to avoid fundamental change?
burutthrow1234 3y ago
[flagged]
orange_joe 3y ago
Since the article talks about the failure of AI in the context of the 10/7 I think it’s worth discussing the situation directly. Everything points to the Israelis not having taken their security seriously beyond the tactical level. I’m certain they thwarted other attacks, but it was an inevitability that a major attack was successful at some point. Such an attack would necessitate a military response. However the Israelis have no strategic vision. They lacked serious plans for such an eventuality and still lack a serious goal for their invasion of Gaza. They haven’t articulated anything that indicates a vision to meaningfully change the situation from the 10/6 state to something more sustainable. Therefore, it doesn’t seem like a reasonable takeaway to say AI failed.
- nkozyra 3y ago
  > Therefore, it doesn’t seem like a reasonable takeaway to say AI failed. There are a lot of reasons - from quite intuitive to conspiratorial - to not take the idea that AI caused or meaningfully contributed to this failure at face value. Or that it was a failure of intelligence in the first place.
- cameldrv 3y ago
  To be fair, the lack of strategic vision has also plagued the U.S. since WWII or Korea. We just keep losing wars because no one ever sets out clear achievable goals. The notable exception was the Powell Doctrine in Desert Storm. For that one, the goal was to kick Iraq out of Kuwait and restore the Kuwaiti monarchy, which was achieved. If you look especially at Afghanistan, Libya, and Syria, there is this magical thinking that if we destroy the Evil Dictator and run an election, that everyone will naturally vote to ally with the U.S. and completely change their social organization to be in accordance with western values. The place we spent the most time in the 21st century, Afghanistan, somehow went from an objective of destroying Al Qaeda to ensuring that girls got a good education and had equal rights. That sort of societal transformation is not possible even with 100,000 troops when they don't even speak the local language. Can you imagine the hubris of trying to tell people in some remote village that the way men and women relate to each other has to change through a translator, because some tall buildings in a place they've never heard of got destroyed? The obvious result was total failure and the Taliban picking up right where they left off in 2001.
deleted 3y ago
[deleted]
nkozyra 3y ago
I think - like a lot of media reporting on the space - this overgeneralizes (heh) artificial intelligence. The predictive aspects of ML have been in use in modern militaries for _decades_, and the opening graf handwavely indicates that an LLM was a bigger chunk of the perceived intelligence failure of the October 7 attack. That an LLM is a part of a system that includes a large amount of ML is not surprising. It's a great human interface. Do I for a second believe that it played a much larger role, such to be implied as responsible in any non-negligble way for missing the attack. Of course not. My point here is that ML continues to play a role, ML continues to both succeed and fail, and ML will continue to be imperfect, even moreso as it competes against adversarial ML. Blaming imperfect tools for inevitable failures is not a useful exercise, and certainly not a "problem" considering the alternative being even more failure-prone humans.
- rossdavidh 3y ago
  Blaming the excessively grand claims that were made for those tools, however, is absolutely a useful exercise.
  - jonchurch_ 3y ago
    But grand claims made by technologists are nothing new. Certainly I don’t know, Ive never been in the military, but aren’t people always trying to sell The Next Big Thing to the military? Is it not the responsibility of those in charge to evaluate the capabilities and limitations of new systems being integrated into their forces? If someone said “we dont need the rigor we used to have anymore, we have AI” I see that as a failure of the org, not an indictment of the claims being put forth by boosters. Corporate Decision Maker #2, sure, theyll get hoodwinked. They and their company may have only 50 years of experience and institutional memory to draw on. But State Militaries? What excuse do they have? War changes, but the armed forces have a long memory, and their poor decisions cost lives. Maybe Im off base, but I would expect each mistake to be an opportunity to learn for that industry. The industry has had plenty of lessons learned over the past 100 years. Why is the latest hype cycle to blame, and not those whose job it is to ensure they maintain capabilities and extensively game out scenarios and responses? Bad bets on tech happen even in institutions with lifetimes of history to draw on, but I see that as a failure of the institution, not on the completely mundane hype cycles which occur naturally. Obviously mistakes happen, and maybe thats what the article is getting at. But if we’re going to point fingers (not saying you are) then lets not let decision makers off the hook whose job is to prevent that hot new thing getting their people killed.
mattnewton 3y ago
I can’t speak for Israeli tech, but the pentagon has an image problem in the valley, I don’t believe they are getting the best recruits even for contracting companies like Palintir. Our generation is closer to Iraq and Vietnam than WW2, and many of the bright minds are first generation immigrants. Despite the more recent image problems ad tech has (now that people are seeing more of how the sausage is made), it’s still sexier to work on big consumer companies than defense. You’d have to pay my colleagues more to work for the US government, even indirectly, instead it’s often less (and often with less freedoms of what they do off the clock). And now, what I’m reading is that if you do go contract for the military in AI, your function is partially some kind of scapegoat insurance. Blame those eggheads with their computers who can be fooled, not the fools who hired them and acted on that signal above others I guess? The idea that a chatGPT model would have been a deciding factor in preventing 10/7 is laughable on its face to anyone who works in the industry, except maybe a consultant selling LLMs to the IDF.
- jajko 3y ago
  I am still not 100% convinced they didnt just let it happen on purpose (and then were surprised just by the scale), having an excuse to raze the place down for good, which is exactly what they are doing. The signs were there, everywhere, and mosad aint bunch of clueless paper pushers. The guy in charge is former spec ops, murder of anybody without battling an eye is part of the deal so dont expect some humanism from that direction. If I didnt read similar stories from other times and places, where it played almost exactly like this... AI is not going to solve political issues, just make them more complex than they already are
- kjellsbells 3y ago
  There's another issue here as well, which is that many of the tech folks who would be ok working for the government, even at reduced rates, cant get through the hiring morass that uncle sam puts up. The fed gov simply isnt set up to quickly acquired talent from industry. They also remain remarkably hidebound by old rules like requiring advanced degrees for senior positions.
  - CoastalCoder 3y ago
AndyMcConachie 3y ago
[flagged]
- shitpostbot 3y ago
  [dead]
A4ET8a8uTh0 3y ago
I personally think this is the most interesting part of the entire article: 'He then focused on defense work, lamenting that people with the relevant tech skills to build the weapons of the future were “largely refusing to work with the defense sector.”' I wonder to what extent that is still true. There is clearly a lot of money flowing and some definitely followed the money ( Palantir exists after all ).
- gamepsys 3y ago
  It's clearly true to some degree, there are documented cases of people that refused to work with the defense sector at great personal costs. The questions are how much resistance is there in the labor force, and how does that impact the ability to recruit talent?
  - aleph_minus_one 3y ago
    > The questions are how much resistance is there in the labor force, and how does that impact the ability to recruit talent? Easy: Give potential employees similar salaries to MAMAA companies, and a similar amount of freedom and independence (at least in the ways in which it is possible at a defense company) as it existed in the early days of Google and Facebook, and I think a lot of potential employees (though of course not all, but this is not necessary) will "forget" their initial moral objections and go for the money. :-)
    - A4ET8a8uTh0 3y ago
      Wasn't DARPA kinda close to that idea ( I am honestly not sure, but it seemed like a lot of interesting stuff came from there )? Still, a person who knows what he/she is building can likely predict how it is going to be used. Would I want to be responsible for popularizing portable black hole generators?
      - aleph_minus_one 3y ago
        > Still, a person who knows what he/she is building can likely predict how it is going to be used. Would I want to be responsible for popularizing portable black hole generators? You just developed an insanely small part of this machinery. Compartmentalization of the work appeases the mind a lot. :-( If you still have bad feelings, there exists the charity-industrial complex: donate some decent paycheck to give a poor, starving child a better life - something that you could not have done if you hadn't accepted the well-paid defense contract.
cameldrv 3y ago
I have no idea how Silicon Valley could be held responsible for an Israeli intelligence failure. Israel is not a part of the U.S. The author exhibits essentially zero knowledge of the advances in military intelligence in the past 10-20 years. He’s talking about problems in the Vietnam war and IBM 360 mainframes as if all of the stuff Macnamara dreamed of weren’t daily reality now.
- luaybs 3y ago
  > Israel is not a part of the U.S. The US sends $3.8B in military aid to Israel yearly...
  - wslh 3y ago
    It is not too much if you think Israel is covering US military mistakes and cybersecurity (e.g. Iran).
    - lostlogin 3y ago
      Isn’t this policing of Iran needed because of the US supporting Israel? I know there are other interests (oil), but the complete freedom to do anything that the US gives Israel is not entirely helpful to the US.
      - wslh 3y ago
        Isn’t this policing of Iran needed because of the US supporting Israel? No, do you remember the terrorists attacks in USA and in Europe? Have you watched the Argo movie? Do you know about terrorist attacks in Latin America where Iran is involved? Seems people don't check history beyond Israel (21k km^2)and Jews (16m).
        sangnoir 3y ago
        I remember the JPCOA and egging on the Trump administration to break the agreement and the subsequent escalations.
neonate 3y ago
http://web.archive.org/web/20240327152111/https://harpers.org/archive/2024/03/the-pentagons-silicon-valley-problem-andrew-cockburn/ http://web.archive.org/web/20240327152111/https://harpers.or...
alephnerd 3y ago
No offense, but this article is MASSIVE BS. There are issues with innovation in the DoD and DHS, but a lot of this is offloaded to private sector vendors anyhow. I notice how the article didn't mention any of the companies I personally know doing stuff in the space, nor actually sourced from members of the VC, Business, or Defense community. The fact that the author took Palantir's marketing at face value is proof enough - the CIA let their contract with Palantir lapse a couple years ago (and I think they only even bought it because of their stake in In-Q-Tel), and they haven't had great success selling to the Fed. I actually work in this space btw. ----- The bigger stumbling block is procurement. Software Procurement by Federal standards is relatively straightforward so a Series E+ startup can make it if they spend around $7-10M and 1-1.5 years on a dedicated roadmap for FedRamp and FIPS compliance. Once you step out of software, procurement becomes paperwork hell. Throw in the paperwork hell from Grantmakers like the DoD and DoE, and you end up with a quasi-Soviet procurement system. Ironically, most of these compliance and regulatory checks were added for good intentions - primarily to minimize corruption and graft, yet it basically clogged up the entire system, and dissuades startups and innovators from working directly with the Defense community. Some projects like DIUx and and In-Q-Tel are trying to change that, but it's too little too late, and our defense base is entirely dependent on firms like Microsoft, Cisco, Crowdstrike, Zscaler, etc acquiring promising startups to evangelize their innovations internally. Fundamentally, this is why I dislike the New America/Khan/Chopra vision of anti-trust. It doesn't actually help innovation from a federal standpoint, as small companies and startups have no reason to work with the Fed given the amount of red tape that exists. If the same effort was put to harmonizing and simplifying procurement across the Federal Government, you could directly make demands on competition. This is what China does, and is a major reason their MIC was able to grow leaps and bounds in just 20 years.
- nceqs3 3y ago
  The way Palantir talks about the CIA really rubs me the wrong way. For years, they would leak to journalists that Palantir "found bin Laden" when, of course, it had nothing to do with finding him. Several CIA employees died trying to find Bin Laden, all for some schmucks in Silicon Valley to try and capitalize on their sacrifice.
outside1234 3y ago
The thing we should all really be terrified about is how Trump and Stephen Miller will use of all of this technology we have built against us when elected.
- deleted 3y ago
  [deleted]
sweeter 3y ago
[flagged]
- yieldcrv 3y ago
  I mean you’ve seen how the people subscribed to that conflict talk about each other, they need a therapist The view is that nobody are civilians, anyone counting deaths is a liar at any count, and surrendering hostages get mistaken for palestinian civilians which means militants Nobody can beat our hostage rescue high score right now so we can just ignore them
- deleted 3y ago
  [deleted]
cess11 3y ago
'Caught by surprise' is a weird description. Israeli press has repeatedly run stories about how frontline analysts sounding the alarm were ignored. That could be due to things like sexism, ageism or discrimination against conscripts, or it could be due to the settler organisations having their people in government and a strong wish to resettle the Gaza strip. Either way, the signals were there, they had been watching the preparations and exercises for a year or so. Even if the resistance groups had kept that secret even a mediocre officer in intelligence or the army should be able to conclude from 'first principles' and what they were doing that there would eventually be a violent response.
- bitcharmer 3y ago
  [flagged]
- Spooky23 3y ago
  This is just a blaming the wrong tools. The people running the Israeli government and army are tools. They fucked up, plain and simple. Whether through malice or just ineptitude and incompetence, they failed.
  - dmurray 3y ago
    Or, they intentionally ignored the intelligence hoping for a casus belli and an excuse to wipe their hated enemies off the map.
    - TacticalCoder 3y ago
      > ... an excuse to wipe their hated enemies off the map 21% of Israel's population are arab-israeli muslims. How many jewish people are living in Iran? How many jewish people are living in the Gaza strip? Who hates who here?
      - javajosh 3y ago
        The composition of the population is less important to this calculus than the composition of Israel's political leadership. It was already known that Netanyahi/Likud allowed Hamas to grow stronger to prevent unification of Gaza and the West Bank. Allowing the Oct. 7 attack gave him every excuse to prosecute total war on the Gazans, while maintaining a great deal of moral and financial support, especially from the US and Britain. Allowing your enemy to take first blood in order to justify annihilating them is a ploy as old as time. Note also that there is a distinction between Hamas and Gaza. Prior to the invasion, Hamas had weak support among Gazans - I think in part because they understood that their extremism was to blame for the blockade and ongoing hardships in the region. It may also be because Hamas systematically embedded its military infrastructure in civilian areas, and they knew what this would mean for them if war broke out. So its particularly evil that Netanyahu propped up a weak Hamas and then invaded with the intention of wiping it out. He prevented the Gazans from voting out the extremists and saving themselves the experience of this atrocity. FWIW Netanyahu (or Israel, as a state) has never spoken once about wiping out "Arabs". Whereas Hamas' stated goal, as with Iran, is to wipe out Jews (and the West).
SpliffnCola 3y ago
[flagged]
cratermoon 3y ago
I was looking for a mention of the Strategic Defense Initiative, aka "Star Wars". Among the technical issues the program never overcame was the ability to adequately recognize incoming missiles and guide anti-missile defenses to the target. Much like the Igloo White and Assault Breaker systems mentioned in the article, it failed to distinguish decoys from real.
- jandrewrogers 3y ago
  > Among the technical issues the program never overcame was the ability to adequately recognize incoming missiles and guide anti-missile defenses to the target. This is factually inaccurate, both of these were proven capabilities several decades ago. The biggest technical issue with ballistic missile intercept was getting the new hypersonic rocket motors they wanted to use to respond to guidance commands with sufficient precision. It was a materials science problem; if you put the same package on a normal rocket motor it (demonstrably) worked just fine.
  - cratermoon 3y ago
    "respond to guidance commands with sufficient precision" is just restating the same problem. "if you put the same package on a normal rocket motor it (demonstrably) worked just fine." Except that the interceptor wouldn't be able to reach the target traveling at hypersonic speeds. In other words, the guidance didn't work. Either the interceptor missed because it was off target or it missed because it was too slow. Dressing it up in technicalities doesn't negate the failure for the purposes of SDI.
Aerbil313 3y ago
I’m shocked by the amount of taxpayer money gone to waste. So many unsuccessful projects, the infamous incompetence of Big Tech looks like nothing compared to US military industrial complex’s. So this was where all the surplus of Western civilization was going to for the last 3/4 of a century. Now the surplus is no more, and soon to turn negative as the critical resources and energy sources run out, I hope the US loses its global dominance as soon as possible. I’m sorry, but at no point in time have they been just rulers over planet Earth. Entire countries of mine have been demolished and entire populations have been killed/forced to migrate, so that you can buy the new Xbox to your child, and your neighbor can buy a new yacht.
- mandmandam 3y ago
  If anything, the numbers in the article undersell the scale of waste. Our Middle East clusterfuck has cost us at least 8 trillion dollars since 2003. Enough to convert the entire US to clean energy, feed every hungry person on the planet, and house every homeless American. With change. Instead it was spent on murdering millions, displacing tens of millions, and riddling generations of children with cancer and birth defects (again). It's so, so far beyond evil and stupid that even Noam Chomsky says there's no word for it.
- nebula8804 3y ago
  To be fair the US is like what 247 years old? Thats a toddler in the grand scheme of things. And the founding fathers did not have a lot of confidence in this thing lasting. [1]:https://www.digitalhistory.uh.edu/disp_textbook.cfm?smtID=2&psid=3245 https://www.digitalhistory.uh.edu/disp_textbook.cfm?smtID=2&... "At the end of the Constitutional Convention, George Washington said, "I do not expect the Constitution to last for more than 20 years." Today, the United States has the oldest written constitution in the world." If anything what the country has achieved over the years is pretty darn good. For a large part of its existence, the US also wasn't this super power that we know of today. The future is unwritten but maybe we will see an isolated US that is left to tend to only its own internal issues.
mschuster91 3y ago
[flagged]
- deleted 3y ago
  [deleted]
underlipton 3y ago
>Nevertheless, Hamas’s devastating attack on October 7 caught Shin Bet and the rest of Israel’s multibillion-dollar defense system entirely by surprise. The intelligence disaster was even more striking considering Hamas carried out much of its preparations in plain sight, including practice assaults on mock-ups of the border fence and Israeli settlements—activities that were openly reported. Hamas-led militant groups even posted videos of their training online. Israelis living close to the border observed and publicized these exercises with mounting alarm, but were ignored in favor of intelligence bureaucracies’ analyses and, by extension, the software that had informed them. Israeli conscripts, mostly young women, monitoring developments through the ubiquitous surveillance cameras along the Gaza border, composed and presented a detailed report on Hamas’s preparations to breach the fence and take hostages, only to have their findings dismissed as “an imaginary scenario.” The Israeli intelligence apparatus had for more than a year been in possession of a Hamas document that detailed the group’s plan for an attack. At some point you have to hazard the notion that they let it happen on purpose. "Wag the dog" trended around that time, and with Netanyahu's various woes, maybe they went ahead and built the Torment Nexus.
- deleted 3y ago
  [deleted]
deleted 3y ago
[deleted]
- deleted 3y ago
  [deleted]
Const-me 3y ago
I think the most important lesson, it’s borderline impossible to design any good system without clear use cases. Ukraine has these use cases, also high motivation to tackle them. Ukrainians are controlling battlefield with commodity computers https://en.defence-ua.com/news/how_the_kropyva_combat_control_system_helps_in_the_most_difficult_situations_fortified_positions_couldnt_save_russian_army-3646.html https://en.defence-ua.com/news/how_the_kropyva_combat_contro... They sunk multiple Russian warships with long-range naval drones https://www.bbc.com/news/world-europe-68528761 https://www.bbc.com/news/world-europe-68528761 They recently started large-scale testing of cheap flying drones with computer vision-based target recognition on board https://www.forbes.com/sites/davidhambling/2024/03/21/ukraine-rolls-out-target-seeking-terminator-drones/ https://www.forbes.com/sites/davidhambling/2024/03/21/ukrain... However, US is at peace. Which is a great thing by itself, but it means it’s too easy for them to waste billions of dollars developing technologies which look awesome in PowerPoint, but useless in practice.
- SteveNuts 3y ago
  This was a huge problem for the Nazis too, Hitler loved hugely complex and massive “super weapons” and wasted immense amounts of money and scientific effort to build them. The allies built practical and easy to maintain equipment in great quantities.
  - wavemode 3y ago
    I mean... the Allies also spent billions developing a superweapon. (And used German scientists to do it!)
    - spanktheuser 3y ago
      And it wasn’t even the most expensive. The Norden Bomb Sight cost slightly more than the Manhattan Project. B-29 development and production cost nearly 3x the cost of the fission bomb.
      - speed_spread 3y ago
kromem 3y ago
I'm getting really tired of writers crapping on 'AI' as if a static self-sufficient offering. Like no, the AI doesn't know everything other than what the terrorist is thinking. It summarizes what it's being fed. If a chatbot was being fed reports concerned about border activities then it's going to raise concern about border activities. This is an unnecessary and misleading angle to the article jumping on a bandwagon. The failure here is a broader failure of human intelligence across Western intelligence services in favor of contracts with third party defense contractors. There's a story for that. For "AI not knowing the terrorist mind" not much of a story.
- rossdavidh 3y ago
  The issue is, that many non-tech (and I'm starting to think also some tech) people believe that "AI" is an accurate label, and therefore that they can expect these algorithms to be able to think intelligently. The reason that it's called "AI" instead of, say, "large language models" (or whatever algorithm is being used), is precisely to create this impression of that capability, so as to sell the product. "Using artificial intelligence, the system analyzes behavior, predicts risks, raises alerts..." No, not very well, it doesn't. And this claim was not at all equivalent to "it summarizes what it's being fed".
  - Eisenstein 3y ago
    > The issue is, that many non-tech (and I'm starting to think also some tech) people believe that "AI" is an accurate label, and therefore that they can expect these algorithms to be able to think intelligently. Until we can define it, I think we should stop using the term 'intelligent' at all. It misleads people precisely because it means different things in different contexts. If something can comprehend language, solve word problems, get a really high score on the SATs and LSATs and translate perfectly from any language to any other, we could definitely say it is 'intelligent' in all of those contexts. Is it 'intelligent' in other contexts? Applying a technology that is really good at many things to things which it is not good at and selling that as a panacea is not a new idea. If we want it to change in this instance, we should start at least defining the terms we use so that we can determine the scope of its relevance to any area. Otherwise people make assumptions to their detriment and we can't agree even on what we are arguing about.
agomez314 3y ago
“no one appears to have noticed that Project Maven fit into the grand tradition of many other high-tech weapons projects: ecstatic claims of prowess coupled with a disregard for real-world experience”
- deleted 3y ago
  [deleted]
j16sdiz 3y ago
I don't understand why this have anything do with silicone valley or AI / AGI. It is just a classical confirmation bias.
OhMeadhbh 3y ago
Finally. someone talking sense about AI.
- deleted 3y ago
  [deleted]
est 3y ago
Looks like op staff were overwelmed by oncall duty false alarms. Yeah the best way to fix errors is to ... just ignore them. I think any sophisticated system that requires a bureaucratic staff to operate is doomed to fail.
- UberFly 3y ago
  That's the truth. In the case of October 7 (and 9/11 for that matter) lots of useful info was coming in, and regardless of the source, it was actively ignored. Human error at it's best. AI analysis is just another tool but ultimately we need competent or empowered people involved in the chain.
- causal 3y ago
  Yes, in any data product scenario it is extremely easy to find a signal and extremely difficult to validate that it's the most important one.
dosinga 3y ago
The examples in the article are rather cherry-picked. Failures in Vietnam can hardly be blamed on an IBM 360 only. The Hamas attack might have surprised Israel but the Iron Dome has been tech working well in recent years. The US warned anybody who wanted to listen (not many) that Russia was about to attack Ukraine. And it was a bunch of rather theoretical physicists who built the atomic bomb.
- deleted 3y ago
  [deleted]
- kurthr 3y ago
  Lots of failures are just human and political. Sure technology can obscure the obvious or highlight the unlikely, but it's just not that commonly influential (at least not yet, the day will come). The US even warned Russia of the attack in Moscow, but it was treated as political interference. That was almost certainly signals intelligence ignored. https://apnews.com/article/russia-intelligence-duty-to-warn-ab89b8b933c1c51b25903166951a1c7a https://apnews.com/article/russia-intelligence-duty-to-warn-...
  - somenameforme 3y ago
    The warning was quite broad, claiming that some group was planning some attack on some large gathering, including concerts, in Moscow, of which there are many. And it warned Americans to avoid large gatherings for the next 48 hours. That was on March 7th. The actual attack would only take place on March 24th. Incidentally, there is speculation that the attack may have been planned for March 9th. One of the terrorists was photographed at Crocus on the 7th, and on the 9th there was a large concert by Shaman - a patriotic Russian singer who's regularly made songs glorifying the war in Ukraine, performed for soldiers in Russia's claimed territories, and so on. This would also have coincided with just before the Russian elections, which happened on the 15th. But security was extremely high during that concert - very possibly in response to the US warning. By contrast when Russia warned the US about the Boston Bomber, the warning was precise to the point of even naming him.
    - 3y ago
hedora 3y ago
The subtitle is rather telling, when combined with the title: How Big Tech is losing the wars of the future The underlying assumption of the article is that we want AI to further centralize military power into the hands of fewer and fewer people. Whenever that goal has been achieved in the past, it has been disastrous for human rights, scientific progress, and things like life expectancies and food security. I’d rather Silicon Valley keep producing stuff like the printing press and gutenberg bible, and not work on reducing the costs of operating a new Spanish Inquisition or an S.S.-style surveillance apparatus. Even if you trust the current Pentagon, there’s some other government that would misuse the technology. Also, you have no way of knowing who will control the Pentagon in 50-100 years.
- deleted 3y ago
  [deleted]
quantum_state 3y ago
It seems difficult to escape from the eternal truth of measure and countermeasure … a fool with a tool is still a fool …
Barrin92 3y ago
It's just the usual technology obsession of military industrial and political types that's been around for decades. The reality is that the most important factor in combat is the human one and every fancy gadget you use just introduces more liability and weak points. The AI marketing hype and lobbying stuff fills the pockets of a few people but it doesn't make soldiers more effective, "cloud computing controls the battlefield" is such a meme worthy sentence I don't understand how anyone can take someone seriously who says that out loud. What you could see in the Israel-Hamas conflict mentioned in the article is what you also see with the Houthis or in Ukraine, that the best technology on the battlefield is cheap, resilient and simple enough to be understood and operated by the least competent soldier, not some 10 billion dollar fantasy tool out of a sci-fi novel. The example in the article of Hamas feeding Israeli informants deliberate misinformation to strengthen the notion that Hamas would not attack, now imagine this amplified by even more gullible LLM powered "intelligence analysts". It's a theme of the "AI age", the people who stand to benefit the most are critically thinking humans able to exploit the tool induced stupidity of everyone else. Hackers, appropriately enough.
gorgoiler 3y ago
”The AI system knows everything about Hamas: what they said, what they published […] it analyzes behavior, predicts risks, and raises alerts.” ”Well aware of this Hamas members fed their enemy the data that they wanted to hear. The AI system, it turned out, knew everything about the terrorist except what he was thinking.” When your opponent can see everything you do and hear everything you say, the only defence is privacy. In the novel The Three Body Problem this is taken to an extreme: the only privacy is inside the human mind and so select individuals are allowed to make decisions based on strategies known only to them which they have never said aloud. Science fiction has become reality.
- lupusreal 3y ago
  (That was from the sequel, The Dark Forest.)
surfingdino 3y ago
Any sufficiently advanced technology can be defeated with sticks and stones.
musha68k 3y ago
What aspects of modern warfare didn't Hideo Kojima foresee? >Another combat veteran, now with a Pentagon agency working on these issues, told me that the AI developers he works with didn’t seem to understand some of the requirements for the technology’s military application. “I don’t know if AI, or the sensors that feed it for that matter, will ever be capable of spontaneity or recognizing spontaneity,” he said. He cited a DARPA experiment in which a squad of Marines defeated an AI-governed robot that had been trained to detect them simply by altering their physical profiles. Two walked inside a large cardboard box. Others somersaulted. One wore the branches of a fir tree. All were able to approach over open ground and touch the robot without detection. Oh.. >I was curious about Palantir, whose stock indeed soared amid the 2023 AI frenzy. I had been told that the Israeli security sector’s AI systems might rely on Palantir’s technology. Furthermore, Shin Bet’s humiliating failure to predict the Hamas assault had not blunted the Israeli Defense Force’s appetite for the technology; the unceasing rain of bombs upon densely packed Gaza neighborhoods, according to a well-sourced report by Israeli reporter Yuval Abraham in +972 Magazine, was in fact partly controlled by an AI target-creation platform called the Gospel. The Gospel produces automatic recommendations for where to strike based on what the technology identifies as being connected with Hamas, such as the private home of a suspected rank-and-file member of the organization. It also calculates how many civilians, including women and children, would die in the process—which, as of this writing, amounted to at least twenty-two thousand people, some 70 percent of them women and children. One of Abraham’s intelligence sources termed the technology a “mass assassination factory.” Despite the high-tech gloss on the massacre, the result has been no different than the slaughter inflicted, with comparatively more primitive means, against Dresden and Tokyo during World War II.
- darkerside 3y ago
  This is horrible news. Blurring the lines of accountability between people and software in the industry of war is a recipe for Armageddon. It's not only genocide laundering, which is atrocious enough. Unchecked, it will lead to a "stand your ground" type of situation where countries may strike first in anticipation of other actions. I fear for the future.
  - 3y ago
thyrsus 3y ago
At the end of the article, Cockburn complains that asking ChatGPT about Palantir work with the IDF gets a hallucination in response. I just queried duckduckgo.com with "IDF Palantir", and receved links to several news articles from relatively mainstream news sources. If the point is that LLMs are currently unreliable, then sure. If the point is that we can't know whether Palantir is working with the IDF, then there is available evidence
bitwize 3y ago
> Nevertheless, Hamas’s devastating attack on October 7 caught Shin Bet and the rest of Israel’s multibillion-dollar defense system entirely by surprise. Somebody high up in the Israeli military was probably like, "After very careful consideration, sir, I've come to the conclusion that your new defense system sucks." https://www.youtube.com/watch?v=fyFB2p1yrQI https://www.youtube.com/watch?v=fyFB2p1yrQI
oglop 3y ago
I’m a combat vet from the infantry. I was a lowly enlisted, not an officer. I attended college after wards and now work in tech out west. I am amazed how confused, historically ignorant, but intelligent my coworkers are. It’s like they’ve never even heard of NATO, world war 1 or 2, and almost each one when they’ve asked what I did has then had the follow up question “what’s the infantry”. This used to seem cute, but lately I’ve wondered if it’s something to be concerned with. If the people who are being paid all this money and building all these products are incredibly intelligent but have zero wisdom, what does that look like when war comes? Do they just step aside, point at a stock price, and tell me I’m barbaric and living in a past century? I wonder about it almost daily now as the conflict looks more and more likely to spread and I see an electorate more and more isolationist and looking to repeat the mistakes of the past. Hmmm, not even sure my point. I guess it’s that I find many Americans insanely smart and intelligent, but they lack any wisdom but act as though they are all Martin Luther king jr when they pronounce a name correctly, or some such morally insignificant thing compared to life and death. Something is totally fucked with how people are valuing things. What I’m not sure, but when a US Republican senator named JD Vance (a fucking veteran!) starts going around literally spouting Kremlin propaganda lines about Ukraine, and conservatives agree and repeat it, I just feel sad about my society. Also, obviously some people do know these things, I’m being loose with language here. But a huge majority of the people have asked me those questions.
smsm42 3y ago
I'm not sure why they discussed Israel at all - except for bullshit accusation in racism which seems to come automatically nowdays. Israeli intelligence failure had zero to do with any technology and everything to do with political and ideological delusion. They had fake data, they had real data, they had all the data. They had tons of opinions and options. They had to choose which concept to believe and which conclusion to make. This is not something that depends on technology, and it can't be, that depends on the person or persons making the decision, and it is as unsolved problem now as if was millenia ago.

309 points by NDAjam 3y ago

> The system knows everything about [the terrorist]: where he went, who his friends are, who his family is, what keeps him busy, what he said and what he published. Using artificial intelligence, the system analyzes behavior, predicts risks, raises alerts. Where does "the terrorist" end and me, you and anyone else just minding our own business get inserted instead? And let's say it's not even the gov doing this but some private company with public data, what's to stop the gov from buying "reports" from that company. 100% legal. That is, no rights being violated, etc. Anyone who says, "I have nothing to hide" is a fool, at best.

humansareok1 3y ago
I don't support ubiquitous spying at all but are you hanging out with known ISIS members or members of White Nationalist Militias regularly? Because I'm pretty sure that's where the line begins.
- 7thaccount 3y ago
  I think they're just saying it's a slippery slope. It starts out with good intentions we all agree on, but then continues to slide and more and more of our freedoms erode as they crank up the boiling pot ever so slowly.
- AlexandrB 3y ago
  There's a lot of grey here. What does "hanging out" mean? If my weird uncle is (unknown to me) in ISIS does spending thanksgiving with him count as "hanging out"? ISIS is at least pretty specific, but what counts as a White Nationalist Militia? Both of these can be redefined to capture more and more of the population if desired.
  - forgotmyinfo 3y ago
    This is what we have attorneys and judges for. And no, obviously Thanksgiving isn't "hanging out". But going to the same weekly meeting and practicing lynching minorities? Yeah, that's a little more than just mashed potatoes and gravy, isn't it. (These contrived "whatabout" gotchas are exhausting. It is abundantly obvious who is and who is not involved with white nationalist militias.)
    - nurple 3y ago
      Yes, all the legal arguments presented before the FISA court by the lawyers working on behalf of those targeted have been really interesting reads!
      - dragonwriter 3y ago
        There aren't legal arguments by the targets in law enforcement search (either physical or wiretap) warrant cases either, that mainly only happens if (as does not always happen) the product of the search is used in criminal prosecution later.
    - sakjur 3y ago
      None of those things seem particularly obvious to me.
    - chasd00 3y ago
      It's really the court of public opinion that has the greatest risk of harm at the day to day level. A non-poc going to the gun range and then posting on social media could cause a "White Nationalist Militia" label to get attached by a jilted coworker and then go viral. That can cause serious harm.
      - mindslight 3y ago
        Maybe gun clubs should implement DEI programs. Then those pictures would have some colorful people in the background.
- A4ET8a8uTh0 3y ago
  If you don't know, who they are and just happen to serve a 'bad guy' ( hate that term ) a burger, should you be in the crosshairs? Because this is where this is heading. If you think I am overselling it, remember that police in US can ask for all users from specific location. To your point, if there is indeed a line, it need to be clearly articulated so that the rules of the game clear.
- hwbehrens 3y ago
  > I'm pretty sure that's where the line begins Based on what? From my perspective, the easiest way to design such a system would be to create entries for every 'actor' in the system, feed in as much data as you can get your hands on, and then let the weights sort themselves out. So for example, if you're hanging out with ISIS members obviously your weights would be higher, but even if you're a server at Applebees you'd still be in the system somewhere. Doing it the other way necessitates some kind of bright-line division, and any such boundary, once defined, becomes susceptible to exploitation. e.g. I won't hang out with the White Nationalist Militia because that puts me "into the system", but I can hang out with insert radical right-wing group where I can talk to 80% of the same people without being flagged. In practice, I imagine that the gradient of extremism is rather gradual and with blurred boundaries.
  - humansareok1 3y ago
    As another poster mentioned this is literally why we have courts. There is a clear line for obtaining a search warrant for example. Precedents exist.
    - raisin_churn 3y ago
      Are you familiar with FISC? I'd say go familiarize yourself with its case law, but you can't, because it's secret. And it authorizes methods much more powerful and invasive than a simple search warrant. Precedent exists, but nobody outside the national security state actually knows what it is.
      - humansareok1 3y ago
        You're maybe proposing another line where no spying is legal at all and we should just submit ourselves to the whims of terrorists or other lunatics? Surely there is actually a line where there are tradeoffs between security and privacy and its probably not 0% security and 100% privacy. Perhaps you think all FISA rulings should be public and any sufficiently savvy malicious actors can just read them to know exactly how to avoid suspicion?
      - 3y ago
- wpietri 3y ago
  Maybe that's where the line starts, but does it stay there? As an example, look at how the US's anti-communist fervor led to things like COINTELPRO: https://en.wikipedia.org/wiki/COINTELPRO https://en.wikipedia.org/wiki/COINTELPRO Especially today, I think we have to look at every power we might give to government and ask, "What happens if the worst people get access to this?" Because they're certainly going to try.
  - chiefalchemist 3y ago
    Did we give it, or was it taken? Yeah, maybe we consented to The Patriot Act. But when it was renewed, it was *expanded*. Too late now.
- chiefalchemist 3y ago
  You've heard of guilt by association, well guilt by co-location isn't that far off. Along the same lines, the rationale is going to be, "We need to track everyone so we can be sure to see all relationships and connections, and connections to connections, and so on. Try to draw the line wherever you want, but they're going to step over it, and never look back.
A4ET8a8uTh0 3y ago
I think, in a sense, that part is already over. Entities that encompass both ends of the spectrum exist and any remaining gaps are filled by public/private partnership ( and hailed as a great thing just about anywhere ). The scenario of a private corporate entity wielding that power has already come to fruition if you look at what Google or Facebook has available on its users. I think that is the main reason why I am not as.. restrictive on use of LLMs and AI, because I see it as a form evening out the playing field at least a little bit.
- nurple 3y ago
  I think one of the things that scares me most about API-accessed LLMs is how powerful they are as data collection tools in their own right. OpenAI, for example, recently updated their terms of use to be more vague about how they work with the gov and I have no doubts that giving the NSA access to conversational feeds is absolutely a requirement to their continued operation as an entity, a la lavabit. In fact, part of me thinks that the Sam/Ilya drama and sam's god complex are at least partially rooted in this, alleged, collaboration. Imagine the questions you could pose to a GPT trained on all the conversations had with users that's been enriched with their biographical data. These conversations are often intimate and curiosity driven in a way that seeking the truth could easily be framed as self-radicalization.
  - A4ET8a8uTh0 3y ago
    That API is one the main reasons companies are not as keen on jumping on the bandwagon. They don't want to have OpenAI to have access to their corporate data. But then, there are options of running models locally.. I think your concerns are valid.
nurple 3y ago
The terrorist ends wherever a threat to the state's power exists, it's been shown quite well that they don't care if you're domestic or not. This, IMO, is why "self-radicalized" and "domestic terrorist" were injected into our vernacular, to normalize and justify the need to surveil the general public. The thing is, and like I mentioned in a post awhile back: technically competent actors, the ones bound to cause the most harm, would absolutely be using a bespoke method of covert communication. There's really little value, IMO, to the countrywide dragnet outside of sentiment analysis and control. The military complex wove itself early into the tech industry in ways that they could intentionally side-step laws meant to keep such public/private collusion from happening[0]. The impetus for the founding of the collaboration was a report on the importance in controlling perception in future wars. We saw the same strategy deployed directly against the American people during the election "fortification" where DHS and social media colluded to control perception with little regard for truth[1]. [0] https://medium.com/insurge-intelligence/how-the-cia-made-google-e836451a959e https://medium.com/insurge-intelligence/how-the-cia-made-goo... [1] https://homeland.house.gov/2023/11/06/chairmen-green-bishop-uncover-new-evidence-of-dhs-big-tech-collaborating-to-censor-americans/ https://homeland.house.gov/2023/11/06/chairmen-green-bishop-...
- chiefalchemist 3y ago
  > The thing is, and like I mentioned in a post awhile back: technically competent actors, the ones bound to cause the most harm, would absolutely be using a bespoke method of covert communication. There's really little value, IMO, to the countrywide dragnet outside of sentiment analysis and control. Agreed. And yet their persistence to surveil continue to expand. They're not trying to watch those who are sure to be hiding. They're watching the rest under the guise of "We gotta get those terrorists."
deleted 3y ago
[deleted]

throwaway4good 3y ago

I don't understand the headline "problem" of the article. Or the "How Big Tech is losing the wars of the future". Silicon Valley has always been a part of the US military complex. Maybe there was a period sometime in the 90es where it was irrational exuberance and don't be evil. But now we are surely back under manners.

selimthegrim 3y ago
TIL ‘under (heavy) manners’
- 082349872349872 3y ago
  TIL 'put manners on' (in combination, it would appear that whatever 'manners' may be, they are located distal of the cranium)

tivert 3y ago

> Nevertheless, Hamas’s devastating attack on October 7 caught Shin Bet and the rest of Israel’s multibillion-dollar defense system entirely by surprise. The intelligence disaster was even more striking considering Hamas carried out much of its preparations in plain sight, including practice assaults on mock-ups of the border fence and Israeli settlements—activities that were openly reported. Hamas-led militant groups even posted videos of their training online. Israelis living close to the border observed and publicized these exercises with mounting alarm, but were ignored in favor of intelligence bureaucracies’ analyses and, by extension, the software that had informed them. Israeli conscripts, mostly young women, monitoring developments through the ubiquitous surveillance cameras along the Gaza border, composed and presented a detailed report on Hamas’s preparations to breach the fence and take hostages, only to have their findings dismissed as “an imaginary scenario.” The Israeli intelligence apparatus had for more than a year been in possession of a Hamas document that detailed the group’s plan for an attack. > Well aware of Israel’s intelligence methods, Hamas members fed their enemy the data that they wanted to hear, using informants they knew would report to the Israelis. They signaled that the ruling group inside Gaza was concentrating on improving the local economy by gaining access to the Israeli job market, and that Hamas had been deterred from action by Israel’s overwhelming military might. Such reports confirmed that Israel’s intelligence system had rigid assumptions of Hamas behavior, overlaid with a racial arrogance that considered Palestinians incapable of such a large-scale operation. AI, it turned out, knew everything about the terrorist except what he was thinking. That sounds a lot like a company that's implementing data-driven "best practices" from some expensive management consultants. It truly is the best system, regardless of how bad the results are. It's best by definition.

deleted 3y ago
[deleted]
wpietri 3y ago
Ooh, very interesting point: > That sounds a lot like a company that's implementing data-driven "best practices" from some expensive management consultants. > > It truly is the best system, regardless of how bad the results are. It's best by definition. Well that rings some bells. It's as if there's a religion where the sacred totem is a graph that goes up and to the right. Some question for the crowd: How do systems like this insulate themselves from failure? Before something goes wrong, what prevents seeing the problem? And after something goes wrong, what are the words and behaviors used to avoid fundamental change?

burutthrow1234 3y ago

[flagged]

orange_joe 3y ago

Since the article talks about the failure of AI in the context of the 10/7 I think it’s worth discussing the situation directly. Everything points to the Israelis not having taken their security seriously beyond the tactical level. I’m certain they thwarted other attacks, but it was an inevitability that a major attack was successful at some point. Such an attack would necessitate a military response. However the Israelis have no strategic vision. They lacked serious plans for such an eventuality and still lack a serious goal for their invasion of Gaza. They haven’t articulated anything that indicates a vision to meaningfully change the situation from the 10/6 state to something more sustainable. Therefore, it doesn’t seem like a reasonable takeaway to say AI failed.

nkozyra 3y ago
> Therefore, it doesn’t seem like a reasonable takeaway to say AI failed. There are a lot of reasons - from quite intuitive to conspiratorial - to not take the idea that AI caused or meaningfully contributed to this failure at face value. Or that it was a failure of intelligence in the first place.
cameldrv 3y ago
To be fair, the lack of strategic vision has also plagued the U.S. since WWII or Korea. We just keep losing wars because no one ever sets out clear achievable goals. The notable exception was the Powell Doctrine in Desert Storm. For that one, the goal was to kick Iraq out of Kuwait and restore the Kuwaiti monarchy, which was achieved. If you look especially at Afghanistan, Libya, and Syria, there is this magical thinking that if we destroy the Evil Dictator and run an election, that everyone will naturally vote to ally with the U.S. and completely change their social organization to be in accordance with western values. The place we spent the most time in the 21st century, Afghanistan, somehow went from an objective of destroying Al Qaeda to ensuring that girls got a good education and had equal rights. That sort of societal transformation is not possible even with 100,000 troops when they don't even speak the local language. Can you imagine the hubris of trying to tell people in some remote village that the way men and women relate to each other has to change through a translator, because some tall buildings in a place they've never heard of got destroyed? The obvious result was total failure and the Taliban picking up right where they left off in 2001.

deleted 3y ago

[deleted]

nkozyra 3y ago

I think - like a lot of media reporting on the space - this overgeneralizes (heh) artificial intelligence. The predictive aspects of ML have been in use in modern militaries for _decades_, and the opening graf handwavely indicates that an LLM was a bigger chunk of the perceived intelligence failure of the October 7 attack. That an LLM is a part of a system that includes a large amount of ML is not surprising. It's a great human interface. Do I for a second believe that it played a much larger role, such to be implied as responsible in any non-negligble way for missing the attack. Of course not. My point here is that ML continues to play a role, ML continues to both succeed and fail, and ML will continue to be imperfect, even moreso as it competes against adversarial ML. Blaming imperfect tools for inevitable failures is not a useful exercise, and certainly not a "problem" considering the alternative being even more failure-prone humans.

rossdavidh 3y ago
Blaming the excessively grand claims that were made for those tools, however, is absolutely a useful exercise.
- jonchurch_ 3y ago
  But grand claims made by technologists are nothing new. Certainly I don’t know, Ive never been in the military, but aren’t people always trying to sell The Next Big Thing to the military? Is it not the responsibility of those in charge to evaluate the capabilities and limitations of new systems being integrated into their forces? If someone said “we dont need the rigor we used to have anymore, we have AI” I see that as a failure of the org, not an indictment of the claims being put forth by boosters. Corporate Decision Maker #2, sure, theyll get hoodwinked. They and their company may have only 50 years of experience and institutional memory to draw on. But State Militaries? What excuse do they have? War changes, but the armed forces have a long memory, and their poor decisions cost lives. Maybe Im off base, but I would expect each mistake to be an opportunity to learn for that industry. The industry has had plenty of lessons learned over the past 100 years. Why is the latest hype cycle to blame, and not those whose job it is to ensure they maintain capabilities and extensively game out scenarios and responses? Bad bets on tech happen even in institutions with lifetimes of history to draw on, but I see that as a failure of the institution, not on the completely mundane hype cycles which occur naturally. Obviously mistakes happen, and maybe thats what the article is getting at. But if we’re going to point fingers (not saying you are) then lets not let decision makers off the hook whose job is to prevent that hot new thing getting their people killed.

mattnewton 3y ago

I can’t speak for Israeli tech, but the pentagon has an image problem in the valley, I don’t believe they are getting the best recruits even for contracting companies like Palintir. Our generation is closer to Iraq and Vietnam than WW2, and many of the bright minds are first generation immigrants. Despite the more recent image problems ad tech has (now that people are seeing more of how the sausage is made), it’s still sexier to work on big consumer companies than defense. You’d have to pay my colleagues more to work for the US government, even indirectly, instead it’s often less (and often with less freedoms of what they do off the clock). And now, what I’m reading is that if you do go contract for the military in AI, your function is partially some kind of scapegoat insurance. Blame those eggheads with their computers who can be fooled, not the fools who hired them and acted on that signal above others I guess? The idea that a chatGPT model would have been a deciding factor in preventing 10/7 is laughable on its face to anyone who works in the industry, except maybe a consultant selling LLMs to the IDF.

jajko 3y ago
I am still not 100% convinced they didnt just let it happen on purpose (and then were surprised just by the scale), having an excuse to raze the place down for good, which is exactly what they are doing. The signs were there, everywhere, and mosad aint bunch of clueless paper pushers. The guy in charge is former spec ops, murder of anybody without battling an eye is part of the deal so dont expect some humanism from that direction. If I didnt read similar stories from other times and places, where it played almost exactly like this... AI is not going to solve political issues, just make them more complex than they already are
kjellsbells 3y ago
There's another issue here as well, which is that many of the tech folks who would be ok working for the government, even at reduced rates, cant get through the hiring morass that uncle sam puts up. The fed gov simply isnt set up to quickly acquired talent from industry. They also remain remarkably hidebound by old rules like requiring advanced degrees for senior positions.
- CoastalCoder 3y ago

AndyMcConachie 3y ago

[flagged]

shitpostbot 3y ago
[dead]

A4ET8a8uTh0 3y ago

I personally think this is the most interesting part of the entire article: 'He then focused on defense work, lamenting that people with the relevant tech skills to build the weapons of the future were “largely refusing to work with the defense sector.”' I wonder to what extent that is still true. There is clearly a lot of money flowing and some definitely followed the money ( Palantir exists after all ).

gamepsys 3y ago
It's clearly true to some degree, there are documented cases of people that refused to work with the defense sector at great personal costs. The questions are how much resistance is there in the labor force, and how does that impact the ability to recruit talent?
- aleph_minus_one 3y ago
  > The questions are how much resistance is there in the labor force, and how does that impact the ability to recruit talent? Easy: Give potential employees similar salaries to MAMAA companies, and a similar amount of freedom and independence (at least in the ways in which it is possible at a defense company) as it existed in the early days of Google and Facebook, and I think a lot of potential employees (though of course not all, but this is not necessary) will "forget" their initial moral objections and go for the money. :-)
  - A4ET8a8uTh0 3y ago
    Wasn't DARPA kinda close to that idea ( I am honestly not sure, but it seemed like a lot of interesting stuff came from there )? Still, a person who knows what he/she is building can likely predict how it is going to be used. Would I want to be responsible for popularizing portable black hole generators?
    - aleph_minus_one 3y ago
      > Still, a person who knows what he/she is building can likely predict how it is going to be used. Would I want to be responsible for popularizing portable black hole generators? You just developed an insanely small part of this machinery. Compartmentalization of the work appeases the mind a lot. :-( If you still have bad feelings, there exists the charity-industrial complex: donate some decent paycheck to give a poor, starving child a better life - something that you could not have done if you hadn't accepted the well-paid defense contract.

cameldrv 3y ago

I have no idea how Silicon Valley could be held responsible for an Israeli intelligence failure. Israel is not a part of the U.S. The author exhibits essentially zero knowledge of the advances in military intelligence in the past 10-20 years. He’s talking about problems in the Vietnam war and IBM 360 mainframes as if all of the stuff Macnamara dreamed of weren’t daily reality now.

luaybs 3y ago
> Israel is not a part of the U.S. The US sends $3.8B in military aid to Israel yearly...
- wslh 3y ago
  It is not too much if you think Israel is covering US military mistakes and cybersecurity (e.g. Iran).
  - lostlogin 3y ago
    Isn’t this policing of Iran needed because of the US supporting Israel? I know there are other interests (oil), but the complete freedom to do anything that the US gives Israel is not entirely helpful to the US.
    - wslh 3y ago
      Isn’t this policing of Iran needed because of the US supporting Israel? No, do you remember the terrorists attacks in USA and in Europe? Have you watched the Argo movie? Do you know about terrorist attacks in Latin America where Iran is involved? Seems people don't check history beyond Israel (21k km^2)and Jews (16m).
      - sangnoir 3y ago
        I remember the JPCOA and egging on the Trump administration to break the agreement and the subsequent escalations.

neonate 3y ago

http://web.archive.org/web/20240327152111/https://harpers.org/archive/2024/03/the-pentagons-silicon-valley-problem-andrew-cockburn/ http://web.archive.org/web/20240327152111/https://harpers.or...

alephnerd 3y ago

No offense, but this article is MASSIVE BS. There are issues with innovation in the DoD and DHS, but a lot of this is offloaded to private sector vendors anyhow. I notice how the article didn't mention any of the companies I personally know doing stuff in the space, nor actually sourced from members of the VC, Business, or Defense community. The fact that the author took Palantir's marketing at face value is proof enough - the CIA let their contract with Palantir lapse a couple years ago (and I think they only even bought it because of their stake in In-Q-Tel), and they haven't had great success selling to the Fed. I actually work in this space btw. ----- The bigger stumbling block is procurement. Software Procurement by Federal standards is relatively straightforward so a Series E+ startup can make it if they spend around $7-10M and 1-1.5 years on a dedicated roadmap for FedRamp and FIPS compliance. Once you step out of software, procurement becomes paperwork hell. Throw in the paperwork hell from Grantmakers like the DoD and DoE, and you end up with a quasi-Soviet procurement system. Ironically, most of these compliance and regulatory checks were added for good intentions - primarily to minimize corruption and graft, yet it basically clogged up the entire system, and dissuades startups and innovators from working directly with the Defense community. Some projects like DIUx and and In-Q-Tel are trying to change that, but it's too little too late, and our defense base is entirely dependent on firms like Microsoft, Cisco, Crowdstrike, Zscaler, etc acquiring promising startups to evangelize their innovations internally. Fundamentally, this is why I dislike the New America/Khan/Chopra vision of anti-trust. It doesn't actually help innovation from a federal standpoint, as small companies and startups have no reason to work with the Fed given the amount of red tape that exists. If the same effort was put to harmonizing and simplifying procurement across the Federal Government, you could directly make demands on competition. This is what China does, and is a major reason their MIC was able to grow leaps and bounds in just 20 years.

nceqs3 3y ago
The way Palantir talks about the CIA really rubs me the wrong way. For years, they would leak to journalists that Palantir "found bin Laden" when, of course, it had nothing to do with finding him. Several CIA employees died trying to find Bin Laden, all for some schmucks in Silicon Valley to try and capitalize on their sacrifice.

outside1234 3y ago

The thing we should all really be terrified about is how Trump and Stephen Miller will use of all of this technology we have built against us when elected.

deleted 3y ago
[deleted]

sweeter 3y ago

[flagged]

yieldcrv 3y ago
I mean you’ve seen how the people subscribed to that conflict talk about each other, they need a therapist The view is that nobody are civilians, anyone counting deaths is a liar at any count, and surrendering hostages get mistaken for palestinian civilians which means militants Nobody can beat our hostage rescue high score right now so we can just ignore them
deleted 3y ago
[deleted]

cess11 3y ago

'Caught by surprise' is a weird description. Israeli press has repeatedly run stories about how frontline analysts sounding the alarm were ignored. That could be due to things like sexism, ageism or discrimination against conscripts, or it could be due to the settler organisations having their people in government and a strong wish to resettle the Gaza strip. Either way, the signals were there, they had been watching the preparations and exercises for a year or so. Even if the resistance groups had kept that secret even a mediocre officer in intelligence or the army should be able to conclude from 'first principles' and what they were doing that there would eventually be a violent response.

bitcharmer 3y ago
[flagged]
Spooky23 3y ago
This is just a blaming the wrong tools. The people running the Israeli government and army are tools. They fucked up, plain and simple. Whether through malice or just ineptitude and incompetence, they failed.
- dmurray 3y ago
  Or, they intentionally ignored the intelligence hoping for a casus belli and an excuse to wipe their hated enemies off the map.
  - TacticalCoder 3y ago
    > ... an excuse to wipe their hated enemies off the map 21% of Israel's population are arab-israeli muslims. How many jewish people are living in Iran? How many jewish people are living in the Gaza strip? Who hates who here?
    - javajosh 3y ago
      The composition of the population is less important to this calculus than the composition of Israel's political leadership. It was already known that Netanyahi/Likud allowed Hamas to grow stronger to prevent unification of Gaza and the West Bank. Allowing the Oct. 7 attack gave him every excuse to prosecute total war on the Gazans, while maintaining a great deal of moral and financial support, especially from the US and Britain. Allowing your enemy to take first blood in order to justify annihilating them is a ploy as old as time. Note also that there is a distinction between Hamas and Gaza. Prior to the invasion, Hamas had weak support among Gazans - I think in part because they understood that their extremism was to blame for the blockade and ongoing hardships in the region. It may also be because Hamas systematically embedded its military infrastructure in civilian areas, and they knew what this would mean for them if war broke out. So its particularly evil that Netanyahu propped up a weak Hamas and then invaded with the intention of wiping it out. He prevented the Gazans from voting out the extremists and saving themselves the experience of this atrocity. FWIW Netanyahu (or Israel, as a state) has never spoken once about wiping out "Arabs". Whereas Hamas' stated goal, as with Iran, is to wipe out Jews (and the West).

SpliffnCola 3y ago

[flagged]

cratermoon 3y ago

I was looking for a mention of the Strategic Defense Initiative, aka "Star Wars". Among the technical issues the program never overcame was the ability to adequately recognize incoming missiles and guide anti-missile defenses to the target. Much like the Igloo White and Assault Breaker systems mentioned in the article, it failed to distinguish decoys from real.

jandrewrogers 3y ago
> Among the technical issues the program never overcame was the ability to adequately recognize incoming missiles and guide anti-missile defenses to the target. This is factually inaccurate, both of these were proven capabilities several decades ago. The biggest technical issue with ballistic missile intercept was getting the new hypersonic rocket motors they wanted to use to respond to guidance commands with sufficient precision. It was a materials science problem; if you put the same package on a normal rocket motor it (demonstrably) worked just fine.
- cratermoon 3y ago
  "respond to guidance commands with sufficient precision" is just restating the same problem. "if you put the same package on a normal rocket motor it (demonstrably) worked just fine." Except that the interceptor wouldn't be able to reach the target traveling at hypersonic speeds. In other words, the guidance didn't work. Either the interceptor missed because it was off target or it missed because it was too slow. Dressing it up in technicalities doesn't negate the failure for the purposes of SDI.

Aerbil313 3y ago

I’m shocked by the amount of taxpayer money gone to waste. So many unsuccessful projects, the infamous incompetence of Big Tech looks like nothing compared to US military industrial complex’s. So this was where all the surplus of Western civilization was going to for the last 3/4 of a century. Now the surplus is no more, and soon to turn negative as the critical resources and energy sources run out, I hope the US loses its global dominance as soon as possible. I’m sorry, but at no point in time have they been just rulers over planet Earth. Entire countries of mine have been demolished and entire populations have been killed/forced to migrate, so that you can buy the new Xbox to your child, and your neighbor can buy a new yacht.

mandmandam 3y ago
If anything, the numbers in the article undersell the scale of waste. Our Middle East clusterfuck has cost us at least 8 trillion dollars since 2003. Enough to convert the entire US to clean energy, feed every hungry person on the planet, and house every homeless American. With change. Instead it was spent on murdering millions, displacing tens of millions, and riddling generations of children with cancer and birth defects (again). It's so, so far beyond evil and stupid that even Noam Chomsky says there's no word for it.
nebula8804 3y ago
To be fair the US is like what 247 years old? Thats a toddler in the grand scheme of things. And the founding fathers did not have a lot of confidence in this thing lasting. [1]:https://www.digitalhistory.uh.edu/disp_textbook.cfm?smtID=2&psid=3245 https://www.digitalhistory.uh.edu/disp_textbook.cfm?smtID=2&... "At the end of the Constitutional Convention, George Washington said, "I do not expect the Constitution to last for more than 20 years." Today, the United States has the oldest written constitution in the world." If anything what the country has achieved over the years is pretty darn good. For a large part of its existence, the US also wasn't this super power that we know of today. The future is unwritten but maybe we will see an isolated US that is left to tend to only its own internal issues.

mschuster91 3y ago

[flagged]

deleted 3y ago
[deleted]

underlipton 3y ago

deleted 3y ago
[deleted]

deleted 3y ago

[deleted]

deleted 3y ago
[deleted]

Const-me 3y ago

I think the most important lesson, it’s borderline impossible to design any good system without clear use cases. Ukraine has these use cases, also high motivation to tackle them. Ukrainians are controlling battlefield with commodity computers https://en.defence-ua.com/news/how_the_kropyva_combat_control_system_helps_in_the_most_difficult_situations_fortified_positions_couldnt_save_russian_army-3646.html https://en.defence-ua.com/news/how_the_kropyva_combat_contro... They sunk multiple Russian warships with long-range naval drones https://www.bbc.com/news/world-europe-68528761 https://www.bbc.com/news/world-europe-68528761 They recently started large-scale testing of cheap flying drones with computer vision-based target recognition on board https://www.forbes.com/sites/davidhambling/2024/03/21/ukraine-rolls-out-target-seeking-terminator-drones/ https://www.forbes.com/sites/davidhambling/2024/03/21/ukrain... However, US is at peace. Which is a great thing by itself, but it means it’s too easy for them to waste billions of dollars developing technologies which look awesome in PowerPoint, but useless in practice.

SteveNuts 3y ago
This was a huge problem for the Nazis too, Hitler loved hugely complex and massive “super weapons” and wasted immense amounts of money and scientific effort to build them. The allies built practical and easy to maintain equipment in great quantities.
- wavemode 3y ago
  I mean... the Allies also spent billions developing a superweapon. (And used German scientists to do it!)
  - spanktheuser 3y ago
    And it wasn’t even the most expensive. The Norden Bomb Sight cost slightly more than the Manhattan Project. B-29 development and production cost nearly 3x the cost of the fission bomb.
    - speed_spread 3y ago

kromem 3y ago

I'm getting really tired of writers crapping on 'AI' as if a static self-sufficient offering. Like no, the AI doesn't know everything other than what the terrorist is thinking. It summarizes what it's being fed. If a chatbot was being fed reports concerned about border activities then it's going to raise concern about border activities. This is an unnecessary and misleading angle to the article jumping on a bandwagon. The failure here is a broader failure of human intelligence across Western intelligence services in favor of contracts with third party defense contractors. There's a story for that. For "AI not knowing the terrorist mind" not much of a story.

rossdavidh 3y ago
The issue is, that many non-tech (and I'm starting to think also some tech) people believe that "AI" is an accurate label, and therefore that they can expect these algorithms to be able to think intelligently. The reason that it's called "AI" instead of, say, "large language models" (or whatever algorithm is being used), is precisely to create this impression of that capability, so as to sell the product. "Using artificial intelligence, the system analyzes behavior, predicts risks, raises alerts..." No, not very well, it doesn't. And this claim was not at all equivalent to "it summarizes what it's being fed".
- Eisenstein 3y ago
  > The issue is, that many non-tech (and I'm starting to think also some tech) people believe that "AI" is an accurate label, and therefore that they can expect these algorithms to be able to think intelligently. Until we can define it, I think we should stop using the term 'intelligent' at all. It misleads people precisely because it means different things in different contexts. If something can comprehend language, solve word problems, get a really high score on the SATs and LSATs and translate perfectly from any language to any other, we could definitely say it is 'intelligent' in all of those contexts. Is it 'intelligent' in other contexts? Applying a technology that is really good at many things to things which it is not good at and selling that as a panacea is not a new idea. If we want it to change in this instance, we should start at least defining the terms we use so that we can determine the scope of its relevance to any area. Otherwise people make assumptions to their detriment and we can't agree even on what we are arguing about.

agomez314 3y ago

“no one appears to have noticed that Project Maven fit into the grand tradition of many other high-tech weapons projects: ecstatic claims of prowess coupled with a disregard for real-world experience”

deleted 3y ago
[deleted]

j16sdiz 3y ago

I don't understand why this have anything do with silicone valley or AI / AGI. It is just a classical confirmation bias.

OhMeadhbh 3y ago

Finally. someone talking sense about AI.

deleted 3y ago
[deleted]

est 3y ago

Looks like op staff were overwelmed by oncall duty false alarms. Yeah the best way to fix errors is to ... just ignore them. I think any sophisticated system that requires a bureaucratic staff to operate is doomed to fail.

UberFly 3y ago
That's the truth. In the case of October 7 (and 9/11 for that matter) lots of useful info was coming in, and regardless of the source, it was actively ignored. Human error at it's best. AI analysis is just another tool but ultimately we need competent or empowered people involved in the chain.
causal 3y ago
Yes, in any data product scenario it is extremely easy to find a signal and extremely difficult to validate that it's the most important one.

dosinga 3y ago

The examples in the article are rather cherry-picked. Failures in Vietnam can hardly be blamed on an IBM 360 only. The Hamas attack might have surprised Israel but the Iron Dome has been tech working well in recent years. The US warned anybody who wanted to listen (not many) that Russia was about to attack Ukraine. And it was a bunch of rather theoretical physicists who built the atomic bomb.

deleted 3y ago
[deleted]
kurthr 3y ago
Lots of failures are just human and political. Sure technology can obscure the obvious or highlight the unlikely, but it's just not that commonly influential (at least not yet, the day will come). The US even warned Russia of the attack in Moscow, but it was treated as political interference. That was almost certainly signals intelligence ignored. https://apnews.com/article/russia-intelligence-duty-to-warn-ab89b8b933c1c51b25903166951a1c7a https://apnews.com/article/russia-intelligence-duty-to-warn-...
- somenameforme 3y ago
  The warning was quite broad, claiming that some group was planning some attack on some large gathering, including concerts, in Moscow, of which there are many. And it warned Americans to avoid large gatherings for the next 48 hours. That was on March 7th. The actual attack would only take place on March 24th. Incidentally, there is speculation that the attack may have been planned for March 9th. One of the terrorists was photographed at Crocus on the 7th, and on the 9th there was a large concert by Shaman - a patriotic Russian singer who's regularly made songs glorifying the war in Ukraine, performed for soldiers in Russia's claimed territories, and so on. This would also have coincided with just before the Russian elections, which happened on the 15th. But security was extremely high during that concert - very possibly in response to the US warning. By contrast when Russia warned the US about the Boston Bomber, the warning was precise to the point of even naming him.
  - 3y ago

hedora 3y ago

The subtitle is rather telling, when combined with the title: How Big Tech is losing the wars of the future The underlying assumption of the article is that we want AI to further centralize military power into the hands of fewer and fewer people. Whenever that goal has been achieved in the past, it has been disastrous for human rights, scientific progress, and things like life expectancies and food security. I’d rather Silicon Valley keep producing stuff like the printing press and gutenberg bible, and not work on reducing the costs of operating a new Spanish Inquisition or an S.S.-style surveillance apparatus. Even if you trust the current Pentagon, there’s some other government that would misuse the technology. Also, you have no way of knowing who will control the Pentagon in 50-100 years.

deleted 3y ago
[deleted]

quantum_state 3y ago

It seems difficult to escape from the eternal truth of measure and countermeasure … a fool with a tool is still a fool …

Barrin92 3y ago

It's just the usual technology obsession of military industrial and political types that's been around for decades. The reality is that the most important factor in combat is the human one and every fancy gadget you use just introduces more liability and weak points. The AI marketing hype and lobbying stuff fills the pockets of a few people but it doesn't make soldiers more effective, "cloud computing controls the battlefield" is such a meme worthy sentence I don't understand how anyone can take someone seriously who says that out loud. What you could see in the Israel-Hamas conflict mentioned in the article is what you also see with the Houthis or in Ukraine, that the best technology on the battlefield is cheap, resilient and simple enough to be understood and operated by the least competent soldier, not some 10 billion dollar fantasy tool out of a sci-fi novel. The example in the article of Hamas feeding Israeli informants deliberate misinformation to strengthen the notion that Hamas would not attack, now imagine this amplified by even more gullible LLM powered "intelligence analysts". It's a theme of the "AI age", the people who stand to benefit the most are critically thinking humans able to exploit the tool induced stupidity of everyone else. Hackers, appropriately enough.

gorgoiler 3y ago

”The AI system knows everything about Hamas: what they said, what they published […] it analyzes behavior, predicts risks, and raises alerts.” ”Well aware of this Hamas members fed their enemy the data that they wanted to hear. The AI system, it turned out, knew everything about the terrorist except what he was thinking.” When your opponent can see everything you do and hear everything you say, the only defence is privacy. In the novel The Three Body Problem this is taken to an extreme: the only privacy is inside the human mind and so select individuals are allowed to make decisions based on strategies known only to them which they have never said aloud. Science fiction has become reality.

lupusreal 3y ago
(That was from the sequel, The Dark Forest.)

surfingdino 3y ago

Any sufficiently advanced technology can be defeated with sticks and stones.

musha68k 3y ago

What aspects of modern warfare didn't Hideo Kojima foresee? >Another combat veteran, now with a Pentagon agency working on these issues, told me that the AI developers he works with didn’t seem to understand some of the requirements for the technology’s military application. “I don’t know if AI, or the sensors that feed it for that matter, will ever be capable of spontaneity or recognizing spontaneity,” he said. He cited a DARPA experiment in which a squad of Marines defeated an AI-governed robot that had been trained to detect them simply by altering their physical profiles. Two walked inside a large cardboard box. Others somersaulted. One wore the branches of a fir tree. All were able to approach over open ground and touch the robot without detection. Oh.. >I was curious about Palantir, whose stock indeed soared amid the 2023 AI frenzy. I had been told that the Israeli security sector’s AI systems might rely on Palantir’s technology. Furthermore, Shin Bet’s humiliating failure to predict the Hamas assault had not blunted the Israeli Defense Force’s appetite for the technology; the unceasing rain of bombs upon densely packed Gaza neighborhoods, according to a well-sourced report by Israeli reporter Yuval Abraham in +972 Magazine, was in fact partly controlled by an AI target-creation platform called the Gospel. The Gospel produces automatic recommendations for where to strike based on what the technology identifies as being connected with Hamas, such as the private home of a suspected rank-and-file member of the organization. It also calculates how many civilians, including women and children, would die in the process—which, as of this writing, amounted to at least twenty-two thousand people, some 70 percent of them women and children. One of Abraham’s intelligence sources termed the technology a “mass assassination factory.” Despite the high-tech gloss on the massacre, the result has been no different than the slaughter inflicted, with comparatively more primitive means, against Dresden and Tokyo during World War II.

darkerside 3y ago
This is horrible news. Blurring the lines of accountability between people and software in the industry of war is a recipe for Armageddon. It's not only genocide laundering, which is atrocious enough. Unchecked, it will lead to a "stand your ground" type of situation where countries may strike first in anticipation of other actions. I fear for the future.
- 3y ago

thyrsus 3y ago

At the end of the article, Cockburn complains that asking ChatGPT about Palantir work with the IDF gets a hallucination in response. I just queried duckduckgo.com with "IDF Palantir", and receved links to several news articles from relatively mainstream news sources. If the point is that LLMs are currently unreliable, then sure. If the point is that we can't know whether Palantir is working with the IDF, then there is available evidence

bitwize 3y ago

> Nevertheless, Hamas’s devastating attack on October 7 caught Shin Bet and the rest of Israel’s multibillion-dollar defense system entirely by surprise. Somebody high up in the Israeli military was probably like, "After very careful consideration, sir, I've come to the conclusion that your new defense system sucks." https://www.youtube.com/watch?v=fyFB2p1yrQI https://www.youtube.com/watch?v=fyFB2p1yrQI

oglop 3y ago

I’m a combat vet from the infantry. I was a lowly enlisted, not an officer. I attended college after wards and now work in tech out west. I am amazed how confused, historically ignorant, but intelligent my coworkers are. It’s like they’ve never even heard of NATO, world war 1 or 2, and almost each one when they’ve asked what I did has then had the follow up question “what’s the infantry”. This used to seem cute, but lately I’ve wondered if it’s something to be concerned with. If the people who are being paid all this money and building all these products are incredibly intelligent but have zero wisdom, what does that look like when war comes? Do they just step aside, point at a stock price, and tell me I’m barbaric and living in a past century? I wonder about it almost daily now as the conflict looks more and more likely to spread and I see an electorate more and more isolationist and looking to repeat the mistakes of the past. Hmmm, not even sure my point. I guess it’s that I find many Americans insanely smart and intelligent, but they lack any wisdom but act as though they are all Martin Luther king jr when they pronounce a name correctly, or some such morally insignificant thing compared to life and death. Something is totally fucked with how people are valuing things. What I’m not sure, but when a US Republican senator named JD Vance (a fucking veteran!) starts going around literally spouting Kremlin propaganda lines about Ukraine, and conservatives agree and repeat it, I just feel sad about my society. Also, obviously some people do know these things, I’m being loose with language here. But a huge majority of the people have asked me those questions.

smsm42 3y ago

I'm not sure why they discussed Israel at all - except for bullshit accusation in racism which seems to come automatically nowdays. Israeli intelligence failure had zero to do with any technology and everything to do with political and ideological delusion. They had fake data, they had real data, they had all the data. They had tons of opinions and options. They had to choose which concept to believe and which conclusion to make. This is not something that depends on technology, and it can't be, that depends on the person or persons making the decision, and it is as unsolved problem now as if was millenia ago.