5 ms·

What's the issue? Everything about this is normal and expected when prototyping new capabilities for DoD. DoD intentionally pushes hard to get testable capabil

What's the issue? Everything about this is normal and expected when prototyping new capabilities for DoD. DoD intentionally pushes hard to get testable capabilities as early as possible to shorten feedback loops, understanding that features ancillary to the capability will be limited, stubbed out, or implemented using a stopgap that you would never use in production. This will all be cleaned up in the production implementation once everyone is happy with how the capability works. Basically an agile customer development approach, similar to what is used in startups. In my experience, the fine-grained control and security features are never implemented in the prototypes. This can be extremely fussy and slow development that isn't needed to evaluate capability. It also requires a lot of customer involvement, so they usually aren't willing to invest the time until they are satisfied that they want to move forward with the capability. The security architecture is demonstrably the kind of thing that can be mechanically added later so DoD takes the view that there is no development risk by not implementing it in the prototype. There may be fair criticisms of the system but it looks like the article is going out of its way to mislead and misrepresent.

ranadomo 1y ago
This is an absurd statement to make about a DOD program. Securing communication is communication in the most adversarial environment imaginable. There are thousands of problems that only emerge once cryptographic authentication and authorization are enabled. This isn't a b2b sass app where you can just add an "auth layer" once the api is built out. It's passing mission-critical messages through signal-jamming and unimaginably hostile imitation scenarios. Many DOD programs are built as prototypes that don't ever factor security in the architecture, and then have massive problems and delays trying to implement it later. DOD cyber acquisition is run by the most incompetent clowns on earth. The only reason they don't know how terrible their software is, is because they're not capable of detecting how fundamentally compromised their systems are, and China and Russia are not exactly white-hats looking for a bounty.
- wyldberry 1y ago
  It is not absurd, this is the standard for rapid prototyping in the DoD. Given that Palantir already has a strong track record for authn/z inside their systems used across the DoD, LEO, and Intelligence Agencies. It's not an untread, uncharted path for these organizations.
- jandrewrogers 1y ago
  I have first-hand experience with the problems of designing systems for adversarial denied environments. These are largely orthogonal to the problem of access controls. The low-level communication security and channel capacity is handled almost exclusively by external trusted modules, systems built on top of them only have to concern themselves with the behavior of these modules. There is a separate concern around denied data environments in the software realm but that is not on many people's radar. Most software devs would not know where to even start to protect systems from this. A tension with access controls is that if you implement it to the level of granularity the most demanding parts of DoD say they want, it never actually gets used because it is too complicated for users to reason about and manage. Or worse, they make mistakes and leak data because it is complicated. A simpler model is always implemented on top of it. At the same time, fine-grained and scalable access controls impose a high operational cost in the software even if they are not being used and some parts of DoD care a lot about software performance. Many parts of DoD are realistic enough to not want to pay for access controls they'll never actually use. On top of this, security architecture is designed to be somewhat pluggable because different users will have mutually exclusive design requirements for the security architecture. It would be nice if this wasn't the case but it is what it is.
  - bitwalker 1y ago
    > There is a separate concern around denied data environments in the software realm but that is not on many people's radar. Most software devs would not know where to even start to protect systems from this. The concept of a denied environment is pretty clear to me when it comes to physical space, or radio communications - but could you clarify what you mean by a "denied data environment"? I have some notion of what you _might_ mean, but I can't find a clear definition of the idea anywhere.
    - jandrewrogers 1y ago
      It is a specific type of sophisticated denial-of-service attack on data infrastructure. Theoretically it is a really interesting attack and difficult to defend against. Essentially all open source and commercial systems are vulnerable to it. Most systems, including military systems, use data from many exogenous sources. Critical systems may use data diodes and formally verified software interfaces to consume this data that make them extremely hardened against outright exploitation. However, these systems are vulnerable in another way: they use data structures and algorithms to serve their purpose, often with a bunch of architecture to allow scalability like multithreading. You can subtly generate or dynamically edit data in exogenous data streams of target systems to produce pathological computer science outcomes that will pass all human inspection and formal verification as legitimate. Nonetheless, it is engineered to enable cascading pathological scenarios in common implementations of data structures and algorithms in popular systems. The attacks are usually targeted against lock graphs and subtle quadratic corner cases in algorithm implementations. Many years ago I engineered a prototype of this for fun targeting a well-known commercial database and it completely locked up the system for more than 10 minutes. For many purposes, that is almost as good as a system kill. The obvious way to recover your system is to disconnect it from data sources and users. There is a requirement in DoD for systems that are designed to automatically detect these types of attacks and to preserve operational performance in these types of adversarial data environments. I think it isn't on anyone's radar because it hasn't been used in any real-world attacks against commercial systems. It definitely requires a sophisticated adversary, random hackers aren't going to pull it off. The "data denied environments" are like the above, where your adversary is injecting these kinds of system attacks in all of your exogenous data feeds. If you have to shut those sources off to keep your systems up, you are running blind.
toss1 1y ago
>>The Army should treat the NGC2 prototype version as “very high risk” because of the “likelihood of an adversary gaining persistent undetectable access,” wrote Gabrielle Chiulli, the Army chief technology officer authorizing official. >>One application revealed 25 high-severity code vulnerabilities What part of the above two quotes are hard to understand? >>"The security architecture is demonstrably the kind of thing that can be mechanically added later so DoD takes the view that there is no development risk by not implementing it in the prototype." Are you actually serious about this? Because to me, that is the most nonsensical thing said in a post of nonsensical things. ARCHITECTURE is the FIRST thing that needs to be determined — what is the structure of the modules, how they communicate, and what underlying tech will be used? Sure, if you are making a true proof-of-concept prototype you can skip it but no one would be discussing the above kinds of issues evaluating a POC. Tacking on some afterthought of a 'security architecture' at the end of a development project is a recipe for disaster in countless ways. And yes, you said "fine details" added later; agree. But even a prototype of a secure communications system the thing must AT LEAST show some security architecture; I mean at least show you can keep Joe-user from accessing Root privs... here, they said anyone can access anything. That is not fine details, that is simply not even close to meeting the spec. (and yes, I do DOD work, and your account is not even close to how I've seen it work)
- dkarl 1y ago
  > no one would be discussing the above kinds of issues evaluating a POC Um, yeah, they would be communicating that it's a bad idea to put sensitive data into it. You have to make that stuff explicit, or somebody will misuse it. I'm looking for the scandal here. Did the contractors misrepresent the prototype systems as being secure in ways they're not? Is the project behind schedule, and proper authorization controls were supposed to be implemented by now? Is the prototype system being improperly used to share sensitive data? Any of those would be concerning, but the article doesn't make any of those claims. All I see is, a prototype version of software doesn't have all the capability that it will need when it's finished.
  - toss1 1y ago
    The scandal here is that multilevel security MUST be a core of the product from the outset The evaluations quoted in my previous comment shows it is VERY obvious the Army expected to see at least the capabilities to prevent "an adversary gaining persistent undetectable access”, expected to see capabilities preventing low-level users from accessing higher-clearance-level materials, and did not expect 25+ high-severity code vulnerabilities, and all of those expectations were failed by the suppliers. True, the article did not cite info about the exact specifications and expected operational readiness level at this date detailed in the contracts. However, we can safely assume the Army officers who wrote the scathing memo do have access to and understand the contractual expectations, and wrote the report in the proper context of the contractual expectations. Instead, you want to do a rhetorical sleight-of-hand to claim since the article necessarily is not a complete report (it's a short news article), the missing bits mean the Army officers are just speaking out of context and there is no scandal. That is wrong
reissbaker 1y ago
I think what's happening here is: 1. The Army gave a reasonable analysis of the security issues with the prototype. 2. This is normal. 3. But most people don't have experience in this field. 4. A reporter got access to the report, and published the most salacious-sounding parts — and if you don't know that this is normal, it sounds really bad. 5. Oh no, the sky is falling. Click here to watch our ads, oops I mean, to read all about it. Generally my read on tech-adjacent reporters is they have very little understanding of what they're reporting on, and mostly try to manufacture outrage to get clicks.
- appreciatorBus 1y ago
  Also true of non-tech-adjacent reporters! The impulse may be stronger in some and weaker in others, and the incentive structure at their outlet may be stronger or weaker, but all of them know, even subconsciously, what type of stories are likelier to capture attention, make $ for the outlet and hopefully return a portion of that to them. Outrage is high the list of qualities that guarantee attention. Since learning about social media algorithms, I've come to think of all news media, even old timey newspapers, in the same light. IMO The only difference between TikTok's For You page and newspaper headlines from 200 years ago was the precision of the engagement feedback and the latency.
  - MountDoom 1y ago
    > Also true of non-tech-adjacent reporters! It's true for everyone. Over the years, I've been on the business end of several "tech outrage" stories that made the headlines on HN, and over time, became a sort of IT canon: "company X did something really bad". And they all had the same origin: a party with an ax to grind latched onto a fairly boring story, distorted the facts to tell a narrative, and that narrative "clicked" with everyone else because it confirmed their preexisting biases. It doesn't mean that there are no companies that are truly evil or incompetent, but I'd wager is <50% of the ones that get raked over the coals here and elsewhere.
    - reissbaker 1y ago
Grimblewald 1y ago
I dunno, the process is normal but the issues raised are extreme. Too extreme. I'd expect things like "in edge cases X vulnerabilities start being an issue" rather than "at any point, anyone, can gain undtected full access to all our live infrastructure, equipment, and troop poaitions" it feels like the bar for what are acceptable toothing issues is set unrealistically low here.
firesteelrain 1y ago
I don’t agree this is normal on a DoD program. You still need to get an IATO or IATT.
- jandrewrogers 1y ago
  Prototypes are routinely pushed into limited production for testing purposes. Someone has to sign-off on doing so but it has never been difficult to get that sign-off in my experience. The operations chosen for that purpose are selected for their limited risk if something goes wrong. DoD wants capability evaluations to be as realistic as possible even if that means taking some risks. They understand the risk/reward tradeoff.
anigbrowl 1y ago
"We cannot control who sees what, we cannot see what users are doing, and we cannot verify that the software itself is secure," the memo says. This is the issue right here. In my experience, the fine-grained control and security features are never implemented in the prototypes. Those features are the whole system. What else is there to prototype, the ability to send and receive signals in the radio spectrum? I'm pretty sure we already have that. It's the secure features that matter to the military. it looks like the article is going out of its way to mislead and misrepresent. Sounds like bullshit to me, and I'd extend that to most of the comments in this subthread that are just based on tropes about the media being mean and lacking technical sophistication. This is a straightforward, soberly written article that provides a reasonable summary of the information. There's no suggestion here that the memo was leaked or is classified in any way.

ranadomo 1y ago

This is an absurd statement to make about a DOD program. Securing communication is communication in the most adversarial environment imaginable. There are thousands of problems that only emerge once cryptographic authentication and authorization are enabled. This isn't a b2b sass app where you can just add an "auth layer" once the api is built out. It's passing mission-critical messages through signal-jamming and unimaginably hostile imitation scenarios. Many DOD programs are built as prototypes that don't ever factor security in the architecture, and then have massive problems and delays trying to implement it later. DOD cyber acquisition is run by the most incompetent clowns on earth. The only reason they don't know how terrible their software is, is because they're not capable of detecting how fundamentally compromised their systems are, and China and Russia are not exactly white-hats looking for a bounty.

wyldberry 1y ago
It is not absurd, this is the standard for rapid prototyping in the DoD. Given that Palantir already has a strong track record for authn/z inside their systems used across the DoD, LEO, and Intelligence Agencies. It's not an untread, uncharted path for these organizations.
jandrewrogers 1y ago
I have first-hand experience with the problems of designing systems for adversarial denied environments. These are largely orthogonal to the problem of access controls. The low-level communication security and channel capacity is handled almost exclusively by external trusted modules, systems built on top of them only have to concern themselves with the behavior of these modules. There is a separate concern around denied data environments in the software realm but that is not on many people's radar. Most software devs would not know where to even start to protect systems from this. A tension with access controls is that if you implement it to the level of granularity the most demanding parts of DoD say they want, it never actually gets used because it is too complicated for users to reason about and manage. Or worse, they make mistakes and leak data because it is complicated. A simpler model is always implemented on top of it. At the same time, fine-grained and scalable access controls impose a high operational cost in the software even if they are not being used and some parts of DoD care a lot about software performance. Many parts of DoD are realistic enough to not want to pay for access controls they'll never actually use. On top of this, security architecture is designed to be somewhat pluggable because different users will have mutually exclusive design requirements for the security architecture. It would be nice if this wasn't the case but it is what it is.
- bitwalker 1y ago
  > There is a separate concern around denied data environments in the software realm but that is not on many people's radar. Most software devs would not know where to even start to protect systems from this. The concept of a denied environment is pretty clear to me when it comes to physical space, or radio communications - but could you clarify what you mean by a "denied data environment"? I have some notion of what you _might_ mean, but I can't find a clear definition of the idea anywhere.
  - jandrewrogers 1y ago
    It is a specific type of sophisticated denial-of-service attack on data infrastructure. Theoretically it is a really interesting attack and difficult to defend against. Essentially all open source and commercial systems are vulnerable to it. Most systems, including military systems, use data from many exogenous sources. Critical systems may use data diodes and formally verified software interfaces to consume this data that make them extremely hardened against outright exploitation. However, these systems are vulnerable in another way: they use data structures and algorithms to serve their purpose, often with a bunch of architecture to allow scalability like multithreading. You can subtly generate or dynamically edit data in exogenous data streams of target systems to produce pathological computer science outcomes that will pass all human inspection and formal verification as legitimate. Nonetheless, it is engineered to enable cascading pathological scenarios in common implementations of data structures and algorithms in popular systems. The attacks are usually targeted against lock graphs and subtle quadratic corner cases in algorithm implementations. Many years ago I engineered a prototype of this for fun targeting a well-known commercial database and it completely locked up the system for more than 10 minutes. For many purposes, that is almost as good as a system kill. The obvious way to recover your system is to disconnect it from data sources and users. There is a requirement in DoD for systems that are designed to automatically detect these types of attacks and to preserve operational performance in these types of adversarial data environments. I think it isn't on anyone's radar because it hasn't been used in any real-world attacks against commercial systems. It definitely requires a sophisticated adversary, random hackers aren't going to pull it off. The "data denied environments" are like the above, where your adversary is injecting these kinds of system attacks in all of your exogenous data feeds. If you have to shut those sources off to keep your systems up, you are running blind.

toss1 1y ago

>>The Army should treat the NGC2 prototype version as “very high risk” because of the “likelihood of an adversary gaining persistent undetectable access,” wrote Gabrielle Chiulli, the Army chief technology officer authorizing official. >>One application revealed 25 high-severity code vulnerabilities What part of the above two quotes are hard to understand? >>"The security architecture is demonstrably the kind of thing that can be mechanically added later so DoD takes the view that there is no development risk by not implementing it in the prototype." Are you actually serious about this? Because to me, that is the most nonsensical thing said in a post of nonsensical things. ARCHITECTURE is the FIRST thing that needs to be determined — what is the structure of the modules, how they communicate, and what underlying tech will be used? Sure, if you are making a true proof-of-concept prototype you can skip it but no one would be discussing the above kinds of issues evaluating a POC. Tacking on some afterthought of a 'security architecture' at the end of a development project is a recipe for disaster in countless ways. And yes, you said "fine details" added later; agree. But even a prototype of a secure communications system the thing must AT LEAST show some security architecture; I mean at least show you can keep Joe-user from accessing Root privs... here, they said anyone can access anything. That is not fine details, that is simply not even close to meeting the spec. (and yes, I do DOD work, and your account is not even close to how I've seen it work)

dkarl 1y ago
> no one would be discussing the above kinds of issues evaluating a POC Um, yeah, they would be communicating that it's a bad idea to put sensitive data into it. You have to make that stuff explicit, or somebody will misuse it. I'm looking for the scandal here. Did the contractors misrepresent the prototype systems as being secure in ways they're not? Is the project behind schedule, and proper authorization controls were supposed to be implemented by now? Is the prototype system being improperly used to share sensitive data? Any of those would be concerning, but the article doesn't make any of those claims. All I see is, a prototype version of software doesn't have all the capability that it will need when it's finished.
- toss1 1y ago
  The scandal here is that multilevel security MUST be a core of the product from the outset The evaluations quoted in my previous comment shows it is VERY obvious the Army expected to see at least the capabilities to prevent "an adversary gaining persistent undetectable access”, expected to see capabilities preventing low-level users from accessing higher-clearance-level materials, and did not expect 25+ high-severity code vulnerabilities, and all of those expectations were failed by the suppliers. True, the article did not cite info about the exact specifications and expected operational readiness level at this date detailed in the contracts. However, we can safely assume the Army officers who wrote the scathing memo do have access to and understand the contractual expectations, and wrote the report in the proper context of the contractual expectations. Instead, you want to do a rhetorical sleight-of-hand to claim since the article necessarily is not a complete report (it's a short news article), the missing bits mean the Army officers are just speaking out of context and there is no scandal. That is wrong

reissbaker 1y ago

I think what's happening here is: 1. The Army gave a reasonable analysis of the security issues with the prototype. 2. This is normal. 3. But most people don't have experience in this field. 4. A reporter got access to the report, and published the most salacious-sounding parts — and if you don't know that this is normal, it sounds really bad. 5. Oh no, the sky is falling. Click here to watch our ads, oops I mean, to read all about it. Generally my read on tech-adjacent reporters is they have very little understanding of what they're reporting on, and mostly try to manufacture outrage to get clicks.

appreciatorBus 1y ago
Also true of non-tech-adjacent reporters! The impulse may be stronger in some and weaker in others, and the incentive structure at their outlet may be stronger or weaker, but all of them know, even subconsciously, what type of stories are likelier to capture attention, make $ for the outlet and hopefully return a portion of that to them. Outrage is high the list of qualities that guarantee attention. Since learning about social media algorithms, I've come to think of all news media, even old timey newspapers, in the same light. IMO The only difference between TikTok's For You page and newspaper headlines from 200 years ago was the precision of the engagement feedback and the latency.
- MountDoom 1y ago
  > Also true of non-tech-adjacent reporters! It's true for everyone. Over the years, I've been on the business end of several "tech outrage" stories that made the headlines on HN, and over time, became a sort of IT canon: "company X did something really bad". And they all had the same origin: a party with an ax to grind latched onto a fairly boring story, distorted the facts to tell a narrative, and that narrative "clicked" with everyone else because it confirmed their preexisting biases. It doesn't mean that there are no companies that are truly evil or incompetent, but I'd wager is <50% of the ones that get raked over the coals here and elsewhere.
  - reissbaker 1y ago

Grimblewald 1y ago

I dunno, the process is normal but the issues raised are extreme. Too extreme. I'd expect things like "in edge cases X vulnerabilities start being an issue" rather than "at any point, anyone, can gain undtected full access to all our live infrastructure, equipment, and troop poaitions" it feels like the bar for what are acceptable toothing issues is set unrealistically low here.

firesteelrain 1y ago

I don’t agree this is normal on a DoD program. You still need to get an IATO or IATT.

jandrewrogers 1y ago
Prototypes are routinely pushed into limited production for testing purposes. Someone has to sign-off on doing so but it has never been difficult to get that sign-off in my experience. The operations chosen for that purpose are selected for their limited risk if something goes wrong. DoD wants capability evaluations to be as realistic as possible even if that means taking some risks. They understand the risk/reward tradeoff.

anigbrowl 1y ago

"We cannot control who sees what, we cannot see what users are doing, and we cannot verify that the software itself is secure," the memo says. This is the issue right here. In my experience, the fine-grained control and security features are never implemented in the prototypes. Those features are the whole system. What else is there to prototype, the ability to send and receive signals in the radio spectrum? I'm pretty sure we already have that. It's the secure features that matter to the military. it looks like the article is going out of its way to mislead and misrepresent. Sounds like bullshit to me, and I'd extend that to most of the comments in this subthread that are just based on tropes about the media being mean and lacking technical sophistication. This is a straightforward, soberly written article that provides a reasonable summary of the information. There's no suggestion here that the memo was leaked or is classified in any way.