5 ms·

Crypto-skeptics downvoted my previous suggestion (https://news.ycombinator.com/item?id=34349576 https://news.ycombinator.com/item?id=34349576) that NOTAM's sort

by gojomo 4y ago

Crypto-skeptics downvoted my previous suggestion (https://news.ycombinator.com/item?id=34349576 https://news.ycombinator.com/item?id=34349576) that NOTAM's sort of simple textual bulletin log – perhaps not even 150MB of data per year! – could be implemented as a distributed-consensus database. But blockchains are specifically designed to keep on appending, no matter the mistakes of individuals, or even motivated sabotage attempts of sophisticated attackers. Essentially, blockchains have a giant, full-coverage set of unit-tests in their shared & public consensus code – reviewed and running across many systems. And many of their designs have proven uptimes in adversarial environments for years. And the task gets even easier when there's a small fixed set of established authorities with privileged keys. Protecting those keys is also, of course, a hard problem. But compromised credentials is probably already a fatal threat for the legacy system(s) that were brought down by "contractors who introduced data errors". So a hypothetical modern crypto-consensus solution's failure modes remain a subset of those of the legacy system.

12baad4db82 4y ago
I don't think it helps to characterise this as a simple textual bulletin log. I am by no means an expert, however to me this system appears to include: a globally agreed upon data format, a transmission protocol hardware distributed across the planet in ground stations and air vehicles, and 70 years of legacy. And all of that is enshrined in legislation across many countries. And while there is a modernisation effort in progress, the nature of this system means that things will move slowly. In 2023 anyone can suggest a better data format and distribution system if they start from scratch. Good luck getting the world to use it and good luck getting anyone to care.
- gojomo 4y ago
  You're right that the main challeges are political & bureaucratic inertia. That shouldn't stop us from honestly & plainly describing its core, & the industry's state-of-the-art for similar systems that don't have the legacy system's encumbrances. The "globally agreed-upon data format" is essentially text blobs with a standard set of abbreviations, that assist search/filtering by flight-paths & planned waypoints. It seems to be a mere 150MB a year. The inherently "move slowly" modernization effort can only be helped by a clearer outline of what's now possible. This is too important to prematurely dumb-down brainstorming ideas, based on low-expectations for the adoption process! And a modern architecture for distributed updates & reliability could trivially mimic the old system's pilot-visible outputs, to remain familiar to tenured personnel.
mgsouth 4y ago
Um, an immutable journal means you can't delete bogus data. If, for example, a system chokes when it sees a 3-byte UTF-8 code point, and you accidentally insert such a thing in the journal, then you're hosed. The system can't come back up until the software is mad-panic patched. Or (again hypothetically) somebody fat-fingers an update, deletes ATL from the airport list, thus causing a crash when software sees now-orphaned NOTAMs. Nor does distributed-consensus seem to apply in this case. For many datasets you want one source of truth (e.g. airport codes, approved abbreviations). Crypto blockchains deal with very, very simplistic global states. Bitcoin, for example, has (a) 256 bits from the prior block, (b) a megabyte or so of transaction data. Every state machine update (mined block) totally replaces the prior state. And it crucially depends upon backtracking never being needed; for example, if a prior block's hash made it impossible for a new block to be created with enough zeroes in the hash (I don't think that's possible), then a global, manual fork would be needed.
- gojomo 4y ago
  If the instructive content is invalid, you send a invalidation message. But if there's a violation of constraints, modern practice (both in and outside crypto projects) can do a pretty good job of precluding it, with test cases, wide review, multiple implementations, formal verification, and so forth. And when something slips by, many eyes & many implementations make a consensus fix very fast. No one has 'fat-fingered' the major blockchains in a long time! Some proprietary/permissioned chains have had problems. And stuff as complicated as smart contracts can go awry in subtle ways. But NOTAM seems to be just a simple text log! Distributed consensus inherently achieves one-source-of-truth without single-point-of-failure. As the notices relevant to individual flight-paths are inherently geo-local, with many independent submitters (like airports), any routing "through the FAA in DC" introduces risks – even if the FAA has final authority over who (which keys) can add entries. Yes, Bitcoin's consensus state is relatively simple – but also far more compicated than NOTAM's even-simpler message. And continually redundantly verified & replicated by many independent actors & codebases, very successfully, for 10+ years.
krisoft 4y ago
> could be implemented as a distributed-consensus database. Yes it can. That is not the question. The question is why do you think a bog standard, but correctly implemented and administered database could not handle the same? Imagine the following: the task is to move two wheels of cheese 5 km from A to B. Your proposal is akin to someone comming and suggesting that a suborbital rocket can do the job. And you are right. It can. But so can a Toyota Corolla without breaking a sweat. So tell me, why should we use a “hypothetical modern crypto-consensus solution” here, with all its complexity, instead of a well understood and bog standard and well implemented database?
- gojomo 4y ago
  Any bog-standard database might work. But if so, curious the FAA hasn't just done that: say, SQLite replicated across many places. But the centralized team running it could still "fat-finger" it. Any "master DB" system has problems that blockchain-like approaches have definitively solved, allowing progress on valid messages when large portions of the usual maintenance systems have gone down or been compromised.
  - tjrDL6MjB2Zwwa 4y ago
    What do you mean might? Is there a major RDBMS out there that struggles with 150MB of data?
    - gojomo 4y ago
      Of course any major RDBMS would have no problem with the data. If the reports here are accurate, you could fit the last 5-20 years of all NOTAM messages into the RAM of a Raspberry Pi running SQLite. But if that's enough, why hasn't the FAA solved the problem already? And even a "major RDBMS" is vulnerable to stodgy management leaving design in a state where it's vulnerable to a single central point of failure, where "2 contractors who introduced data errors" (!) can bring it down. Modern highly-distributed consensus systems are not vulnerable to that. Neither accident nor malice from 2 government contractors could prevent all authorized updates worldwide in a distributed consensus system like Ethereum. DC & northern Virginia (or wherever the FAA's systems hosted) could even spontaneously disappear into a lava pit and the official consensus set of NOTAMs would continue to update.

by gojomo 4y ago

12baad4db82 4y ago

I don't think it helps to characterise this as a simple textual bulletin log. I am by no means an expert, however to me this system appears to include: a globally agreed upon data format, a transmission protocol hardware distributed across the planet in ground stations and air vehicles, and 70 years of legacy. And all of that is enshrined in legislation across many countries. And while there is a modernisation effort in progress, the nature of this system means that things will move slowly. In 2023 anyone can suggest a better data format and distribution system if they start from scratch. Good luck getting the world to use it and good luck getting anyone to care.

gojomo 4y ago
You're right that the main challeges are political & bureaucratic inertia. That shouldn't stop us from honestly & plainly describing its core, & the industry's state-of-the-art for similar systems that don't have the legacy system's encumbrances. The "globally agreed-upon data format" is essentially text blobs with a standard set of abbreviations, that assist search/filtering by flight-paths & planned waypoints. It seems to be a mere 150MB a year. The inherently "move slowly" modernization effort can only be helped by a clearer outline of what's now possible. This is too important to prematurely dumb-down brainstorming ideas, based on low-expectations for the adoption process! And a modern architecture for distributed updates & reliability could trivially mimic the old system's pilot-visible outputs, to remain familiar to tenured personnel.

mgsouth 4y ago

Um, an immutable journal means you can't delete bogus data. If, for example, a system chokes when it sees a 3-byte UTF-8 code point, and you accidentally insert such a thing in the journal, then you're hosed. The system can't come back up until the software is mad-panic patched. Or (again hypothetically) somebody fat-fingers an update, deletes ATL from the airport list, thus causing a crash when software sees now-orphaned NOTAMs. Nor does distributed-consensus seem to apply in this case. For many datasets you want one source of truth (e.g. airport codes, approved abbreviations). Crypto blockchains deal with very, very simplistic global states. Bitcoin, for example, has (a) 256 bits from the prior block, (b) a megabyte or so of transaction data. Every state machine update (mined block) totally replaces the prior state. And it crucially depends upon backtracking never being needed; for example, if a prior block's hash made it impossible for a new block to be created with enough zeroes in the hash (I don't think that's possible), then a global, manual fork would be needed.

gojomo 4y ago
If the instructive content is invalid, you send a invalidation message. But if there's a violation of constraints, modern practice (both in and outside crypto projects) can do a pretty good job of precluding it, with test cases, wide review, multiple implementations, formal verification, and so forth. And when something slips by, many eyes & many implementations make a consensus fix very fast. No one has 'fat-fingered' the major blockchains in a long time! Some proprietary/permissioned chains have had problems. And stuff as complicated as smart contracts can go awry in subtle ways. But NOTAM seems to be just a simple text log! Distributed consensus inherently achieves one-source-of-truth without single-point-of-failure. As the notices relevant to individual flight-paths are inherently geo-local, with many independent submitters (like airports), any routing "through the FAA in DC" introduces risks – even if the FAA has final authority over who (which keys) can add entries. Yes, Bitcoin's consensus state is relatively simple – but also far more compicated than NOTAM's even-simpler message. And continually redundantly verified & replicated by many independent actors & codebases, very successfully, for 10+ years.

krisoft 4y ago

> could be implemented as a distributed-consensus database. Yes it can. That is not the question. The question is why do you think a bog standard, but correctly implemented and administered database could not handle the same? Imagine the following: the task is to move two wheels of cheese 5 km from A to B. Your proposal is akin to someone comming and suggesting that a suborbital rocket can do the job. And you are right. It can. But so can a Toyota Corolla without breaking a sweat. So tell me, why should we use a “hypothetical modern crypto-consensus solution” here, with all its complexity, instead of a well understood and bog standard and well implemented database?

gojomo 4y ago
Any bog-standard database might work. But if so, curious the FAA hasn't just done that: say, SQLite replicated across many places. But the centralized team running it could still "fat-finger" it. Any "master DB" system has problems that blockchain-like approaches have definitively solved, allowing progress on valid messages when large portions of the usual maintenance systems have gone down or been compromised.
- tjrDL6MjB2Zwwa 4y ago
  What do you mean might? Is there a major RDBMS out there that struggles with 150MB of data?
  - gojomo 4y ago
    Of course any major RDBMS would have no problem with the data. If the reports here are accurate, you could fit the last 5-20 years of all NOTAM messages into the RAM of a Raspberry Pi running SQLite. But if that's enough, why hasn't the FAA solved the problem already? And even a "major RDBMS" is vulnerable to stodgy management leaving design in a state where it's vulnerable to a single central point of failure, where "2 contractors who introduced data errors" (!) can bring it down. Modern highly-distributed consensus systems are not vulnerable to that. Neither accident nor malice from 2 government contractors could prevent all authorized updates worldwide in a distributed consensus system like Ethereum. DC & northern Virginia (or wherever the FAA's systems hosted) could even spontaneously disappear into a lava pit and the official consensus set of NOTAMs would continue to update.