6 ms·

Great, another Nix tutorial. Sounds great, isn't. A lot of magical thinking going on in that group. If you know anything about computers, you know for them to

by than3 4y ago

Great, another Nix tutorial. Sounds great, isn't. A lot of magical thinking going on in that group. If you know anything about computers, you know for them to operate deterministically, uniqueness of the next potential state is an important characteristic. They use hashes (galois fields) that aren't unique to ID packages (by definition), and as far as I have been told (when asking about this), they don't do anything to check for collisions. Yes the probability is extremely remote, but probability isn't likelihood. What happens when a collision does occur, maybe the software won't work. So how critical is the software? Will it break the system? Can the system clean it up and recover (no, because its non-deterministic, more time). These are questions anyone worth their salt would ask when they value stability and many Linux systems run for months of uptime or more. They have two fundamental problems, they violate determinism, but that failure doesn't hit until there is a package collision, and its non-deterministic which means its not easily characterized (you spend a lot of time), and you need some form of mutability for package security updates, and configurations. If you can't adjust your software easily its stealing your time.

stouset 4y ago
I don’t think you quite appreciate how remote the possibility of a SHA-256 collision is. Barring a catastrophic break, there will never be a collision witnessed across the full lifespan of humanity.
- than3 4y ago
  I don't think you understand the difference and fluidity of what a distribution is and how it differs from likelihood curves. You've made some significant assumptions, one of those assumptions is that the brute force way is the most efficient way or put another way there is no optimal way to calculate collisions better than brute force. That line of thinking is very flawed. Has shown to be flawed many times in the past several decades, and shouldn't even be up for debate given the evidence and deprecation of various cryptographic functions (as vulnerabilities become known publicly). We all know vulnerabilities were known before the deprecation because detection is a lagging process. Also note, as far as the respondents of the project I've spoken with have said, they do not even check for existing collisions. So if a collision did occur, you wouldn't even know it without specialized knowledge.
  - anon291 4y ago
    What kind of collision are you envisioning? And what's the failure case? Two packages that suddenly have the same hash are still going to differ in the package name and version string, and thus different directories on the nix store. You can never have something compromised injected onto your computer this way either because nix requires signed builds to download from a remote repository. If you trust a malicious repository... that's on you. By default, only nixpkgs is trusted with the default hydra.
    - than3 4y ago
      package name and version string are part of what, the metadata manifest provided right? As a thought experiment, say a collision is found for libc or the standard library, thanks to all that research on primes that's been released recently, assume a more efficient search is found. What would the failure domains be for those two libraries if, a) the library was non-functional suddenly, or b) the library remained functional but malicious. Would Nix detect the collision, or would Nix simply recreate the dependency graph overwriting the old but secure with this new malicious software. Almost all of the security primitives are implemented at domain edges, which means local access, or remote exploitation would have equal effect, and a watering hole/supply chain attack may not even be necessary.
      - the_gipsy 4y ago
        Nothing would happen. Like the other commenter said, you trust your package repo like with any other distro.
      - shakow 4y ago
        So, basically, you're saying that Nix is insecure, because you need hash collision + signature forgery + malicious access to repos; whereas in a standard distro, you only need signature forgery + malicious access to repos? I really don't get your point.
      - viraptor 4y ago
        > thanks to all that research on primes that's been released recently, assume a more efficient search is found. That's just trolling. Primes have nothing to do with sha256 beyond the initial value. It's just bitwise operations and additions.
zdragnar 4y ago
What are the alternatives which are properly deterministic?
- than3 4y ago
  Any of the existing packaging solutions work to a greater or lesser degree so long as the namespaces guarantee uniqueness in the package identity and dependency graphs which is what most packagers do when they put it together for other distros. As for automating the identity hash generation so it is perfectly deterministic, its not possible so long as you map onto Galois Fields due to their finite field length (and inherent non-unique solutions). I'm not aware of any mathematical alternative for cryptography, to finite fields, and a hash is a cryptographic function. Strictly speaking, you have a tradeoff in terms of stability as well as additional cost from the language complexity, and non-deterministic behavior is the fallback when discrete time invariance or determinism is violated. Many people like Nix, but many people also don't understand how it works, and thus it will work until it doesn't, and then you'll be stuck. Its not like there is tooling to check determinism, and if time invariance fails because of say a cache (memory property), then you can't even troubleshoot because everything we do to troubleshoot relies on those fundamental system properties with exception to Guess and Check which to be successful requires the highest specialization and understanding of the system.
  - anon291 4y ago
    > Many people like Nix, but many people also don't understand how it works, and thus it will work until it doesn't, and then you'll be stuck. Until what doesn't work? The base system is verified continuously, so there's no chance some two packages are linked against each other causing a boot problem. If there's really a problem ever, then you just find the old system and start it on boot to one that doesn't include that package. This is straightforwards because most nix users have multiple nixos versions installed at once. Then, you update the offending package to not hash to the same thing (add a comment to a build script).
    - than3 4y ago
smitop 4y ago
What happens is you'll write a paper titled "The first SHA-256 collision: SHA-256 is broken in practice", since there are currently no known collisions (despite Bitcoin miners computing exahashes per second). The chances of you finding the first known collision by accident are overwhelming slim, and the benefits of showing that a widely used hash function is broken would greatly exceed the annoyance of a broken system.
- than3 4y ago
  No one I know of ever breaks these systems with exhaustive searches. Holding an exhaustive search up as some kind of bedrock of security is foolhardy. As for cryptography, its all based on primes, and have you been watching what papers are being published regarding more efficient methods and searches for primes? Ethical people would publish, unethical people would sell it to everyone elses detriment.
  - hash-no-gal-fld 4y ago
    Cryptography is not all based on primes. You are making a lot of over-general assumptions in your posts.
    - _siis 4y ago
      The modern cryptographic functions we were talking about are all based on primes. Unless you are suddenly trying to shift gears to muddy the water for some other purpose and apply what I said to 'all cryptography', it was pretty clear what we were talking about prior to this response. Perhaps you should do a bit more research into the origin of those magic constants that get used to initialize those functions. Where do you think they come from... yes you could change them but then you wouldn't be following the specification, and you can't legitimately call it the same thing. AFAIK, they are all in some form related to primes, whether that's a truncation of a floating point representation of a prime or some other operation like a square root of a sequence of primes, because evidence has shown that the chaotic nature of primes works best and this is a property we want in cryptographic systems.
hexo 4y ago
Nix has so many problems with misdesign that this is the last nail. first: i know some folks who use nix and it works very well for them. next: but not me tho. the experience: To be clear, I had a laptop with Nixos as daily driver for about 9 months and it was total hell. The thing that finally forced me to switch to ubuntu (yes, that. at that time i needed working software for 7hours already. studying another five A3 pages of what could have hypothetically go wrong and how to maybe fix it was far beyond the scope what i needed to do.) was some component needed by pyqt4 and pyqt5 and nix could not handle it in any way. i've reported the issue which was subsequently fixed about 8 months later. thanks but not again. the nix language itself is a hard no. 3 days worth of figuring out stuff instead of having it done in 15 minutes is a usability joke. then there is guix that has the very same problems you mentioned, because the re-used that hashing nonsense from nix. looking into /nix/store makes me want to cry every time. i just cant believe the current state (implementation) could be taken seriously by anyone. i have a strong feeling that after 19 years of development nix got into stage when it needs complete redesign and rethink way of doing stuff.
- anon291 4y ago
  Nope. I stopped using Ubuntu after realizing it offers no benefits over windows. You still have to reinstall the thing every few years when it ends up breaking, and good luck and Godspeed if you ever die in the middle of an upgrade. Oh and it just accumulates cruft and random configuration settings after a while. Thanks, but no thanks. I've had too many ubuntu systems end up in states where something doesn't work with no guide on the internet. In nix, I restart and choose an older generation and I'm left with a clean system. That's the way a computer should work. Are there problems sure? But the class of problem is wholly different.
  - hexo 4y ago
    Ubuntu isnt good at all, compared to windows its far better os, but still crap. Nix? Where not even regular bash script works? What a joke. One big hack. Benchmark of symlinking a dynlinker with no benefit to users, just making things mega complicated, slow, non-configurable or outright not working as (I) used to with no clear workaround. Yea, tell that Nix is good to admins. They might appreciate. Desktop? No way. Devs? Not even remote. I'd give it 2 out of 10.
hash-no-gal-fld 4y ago
> hashes (galois fields) Huh? What do hashes like Sha256 have to do with Galois fields?
- than3 4y ago
  Math, the hash function use magic numbers which in turn initialize and eventually map to a specific Galois Field, they are also called finite fields. You have a set field size as you go over it, it loops and so output fields are never unique, you have a 1:infinite relationship. That property to determinism, there are problems computers cannot solve but we can. A computer requires determinism to do work, which in effect is that 1:1 state->next state mapping on the state graph. There's a lot of computer science theory and literature on this if you want to dig deeper. MIT has an OCW course in signals and systems. Anywhere you see modulus math happening, its working with finite fields.
  - deadbeef57 4y ago
    That's flat out wrong. Working modulo 4 is not working in a finite field, because 22 = 0 when you work mod 4. When you work modulo a prime, then you are working in a finite field. Working in the finite field of 4 elements is not* the same as working mod 4. Finite fields of size p^n are not the same as (not isomorphic to!) the ring Z/(p^nZ) whenever n > 1. Just because there exists a finite field of size 2^n doesn't mean that any length-n-bit-string that is the output of a hash function lives in a finite field of size 2^n. Adding and multiplying hashes doesn't make any sense. If it does then you have a very weird hash function. Not one of the standard cryptographic hashes. So saying that a hash function maps onto a finite field sounds pretty confused to me.
    - deadbeef57 4y ago
      2 times 2 = 0 mod 4. Sorry for the markdown mess-up.

by than3 4y ago

stouset 4y ago

I don’t think you quite appreciate how remote the possibility of a SHA-256 collision is. Barring a catastrophic break, there will never be a collision witnessed across the full lifespan of humanity.

than3 4y ago
I don't think you understand the difference and fluidity of what a distribution is and how it differs from likelihood curves. You've made some significant assumptions, one of those assumptions is that the brute force way is the most efficient way or put another way there is no optimal way to calculate collisions better than brute force. That line of thinking is very flawed. Has shown to be flawed many times in the past several decades, and shouldn't even be up for debate given the evidence and deprecation of various cryptographic functions (as vulnerabilities become known publicly). We all know vulnerabilities were known before the deprecation because detection is a lagging process. Also note, as far as the respondents of the project I've spoken with have said, they do not even check for existing collisions. So if a collision did occur, you wouldn't even know it without specialized knowledge.
- anon291 4y ago
  What kind of collision are you envisioning? And what's the failure case? Two packages that suddenly have the same hash are still going to differ in the package name and version string, and thus different directories on the nix store. You can never have something compromised injected onto your computer this way either because nix requires signed builds to download from a remote repository. If you trust a malicious repository... that's on you. By default, only nixpkgs is trusted with the default hydra.
  - than3 4y ago
    package name and version string are part of what, the metadata manifest provided right? As a thought experiment, say a collision is found for libc or the standard library, thanks to all that research on primes that's been released recently, assume a more efficient search is found. What would the failure domains be for those two libraries if, a) the library was non-functional suddenly, or b) the library remained functional but malicious. Would Nix detect the collision, or would Nix simply recreate the dependency graph overwriting the old but secure with this new malicious software. Almost all of the security primitives are implemented at domain edges, which means local access, or remote exploitation would have equal effect, and a watering hole/supply chain attack may not even be necessary.
    - the_gipsy 4y ago
      Nothing would happen. Like the other commenter said, you trust your package repo like with any other distro.
    - shakow 4y ago
      So, basically, you're saying that Nix is insecure, because you need hash collision + signature forgery + malicious access to repos; whereas in a standard distro, you only need signature forgery + malicious access to repos? I really don't get your point.
    - viraptor 4y ago
      > thanks to all that research on primes that's been released recently, assume a more efficient search is found. That's just trolling. Primes have nothing to do with sha256 beyond the initial value. It's just bitwise operations and additions.

zdragnar 4y ago

What are the alternatives which are properly deterministic?

than3 4y ago
Any of the existing packaging solutions work to a greater or lesser degree so long as the namespaces guarantee uniqueness in the package identity and dependency graphs which is what most packagers do when they put it together for other distros. As for automating the identity hash generation so it is perfectly deterministic, its not possible so long as you map onto Galois Fields due to their finite field length (and inherent non-unique solutions). I'm not aware of any mathematical alternative for cryptography, to finite fields, and a hash is a cryptographic function. Strictly speaking, you have a tradeoff in terms of stability as well as additional cost from the language complexity, and non-deterministic behavior is the fallback when discrete time invariance or determinism is violated. Many people like Nix, but many people also don't understand how it works, and thus it will work until it doesn't, and then you'll be stuck. Its not like there is tooling to check determinism, and if time invariance fails because of say a cache (memory property), then you can't even troubleshoot because everything we do to troubleshoot relies on those fundamental system properties with exception to Guess and Check which to be successful requires the highest specialization and understanding of the system.
- anon291 4y ago
  > Many people like Nix, but many people also don't understand how it works, and thus it will work until it doesn't, and then you'll be stuck. Until what doesn't work? The base system is verified continuously, so there's no chance some two packages are linked against each other causing a boot problem. If there's really a problem ever, then you just find the old system and start it on boot to one that doesn't include that package. This is straightforwards because most nix users have multiple nixos versions installed at once. Then, you update the offending package to not hash to the same thing (add a comment to a build script).
  - than3 4y ago

smitop 4y ago

What happens is you'll write a paper titled "The first SHA-256 collision: SHA-256 is broken in practice", since there are currently no known collisions (despite Bitcoin miners computing exahashes per second). The chances of you finding the first known collision by accident are overwhelming slim, and the benefits of showing that a widely used hash function is broken would greatly exceed the annoyance of a broken system.

than3 4y ago
No one I know of ever breaks these systems with exhaustive searches. Holding an exhaustive search up as some kind of bedrock of security is foolhardy. As for cryptography, its all based on primes, and have you been watching what papers are being published regarding more efficient methods and searches for primes? Ethical people would publish, unethical people would sell it to everyone elses detriment.
- hash-no-gal-fld 4y ago
  Cryptography is not all based on primes. You are making a lot of over-general assumptions in your posts.
  - _siis 4y ago
    The modern cryptographic functions we were talking about are all based on primes. Unless you are suddenly trying to shift gears to muddy the water for some other purpose and apply what I said to 'all cryptography', it was pretty clear what we were talking about prior to this response. Perhaps you should do a bit more research into the origin of those magic constants that get used to initialize those functions. Where do you think they come from... yes you could change them but then you wouldn't be following the specification, and you can't legitimately call it the same thing. AFAIK, they are all in some form related to primes, whether that's a truncation of a floating point representation of a prime or some other operation like a square root of a sequence of primes, because evidence has shown that the chaotic nature of primes works best and this is a property we want in cryptographic systems.

hexo 4y ago

Nix has so many problems with misdesign that this is the last nail. first: i know some folks who use nix and it works very well for them. next: but not me tho. the experience: To be clear, I had a laptop with Nixos as daily driver for about 9 months and it was total hell. The thing that finally forced me to switch to ubuntu (yes, that. at that time i needed working software for 7hours already. studying another five A3 pages of what could have hypothetically go wrong and how to maybe fix it was far beyond the scope what i needed to do.) was some component needed by pyqt4 and pyqt5 and nix could not handle it in any way. i've reported the issue which was subsequently fixed about 8 months later. thanks but not again. the nix language itself is a hard no. 3 days worth of figuring out stuff instead of having it done in 15 minutes is a usability joke. then there is guix that has the very same problems you mentioned, because the re-used that hashing nonsense from nix. looking into /nix/store makes me want to cry every time. i just cant believe the current state (implementation) could be taken seriously by anyone. i have a strong feeling that after 19 years of development nix got into stage when it needs complete redesign and rethink way of doing stuff.

anon291 4y ago
Nope. I stopped using Ubuntu after realizing it offers no benefits over windows. You still have to reinstall the thing every few years when it ends up breaking, and good luck and Godspeed if you ever die in the middle of an upgrade. Oh and it just accumulates cruft and random configuration settings after a while. Thanks, but no thanks. I've had too many ubuntu systems end up in states where something doesn't work with no guide on the internet. In nix, I restart and choose an older generation and I'm left with a clean system. That's the way a computer should work. Are there problems sure? But the class of problem is wholly different.
- hexo 4y ago
  Ubuntu isnt good at all, compared to windows its far better os, but still crap. Nix? Where not even regular bash script works? What a joke. One big hack. Benchmark of symlinking a dynlinker with no benefit to users, just making things mega complicated, slow, non-configurable or outright not working as (I) used to with no clear workaround. Yea, tell that Nix is good to admins. They might appreciate. Desktop? No way. Devs? Not even remote. I'd give it 2 out of 10.

hash-no-gal-fld 4y ago

> hashes (galois fields) Huh? What do hashes like Sha256 have to do with Galois fields?

than3 4y ago
Math, the hash function use magic numbers which in turn initialize and eventually map to a specific Galois Field, they are also called finite fields. You have a set field size as you go over it, it loops and so output fields are never unique, you have a 1:infinite relationship. That property to determinism, there are problems computers cannot solve but we can. A computer requires determinism to do work, which in effect is that 1:1 state->next state mapping on the state graph. There's a lot of computer science theory and literature on this if you want to dig deeper. MIT has an OCW course in signals and systems. Anywhere you see modulus math happening, its working with finite fields.
- deadbeef57 4y ago
  That's flat out wrong. Working modulo 4 is not working in a finite field, because 22 = 0 when you work mod 4. When you work modulo a prime, then you are working in a finite field. Working in the finite field of 4 elements is not* the same as working mod 4. Finite fields of size p^n are not the same as (not isomorphic to!) the ring Z/(p^nZ) whenever n > 1. Just because there exists a finite field of size 2^n doesn't mean that any length-n-bit-string that is the output of a hash function lives in a finite field of size 2^n. Adding and multiplying hashes doesn't make any sense. If it does then you have a very weird hash function. Not one of the standard cryptographic hashes. So saying that a hash function maps onto a finite field sounds pretty confused to me.
  - deadbeef57 4y ago
    2 times 2 = 0 mod 4. Sorry for the markdown mess-up.