6 ms·
The implication that any software is "mysterious" is problematic - there is no "woo" here - the exact state of the machine running the software may be determine
by EncomLab 2y ago
The implication that any software is "mysterious" is problematic - there is no "woo" here - the exact state of the machine running the software may be determined at every cycle. The exact instruction and the data it executed with may be precisely determined, as can the next instruction. The entire mythos of any software being a "black box" is just so much advertising jargon, perpetuated by tech bros who want to believe they are part of some Mr. Robot self-styled priestly class.
- margalabargala 2y agoYou're misunderstanding. A level of abstraction is necessary for operation of modern systems. There is no human alive who, given an intermediate step in the middle of some running learning algorithm, is able to understand and mentally model the full system at full man-made resolution, that is, down to the transistor level, on a modern CPU. Someone wishing to understand a piece of software in 2025 is forced to, at some point, accept that something somewhere "does what it says on the tin" and model it thusly rather than having a full understanding.
- EncomLab 2y agoIt's not misunderstanding at all - but your response is certainly an attempt to obfuscate the point being made. The moment you represent anything in code, you are abstracting a real thing into it's digital representation. That digital representation if fully formed at every cycle of the digital system processing it, and the state of the system - all the way down to the transistor level may be precisely determined. To say otherwise is to make the same error as those who claim that consciousness or understanding are indefinable "extra-ordinary" things that we have to just accept exist without any justification or evidence.
- margalabargala 2y agoOkay, then, you're just using your own personal definition of "black box" instead of the one everyone else uses. Something that's a black box is unknown to the speaker. It's not understood to be unknowable to anyone.
- EncomLab 2y agoSo your claim is that there are instructions, data, or both that are unable to be determined in what, is by definition, a fully deterministic machine?
- margalabargala 2y agoBy an individual person, yes. I claim that there exists no single human capable of fully understanding the totality of the software and hardware down to the individual transistor level.
- EncomLab 2y agoI agree and never claimed that "a single person" could - but just because something is too complex for a single person to fully understand does not make it "mysterious" or a "black box". So what is the claim you are making? Anything beyond the complexity of a single person to understand = magic?
- margalabargala 2y agoWe're just using different definitions for "black box". My definition is that it's something unknown, yours is that it's something unknowable.
- arkadiytehgraet 2y agoThat's a very wrong statement. Pretty sure I could explain all the maths, all the physics, all the electronics, all the operating systems and all the user space of a single high level language operation, when I was a fresh graduate. Now, I have forgotten most of the physics and electronics, since the university was quite some time ago, but feel free to ask any decent student of an IT bachelor, they should be able to pretty much build the PC from scratch. Sure, modern processors and whatnot add a bunch of optimizations, but you seem to really overstate the complexity of the computer.
- 2y ago
- 01HNNWZ0MV43FF 2y agoBut the weights trained from machine learning are a black box, in the sense that no human designed e.g. the image processing kernels that those weights represent. That is one reason people are skeptical of them, not only is training a large model at home expensive, not only is the data too big to trivially store, but the weights are not trivial to debug either
- xmprt 2y agoThe mystery was never in the "how do computers calculate the probabilities of next tokens" but rather in the "why is it able to work so well" and "what does this individual neuron contribute to the whole model"
- UltraSane 2y agoThe mystery is in how the data is encoded in the parameters and why LLMs performance scales so well with parameters. The key seems to be almost orthogonal vectors that allow neural networks to store so much data. They allow 2^(cn) vectors to be learned in an n-dimensional space with c being a constant.Since almost orthogonal vectors have very small dot products, they minimally interfere with each other, allowing many concepts to coexist with limited cross talk which enables superposition
- tlb 2y agoI don't know any serious programmer who thinks that, just because each operation is simple, the operation of the whole thing can't be mysterious.