nyrikki

55 ms·

661.
by nyrikki 3y ago
Well-founded relations is probably a good lens on why the above is important.
662.
by nyrikki 3y ago
Cantor's proofs showing that Z and Q are countable and R is uncountable. Cantor's "diagonalization proof" showed that. Turing extended to the computable numbers K, which can be conceptualized as a number where you can wr
663.
by nyrikki 3y ago
RLHF works on problems that are difficult to specify yet easy to judge. While RLHF will help improve systems, code correctness is not easy to judge outside of the simplest cases. Note how on OpenAI's technical report, they admit perfor
664.
by nyrikki 3y ago
Architectural erosion is an ongoing problem for humans, but they don't produce tightly coupled low cohesion code by default at the SWE level the majority of the time. With this quality of changes it won't be long until violations
665.
by nyrikki 3y ago
To add to this. I was going through devin's 'pass' diffs from SWE bench. Every one I ended up tracing to actual issues caused changes that would reduce maintainablity or introduced potential side effects. I think it may be us
666.
by nyrikki 3y ago
They don't work in some axiomized higher logic because they chose the rules to avoid them. HoL not having traditional NOT is an example. Even in FoL, Peano arithmetic uses the SoL induction to be usable. There is no free lunch.
667.
by nyrikki 3y ago
Implementing the Blum Blum Shub algorithm was my first real programming task for a gaming website in 95. Way too slow on Alphas at the time for primary use but OK for seed at the time. I still remember SGI publishing LavaRand, using lava la
668.
by nyrikki 3y ago
Transformers are highly parallelized, and generative AI appends. Even if you ignore that graph problems typically have more time complexity than LLMs offer during inference, they are circuit like and not turing machine like. Without getting
669.
by nyrikki 3y ago
ZFC was specifically designed to be immune to the classic paradoxes of naive set theory: Russell's paradox, the Burali-Forti paradox, and Cantor's paradox. You are arguing that the ground moves to perfectly fit the shape of a pudd
670.
by nyrikki 3y ago
Different hardware cache-hits differently for one. Level 3 BLAS functions like matrix on matrix products perform O(N^3) operations on O(N^2) data. The main way BLAS boosts Level 3 functions is cache optimization which is hardware dependent.
671.
by nyrikki 3y ago
Chess is decidable, it may be PSPACE-hard or EXPTIME-hard, but there are reductions. Entscheidungsproblem and Halt are not decidable in the general case. While you have to find reductions, decidable problems having access to both yes-instan
672.
by nyrikki 3y ago
Chess is a Perfect information game with a finite tree. Very different problem than programming.
673.
Why Are Sensitive Functions Hard for Transformers?(arxiv.org)
1 points by nyrikki 3y ago|1 comments
674.
by nyrikki 3y ago
This may be a positive or negative depending on your needs. We will see if it holds.
675.
by nyrikki 3y ago
The deduction principle turns out to be invalid for weak rejections and strong AI requires it. Consider strong negation: Did Homer write the Iliad? No, he did not. VS a too computationally expensive system with true,false,other with no kno
676.
by nyrikki 3y ago
While TC^0 can divide or multiply through say FFT, it actually can't do counters or addition which I am pretty sure requires TC^1 or Log-depth Threshold Circuits. LTSM is much more powerful.
677.
by nyrikki 3y ago
> We prove that the time complexity of self-attention is necessarily quadratic in the input length, unless the Strong Exponential Time Hypothesis (SETH) is false. This argument holds even if the attention computation is performed only ap
678.
by nyrikki 3y ago
> With a little more effort we can establish the same for any enumerable set of axioms. This is a case of presentism, which due to the work of Gödel we accept the costs of the implications of his findings. RE is semi-decidable, meaning w
679.
by nyrikki 3y ago
Using google lens with the tick marks on the image provide lots and lots of training examples that could work for that via simple pre-training and pattern matching. In this specific image case there are even games on Baamboozle called &quot
680.
by nyrikki 3y ago
LLMs are trained to sound confident. But they can also only do negation through exhaustion, known unknowns, future unknowns, etc... That is the pain of the Entscheidungsproblem. Even in Presburger arithmetic, Natural numbers will addition a
681.
by nyrikki 3y ago
No, LLMs with soft attention use compression, and actually has no mechanism for ground truth. They are simply pattern finding and matching. More correctly, they are uniform consent depth threshold circuits. Basically parallel operations on
682.
by nyrikki 3y ago
Really simple explanation is that for inference, feed forward networks are threshold circuits and by their nature ANNs are binary output, outputting true and false (same as being a threshold circuit) So if you train your models with that in
683.
by nyrikki 3y ago
For colloquial use just replace it with overconfidence which is how this post, and most people use it. It is a real thing for humans. https://www.sciencedirect.com/topics/psychology/overconfiden...
684.
by nyrikki 3y ago
Common usage doesn't typically meet that specific definition: “If you are really, really stupid, then it’s impossible for you to know you are really, really stupid.” Typically it is used to describe when people who don't have comp
685.
by nyrikki 3y ago
In context of This paper, where they are talking about Total forms of pure functions, yes But in general no. I could say have a total function with an internal counter that adds 1 each time it was run. That would still be a total function,
686.
by nyrikki 3y ago
If your language is TC and thus supports general recursion, no algorithm can possibly exist for deciding whether a function is in fact total in the general case. As all primitive recursive functions are provably total, that is a place a com
687.
by nyrikki 3y ago
Very big difference between pure and total functions. Pure functions are memoryless and deterministic. Total functions are defined for all elements in its domain. Square (x²) is a total function. Reciprocal (1/x) is not, since 0 is a r
688.
by nyrikki 3y ago
Division isn't closed on the natural numbers. Total functions have to return either true or false explicitly, failure as negation doesn't work. That is where you get Gödel's 'this statement is false' from. This is t
689.
by nyrikki 3y ago
Many graph problems also end up having lower bounds, being subject to conjectures like SETH or 3sum SETH has a sub quadratic lower bound and several other graph problems have cubic lower bounds. Many real systems are often saved because it
690.
by nyrikki 3y ago
Even pass alignment and bias considerations, the fact that larger models saw such a rise of covert racism should worry us about a general regression to the mean in other contexts. There was a post last week about the problem of LLMs being b

More ›