haltist

52 ms·

331.
Puzzles for Automatic Differentiation(github.com)
1 points by haltist 3y ago|0 comments
332.
by haltist 3y ago
Code is logic. LLMs are expressed with logical syntax. I recommend meditating on this fact for some time and trying to understand its implications instead of buying into the hype about LLMs being world models.
333.
by haltist 3y ago
Then why call it a world model? It's a "data" model and not actually a "world" model. World models would need to have logical relationships between the entities being modeled which would include non-trivial logical
334.
by haltist 3y ago
> coherent model of the data generating process -- a world model. I'd never seen a definition of world model but this seems deficient in several ways because it does not mention anything about abstraction and logical reasoning.
335.
by haltist 3y ago
Lots of good insights and quotes.
336.
by haltist 3y ago
Good question. I have no idea why people allow their attentionto be hijacked by algorithms. My guess is they just don't realize what is happening.
337.
by haltist 3y ago
Sometimes I will be having a conversation with someone and then their phone will buzz or make some sound and the person's attention will shift entirely to the phone. I don't think people realize the effect these gadgets have on th
338.
by haltist 3y ago
Very cool topic list.
339.
Language Modeling Is Compression(huggingface.co)
27 points by haltist 3y ago|3 comments
340.
by haltist 3y ago
Alex Karp sells software services and custom applications operating on Palantir's proprietary ontology management tools. One should take whatever he has to say about AI with a grain of salt.
341.
by haltist 3y ago
It doesn't know anything. Large language models are basically Markov chains with a large context for conditional probabilities. If the output contains the current date then it is supplied out of band in some other way. It could be part
342.
by haltist 3y ago
I am saying even in the case that the date was given unless you have direct access to the relevant data you can not conclude the date in the output was included anywhere in the input prompts (system or otherwise).
343.
by haltist 3y ago
It's possible the date is hallucinated. There is no reason that a combination of system prompt and regular prompt when combined can not generate a hallucinated cut off date that does not match the actual date. LLMs are statistical mode
344.
by haltist 3y ago
Hallucinations are a result of how LLMs simply generate sequences of probable tokens. Depending on instruction fine tuning and how your prompt was related to the instruction tuning dataset it might be the case that the most statistically li
345.
Confronting the Technological Society [pdf](github.com)
2 points by haltist 3y ago|0 comments
346.
by haltist 3y ago
How do you validate semantic equivalence?
347.
by haltist 3y ago
Interesting. AI code has no loops so the problem they mention in the beginning about unrolling loops to a certain depth is a non-issue.
348.
by haltist 3y ago
Next step is to add verification for optimized code from the LLM with an SMT solver (like Z3) to remove "hallucinations". If the input and output code can be verified to be equivalent then this would be a great addition to an opti
349.
by haltist 3y ago
How else would you learn about neural networks? You have to write code that expresses the network with some kind of executable semantics, e.g. PyTorch, JAX, Keras, &etc.
350.
Make a digital AI clone of yourself(withdelphi.com)
2 points by haltist 3y ago|0 comments
351.
by haltist 3y ago
I just sampled the cars during a 1 minute window by recording with my phone camera and then counted cars in slow motion. I guess if I had access to the cameras at the intersection I could feed the data through an ML model and get a more acc
352.
by haltist 3y ago
Children have no explict conceptions of numbers and logic so they obviously do not plan the same way someone with explicit knowledge and understanding of logic reasons and plans. One could argue they have implicit understanding as members o
353.
by haltist 3y ago
Digital storage density is increasing and most developed nations have 5G deployments. I imagine the ad models could probably be updated in real time because neither storage nor bandwidth are real bottlenecks so if you are frustrated during
354.
by haltist 3y ago
Yesterday I did some amateur science and measured how many cars pass through a relatively busy intersection during rush hour. I knew it was pretty bad and did some basic math to confirm that things are indeed very bad. Expect things to cont
355.
by haltist 3y ago
Statistical algorithms can be used to segment drivers and if it is possible to figure out that someone in some segment is of a specific orientation then the probability that others in that segments are of the same orientation goes up as wel
356.
by haltist 3y ago
Very interesting. This could be applied recursively to AI compilers to accelerate the training pipeline. After all, GPU code is just a bunch of kernels expressed in some syntax that is very similar to LLVM low level representation so the sa
357.
by haltist 3y ago
LLMs can not plan. There is no LLM that can solve sudoku puzzles by executing the obvious constraint propagation algorithm with backtracking. Therefore, LLMs can neither reason nor plan. Software is not magic and the fact that a lot of peop
358.
by haltist 3y ago
Seems like that should be even more reason for Texas to upgrade their energy infrastructure like California.
359.
by haltist 3y ago
So it sounds like you don't think most of the US will adapt.
360.
by haltist 3y ago
What states do you think will manage to deal with climate change if not California?

More ›