zby

11 ms·

121.
by zby 1y ago
Lem predicted this in The Washing Machine Tragedy.
122.
by zby 1y ago
We need personal relationships - but we also prefer impersonal relationships - because it frees us from the social calculations about what is fair.
123.
by zby 1y ago
The root problem is not misunderstanding but not trusting each other. When the team implementing something says it is two weeks while the other team believes it should be just a day - in a world with full trust the second team would just ac
124.
by zby 1y ago
The novelty part is a hard one - but maybe in many cases we could substitute something else for it? If an idea promises to beat state of the art in some field - and it is not yet actively researched - then it is novel. But most promising wo
125.
by zby 1y ago
I assume that it would mostly help beginners (and maybe instructors) - but we'll see. If you send me an audio file I can generate the counting video (with the current model) - I am curious of your opinion.
126.
by zby 1y ago
I am working on an app for rhythm training for dancing. The first step is beat detection in music - here is my code for generating videos with beat counting: https://github.com/zby/beat_counter (vibe coding warning). B
127.
by zby 2y ago
:) I was just to add these. I visited the Georgian ones - they really look otherwordly - some towns are packed with them.
128.
by zby 2y ago
" we are going to move it off AMD to our own or partner silicon. We have developed it to be very portable." https://x.com/__tinygrad__/status/1879617702526087346
129.
by zby 2y ago
Not only that - if you have {0,0,0,0.01} - then the probability that you would get any reward at one shot would be very low. And also I have the intuition that giving the rewards to traces at the edge is more efficient - because the model n
130.
by zby 2y ago
I think is where the relative rewards come to play - they sample many thinking traces and reward those that are correct. This works at the current 'cutting edge' for the model - exactly where it could be improved.
131.
by zby 2y ago
I think the reward is relative to other sampled answers for the same question. This way the signal is strong at the very margin of what is possible with a given model and there is less noise in it with impossible or too easy questions. Ther
132.
HawkinsDB: Neuroscience-Inspired Memory Layer for LLM Applications(github.com)
3 points by zby 2y ago|1 comments
133.
by zby 2y ago
Well - it does not need to 'know' anything - it just needs to generate the string "I don't know" when it does not have better connections.
134.
by zby 2y ago
I've just published to Github my own LLM logging and debugging tool with local storage: https://github.com/zby/llm_recorder It is more for debugging than observability in production like your package. I think I am
135.
by zby 2y ago
My wish list for LLM APIs to make them more useful for 'agentic' workflows: Finer grained control over the tools the LLM is supposed to use. The 'tool_choice' should allow giving a list of tools to choose. The point is t
136.
by zby 2y ago
Ah yeah - actually I tested that taking out of context. This is the thing that surprised me - I thought it is about 'writing itself into a corner - but even in a completely different context the LLM is consistently doing an obvious mis
137.
by zby 2y ago
IMHO with a simple loop LLMs are already capable of some meta thinking, even without any internal new architectures. For me where it still fails is that LLMs cannot catch their own mistakes even some obvious ones. Like with GPT 3.5 I had a
138.
by zby 2y ago
ESR has a theory: "The factual piece missing from the article is that sacrificial victims were drugged into semi-consciousness so they wouldn't struggle and disrupt the ceremony. This meant they didn't scream as obsidian kniv
139.
by zby 2y ago
I did not read the article - but I guess it all depends on the level of abstraction we are talking about. There is a very abstract level where you can say that AI learns like a biological brain and there is a level where you would say that
140.
by zby 2y ago
OK - I stand corrected on the point of consuming carbs. But our body produces glucose anyway - so consuming or not does not change much. Brain needs glucose.
141.
by zby 2y ago
That article starts with > The ketogenic diet is a high-fat, adequate-protein, low-carbohydrate dietary therapy Keto diet is low-carbs - not completely carbs free.
142.
by zby 2y ago
Actually carbs are necessary and carbs are sugars. In the past people with diabetes tried to live on a completely carb free diet - but you cannot do that for long. Personally I am not sure I am buying the narrative about fructose - but it i
143.
by zby 2y ago
The definition of entropy (from Wolfram Alpha): > The (Shannon) entropy of a variable X is defined as > H(X)=-sum_(x)P(x)log_2[P(x)] > bits, where P(x) is the probability that X is in the state x, and Plog_2P is defined as 0 if P=0
144.
by zby 2y ago
There are claims that it improves the LLMs on an array of benchmarks - if that is confirmed - wouldn't it be more important than the theory?
145.
by zby 2y ago
These sampling based techniques is a rare occasion where experimenting with consumer hardware can let you improve on SOTA models. I don't think it will last - the end game surely will be a trainable sampler. But for now - enjoy tinkeri
146.
by zby 2y ago
Can it be mixed with the sampling based approaches from optillm ( https://github.com/codelion/optillm )?
147.
LLMs and Reasoning(zzbbyy.substack.com)
1 points by zby 2y ago|0 comments
148.
by zby 2y ago
The paper shows that LLMs do pattern matching - IMHO the reasoning in LLMs emerges from pattern matching: https://open.substack.com/pub/zzbbyy/p/llms-and-reasoning Sometimes this fails - this is all that pape
149.
O1, inference time, Turing Machines and stuff(zzbbyy.substack.com)
1 points by zby 2y ago|0 comments
150.
by zby 2y ago
I am always looking for definitions of "reasoning". My theory is that if we find a good definition - then it will turn out that we can build systems that would combine fuzzy llm thinking with classical algorithms to solve "re

More ›