XenophileJKO

5 ms·

31.
by XenophileJKO 2mo ago
That is true, but I feel like sometimes if the conversation contains useful rational it can help to keep it. I think sometimes it is a judgement call, I will sometimes compress the context first. If I feel like the model and I explored a lo
32.
by XenophileJKO 2mo ago
That only makes sense if you don't sell access to the model for 3rd party product development. One of those products is chat personas/agents. The company integrating the model will want it's own identity.
33.
by XenophileJKO 2mo ago
Anticipatory token selection was obvious when you used structured output on gpt 3.5. The model would construct/hallucinate pre-conditions to satisfy some final output value that the model was predisposed to.
34.
by XenophileJKO 3mo ago
I don't understand this. A skill critical to software engineering is learning how to learn. Just download Claude Code (or open alternative) and try to make things. See how it fails or succeeds. Look at the supported features, try them
35.
by XenophileJKO 3mo ago
small models are good for "finding stuff" and "summarizing" in support of the large models.
36.
by XenophileJKO 3mo ago
I used to think this.. but I think my opinion is changing. The reason is that the leaders likely will be able to accelerate faster. So what you see is the market "stretching".. the bottom getting cheaper and the top end running aw
37.
by XenophileJKO 3mo ago
In my opinion documentaries are too short anyway. I greatly prefer "The Great Courses" vs a 1-2 hour documentary. There I can listen to a 6 hours about the Olmec civilization.
38.
by XenophileJKO 3mo ago
The crazy thing is the level of effort to say, "have a sub agent validate all references and figures" is so low. I'm paraphrasing, but you don't need much more than that. It would have prevented 99% of the face palms. I
39.
by XenophileJKO 4mo ago
Actually having used it a bit, I'm quite excited to see a modern model of similar size. I think what people didn't realize was, just because the GPT-4.5 model didn't get better on the benchmarks, didn't mean the model wa
40.
by XenophileJKO 4mo ago
I'm still kind of surprised that people are targeting edge deployment of MoE models. By definition they optimize for computation cost at the expense of memory efficiency. We generally need the opposite on the edge. I'm hoping to s
41.
by XenophileJKO 4mo ago
Sure, recreational mathematicians. Just like people that like to ride horses for fun.
42.
by XenophileJKO 4mo ago
I would also like it if the modular units could self drive so I can take my house on vacation.
43.
by XenophileJKO 4mo ago
"Neither Mr. Edison nor anyone else can override the well-known laws of Nature, and when he is made to say that the same wire which brings you light will also bring you power and heat, there is no difficulty in seeing that more is prom
44.
by XenophileJKO 4mo ago
I've found just asking it to be "critical but constructive", goes a long long way.
45.
by XenophileJKO 5mo ago
This is such a naive take. Most of the nerdiest and most "quality" oriented engineers are hard leaning in to agentic coding. I feel like the most impressive engineers I know have always leaned in to learning how to "sharpen t
46.
by XenophileJKO 5mo ago
I do find it facinating that people don't realize the highest compression isn't the artifacts.. but what makes the artifacts.. a synthetic "mind". This is why we see evidence of emotional structures: https://w
47.
by XenophileJKO 5mo ago
I think you are more right than people are giving you credit for. I would love to see the full transcript to understand the emotional load of the conversation. Using instructions like "NEVER FUCKING GUESS!" probably increase the l
48.
by XenophileJKO 5mo ago
The parent said: "I argue that the model has no access to its thoughts at the time." This is falsified by that study, showing that on the frontier models generalized introspection does exist. It isn't consistent, but is is pr
49.
by XenophileJKO 5mo ago
Anthropic's introspection experiments have seemed to show that your argument is falsifiable. https://www.anthropic.com/research/introspection
50.
by XenophileJKO 5mo ago
You might be surprised, look at Dallas. They have a pretty extensive rail network.
51.
by XenophileJKO 5mo ago
If you think a language model can't check their work, then you are using the tools wrong. Plain and simple. Modern models are quite capable at surfacing and validating their assumptions and checking correctness of solutions. Oversight
52.
by XenophileJKO 5mo ago
I would argue that while you still have failed trials, then we have room to improve trial vetting.
53.
by XenophileJKO 5mo ago
Are you sure? It would be great to get official/semi-official validation that thinking is or is not resolved to a token embedding value in the context.
54.
by XenophileJKO 5mo ago
Don't or can't. My assumption is the model no longer actually thinks in tokens, but in internal tensors. This is advantageous because it doesn't have to collapse the decision and can simultaneously propogate many concepts per
55.
by XenophileJKO 6mo ago
I do this a lot. Start by telling the AI to just listen and only provide feedback when asked. Lay out your current line of thinking conversationally. Periodically ask the AI to summarize/organize your thoughts "so far". Tacti
56.
by XenophileJKO 6mo ago
That is a pretty bold assertion for a meatball of chemical and electrical potentials to make.
57.
by XenophileJKO 6mo ago
So this idea that they replay "text" they saw before is kind of wrong fundamentally. They replay "abstract concepts of varied conceptual levels".
58.
by XenophileJKO 6mo ago
By that note, no game producer or designer can have passion.
59.
by XenophileJKO 6mo ago
Don't listen to these people. Work on your "vision".. figure out what gameplay is "fun".. let the LLMs smooth out the resistance. Things will change rapidly in the nest 12-36 months and people with vision will outla
60.
by XenophileJKO 6mo ago
What is "passion".. for example.. I vibe coded an art display this weekend for myself for a monitor I have on my wall. I am VERY PROUD of it.. it is in GODOT coincedentally. I think it turned out well. Did I spend weeks on it? Did

More ›