4 ms·

>one wonders if stoking the model to be unconventional is part of the success I've long suspected that a lot of these model's real capabilities are still locke

>one wonders if stoking the model to be unconventional is part of the success I've long suspected that a lot of these model's real capabilities are still locked behind certain prompts, despite the big labs spending tons of effort on making default responses to simple prompts better. Even really dumb shit like "Answer this: ..." vs "Question: ..." vs "... you'll be judged by <competitor>" that should have zero impact in an ideal world can significantly impact benchmark results. The problem is that you can waste a ton of time finding the right prompt using these "dumb" approaches, while the model actually just required some very specific context that was obvious to you and not to it in many day-to-day situations. My go to method is still to have the model ask me questions as the very first step to any of these problems. They kind of tried that with deep research since the early o-series, but it still needs improvement.

burnerRhodov2 6mo ago
Just the right "prompt" is exactly what happened here. Lean has been developed and incorporated into it's data set. Also, token responses only vaguely correlate to "human language" and it's been proven transformers develop their own internal representation that has created a whole field called machanistic interpretation. Being able to more correctly "parse", AKA using Lean and the right "Prompts, insights and suggestions", will take a whole new meaning in the future.
- bonesss 6mo ago
  > machanistic interpretation Awesome term/info, and (completely orthogonal to whether they’ll take err jerbs): I’m really excited about the social/civic picture that might be enabled by a defined and verifiable ontological and taxonomical foundation shared across humanity, particularly coupled with potential ‘legislation as code’ or ‘legal system as code’ solutions. I’m thinking on a time horizon a bit past my own lifespan, but: even the possibility to objectively map out some specific aspect of a regional approach to social rights in a given time period and consider it with another social framework, alongside automated & verifiable execution of policy, irrespective of the language of origin is incredible. Instead of hundreds and thousands of incommensurate legislative silos we might create a bazaar of shared improvement and governance efficiency. Turnkey mature governance and anti-corruption measures for newborn nations and countries trying to break out of vicious historical exploitation cycles. Fingers crossed.
  - dalmo3 6mo ago
    Ah, yes, 2001 but on land.
    - bitwize 6mo ago
      I consider the scene with Dr. Chandra and SAL 9000 to be a fairly realistic predictive description of how experts interact with LLMs. SAL even has a somewhat obsequious personality.
  - balamatom 6mo ago
    Moldbug called, asked for his mold and bugs back.
  - anon7725 6mo ago
    Do you think the root cause of social/civic failures has been an inadequate policy repository and lack of a map between policy representations? If so, I have a bridge in Alaska for you to encode into your representation scheme.
muzani 6mo ago
They're tuned to target a certain customer demographic solving for certain problems. I've seen standard AI models to absolutely brilliant things sometimes. But the prompts to get it to perform like it did with GPT-3 seem to get lengthier and lengthier in time. At some point we'll probably just snip out smaller, specialized models to do certain things.
omcnoe 6mo ago
Model output reflects on your input, and the effect is self reinforcing over the course of a whole conversation. Color you add around a problem influences the model behavior. A "dumber"/vague framing will get a less insightful solution, or possibly no solution at all. I don't even necessarily think this is a critical flaw - in general it's just the model tuning it's responses to your style of prompt. People utilize LLMs for all kinds of different tasks, and the "modes of thought" for responding to an Erdos problem versus software engineering versus a more human/soft skills topic are all very different. I think the "prompt sensitivity" issue is just coming bundled along with this general behavior.
- WarmWash 6mo ago
  Keeping a pristine context is so important that I used two separate conversations whenever doing something meaningful. One is the main task executor, and the other is for me to bounce random problems, thoughts, and ideas off of while doing everything to keep a pristine context in the executor instance. It's sort of an agentic loop where I am one of the agents
AlienRobot 6mo ago
Yes, it's extremely awkward! Why is a model that can solve problems in scientific literature the same model that can generate random code, write poems in pirate speech, and do all sorts of other random tasks? It feels like there is a lot of untapped power for specialized LLM tasks if they were created for specialists instead of the general populace prompting from a smartphone.

burnerRhodov2 6mo ago

Just the right "prompt" is exactly what happened here. Lean has been developed and incorporated into it's data set. Also, token responses only vaguely correlate to "human language" and it's been proven transformers develop their own internal representation that has created a whole field called machanistic interpretation. Being able to more correctly "parse", AKA using Lean and the right "Prompts, insights and suggestions", will take a whole new meaning in the future.

bonesss 6mo ago
> machanistic interpretation Awesome term/info, and (completely orthogonal to whether they’ll take err jerbs): I’m really excited about the social/civic picture that might be enabled by a defined and verifiable ontological and taxonomical foundation shared across humanity, particularly coupled with potential ‘legislation as code’ or ‘legal system as code’ solutions. I’m thinking on a time horizon a bit past my own lifespan, but: even the possibility to objectively map out some specific aspect of a regional approach to social rights in a given time period and consider it with another social framework, alongside automated & verifiable execution of policy, irrespective of the language of origin is incredible. Instead of hundreds and thousands of incommensurate legislative silos we might create a bazaar of shared improvement and governance efficiency. Turnkey mature governance and anti-corruption measures for newborn nations and countries trying to break out of vicious historical exploitation cycles. Fingers crossed.
- dalmo3 6mo ago
  Ah, yes, 2001 but on land.
  - bitwize 6mo ago
    I consider the scene with Dr. Chandra and SAL 9000 to be a fairly realistic predictive description of how experts interact with LLMs. SAL even has a somewhat obsequious personality.
- balamatom 6mo ago
  Moldbug called, asked for his mold and bugs back.
- anon7725 6mo ago
  Do you think the root cause of social/civic failures has been an inadequate policy repository and lack of a map between policy representations? If so, I have a bridge in Alaska for you to encode into your representation scheme.

muzani 6mo ago

They're tuned to target a certain customer demographic solving for certain problems. I've seen standard AI models to absolutely brilliant things sometimes. But the prompts to get it to perform like it did with GPT-3 seem to get lengthier and lengthier in time. At some point we'll probably just snip out smaller, specialized models to do certain things.

omcnoe 6mo ago

Model output reflects on your input, and the effect is self reinforcing over the course of a whole conversation. Color you add around a problem influences the model behavior. A "dumber"/vague framing will get a less insightful solution, or possibly no solution at all. I don't even necessarily think this is a critical flaw - in general it's just the model tuning it's responses to your style of prompt. People utilize LLMs for all kinds of different tasks, and the "modes of thought" for responding to an Erdos problem versus software engineering versus a more human/soft skills topic are all very different. I think the "prompt sensitivity" issue is just coming bundled along with this general behavior.

WarmWash 6mo ago
Keeping a pristine context is so important that I used two separate conversations whenever doing something meaningful. One is the main task executor, and the other is for me to bounce random problems, thoughts, and ideas off of while doing everything to keep a pristine context in the executor instance. It's sort of an agentic loop where I am one of the agents

AlienRobot 6mo ago

Yes, it's extremely awkward! Why is a model that can solve problems in scientific literature the same model that can generate random code, write poems in pirate speech, and do all sorts of other random tasks? It feels like there is a lot of untapped power for specialized LLM tasks if they were created for specialists instead of the general populace prompting from a smartphone.