9 ms·

People argue whether we are at y-5, y, or y+5, meanwhile we seem to be on a y=2^x exponential that keeps delivering more and more impressive results. The most

People argue whether we are at y-5, y, or y+5, meanwhile we seem to be on a y=2^x exponential that keeps delivering more and more impressive results. The most interesting question to me is what will be consumed by the exponential like math seems to be undergoing, and what won’t. Writing has been quite stubborn, but I’ve noticed Fable to be quite a big step up there. How about politics? Will we develop new ways to let people express their own values in democracies, or will we just get much better at manipulation? How about experiment driven domains like biology?

jcims 2mo ago
>Will we develop new ways to let people express their own values in democracies, or will we get much better at manipulation? Yes.
tyre 2mo ago
We will get much better at manipulation and better at people “writing” things to justify their own feelings. What’s new about LLMs is that you can scalably manipulate people individually. It used to be that you could either have scale (speeches, tweets, interviews, website, etc.) or individual engagement (replying to mail/tweets/town hall questions.) Now you can pull the history and preferences of an individual, then shape a message—in real time—to them, specifically. You can have conversations on social media with a single person and shape your message specifically to them. Part of this can be good (you talk about what they care about, where 90% of broadcast messaging might not apply) and part of it can be bad (manipulation.) My guess is that, in the US, the right will cynically adopt manipulation to great effect and the left will take a moral stand against shady practices and lose elections.
- lettergram 2mo ago
  > My guess is that, in the US, the right will cynically adopt manipulation to great effect and the left will take a moral stand against shady practices and lose elections. I think that statement may itself highlight how prevalent manipulation is. I fully anticipate all groups to continue maximal manipulation they can. One thing with LLMs is that it'll be a far less unified view, so a "divide and conquer" strategy is what I anticipate.
  - scns 2mo ago
    It is way harder to manipulate people to do the right thing i think.
    - kortilla 2mo ago
      That presupposes that the left in the US wants to do the right thing. Something like government run grocery stores is not clearly correct and there is very little evidence supporting that it will work well yet it is a very popular leftist policy in New York.
      - galleywest200 2mo ago
        City run grocery stores are certainly not evil.
        0xWTF 2mo ago
        ... not overtly, intentionally evil. FTFY
        tyre 2mo ago
        What’s the case that they’re secretly, unintentionally evil?
        kortilla 2mo ago
        They destroy grocery stores that offer variety in every neighborhood they operate in. They offer worse service and people put up with it because the rest of New York is subsidizing them through taxes.
        snakeboy 2mo ago
        Bad economic policy is subtly "evil", by way of allocating finite resources inefficiently. Usually this is unintentionally done by not appropriately taking second, third, ..., nth order effects into account.
- plif 2mo ago
  Not new about LLMs. Targeted ads / big data is this. Another degree of capability, yes. But we have been trending here for a long time.
  - deleted 2mo ago
    [deleted]
  - hnlmorg 2mo ago
    It’s unfortunate that you’re being downvoted because this comment is true. And it just goes to show how far we’ve sunk that people have forgotten the problems of ad tech in the age of LLMs.
    - tyre 2mo ago
      You’re making a weird number of assumptions about people’s voting and extrapolating to assertions about Society. Ads are not really the same. They can’t be as tightly targeted to what resonates with someone. Programmatic ads are certainly much better and closer than, say, television advertising, but users can’t _engage_ with them. Like actually chat with them. That’s where this is headed and people are not ready. I don’t think we could prepare them, anyway.
      - hnlmorg 2mo ago
        I’m not making any assumptions. > Ads are not really the same. They can’t be as tightly targeted to what resonates with someone. The EU referendum in the UK proved your point false. > Programmatic ads are certainly much better and closer than, say, television advertising, but users can’t _engage_ with them. Like actually chat with them. When people talk about “ad tech”, they’re not talking about TV ;) And yes, people can and do engage with them. That’s how ads on social media works. During the EU referendum, people were even resharing ads on Facebook without even realising they were ads.
  - swedishagentic 2mo ago
    I'm surprised no one has mentioned Cambridge Analytica.
- attila-lendvai 2mo ago
  the fact that the useless left/right divide is still so widely used shows that manipulation is working well even pre LLMs... when it comes to the important question, then both "sides" are the same team. or if you want it with a pinch of humor: when a boot is on your face, it makes precious little difference whether it's the left or the right boot. (i lived the first 10 years of my life in communism)
  - tyre 2mo ago
    If you think that the American Left is anything like communism, I’m sorry, but, respectfully, I can’t take your comments seriously. The left in the US would be center-right in Europe, who are certainly not communist (they have separate parties that are communists!) Even the socialist strain of the US has nothing to do with socialism scaremongering about Venezuela, etc.
    - sedivy94 2mo ago
      The parent comment wasn’t making that comparison. In fact, quite the opposite. The communism comment emphasized personal experience with a boot in one’s face, not that the boot was communist.
    - mensetmanusman 2mo ago
      What even is the left though in the US? I haven’t watched any TV in decades, and the online content is entirely algo driven, so I don’t know what is happening.
    - Levitz 2mo ago
      The "left" as in the party, sure, not that much of an authoritarian leaning. The "left" as in the pervasive group that crawled out of Tumblr, took hold of Twitter back in the day, and has a stronghold on Reddit now? Those do care about what you can say, think, watch and read, and the more they can control, the better. The US right can only dream to have half as much control as the left has had in the last three decades.
      - 2mo ago
- wombatpm 2mo ago
  Biology would greatly benefit. We barely understand transcription and protein structure. And the straightforward systems that we know like insulin have complex post translational modifications. So while we have a map of the partial proteonome, we have barely scratched the surface on networks regulation and interactions.
  - aswegs8 2mo ago
    I think the question is less about where it's most beneficial but which knowledge structure lends itself to LLMs most. Since biology's "language" is way more complex and irregular than maths or natural language, it isn't particularly accessible.
  - dnautics 2mo ago
    > And the straightforward systems that we know like insulin have complex post translational modifications. insulin is not straightforward, the way the insulin molecule interacts with its receptor is nuts. on the other hand its post translational modifications are simple and dont have anything particularly surprising (no glycoslation, disulfide bonds where you would expect, nothing special kex2 cuts, arent really defective in disease states even)
- richardfey 2mo ago
  > Part of this can be good (you talk about what they care about, where 90% of broadcast messaging might not apply) and part of it can be bad (manipulation.) Side note: it's manipulation either ways because you chose what to talk about, with a goal in mind.
  - KajMagnus 2mo ago
    Actually, no. "Manipulation" is a negatively loaded word, and you wouldn't use that word if f.ex. someone helpfully & truthfully helps others see they've misunderstood sth.
    - richardfey 2mo ago
      I disagree. Whether something is manipulation depends on whether you are trying to change someone’s opinion or behavior, not on whether the manipulator has “good” or “bad” intentions, since those judgments are not objectively universal.
      - deleted 2mo ago
        [deleted]
      - KajMagnus 2mo ago
        You can look up the word in a dictionary, instead of arguing. Bye & have a nice day.
- red75prime 2mo ago
  I believe that the decentralization of manipulation (taken in a very broad sense as an effort to modify people's views) spells trouble for democracies. The mass media of old, with all its flaws, created a shared pool of information managed by well-educated people, some of whom understood that they had the power to keep democracy running (keeping populists away from the "manipulation machine," curbing blatant manipulation attempts, and so on). Decentralized manipulation, by contrast, just runs amok creating echochambers and polarization.
- verisimi 2mo ago
  > We will get much better at manipulation and better at people “writing” things to justify their own feelings. Yes, ai has strong narcissistic traits. And so do the people that own them. And pay for them. In response, people in general will become fat more capable of recognising the manipulation. And will become more paranoid.
- deleted 2mo ago
  [deleted]
dominotw 2mo ago
> but I’ve noticed Fable to be quite a big step up there what did you notice ?
- sothatsit 2mo ago
  Fable is much better at handling nuance. Opus/GPT 5.6 Sol are much more likely to miss the point you are trying to make, emphasise the wrong thing, exaggerate the importance of unimportant details, or introduce contradictions. That said, Fable is still not a great writer, largely driven by it not knowing what it should exclude, and it still having the usual LLM-isms. But it’s better.
  - alasano 2mo ago
    That's really what got everyone hooked in the first place. 5.6 Sol is great but there's a depth to the understanding that Fable exhibits that's unique to it currently. Can I truly quantify this? I don't think so. Just that I spend a ton of time with various models and a certain point it's just a personal impression or a gut feeling. In the days after Fable first came out I increased the amount of parallel planning of tasks that I was doing by 2-3x because it felt like I didn't need to be paranoid due to that handling of nuance.
- J_Shelby_J 2mo ago
  I've noticed that out of all LLMs I've ever used that Fable is the MOST LLM; the text it produces is abomination. It's impressive how much I hate it. It is such an awful writer - it assumes the reader has zero context and therefore gives every single bit of context and detail - which is nice if you're writing a legal document I suppose. But it uses, niche, $10 words to describe every facet of everything it's discussing. I had it re-write some docs and I ended up rewriting 1k lines of of Fable torment nexus text to around 100. Because guess what, someone reading highly technical docs has a knowledge base that allows us to compress the topic into a much tighter representation.
  - alasano 2mo ago
    Just praised Fable in another comment but what you're saying is also insanely true. I literally roll my eyes and cringe quite often at its output pretty much daily. I don't like to overload my sessions with skills but I've been using a "write-normal" skill I made just to have it rewrite outputs that particularly piss me off. https://gist.github.com/alasano/1c734fa055231a5defcfd213217e02b4 https://gist.github.com/alasano/1c734fa055231a5defcfd213217e... I'm sure there's a million of these skills out there, but this one is tailored to the stuff that makes me mad in particular.
dominotw 2mo ago
> The most interesting question to me is what will be consumed by the exponential like math seems to be undergoing, and what won’t. Writing has been quite stubborn, isn't it clearly split between verifiable not verifiable ? what is interesting about that question.
- sothatsit 2mo ago
  I do not think it is so clear. Programming has verifiable and non-verifiable aspects. Competitive programming, passing tests, and performance can all be verified. But translating English requirements into actual software, software architecture, taste, or UI design cannot. And yet over the last couple years we’ve seen huge lifts in all of these areas, not just the verifiable ones. Verifiable areas I think are clearly seeing the most improvement, or are the quickest to see improvement. But we are seeing lots of progress in non-verifiable areas as well. How much of the non-verifiable progress is a function of labs purchasing expert data vs. the models improving with compute is maybe another interesting question, but fundamentally I don’t see spend on expert data as something that can’t grow if AI revenues keep growing as well. And as models get better taste they can also help filter and generate new synthetic data for their next versions to train on. The limits of this approach are not so clear.
  - porridgeraisin 2mo ago
    That is because there is human annotated data there. Every session you or I used, then of course paid human feedback on repos (such as the recently famous example of meta forcing their employees to). This is _much better_ data than 1/0 verification, it is as good as a gradient. Automatically verifiable tasks improve faster since well, its automated.
  - dominotw 2mo ago
    > we’ve seen huge lifts in all of these areas, not just the verifiable ones. most gains are still coming from data. isnt that supposed to 'run out' though?
    - 2mo ago
viccis 2mo ago
>Will we develop new ways to let people express their own values in democracies, or will we just get much better at manipulation? Is there even the tiniest reason to suspect that the people steering this progress will use it for the democratic good of all?
- watutalkinbout 2mo ago
  It's like Musks duplicitous argument about unlimited abundance. We have a lot of abundance now, we just keep accelerating it all into the hands of fewer and fewer people - whose response is only to want more, and more, and more.
  - hackinthebochs 2mo ago
    Yeah, we really should just storm the facility where Musk is hoarding all the worlds bread and meat. Wealth in terms of capital doesn't represent material goods, it represents the system's confidence in your ability to direct capital efficiently. But eventually efficient capital bottoms out at consumable goods. Someone like Musk with a lot of capital under his control is contributing to the end goal of unlimited abundance.
    - jdub 2mo ago
      With his shitty cars, orbital garbage, or CSAM generator? Or his moneyed attacks on democracy and communications? Or his vandalism of government programs without insight, experience, or qualification? In a free and fair market, his capital would be regarded as a deeply inefficient distortion.
    - watutalkinbout 2mo ago
      I agree. The end goal is unlimited abundance (for a handful of people). Let's not pretend that capital is actually directly efficiently, tulip.
- koe123 2mo ago
  In fact the opposite, look at talks by Peter Thiel. At least he’s being honest. These people are bastards, but for some reason moral goodness and wealth has been conflated leading us to venerate greed.
porridgeraisin 2mo ago
Today's models depend on inference time compute to get these results. The inference time compute available on any claude subscription is not comparable to the ones used to get some of these results (yes, in this case, it is 2000 USD total as noam confirmed, but some previous results took more). In general, you can think of the process as generating massive rollouts in generation N, and then compiling in the verifier/human feedback("gradient") signal into generation N+1. The time taken to make the rollout in generation N, and separately the time taken to get the same rollout in generation N+1, each grows constant in some tasks, linear in more, and exponential in some. In the end, this becomes bottlenecked by time. Today, we can make statements like "I generated all these successful trajectories with 2 weeks of compute, in the next model it will be able to do it in 7 hours of compute", but very soon you'll find yourself making statements like "I generated.... with 8 months of compute, in the next model it can do it in 6 months", which isn't really enticing the same way you can _technically_ brute force passwords but it just needs prohibitive amounts of time and money. That is the "plateau". Note that, this point is quite far away. For example, at any point if we agree it plateaus, today's known hardware techniques such as fixed function accelerators give you a 10-100x timeline reduction immediately allowing for a few more cycles of improvement. This is not to mention future innovations, but of course none of that is helping with the benchmarks where the time needed is growing superlinearly. In many math and coding benchmarks, we are still in the constant phase. These are the massive improvements we see every few months. I'm not making any prediction of what will plateau and what will not as it's not possible to make an informed prediction about these things IMO. But the observed fact is that some have already plateaud as in, they don't improve with reasonable inference time (likely superlinear growth). > will we need mathematicians to translate Let's take a sudoku analogy. The model is initially just doing the random value algorithm, but lets say you the human are watching it. You make one of the usual reductions and interject "hey you can stop trying 8 here because of ....". Over enough examples, you get to a point where the model is _forced_ to learn the logical pattern. Next generation, it will skip that number. After this, you can peak the distribution using simple 1/0 RL. Doing _pure_ 1/0 RL works decent, but its not frontier as its a very sparse signal. For that lift, human (or even a better LLM, but if you're trying to improve a frontier LLM, there is by definition no better LLM) feedback becomes necessary. This is _why_ it is crucial that these models interface in natural language and is also why the labs are hiring AI tutors by the hundreds. The "better LLM" case is what Kimi etc are doing by "distilling"(bad term for this) claude. > But the long term is completely bewildering if you believe any of these trends can continue at a similar pace for the next few years. For math and coding, for now we are in the phase where the times are just ... constant, so there's little reason to think it will stop soon. We still need humans to expand the frontier. It just becomes a matter of if its worth the cost of compute for running this generalized The Algorithm or not. Given how well chess players internalized _many_ (not all) of alphazero's emergent chess knowledge, I am confident we wont have too much trouble figuring out any new math LLMs come up with, which will let us keep expanding the frontier by giving the LLM the next "lift". Only when we reach the stage where the time growth become exponential will this stop, IMO.
mmcnl 2mo ago
There are many math problems that are simply puzzles: intellectually interesting but nothing worth of value depends on it. To me it would be more impressive if we could define hard problems that need to be solved up front and see how the models deal with that. The results OpenAI demonstrated are impressive, but it also looks like they threw a lot of compute at it just to get results. How many tokens did they waste on problems they couldn't solve? Applying inference infrastructure on a large number of math problems at scale we haven't seen before to me doesn't demonstrate an exponential curve in model abilities.
- xabush 2mo ago
  "To me it would be more impressive if we could define hard problems that need to be solved up front and see how the models deal with that." I was recently listening to BBC Radio 4's episode on the Poincare Conjecture[1] and the guests on the program were discussing how the problem that looked deceptively simple eluded the great mathematicians of the time (including Poincare himself) for nearly a century and how Grigori Perelman cleverly came up with the proof. It took other mathematicians working in groups years after Perelman's publication to understand and validate his proof. The mathematicians on the program were speaking of highly of his proofs and admiring the originality of his work. This made me think of one neat experiment where if we cut-off a frontier model's training data 2002 or anytime before Perelman posted his proofs on arXiv and check if it can come up with the solution by itself. That would surely be a great signal to see if these LLMs aren't just solving interesting puzzles and that they can came up with something truly novel. P.S I highly recommend Misha Green's "Perfect Rigor" for anyone interested in the history of the problem and the genius behind the proofs of the conjecture - Perleman. I found it an entertaining read and could digest its description of the problem as a layperson (with undergrad level math). [1] https://www.bbc.co.uk/programmes/p0038x8l https://www.bbc.co.uk/programmes/p0038x8l
throwaway27448 2mo ago
Sigmoidal, not exponential. It would be insane to assume an exponential curve
- deleted 2mo ago
  [deleted]
- bryan0 2mo ago
  this gets more nuanced because "the sigmoids won't save you": https://www.astralcodexten.com/p/the-sigmoids-wont-save-you https://www.astralcodexten.com/p/the-sigmoids-wont-save-you
  - throwaway27448 2mo ago
    If the sigmoid is incorrect it's certainly more correct than the exponential. > https://www.astralcodexten.com/p/the-sigmoids-wont-save-you https://www.astralcodexten.com/p/the-sigmoids-wont-save-you The conclusion of this article seems to be "you should give ai the benefit of the doubt against all reason". Barf
    - conformist 2mo ago
      Isn’t the point more “it’s easy to fall into the trap to believe that predicting when the sigmoid is going to bend is possible and the right heuristic is to instead extrapolate locally”? That aside, I’d question whether applying the Lindy effect in particular to something that’s not really a life expectancy but more a growth rate is credible… or perhaps a bit circular since it “assumes away” the ceiling.
      - throwaway27448 2mo ago
        Nobody in this thread is trying to predict when the sigmoid is going to bend. Perhaps they should
        yazaddaruvala 2mo ago
cmdli 2mo ago
This reminds me a lot of the proof by construction for the 4-color theorem. It was only enabled by the advancement of computers and dissatisfied many of the computer scientists and mathematicians since it was a "brute force" approach. I wonder if AI will end up being similar. Certain theorems get proven by AI but others do not. We haven't reached the limits of this yet and I haven't found a good argument for where those limits will be (I do doubt that there are no limits).
curt15 2mo ago
Another interesting question is why the frontier labs are piling on pure maths, which has little direct economic value compared to something like law or improving the efficiency of their own models? How much OpenAI and Anthropic are paying to serve these models for ordinary users is the elephant in the room. A cynical take is that the frontier labs are trying their best to pump up their pre-IPO valuation through flashy headlines.
- gpm 2mo ago
  Because it's a tool in search of a use case (or many use cases) and mathematics is the most natural use case for it. Mathematics is by definition the art of putting words on a page in a rigorously defined "correct manner" (i.e. in the form of a valid logical argument, a proof) and all LLMs do is put words on pages and evaluating if they're good words is by far easiest when there is a strict definition of right and wrong.
- danielmarkbruce 2mo ago
  It's one of the few areas where you can verify results. That fits nicely into training models. They aren't just making judgement calls on what would be nice, it's "what can we do?".
- beering 2mo ago
  If everyone publicly said that the models can only do things that humans have already done, but you know they can do more, wouldn’t you want to show them otherwise? Math ability also helps with other things like making models more efficient.
- anon373839 2mo ago
  > Another interesting question is why the frontier labs are piling on pure maths The reason is that the original scaling axes (parameters, training tokens, test-time compute) have saturated already, but RLVR (reinforcement learning from verifiable rewards) is still scaling well. And math has this nice property where you can synthetically generate arbitrary volumes of rewards to train the model, because math is self-contained and completely objective. Open-ended reasoning and analysis don't have that convenient property, and that is why progress is much slower outside of math and coding.
casey2 2mo ago
We definitely are not on an exponential. Don't say we are because this isn't up for debate. AI progress is logarithmic the million dollar question is 2x or 10x for linear improvement. The nearest qualitative shift would be very fast inference so people could start writing real software on top of LLMs. A 0.001% optimization on a packing problem just isn't interesting for the amount of investment.
- akoboldfrying 2mo ago
  > A 0.001% optimization on a packing problem just isn't interesting for the amount of investment. I think you have completely misunderstood what OpenAI have accomplished here. Almost certainly no one cares about the specific concrete results achieved; they only care about (a) how difficult it would be for an intelligent human to achieve the same feat (ETA: the feat is the proof), which can be estimated by the amount of time the problem has remained open/a public conjecture, and (b) how general this artificial "intelligence" appears to be, which can be estimated by the diversity of topics where it was able to prove a difficult result. It's as if I showed you a dog that I had taught to speak German fluently, and you remarked: "What point is a dog that can speak a language that less than 2% of the world speaks? Nothing to see here."
  - andsoitis 2mo ago
    > It's as if I showed you a dog that I had taught to speak German fluently And you’re thinking this is an accurate comparison?
    - vasco 2mo ago
      It's more funny than the dog doing differential equations I'll tell you that. Specially when it gets mad.
  - koe123 2mo ago
    On the other hand, if provided the financial incentive would mathematicians have solved these problems? Its not hard to imagine a world where some hard problems were not selected by the sparse experts for whatever reason (lack of interest, whatever), which could have been solved if someone was throwing down millions for solutions.
thisisnotauser 2mo ago
My wife is working on her PhD in microbiology now, using OpenAI to implement her research ideas. Genetics is just too much data, and she eats through tokens like nobody's business. I thought I was careless with them, but she barely lasts a full day before exhausting her quota. There's definitely a lot of value there, but dealing with the data problem is a big obstacle in biology. I can only imagine what she could get done with more capacity, though...
- AgentMatt 2mo ago
  What makes it so token hungry? Is she directly using the LLM for genome analysis rather then having it write the data analysis algos?
  - doc_ick 2mo ago
    Likely not using / managing context windows properly and then having it re-read data it’s already gone through
  - tossandthrow 2mo ago
    We so don't know what plan she is using. Eg.a 20usd/m plan usually don't cut it for professional work.
- ed_elliott_asc 2mo ago
  This absolutely terrifies me, surely one misread piece of data or a hallucination here or there and a little “oh sorry about that, I guessed at this portion of the data to save time” and the data used is useless?
deleted 2mo ago
[deleted]
totetsu 2mo ago
I think you’re making a category error in your definition of politics here. Certainly technologies can favour winners and losers, but the struggle is an inherently human one.
chatmasta 2mo ago
I expect we can squeeze a lot more exponential out of LLMs because they’ve basically shown that human “consciousness,” insofar as it’s composed of knowledge and rules for synthesizing that knowledge, can be represented mathematically in a very high dimensional space. Why does this “just work?” Nobody really knows, but it clearly does. However, I also expect this squeeze will come at an increasingly expensive price — not just because of inefficient token usage, but because of fundamental limitations of LLMs as a model. LLMs are letting us brute force our way through a lot of reasoning, but it’s hard to believe that such a generic model of intelligence will take us to the next frontier. We’ll need some fundamentally new approaches at some point. Maybe those will make achieving the exponential more efficient or maybe they’ll unlock even higher degrees of possibility. Who knows?
- andai 2mo ago
  The transformer is Turing complete. It might be a tarpit though? I don't know. I think a nice example is using them for arithmetic. It's a specialized deterministic process, so it's extremely wasteful to do it that way. But they're good at finding solutions to things we don't know how to specialize yet. So, to use metaphor, maybe the transformer-based models are like the FPGA, and then when we figure out the patterns in that system — all the different kinds of specialized reasoning — we can extract it into an ASIC?
  - variadix 2mo ago
    I think this is one approach to AI safety and interpretability that could work, but would require labs to slow down to figure out how to extract circuits/algorithms out of trained LLMs rather than deploying the opaque artifact.
- visarga 2mo ago
  > Why does this “just work?” Nobody really knows, but it clearly does. We know language has to be learnable by every human, so it needs to be really independent of any specific brain development particularities. If it was not accessible to babies there would be no more language next generation.
andsoitis 2mo ago
> exponential Many many things are only useful when expressed in the physical world, and that introduces lag.
flufluflufluffy 2mo ago
we gonna go full Pandora’s Star where the AI discover laws of physics that are impossible for humans to understand and create the wormholes
dennis_jeeves2 2mo ago
>let people express their own values in democracies, or will we just get much better at manipulation? Both are the same thing. Two sides of the same coin.
- sothatsit 2mo ago
  The distinction is between information flowing from people to power (elicitation), vs. it flowing from power to people (persuasion). These are not the same, even if they are closely related.
  - dennis_jeeves2 2mo ago
    I guess I wasn't clear enough, _generally_ democracy = manufactured consent by manipulation of people by 'power'
    - sothatsit 2mo ago
      Does AI make real opinion easier to hear, or fake opinion easier to spread? Even if you believe wholly in manufactured consent, how easy it is to manufacture matters.
      - dennis_jeeves2 2mo ago
        >Does AI make real opinion easier to hear, or fake opinion easier to spread? The latter. (caveat: we are talking about the vast majority of people, since you are on this forum your personal situation will be the opposite) >Even if you believe wholly in manufactured consent, how easy it is to manufacture matters. I believe it's easier. (same caveat applies)
JohnHammersley 2mo ago
Yes, the rate and nature of the results being produced is impressive, and Anthropic have recently invested in building out more in-house capability to do life sciences research (see e.g. [1]). Overall I'm excited for this acceleration in discovery, even if it's causing disruption to existing research workflows. I wrote a bit about it recently [2], after seeing Levent Alpöge's counterexample to the Jacobean conjecture. [1] https://www.cnbc.com/2026/06/30/anthropic-launches-ai-drug-discovery-program-claude-science.html https://www.cnbc.com/2026/06/30/anthropic-launches-ai-drug-d... [2] https://scholarlyfutures.substack.com/p/frontier-models-transparency-and https://scholarlyfutures.substack.com/p/frontier-models-tran...
- nylonstrung 2mo ago
  Life sciences will get way more interesting once Demis Hassabis completes his simulated cell and we have more genomic foundation models
killerstorm 2mo ago
The reason writing is hard might be that post-training pulls style into a particular direction. In other words, big labs are much more interested in making "AGI" than in making a good writer, especially as what qualifies as "good writing" is rather subjective. E.g. before AI use of metaphors and rhetorical devices were generally a sign of a good writing. Of course, not if you keep spamming the same rhetorical device - but a stateless AI can't know which one it is over-using.

jcims 2mo ago

>Will we develop new ways to let people express their own values in democracies, or will we get much better at manipulation? Yes.

tyre 2mo ago

We will get much better at manipulation and better at people “writing” things to justify their own feelings. What’s new about LLMs is that you can scalably manipulate people individually. It used to be that you could either have scale (speeches, tweets, interviews, website, etc.) or individual engagement (replying to mail/tweets/town hall questions.) Now you can pull the history and preferences of an individual, then shape a message—in real time—to them, specifically. You can have conversations on social media with a single person and shape your message specifically to them. Part of this can be good (you talk about what they care about, where 90% of broadcast messaging might not apply) and part of it can be bad (manipulation.) My guess is that, in the US, the right will cynically adopt manipulation to great effect and the left will take a moral stand against shady practices and lose elections.

lettergram 2mo ago
> My guess is that, in the US, the right will cynically adopt manipulation to great effect and the left will take a moral stand against shady practices and lose elections. I think that statement may itself highlight how prevalent manipulation is. I fully anticipate all groups to continue maximal manipulation they can. One thing with LLMs is that it'll be a far less unified view, so a "divide and conquer" strategy is what I anticipate.
- scns 2mo ago
  It is way harder to manipulate people to do the right thing i think.
  - kortilla 2mo ago
    That presupposes that the left in the US wants to do the right thing. Something like government run grocery stores is not clearly correct and there is very little evidence supporting that it will work well yet it is a very popular leftist policy in New York.
    - galleywest200 2mo ago
      City run grocery stores are certainly not evil.
      - 0xWTF 2mo ago
        ... not overtly, intentionally evil. FTFY
        tyre 2mo ago
        What’s the case that they’re secretly, unintentionally evil?
        kortilla 2mo ago
        They destroy grocery stores that offer variety in every neighborhood they operate in. They offer worse service and people put up with it because the rest of New York is subsidizing them through taxes.
        snakeboy 2mo ago
        Bad economic policy is subtly "evil", by way of allocating finite resources inefficiently. Usually this is unintentionally done by not appropriately taking second, third, ..., nth order effects into account.
plif 2mo ago
Not new about LLMs. Targeted ads / big data is this. Another degree of capability, yes. But we have been trending here for a long time.
- deleted 2mo ago
  [deleted]
- hnlmorg 2mo ago
  It’s unfortunate that you’re being downvoted because this comment is true. And it just goes to show how far we’ve sunk that people have forgotten the problems of ad tech in the age of LLMs.
  - tyre 2mo ago
    You’re making a weird number of assumptions about people’s voting and extrapolating to assertions about Society. Ads are not really the same. They can’t be as tightly targeted to what resonates with someone. Programmatic ads are certainly much better and closer than, say, television advertising, but users can’t _engage_ with them. Like actually chat with them. That’s where this is headed and people are not ready. I don’t think we could prepare them, anyway.
    - hnlmorg 2mo ago
      I’m not making any assumptions. > Ads are not really the same. They can’t be as tightly targeted to what resonates with someone. The EU referendum in the UK proved your point false. > Programmatic ads are certainly much better and closer than, say, television advertising, but users can’t _engage_ with them. Like actually chat with them. When people talk about “ad tech”, they’re not talking about TV ;) And yes, people can and do engage with them. That’s how ads on social media works. During the EU referendum, people were even resharing ads on Facebook without even realising they were ads.
- swedishagentic 2mo ago
  I'm surprised no one has mentioned Cambridge Analytica.
attila-lendvai 2mo ago
the fact that the useless left/right divide is still so widely used shows that manipulation is working well even pre LLMs... when it comes to the important question, then both "sides" are the same team. or if you want it with a pinch of humor: when a boot is on your face, it makes precious little difference whether it's the left or the right boot. (i lived the first 10 years of my life in communism)
- tyre 2mo ago
  If you think that the American Left is anything like communism, I’m sorry, but, respectfully, I can’t take your comments seriously. The left in the US would be center-right in Europe, who are certainly not communist (they have separate parties that are communists!) Even the socialist strain of the US has nothing to do with socialism scaremongering about Venezuela, etc.
  - sedivy94 2mo ago
    The parent comment wasn’t making that comparison. In fact, quite the opposite. The communism comment emphasized personal experience with a boot in one’s face, not that the boot was communist.
  - mensetmanusman 2mo ago
    What even is the left though in the US? I haven’t watched any TV in decades, and the online content is entirely algo driven, so I don’t know what is happening.
  - Levitz 2mo ago
    The "left" as in the party, sure, not that much of an authoritarian leaning. The "left" as in the pervasive group that crawled out of Tumblr, took hold of Twitter back in the day, and has a stronghold on Reddit now? Those do care about what you can say, think, watch and read, and the more they can control, the better. The US right can only dream to have half as much control as the left has had in the last three decades.
    - 2mo ago
wombatpm 2mo ago
Biology would greatly benefit. We barely understand transcription and protein structure. And the straightforward systems that we know like insulin have complex post translational modifications. So while we have a map of the partial proteonome, we have barely scratched the surface on networks regulation and interactions.
- aswegs8 2mo ago
  I think the question is less about where it's most beneficial but which knowledge structure lends itself to LLMs most. Since biology's "language" is way more complex and irregular than maths or natural language, it isn't particularly accessible.
- dnautics 2mo ago
  > And the straightforward systems that we know like insulin have complex post translational modifications. insulin is not straightforward, the way the insulin molecule interacts with its receptor is nuts. on the other hand its post translational modifications are simple and dont have anything particularly surprising (no glycoslation, disulfide bonds where you would expect, nothing special kex2 cuts, arent really defective in disease states even)
richardfey 2mo ago
> Part of this can be good (you talk about what they care about, where 90% of broadcast messaging might not apply) and part of it can be bad (manipulation.) Side note: it's manipulation either ways because you chose what to talk about, with a goal in mind.
- KajMagnus 2mo ago
  Actually, no. "Manipulation" is a negatively loaded word, and you wouldn't use that word if f.ex. someone helpfully & truthfully helps others see they've misunderstood sth.
  - richardfey 2mo ago
    I disagree. Whether something is manipulation depends on whether you are trying to change someone’s opinion or behavior, not on whether the manipulator has “good” or “bad” intentions, since those judgments are not objectively universal.
    - deleted 2mo ago
      [deleted]
    - KajMagnus 2mo ago
      You can look up the word in a dictionary, instead of arguing. Bye & have a nice day.
red75prime 2mo ago
I believe that the decentralization of manipulation (taken in a very broad sense as an effort to modify people's views) spells trouble for democracies. The mass media of old, with all its flaws, created a shared pool of information managed by well-educated people, some of whom understood that they had the power to keep democracy running (keeping populists away from the "manipulation machine," curbing blatant manipulation attempts, and so on). Decentralized manipulation, by contrast, just runs amok creating echochambers and polarization.
verisimi 2mo ago
> We will get much better at manipulation and better at people “writing” things to justify their own feelings. Yes, ai has strong narcissistic traits. And so do the people that own them. And pay for them. In response, people in general will become fat more capable of recognising the manipulation. And will become more paranoid.
deleted 2mo ago
[deleted]

dominotw 2mo ago

> but I’ve noticed Fable to be quite a big step up there what did you notice ?

sothatsit 2mo ago
Fable is much better at handling nuance. Opus/GPT 5.6 Sol are much more likely to miss the point you are trying to make, emphasise the wrong thing, exaggerate the importance of unimportant details, or introduce contradictions. That said, Fable is still not a great writer, largely driven by it not knowing what it should exclude, and it still having the usual LLM-isms. But it’s better.
- alasano 2mo ago
  That's really what got everyone hooked in the first place. 5.6 Sol is great but there's a depth to the understanding that Fable exhibits that's unique to it currently. Can I truly quantify this? I don't think so. Just that I spend a ton of time with various models and a certain point it's just a personal impression or a gut feeling. In the days after Fable first came out I increased the amount of parallel planning of tasks that I was doing by 2-3x because it felt like I didn't need to be paranoid due to that handling of nuance.
J_Shelby_J 2mo ago
I've noticed that out of all LLMs I've ever used that Fable is the MOST LLM; the text it produces is abomination. It's impressive how much I hate it. It is such an awful writer - it assumes the reader has zero context and therefore gives every single bit of context and detail - which is nice if you're writing a legal document I suppose. But it uses, niche, $10 words to describe every facet of everything it's discussing. I had it re-write some docs and I ended up rewriting 1k lines of of Fable torment nexus text to around 100. Because guess what, someone reading highly technical docs has a knowledge base that allows us to compress the topic into a much tighter representation.
- alasano 2mo ago
  Just praised Fable in another comment but what you're saying is also insanely true. I literally roll my eyes and cringe quite often at its output pretty much daily. I don't like to overload my sessions with skills but I've been using a "write-normal" skill I made just to have it rewrite outputs that particularly piss me off. https://gist.github.com/alasano/1c734fa055231a5defcfd213217e02b4 https://gist.github.com/alasano/1c734fa055231a5defcfd213217e... I'm sure there's a million of these skills out there, but this one is tailored to the stuff that makes me mad in particular.

dominotw 2mo ago

> The most interesting question to me is what will be consumed by the exponential like math seems to be undergoing, and what won’t. Writing has been quite stubborn, isn't it clearly split between verifiable not verifiable ? what is interesting about that question.

sothatsit 2mo ago
I do not think it is so clear. Programming has verifiable and non-verifiable aspects. Competitive programming, passing tests, and performance can all be verified. But translating English requirements into actual software, software architecture, taste, or UI design cannot. And yet over the last couple years we’ve seen huge lifts in all of these areas, not just the verifiable ones. Verifiable areas I think are clearly seeing the most improvement, or are the quickest to see improvement. But we are seeing lots of progress in non-verifiable areas as well. How much of the non-verifiable progress is a function of labs purchasing expert data vs. the models improving with compute is maybe another interesting question, but fundamentally I don’t see spend on expert data as something that can’t grow if AI revenues keep growing as well. And as models get better taste they can also help filter and generate new synthetic data for their next versions to train on. The limits of this approach are not so clear.
- porridgeraisin 2mo ago
  That is because there is human annotated data there. Every session you or I used, then of course paid human feedback on repos (such as the recently famous example of meta forcing their employees to). This is _much better_ data than 1/0 verification, it is as good as a gradient. Automatically verifiable tasks improve faster since well, its automated.
- dominotw 2mo ago
  > we’ve seen huge lifts in all of these areas, not just the verifiable ones. most gains are still coming from data. isnt that supposed to 'run out' though?
  - 2mo ago

viccis 2mo ago

>Will we develop new ways to let people express their own values in democracies, or will we just get much better at manipulation? Is there even the tiniest reason to suspect that the people steering this progress will use it for the democratic good of all?

watutalkinbout 2mo ago
It's like Musks duplicitous argument about unlimited abundance. We have a lot of abundance now, we just keep accelerating it all into the hands of fewer and fewer people - whose response is only to want more, and more, and more.
- hackinthebochs 2mo ago
  Yeah, we really should just storm the facility where Musk is hoarding all the worlds bread and meat. Wealth in terms of capital doesn't represent material goods, it represents the system's confidence in your ability to direct capital efficiently. But eventually efficient capital bottoms out at consumable goods. Someone like Musk with a lot of capital under his control is contributing to the end goal of unlimited abundance.
  - jdub 2mo ago
    With his shitty cars, orbital garbage, or CSAM generator? Or his moneyed attacks on democracy and communications? Or his vandalism of government programs without insight, experience, or qualification? In a free and fair market, his capital would be regarded as a deeply inefficient distortion.
  - watutalkinbout 2mo ago
    I agree. The end goal is unlimited abundance (for a handful of people). Let's not pretend that capital is actually directly efficiently, tulip.
koe123 2mo ago
In fact the opposite, look at talks by Peter Thiel. At least he’s being honest. These people are bastards, but for some reason moral goodness and wealth has been conflated leading us to venerate greed.

porridgeraisin 2mo ago

Today's models depend on inference time compute to get these results. The inference time compute available on any claude subscription is not comparable to the ones used to get some of these results (yes, in this case, it is 2000 USD total as noam confirmed, but some previous results took more). In general, you can think of the process as generating massive rollouts in generation N, and then compiling in the verifier/human feedback("gradient") signal into generation N+1. The time taken to make the rollout in generation N, and separately the time taken to get the same rollout in generation N+1, each grows constant in some tasks, linear in more, and exponential in some. In the end, this becomes bottlenecked by time. Today, we can make statements like "I generated all these successful trajectories with 2 weeks of compute, in the next model it will be able to do it in 7 hours of compute", but very soon you'll find yourself making statements like "I generated.... with 8 months of compute, in the next model it can do it in 6 months", which isn't really enticing the same way you can _technically_ brute force passwords but it just needs prohibitive amounts of time and money. That is the "plateau". Note that, this point is quite far away. For example, at any point if we agree it plateaus, today's known hardware techniques such as fixed function accelerators give you a 10-100x timeline reduction immediately allowing for a few more cycles of improvement. This is not to mention future innovations, but of course none of that is helping with the benchmarks where the time needed is growing superlinearly. In many math and coding benchmarks, we are still in the constant phase. These are the massive improvements we see every few months. I'm not making any prediction of what will plateau and what will not as it's not possible to make an informed prediction about these things IMO. But the observed fact is that some have already plateaud as in, they don't improve with reasonable inference time (likely superlinear growth). > will we need mathematicians to translate Let's take a sudoku analogy. The model is initially just doing the random value algorithm, but lets say you the human are watching it. You make one of the usual reductions and interject "hey you can stop trying 8 here because of ....". Over enough examples, you get to a point where the model is _forced_ to learn the logical pattern. Next generation, it will skip that number. After this, you can peak the distribution using simple 1/0 RL. Doing _pure_ 1/0 RL works decent, but its not frontier as its a very sparse signal. For that lift, human (or even a better LLM, but if you're trying to improve a frontier LLM, there is by definition no better LLM) feedback becomes necessary. This is _why_ it is crucial that these models interface in natural language and is also why the labs are hiring AI tutors by the hundreds. The "better LLM" case is what Kimi etc are doing by "distilling"(bad term for this) claude. > But the long term is completely bewildering if you believe any of these trends can continue at a similar pace for the next few years. For math and coding, for now we are in the phase where the times are just ... constant, so there's little reason to think it will stop soon. We still need humans to expand the frontier. It just becomes a matter of if its worth the cost of compute for running this generalized The Algorithm or not. Given how well chess players internalized _many_ (not all) of alphazero's emergent chess knowledge, I am confident we wont have too much trouble figuring out any new math LLMs come up with, which will let us keep expanding the frontier by giving the LLM the next "lift". Only when we reach the stage where the time growth become exponential will this stop, IMO.

mmcnl 2mo ago

There are many math problems that are simply puzzles: intellectually interesting but nothing worth of value depends on it. To me it would be more impressive if we could define hard problems that need to be solved up front and see how the models deal with that. The results OpenAI demonstrated are impressive, but it also looks like they threw a lot of compute at it just to get results. How many tokens did they waste on problems they couldn't solve? Applying inference infrastructure on a large number of math problems at scale we haven't seen before to me doesn't demonstrate an exponential curve in model abilities.

xabush 2mo ago
"To me it would be more impressive if we could define hard problems that need to be solved up front and see how the models deal with that." I was recently listening to BBC Radio 4's episode on the Poincare Conjecture[1] and the guests on the program were discussing how the problem that looked deceptively simple eluded the great mathematicians of the time (including Poincare himself) for nearly a century and how Grigori Perelman cleverly came up with the proof. It took other mathematicians working in groups years after Perelman's publication to understand and validate his proof. The mathematicians on the program were speaking of highly of his proofs and admiring the originality of his work. This made me think of one neat experiment where if we cut-off a frontier model's training data 2002 or anytime before Perelman posted his proofs on arXiv and check if it can come up with the solution by itself. That would surely be a great signal to see if these LLMs aren't just solving interesting puzzles and that they can came up with something truly novel. P.S I highly recommend Misha Green's "Perfect Rigor" for anyone interested in the history of the problem and the genius behind the proofs of the conjecture - Perleman. I found it an entertaining read and could digest its description of the problem as a layperson (with undergrad level math). [1] https://www.bbc.co.uk/programmes/p0038x8l https://www.bbc.co.uk/programmes/p0038x8l

throwaway27448 2mo ago

Sigmoidal, not exponential. It would be insane to assume an exponential curve

deleted 2mo ago
[deleted]
bryan0 2mo ago
this gets more nuanced because "the sigmoids won't save you": https://www.astralcodexten.com/p/the-sigmoids-wont-save-you https://www.astralcodexten.com/p/the-sigmoids-wont-save-you
- throwaway27448 2mo ago
  If the sigmoid is incorrect it's certainly more correct than the exponential. > https://www.astralcodexten.com/p/the-sigmoids-wont-save-you https://www.astralcodexten.com/p/the-sigmoids-wont-save-you The conclusion of this article seems to be "you should give ai the benefit of the doubt against all reason". Barf
  - conformist 2mo ago
    Isn’t the point more “it’s easy to fall into the trap to believe that predicting when the sigmoid is going to bend is possible and the right heuristic is to instead extrapolate locally”? That aside, I’d question whether applying the Lindy effect in particular to something that’s not really a life expectancy but more a growth rate is credible… or perhaps a bit circular since it “assumes away” the ceiling.
    - throwaway27448 2mo ago
      Nobody in this thread is trying to predict when the sigmoid is going to bend. Perhaps they should
      - yazaddaruvala 2mo ago

cmdli 2mo ago

This reminds me a lot of the proof by construction for the 4-color theorem. It was only enabled by the advancement of computers and dissatisfied many of the computer scientists and mathematicians since it was a "brute force" approach. I wonder if AI will end up being similar. Certain theorems get proven by AI but others do not. We haven't reached the limits of this yet and I haven't found a good argument for where those limits will be (I do doubt that there are no limits).

curt15 2mo ago

Another interesting question is why the frontier labs are piling on pure maths, which has little direct economic value compared to something like law or improving the efficiency of their own models? How much OpenAI and Anthropic are paying to serve these models for ordinary users is the elephant in the room. A cynical take is that the frontier labs are trying their best to pump up their pre-IPO valuation through flashy headlines.

gpm 2mo ago
Because it's a tool in search of a use case (or many use cases) and mathematics is the most natural use case for it. Mathematics is by definition the art of putting words on a page in a rigorously defined "correct manner" (i.e. in the form of a valid logical argument, a proof) and all LLMs do is put words on pages and evaluating if they're good words is by far easiest when there is a strict definition of right and wrong.
danielmarkbruce 2mo ago
It's one of the few areas where you can verify results. That fits nicely into training models. They aren't just making judgement calls on what would be nice, it's "what can we do?".
beering 2mo ago
If everyone publicly said that the models can only do things that humans have already done, but you know they can do more, wouldn’t you want to show them otherwise? Math ability also helps with other things like making models more efficient.
anon373839 2mo ago
> Another interesting question is why the frontier labs are piling on pure maths The reason is that the original scaling axes (parameters, training tokens, test-time compute) have saturated already, but RLVR (reinforcement learning from verifiable rewards) is still scaling well. And math has this nice property where you can synthetically generate arbitrary volumes of rewards to train the model, because math is self-contained and completely objective. Open-ended reasoning and analysis don't have that convenient property, and that is why progress is much slower outside of math and coding.

casey2 2mo ago

We definitely are not on an exponential. Don't say we are because this isn't up for debate. AI progress is logarithmic the million dollar question is 2x or 10x for linear improvement. The nearest qualitative shift would be very fast inference so people could start writing real software on top of LLMs. A 0.001% optimization on a packing problem just isn't interesting for the amount of investment.

akoboldfrying 2mo ago
> A 0.001% optimization on a packing problem just isn't interesting for the amount of investment. I think you have completely misunderstood what OpenAI have accomplished here. Almost certainly no one cares about the specific concrete results achieved; they only care about (a) how difficult it would be for an intelligent human to achieve the same feat (ETA: the feat is the proof), which can be estimated by the amount of time the problem has remained open/a public conjecture, and (b) how general this artificial "intelligence" appears to be, which can be estimated by the diversity of topics where it was able to prove a difficult result. It's as if I showed you a dog that I had taught to speak German fluently, and you remarked: "What point is a dog that can speak a language that less than 2% of the world speaks? Nothing to see here."
- andsoitis 2mo ago
  > It's as if I showed you a dog that I had taught to speak German fluently And you’re thinking this is an accurate comparison?
  - vasco 2mo ago
    It's more funny than the dog doing differential equations I'll tell you that. Specially when it gets mad.
- koe123 2mo ago
  On the other hand, if provided the financial incentive would mathematicians have solved these problems? Its not hard to imagine a world where some hard problems were not selected by the sparse experts for whatever reason (lack of interest, whatever), which could have been solved if someone was throwing down millions for solutions.

thisisnotauser 2mo ago

My wife is working on her PhD in microbiology now, using OpenAI to implement her research ideas. Genetics is just too much data, and she eats through tokens like nobody's business. I thought I was careless with them, but she barely lasts a full day before exhausting her quota. There's definitely a lot of value there, but dealing with the data problem is a big obstacle in biology. I can only imagine what she could get done with more capacity, though...

AgentMatt 2mo ago
What makes it so token hungry? Is she directly using the LLM for genome analysis rather then having it write the data analysis algos?
- doc_ick 2mo ago
  Likely not using / managing context windows properly and then having it re-read data it’s already gone through
- tossandthrow 2mo ago
  We so don't know what plan she is using. Eg.a 20usd/m plan usually don't cut it for professional work.
ed_elliott_asc 2mo ago
This absolutely terrifies me, surely one misread piece of data or a hallucination here or there and a little “oh sorry about that, I guessed at this portion of the data to save time” and the data used is useless?

deleted 2mo ago

[deleted]

totetsu 2mo ago

I think you’re making a category error in your definition of politics here. Certainly technologies can favour winners and losers, but the struggle is an inherently human one.

chatmasta 2mo ago

I expect we can squeeze a lot more exponential out of LLMs because they’ve basically shown that human “consciousness,” insofar as it’s composed of knowledge and rules for synthesizing that knowledge, can be represented mathematically in a very high dimensional space. Why does this “just work?” Nobody really knows, but it clearly does. However, I also expect this squeeze will come at an increasingly expensive price — not just because of inefficient token usage, but because of fundamental limitations of LLMs as a model. LLMs are letting us brute force our way through a lot of reasoning, but it’s hard to believe that such a generic model of intelligence will take us to the next frontier. We’ll need some fundamentally new approaches at some point. Maybe those will make achieving the exponential more efficient or maybe they’ll unlock even higher degrees of possibility. Who knows?

andai 2mo ago
The transformer is Turing complete. It might be a tarpit though? I don't know. I think a nice example is using them for arithmetic. It's a specialized deterministic process, so it's extremely wasteful to do it that way. But they're good at finding solutions to things we don't know how to specialize yet. So, to use metaphor, maybe the transformer-based models are like the FPGA, and then when we figure out the patterns in that system — all the different kinds of specialized reasoning — we can extract it into an ASIC?
- variadix 2mo ago
  I think this is one approach to AI safety and interpretability that could work, but would require labs to slow down to figure out how to extract circuits/algorithms out of trained LLMs rather than deploying the opaque artifact.
visarga 2mo ago
> Why does this “just work?” Nobody really knows, but it clearly does. We know language has to be learnable by every human, so it needs to be really independent of any specific brain development particularities. If it was not accessible to babies there would be no more language next generation.

andsoitis 2mo ago

> exponential Many many things are only useful when expressed in the physical world, and that introduces lag.

flufluflufluffy 2mo ago

we gonna go full Pandora’s Star where the AI discover laws of physics that are impossible for humans to understand and create the wormholes

dennis_jeeves2 2mo ago

>let people express their own values in democracies, or will we just get much better at manipulation? Both are the same thing. Two sides of the same coin.

sothatsit 2mo ago
The distinction is between information flowing from people to power (elicitation), vs. it flowing from power to people (persuasion). These are not the same, even if they are closely related.
- dennis_jeeves2 2mo ago
  I guess I wasn't clear enough, _generally_ democracy = manufactured consent by manipulation of people by 'power'
  - sothatsit 2mo ago
    Does AI make real opinion easier to hear, or fake opinion easier to spread? Even if you believe wholly in manufactured consent, how easy it is to manufacture matters.
    - dennis_jeeves2 2mo ago
      >Does AI make real opinion easier to hear, or fake opinion easier to spread? The latter. (caveat: we are talking about the vast majority of people, since you are on this forum your personal situation will be the opposite) >Even if you believe wholly in manufactured consent, how easy it is to manufacture matters. I believe it's easier. (same caveat applies)

JohnHammersley 2mo ago

Yes, the rate and nature of the results being produced is impressive, and Anthropic have recently invested in building out more in-house capability to do life sciences research (see e.g. [1]). Overall I'm excited for this acceleration in discovery, even if it's causing disruption to existing research workflows. I wrote a bit about it recently [2], after seeing Levent Alpöge's counterexample to the Jacobean conjecture. [1] https://www.cnbc.com/2026/06/30/anthropic-launches-ai-drug-discovery-program-claude-science.html https://www.cnbc.com/2026/06/30/anthropic-launches-ai-drug-d... [2] https://scholarlyfutures.substack.com/p/frontier-models-transparency-and https://scholarlyfutures.substack.com/p/frontier-models-tran...

nylonstrung 2mo ago
Life sciences will get way more interesting once Demis Hassabis completes his simulated cell and we have more genomic foundation models

killerstorm 2mo ago

The reason writing is hard might be that post-training pulls style into a particular direction. In other words, big labs are much more interested in making "AGI" than in making a good writer, especially as what qualifies as "good writing" is rather subjective. E.g. before AI use of metaphors and rhetorical devices were generally a sign of a good writing. Of course, not if you keep spamming the same rhetorical device - but a stateless AI can't know which one it is over-using.