10 ms·

> These are not going to be problems tomorrow because the technology will shift. As it happened many time in the span of the last 2 years. What technology shif

> These are not going to be problems tomorrow because the technology will shift. As it happened many time in the span of the last 2 years. What technology shifts have happened for LLMs in the last 2 years?

echelon 11mo ago
We started putting them in image and video models and now image and video models are insane. I think the next period of high and rapid growth will be in media (image, video, sound, 3D), not text. It's much harder to adapt LLMs to solving business use cases with text. Each problem is niche, you have to custom tailor the solution, and the tooling is crude. The media use cases, by contrast, are low hanging fruit and result in 10,000x speedups and cost reductions almost immediately. The models are pure magic. I think more companies would be wise to ignore text for now and focus on visual domain problems. Nano Banana has so much more utility than agents. And there are so many low hanging fruit ways to make lots of money. Don't sleep on image and video. That's where the growth salient is.
- wild_egg 11mo ago
  > Nano Banana has so much more utility than agents. I am so far removed from multimedia spaces that I truly can't imagine a universe where this could be true. Agents have done incredible things for me and Nano Banana has been a cool gimmick for making memes. Anyone have a use case for media models that'll expand my mind here?
  - echelon 11mo ago
    We now have capacity to program and automate in the optics, signals, and spatial domains. As someone in the film space, here's just one example: we are getting extremely close to being able to make films with only AI tools. Nano Banana makes it easy to create character and location consistent shots that adhere to film language and the rules of storytelling. This still isn't "one shot", and considerable effort still needs to be put in by humans. Not unlike AI assistance in IDEs requiring a human engineer pilot. We're entering the era of two person film studios. You'll undoubtedly start seeing AI short films next year. I had one art school professor tell me that film seems like it's turning into animation, and that "photorealism" is just style transfer or an aesthetic choice. The film space is hardly the only space where these models have utility. There are so many domains. News, shopping, gaming, social media, phone and teleconference, music, game NPCs, GIS, design, marketing, sales, pitching, fashion, sports, all of entertainment, consumer, CAD, navigation, industrial design, even crazy stuff like VTubing, improv, and LARPing. So much of what we do as humans is non-text based. We haven't had effective automation for any of this until this point. This is a huge percentage of the economy. This is actually the beating heart of it all.
    - yunwal 11mo ago
      > we are getting extremely close to being able to make films with only AI tools AI still can’t reliably write text on background details. It can’t get shadows right. If you ask it to shoot things from a head on perspective, for example a bookshelf, it fails to keep proportions accurate enough. The bookshelf will not have parallel shelves. The books won’t have text. If in a library, the labels will not be in Dewey decimal order. It still lacks a huge amount of understanding about how the world works necessary to make a film. It has its uses, but pretending like it can make a whole movie is laughable.
      - wild_egg 11mo ago
        I don't think they're suggesting AI could one-shot a whole movie. It would be iterative, just like programming.
        echelon 11mo ago
        Exactly. You can still open the generations in Photoshop. I'd say the image and video tools are much further along and much more useful than AI code gen (not to dunk on code autocomplete). They save so much time and are quite incredible at what they can do.
        yunwal 11mo ago
        Still impossible at this point and for the foreseeable future. You can prompt all you want, you cannot get an image or video model to create certain scenes, even with a well defined spec
      - gabriel666smith 11mo ago
        I don't think equating "extremely close" with "pretending like it can" is a fair way to frame the sentiment of the comment you were replying to. Saying something is close to doing something is not the same as saying it already can. In terms of cinema tech, it took us arguably until the early 1940s to achieve "deep focus in artificial light". About 50 years! The last couple of years of development in generative video looks, to me, like the tech is improving more quickly than the tech it is mimicking did. This seems unsurprising - one was definitely a hardware problem, and the other is most likely a mixture of hardware and software problems. Your complaints (or analogous technical complaints) would have been acceptable issues - things one had to work around - for a good deal of cinema history. We've already reached people complaining about "these book spines are illegible", which feels very close to "it's difficult to shoot in focus, indoors". Will that take four or five decades to achieve, based on the last 3 - 5 years of development? The tech certainly isn't there yet, nor am I pretending like it is, and nor was the comment you replied to. To call it close is not laughable, though, in the historical context. The much more interesting question is: At what point is there an audience for the output? That's the one that will actually matter - not whether it's possible to replicate Citizen Kane.
WA 11mo ago
Not the LLMs. The APIs got more capabilities such as tool/function calling, explicit caching etc.
- dcre 11mo ago
  It is the LLMs because they have to be RLed to be good at these things.
dcre 11mo ago
One example is that there used to be a whole complex apparatus around getting models to do chain of thought reasoning, e.g., LangChain. Now that is built in as reasoning and they are heavily trained to do it. Same with structured outputs and tool calls — you used to have to do a bunch of stuff to get models to produce valid JSON in the shape you want, now it’s built in and again, they are specifically trained around it. It used to be you would have to go find all relevant context up front and give it to the model. Now agent loops can dynamically figure out what they need and make the tool calls to retrieve it. Etc etc.
- mewpmewp2 11mo ago
  LangChain generally felt pointless for me to use, not a good abstraction. It would rather keep you from the most important thing that you need in this fast evolving ecosystem, and it's direct prompt level (if you can even call that low level) understanding of what is going on.
- vinibrito 11mo ago
  For JSON I agree, now I can just mention JSON and provide examples and the response always comes in the right format, but for tool calling and information retrieval I have never seen a system actually work, nor in my tests have these worked. Now, I'm open to the idea that I am just using it wrong, but I have seen several reports around the web that the most that people got in tool calling accuracy is 80%, which is unusable for any production system, also for info retrieval I have seen it lose coherence the more data is available overall. Is there a model that actually achieved 100% tool calling accuracy? So far I built systems for that myself, surrounding the LLM, and only like this it worked well in production.
postalcoder 11mo ago
If we expand this to 3 years, the single biggest shift that totally changed LLM development is the increase in size of context windows from 4,000 to 16,000 to 128,000 to 256,000. When we were at 4,000 and 16,000 context windows, a lot of effort was spent on nailing down text splitting, chunking, and reduction. For all intents and purposes, the size of current context windows obviates all of that work. What else changed? - Multimodal LLMs - Text extraction from PDFs was a major issue for rag/document intelligence. A lot of time was wasted trying to figure out custom text extraction strategies for documents. Now, you can just feed the image of a PDF page into an LLM and get back a better transcription. - Reduced emphasis on vector search. People have found that for most purposes, having an agent grep your documents is cheaper and better than using a more complex rag pipeline. Boris Cherny created a stir when he talked about claude code doing it that way[0] https://news.ycombinator.com/item?id=43163011#43164253 https://news.ycombinator.com/item?id=43163011#43164253
- yesb 11mo ago
  >For all intents and purposes, the size of current context windows obviates all of that work. Large context windows can make some problems easier or go away for sure. But you may still have the same issue of getting the right information to the model. If your data is much larger than e.g. 256k tokens you still need to filter it. Either way, it can still be beneficial (cost, performance, etc.) to filter out most of the irrelevant information. >Reduced emphasis on vector search. People have found that for most purposes, having an agent grep your documents is cheaper and better than using a more complex rag pipeline This has been obvious from the beginning for anyone familiar with information retrieval (R in RAG). It's very common that search queries are looking for exact matches, not just anything with similar meaning. Your linked example is code search. Exact matches/regex type of searches are generally what you are looking for there.
throwaway13337 11mo ago
I'm amazed at this question and the responses you're getting. These last few years, I've noticed that the tone around AI on HN changes quite a bit by waking time zone. EU waking hours have comments that seem disconnected from genAI. And, while the US hours show a lot of resistance, it's more fear than a feeling that the tools are worthless. It's really puzzling to me. This is the first time I noticed such a disconnect in the community about what the reality of things are. To answer your question personally, genAI has changed the way I code drastically about every 6 months in the last two years. The subtle capability differences change what sorts of problems I can offload. The tasks I can trust them with get larger and larger. It started with better autocomplete, and now, well, agents are writing new features as I write this comment.
- the_mitsuhiko 11mo ago
  > EU waking hours have comments that seem disconnected from genAI. And, while the US hours show a lot of resistance, it's more fear than a feeling that the tools are worthless. I don't think it's because the audience is different but because the moderators are asleep when Europeans are up. There are certain topics which don't really survive on the frontpage when moderators are active.
  - jagged-chisel 11mo ago
    I'm unsure how you're using "moderators." We, the audience, are all 'moderators' if we have the karma. The operators of the site are pretty hands-off as far as content in general. This would mean it is because the audience is different.
    - uoaei 11mo ago
      The people who "operate" the website are different from the people who "moderate" the website but both are paid positions. This fru-fru about how "we all play a part" is only serving to obscure the reality.
      - 11mo ago
deepdarkforest 11mo ago
On the foundational level, test time compute(reasoning), heavy RL post training, 1M+ plus context length etc. On the application layer, connecting with sandboxes/VM's is one of the biggest shifts. (Cloudfares codemode etc). Giving an llm a sandbox unlocks on the fly computation, calculations, RPA, anything really. MCP's, or rather standardized function calling is another one. Also, local llm's are becoming almost viable because of better and better distillation, relying on quick web search for facts etc.
- vrighter 11mo ago
  test-time-compute(reasoning): Running the thing in a loop so it can hallucinate on its own words. Let's not kid ourselves as to what it actually is.

echelon 11mo ago

We started putting them in image and video models and now image and video models are insane. I think the next period of high and rapid growth will be in media (image, video, sound, 3D), not text. It's much harder to adapt LLMs to solving business use cases with text. Each problem is niche, you have to custom tailor the solution, and the tooling is crude. The media use cases, by contrast, are low hanging fruit and result in 10,000x speedups and cost reductions almost immediately. The models are pure magic. I think more companies would be wise to ignore text for now and focus on visual domain problems. Nano Banana has so much more utility than agents. And there are so many low hanging fruit ways to make lots of money. Don't sleep on image and video. That's where the growth salient is.

wild_egg 11mo ago
> Nano Banana has so much more utility than agents. I am so far removed from multimedia spaces that I truly can't imagine a universe where this could be true. Agents have done incredible things for me and Nano Banana has been a cool gimmick for making memes. Anyone have a use case for media models that'll expand my mind here?
- echelon 11mo ago
  We now have capacity to program and automate in the optics, signals, and spatial domains. As someone in the film space, here's just one example: we are getting extremely close to being able to make films with only AI tools. Nano Banana makes it easy to create character and location consistent shots that adhere to film language and the rules of storytelling. This still isn't "one shot", and considerable effort still needs to be put in by humans. Not unlike AI assistance in IDEs requiring a human engineer pilot. We're entering the era of two person film studios. You'll undoubtedly start seeing AI short films next year. I had one art school professor tell me that film seems like it's turning into animation, and that "photorealism" is just style transfer or an aesthetic choice. The film space is hardly the only space where these models have utility. There are so many domains. News, shopping, gaming, social media, phone and teleconference, music, game NPCs, GIS, design, marketing, sales, pitching, fashion, sports, all of entertainment, consumer, CAD, navigation, industrial design, even crazy stuff like VTubing, improv, and LARPing. So much of what we do as humans is non-text based. We haven't had effective automation for any of this until this point. This is a huge percentage of the economy. This is actually the beating heart of it all.
  - yunwal 11mo ago
    > we are getting extremely close to being able to make films with only AI tools AI still can’t reliably write text on background details. It can’t get shadows right. If you ask it to shoot things from a head on perspective, for example a bookshelf, it fails to keep proportions accurate enough. The bookshelf will not have parallel shelves. The books won’t have text. If in a library, the labels will not be in Dewey decimal order. It still lacks a huge amount of understanding about how the world works necessary to make a film. It has its uses, but pretending like it can make a whole movie is laughable.
    - wild_egg 11mo ago
      I don't think they're suggesting AI could one-shot a whole movie. It would be iterative, just like programming.
      - echelon 11mo ago
        Exactly. You can still open the generations in Photoshop. I'd say the image and video tools are much further along and much more useful than AI code gen (not to dunk on code autocomplete). They save so much time and are quite incredible at what they can do.
      - yunwal 11mo ago
        Still impossible at this point and for the foreseeable future. You can prompt all you want, you cannot get an image or video model to create certain scenes, even with a well defined spec
    - gabriel666smith 11mo ago
      I don't think equating "extremely close" with "pretending like it can" is a fair way to frame the sentiment of the comment you were replying to. Saying something is close to doing something is not the same as saying it already can. In terms of cinema tech, it took us arguably until the early 1940s to achieve "deep focus in artificial light". About 50 years! The last couple of years of development in generative video looks, to me, like the tech is improving more quickly than the tech it is mimicking did. This seems unsurprising - one was definitely a hardware problem, and the other is most likely a mixture of hardware and software problems. Your complaints (or analogous technical complaints) would have been acceptable issues - things one had to work around - for a good deal of cinema history. We've already reached people complaining about "these book spines are illegible", which feels very close to "it's difficult to shoot in focus, indoors". Will that take four or five decades to achieve, based on the last 3 - 5 years of development? The tech certainly isn't there yet, nor am I pretending like it is, and nor was the comment you replied to. To call it close is not laughable, though, in the historical context. The much more interesting question is: At what point is there an audience for the output? That's the one that will actually matter - not whether it's possible to replicate Citizen Kane.

WA 11mo ago

Not the LLMs. The APIs got more capabilities such as tool/function calling, explicit caching etc.

dcre 11mo ago
It is the LLMs because they have to be RLed to be good at these things.

dcre 11mo ago

One example is that there used to be a whole complex apparatus around getting models to do chain of thought reasoning, e.g., LangChain. Now that is built in as reasoning and they are heavily trained to do it. Same with structured outputs and tool calls — you used to have to do a bunch of stuff to get models to produce valid JSON in the shape you want, now it’s built in and again, they are specifically trained around it. It used to be you would have to go find all relevant context up front and give it to the model. Now agent loops can dynamically figure out what they need and make the tool calls to retrieve it. Etc etc.

mewpmewp2 11mo ago
LangChain generally felt pointless for me to use, not a good abstraction. It would rather keep you from the most important thing that you need in this fast evolving ecosystem, and it's direct prompt level (if you can even call that low level) understanding of what is going on.
vinibrito 11mo ago
For JSON I agree, now I can just mention JSON and provide examples and the response always comes in the right format, but for tool calling and information retrieval I have never seen a system actually work, nor in my tests have these worked. Now, I'm open to the idea that I am just using it wrong, but I have seen several reports around the web that the most that people got in tool calling accuracy is 80%, which is unusable for any production system, also for info retrieval I have seen it lose coherence the more data is available overall. Is there a model that actually achieved 100% tool calling accuracy? So far I built systems for that myself, surrounding the LLM, and only like this it worked well in production.

postalcoder 11mo ago

If we expand this to 3 years, the single biggest shift that totally changed LLM development is the increase in size of context windows from 4,000 to 16,000 to 128,000 to 256,000. When we were at 4,000 and 16,000 context windows, a lot of effort was spent on nailing down text splitting, chunking, and reduction. For all intents and purposes, the size of current context windows obviates all of that work. What else changed? - Multimodal LLMs - Text extraction from PDFs was a major issue for rag/document intelligence. A lot of time was wasted trying to figure out custom text extraction strategies for documents. Now, you can just feed the image of a PDF page into an LLM and get back a better transcription. - Reduced emphasis on vector search. People have found that for most purposes, having an agent grep your documents is cheaper and better than using a more complex rag pipeline. Boris Cherny created a stir when he talked about claude code doing it that way[0] https://news.ycombinator.com/item?id=43163011#43164253 https://news.ycombinator.com/item?id=43163011#43164253

yesb 11mo ago
>For all intents and purposes, the size of current context windows obviates all of that work. Large context windows can make some problems easier or go away for sure. But you may still have the same issue of getting the right information to the model. If your data is much larger than e.g. 256k tokens you still need to filter it. Either way, it can still be beneficial (cost, performance, etc.) to filter out most of the irrelevant information. >Reduced emphasis on vector search. People have found that for most purposes, having an agent grep your documents is cheaper and better than using a more complex rag pipeline This has been obvious from the beginning for anyone familiar with information retrieval (R in RAG). It's very common that search queries are looking for exact matches, not just anything with similar meaning. Your linked example is code search. Exact matches/regex type of searches are generally what you are looking for there.

throwaway13337 11mo ago

I'm amazed at this question and the responses you're getting. These last few years, I've noticed that the tone around AI on HN changes quite a bit by waking time zone. EU waking hours have comments that seem disconnected from genAI. And, while the US hours show a lot of resistance, it's more fear than a feeling that the tools are worthless. It's really puzzling to me. This is the first time I noticed such a disconnect in the community about what the reality of things are. To answer your question personally, genAI has changed the way I code drastically about every 6 months in the last two years. The subtle capability differences change what sorts of problems I can offload. The tasks I can trust them with get larger and larger. It started with better autocomplete, and now, well, agents are writing new features as I write this comment.

the_mitsuhiko 11mo ago
> EU waking hours have comments that seem disconnected from genAI. And, while the US hours show a lot of resistance, it's more fear than a feeling that the tools are worthless. I don't think it's because the audience is different but because the moderators are asleep when Europeans are up. There are certain topics which don't really survive on the frontpage when moderators are active.
- jagged-chisel 11mo ago
  I'm unsure how you're using "moderators." We, the audience, are all 'moderators' if we have the karma. The operators of the site are pretty hands-off as far as content in general. This would mean it is because the audience is different.
  - uoaei 11mo ago
    The people who "operate" the website are different from the people who "moderate" the website but both are paid positions. This fru-fru about how "we all play a part" is only serving to obscure the reality.
    - 11mo ago

deepdarkforest 11mo ago

On the foundational level, test time compute(reasoning), heavy RL post training, 1M+ plus context length etc. On the application layer, connecting with sandboxes/VM's is one of the biggest shifts. (Cloudfares codemode etc). Giving an llm a sandbox unlocks on the fly computation, calculations, RPA, anything really. MCP's, or rather standardized function calling is another one. Also, local llm's are becoming almost viable because of better and better distillation, relying on quick web search for facts etc.

vrighter 11mo ago
test-time-compute(reasoning): Running the thing in a loop so it can hallucinate on its own words. Let's not kid ourselves as to what it actually is.