6 ms·

I've started a company in this space about 2 years ago. We are doing fine. What we've learned so far is that a lot of these techniques are simply optimisations

by _pdp_ 11mo ago

I've started a company in this space about 2 years ago. We are doing fine. What we've learned so far is that a lot of these techniques are simply optimisations to tackle some deficiency in LLMs that is a problem "today". These are not going to be problems tomorrow because the technology will shift. As it happened many time in the span of the last 2 years. So yah, cool, caching all of that... but give it a couple of months and a better technique will come out - or more capable models. Many years ago when disc encryption on AWS was not an option, my team and I had to spend 3 months to come up with a way to encrypt the discs and do so well because at the time there was no standard way. It was very difficult as that required pushing encrypted images (as far as I remember). Soon after we started, AWS introduced standard disc encryption that you can turn on by clicking a button. We wasted 3 months for nothing. We should have waited! What I've learned from this is that often times it is better to do absolutely nothing.

gchamonlive 11mo ago
I think knowing when to do nothing is being able to evaluate if the problem the team is tackling is essential or tangential to the core focus of the project, and also whether the problem is something new or if it's been around for a while and there is still no standard way to solve it.
- gessha 11mo ago
  Yeah, that will be the make it to brake it moment because if it’s too essential, it will be implemented but if it’s not, it may become a competitive advantage
an0malous 11mo ago
> These are not going to be problems tomorrow because the technology will shift. As it happened many time in the span of the last 2 years. What technology shifts have happened for LLMs in the last 2 years?
- echelon 11mo ago
  We started putting them in image and video models and now image and video models are insane. I think the next period of high and rapid growth will be in media (image, video, sound, 3D), not text. It's much harder to adapt LLMs to solving business use cases with text. Each problem is niche, you have to custom tailor the solution, and the tooling is crude. The media use cases, by contrast, are low hanging fruit and result in 10,000x speedups and cost reductions almost immediately. The models are pure magic. I think more companies would be wise to ignore text for now and focus on visual domain problems. Nano Banana has so much more utility than agents. And there are so many low hanging fruit ways to make lots of money. Don't sleep on image and video. That's where the growth salient is.
  - wild_egg 11mo ago
    > Nano Banana has so much more utility than agents. I am so far removed from multimedia spaces that I truly can't imagine a universe where this could be true. Agents have done incredible things for me and Nano Banana has been a cool gimmick for making memes. Anyone have a use case for media models that'll expand my mind here?
    - echelon 11mo ago
      We now have capacity to program and automate in the optics, signals, and spatial domains. As someone in the film space, here's just one example: we are getting extremely close to being able to make films with only AI tools. Nano Banana makes it easy to create character and location consistent shots that adhere to film language and the rules of storytelling. This still isn't "one shot", and considerable effort still needs to be put in by humans. Not unlike AI assistance in IDEs requiring a human engineer pilot. We're entering the era of two person film studios. You'll undoubtedly start seeing AI short films next year. I had one art school professor tell me that film seems like it's turning into animation, and that "photorealism" is just style transfer or an aesthetic choice. The film space is hardly the only space where these models have utility. There are so many domains. News, shopping, gaming, social media, phone and teleconference, music, game NPCs, GIS, design, marketing, sales, pitching, fashion, sports, all of entertainment, consumer, CAD, navigation, industrial design, even crazy stuff like VTubing, improv, and LARPing. So much of what we do as humans is non-text based. We haven't had effective automation for any of this until this point. This is a huge percentage of the economy. This is actually the beating heart of it all.
exe34 11mo ago
if we wait long enough, we just end up dead, so it turns out we didn't need to do anything at all whatsoever. of course there's a balance - often times starting out and growing up with the technology gives you background and experience that gives you an advantage when it hits escape velocity.
siva7 11mo ago
This is the most important observation. I'm getting so many workshop invitations from my corporate colleagues about AI and agents. What most people don't get that these clever patterns they "invented" will be obsolete next week. This nice company blog about agents - one which got viral recently - will be obsolete next month. It's hard to swallow for my colleagues that in these age - like when you studied gang of four or a software architecture pattern book that you have learned a common language - no, these days the half-life of a pattern for AI is about a week. Even when you ask 10 professionals what an agent actually is - you will get 10 different answers yet they assume that how they use it is the common understanding.
- hibikir 11mo ago
  Note that even many of those "long knowledge" things people learned are today obsolete, but people that follow them just haven't figured it out yet. See how many of those object oriented design patters just look very silly the minute you use immutable data structures, and have access to functional programming constructs in your language. And nowadays most do. Many seminal books on how to program in the early 2000s, especially those covering "pure" OO, look quite silly today.
- Vinnl 11mo ago
  This is also why it's perfectly fine to wait out this AI hype and see what sticks afterward. It probably won't cost too much time to catch up, because at that point everyone who knows what they're doing only learned that a month or two ago anyway.
  - credit_guy 11mo ago
    > It probably won't cost too much time to catch up That's a risky bet. It is more likely that the user interface of AI will evolve. Some things will stick, some will not. Three years from now, many things that are clunky now will be replaced by more intuitive things. But some things that already work now will still be in place. People who have been heavy users of AI between now and then will definitely have a leg start on those who will just start then.
nowittyusername 11mo ago
I agree with the sentiment. things are moving so fast that waiting now is a legitimate strategy. though it is also easy to fall in to the trap of. well if we continue along these lines might as well wait 4-5 years and we get agi. which still true imo does feel off as you arent participating in the process.
lelanthran 11mo ago
> I've started a company in this space about 2 years ago. We are doing fine. You have a positive cash flow from sales of agents? Your revenue exceeds your operating costs? I've been very skeptical that it is possible to make money from agents, having seen how difficult it was for the current well-known players to do so. What is your secret sauce?
- nvader 11mo ago
  Bumping for interest too. Would love to hear what you believe is correlated to success.
- yayitswei 11mo ago
  I'm cash flow positive on my SMS sales agent, it serves just one client and my revenue is at least 3x the cost of inference/hosting. Imo the key is to serve one use case really well rather than overgeneralize.
verdverm 11mo ago
Vendor choice matters. You could use the like of Amazon / Anthropic, or use Google who has had transparent disk encryption for 10+ years, and Gemini which already had the transparent caching discussed built in.
- te_chris 11mo ago
  If you’ve spent any time with the vertex LLM apis you wouldn’t be so enthusiastic about using Google’s platform (I say this as someone who prefers GCP to aws for compute and networking).
  - verdverm 11mo ago
    been using it for years, no idea what you are getting at I've never had the downtime or service busy situations I've heard others complain about with other vendors. They did pricing based on chars back in the day, but now they are token based like everyone else. I like that they are building custom hardware that is industry leading in terms of limiting how much my AI usage impacts the environment. What do you think I shouldn't be enthusiastic about?
toddmorey 11mo ago
In some ways, the fact that the technology will shift is the problem as model behavior keeps changing. It's rather maddening unstable ground to build on. Really hard to gauge the impact to customer experience from a new model.
- ares623 11mo ago
  For a JS dev, it’s just another Tuesday
  - cheschire 11mo ago
    Is JS dev really still so mercurial as it was 5 to 10 years ago? I'm not so sure. Back then, there would be a new topic daily about some new JS framework etc etc. I still occasionally see a blip of activity but I can't say it's anything like what we witnessed in the past. Though I will agree that gen AI trends feel reminiscent of that period of JS dev history.
    - snoman 11mo ago
      If it is, it’s entirely self inflicted today. There’s some tentpole tech that is reliable enough to stick with and get things done. Has been for a while.
    - biztos 11mo ago
      I’m working on a couple apps using Typescript and for me (ex-JS hacker coming back to it after some years) it’s still an insane menu of bad choices and new “better” frameworks, some of which are abandoned before you get done reading the docs. Though I get that it probably moved faster a few years ago. I settled on what seemed like the most “standard” set of things (marketable skills blabla) and every week I read an article about how that stack is dead, and everybody supposedly uses FancyStack now. Adding insult to injury, I have relearned the fine art of inline styles. I assume table layouts are next. To lurch back on topic: I’m doing this for AI-related stuff and yes, the AI pace of change is much worse, but they sure do make a nice feedback loop.
jFriedensreich 11mo ago
exactly what my experience is too. we focus all our energy on the parts that will not be solved by someone else in a few months.
sethev 11mo ago
I suspect you're right, but it's a bit discouraging to consider that an alternative way of framing this is that companies like OpenAI have a huge advantage in this landscape and anything that works will end up behind their API.
nrhrjrjrjtntbt 11mo ago
If you wait long enough in AI they may not need your agent they just use OpenAI directly.
- DrewADesign 11mo ago
  These days it seems like training yourself into a specialty that provides steadyish income for a year before someone obliterates your professional/corporate/field’s scaffolding with AI and you have to start over is kind of a win. Doesn’t it feel like a win? Look at the efficiency!
ramraj07 11mo ago
Vehement disagree. We implemented our own context editing features 4 months back. Claude released a very similar featureset we had all along last month. We were still glad we did it because (A) it took me half a day to do that work (B) our solution is still more powerful for our use case (C) our solution works on other models as well. It all comes down to trying to predict what will be your vendors' roadmap (or if youre savvy, get a peek into it) and whether the feature you want to create is fundamental to your applications behavior (I doubt encryption is unless youre a storage company).
popcorncowboy 11mo ago
This is the "Wait Calculation" and it's fiendish because there exists only some small, finite window in which it is indeed better to start before the tech is "better" in order to "win" (i.e. get "there" first, wherever "there" is in your scenario). Here's a nice article about it: https://www.oneusefulthing.org/p/the-lazy-tyranny-of-the-wait-calculation https://www.oneusefulthing.org/p/the-lazy-tyranny-of-the-wai...
wolttam 11mo ago
This has been my intuition with these models since close to the beginning. Any framework you build around the model is just behaviour that can be trained into the model itself

by _pdp_ 11mo ago

gchamonlive 11mo ago

I think knowing when to do nothing is being able to evaluate if the problem the team is tackling is essential or tangential to the core focus of the project, and also whether the problem is something new or if it's been around for a while and there is still no standard way to solve it.

gessha 11mo ago
Yeah, that will be the make it to brake it moment because if it’s too essential, it will be implemented but if it’s not, it may become a competitive advantage

an0malous 11mo ago

> These are not going to be problems tomorrow because the technology will shift. As it happened many time in the span of the last 2 years. What technology shifts have happened for LLMs in the last 2 years?

echelon 11mo ago
We started putting them in image and video models and now image and video models are insane. I think the next period of high and rapid growth will be in media (image, video, sound, 3D), not text. It's much harder to adapt LLMs to solving business use cases with text. Each problem is niche, you have to custom tailor the solution, and the tooling is crude. The media use cases, by contrast, are low hanging fruit and result in 10,000x speedups and cost reductions almost immediately. The models are pure magic. I think more companies would be wise to ignore text for now and focus on visual domain problems. Nano Banana has so much more utility than agents. And there are so many low hanging fruit ways to make lots of money. Don't sleep on image and video. That's where the growth salient is.
- wild_egg 11mo ago
  > Nano Banana has so much more utility than agents. I am so far removed from multimedia spaces that I truly can't imagine a universe where this could be true. Agents have done incredible things for me and Nano Banana has been a cool gimmick for making memes. Anyone have a use case for media models that'll expand my mind here?
  - echelon 11mo ago
    We now have capacity to program and automate in the optics, signals, and spatial domains. As someone in the film space, here's just one example: we are getting extremely close to being able to make films with only AI tools. Nano Banana makes it easy to create character and location consistent shots that adhere to film language and the rules of storytelling. This still isn't "one shot", and considerable effort still needs to be put in by humans. Not unlike AI assistance in IDEs requiring a human engineer pilot. We're entering the era of two person film studios. You'll undoubtedly start seeing AI short films next year. I had one art school professor tell me that film seems like it's turning into animation, and that "photorealism" is just style transfer or an aesthetic choice. The film space is hardly the only space where these models have utility. There are so many domains. News, shopping, gaming, social media, phone and teleconference, music, game NPCs, GIS, design, marketing, sales, pitching, fashion, sports, all of entertainment, consumer, CAD, navigation, industrial design, even crazy stuff like VTubing, improv, and LARPing. So much of what we do as humans is non-text based. We haven't had effective automation for any of this until this point. This is a huge percentage of the economy. This is actually the beating heart of it all.

exe34 11mo ago

if we wait long enough, we just end up dead, so it turns out we didn't need to do anything at all whatsoever. of course there's a balance - often times starting out and growing up with the technology gives you background and experience that gives you an advantage when it hits escape velocity.

siva7 11mo ago

This is the most important observation. I'm getting so many workshop invitations from my corporate colleagues about AI and agents. What most people don't get that these clever patterns they "invented" will be obsolete next week. This nice company blog about agents - one which got viral recently - will be obsolete next month. It's hard to swallow for my colleagues that in these age - like when you studied gang of four or a software architecture pattern book that you have learned a common language - no, these days the half-life of a pattern for AI is about a week. Even when you ask 10 professionals what an agent actually is - you will get 10 different answers yet they assume that how they use it is the common understanding.

hibikir 11mo ago
Note that even many of those "long knowledge" things people learned are today obsolete, but people that follow them just haven't figured it out yet. See how many of those object oriented design patters just look very silly the minute you use immutable data structures, and have access to functional programming constructs in your language. And nowadays most do. Many seminal books on how to program in the early 2000s, especially those covering "pure" OO, look quite silly today.
Vinnl 11mo ago
This is also why it's perfectly fine to wait out this AI hype and see what sticks afterward. It probably won't cost too much time to catch up, because at that point everyone who knows what they're doing only learned that a month or two ago anyway.
- credit_guy 11mo ago
  > It probably won't cost too much time to catch up That's a risky bet. It is more likely that the user interface of AI will evolve. Some things will stick, some will not. Three years from now, many things that are clunky now will be replaced by more intuitive things. But some things that already work now will still be in place. People who have been heavy users of AI between now and then will definitely have a leg start on those who will just start then.

nowittyusername 11mo ago

I agree with the sentiment. things are moving so fast that waiting now is a legitimate strategy. though it is also easy to fall in to the trap of. well if we continue along these lines might as well wait 4-5 years and we get agi. which still true imo does feel off as you arent participating in the process.

lelanthran 11mo ago

> I've started a company in this space about 2 years ago. We are doing fine. You have a positive cash flow from sales of agents? Your revenue exceeds your operating costs? I've been very skeptical that it is possible to make money from agents, having seen how difficult it was for the current well-known players to do so. What is your secret sauce?

nvader 11mo ago
Bumping for interest too. Would love to hear what you believe is correlated to success.
yayitswei 11mo ago
I'm cash flow positive on my SMS sales agent, it serves just one client and my revenue is at least 3x the cost of inference/hosting. Imo the key is to serve one use case really well rather than overgeneralize.

verdverm 11mo ago

Vendor choice matters. You could use the like of Amazon / Anthropic, or use Google who has had transparent disk encryption for 10+ years, and Gemini which already had the transparent caching discussed built in.

te_chris 11mo ago
If you’ve spent any time with the vertex LLM apis you wouldn’t be so enthusiastic about using Google’s platform (I say this as someone who prefers GCP to aws for compute and networking).
- verdverm 11mo ago
  been using it for years, no idea what you are getting at I've never had the downtime or service busy situations I've heard others complain about with other vendors. They did pricing based on chars back in the day, but now they are token based like everyone else. I like that they are building custom hardware that is industry leading in terms of limiting how much my AI usage impacts the environment. What do you think I shouldn't be enthusiastic about?

toddmorey 11mo ago

In some ways, the fact that the technology will shift is the problem as model behavior keeps changing. It's rather maddening unstable ground to build on. Really hard to gauge the impact to customer experience from a new model.

ares623 11mo ago
For a JS dev, it’s just another Tuesday
- cheschire 11mo ago
  Is JS dev really still so mercurial as it was 5 to 10 years ago? I'm not so sure. Back then, there would be a new topic daily about some new JS framework etc etc. I still occasionally see a blip of activity but I can't say it's anything like what we witnessed in the past. Though I will agree that gen AI trends feel reminiscent of that period of JS dev history.
  - snoman 11mo ago
    If it is, it’s entirely self inflicted today. There’s some tentpole tech that is reliable enough to stick with and get things done. Has been for a while.
  - biztos 11mo ago
    I’m working on a couple apps using Typescript and for me (ex-JS hacker coming back to it after some years) it’s still an insane menu of bad choices and new “better” frameworks, some of which are abandoned before you get done reading the docs. Though I get that it probably moved faster a few years ago. I settled on what seemed like the most “standard” set of things (marketable skills blabla) and every week I read an article about how that stack is dead, and everybody supposedly uses FancyStack now. Adding insult to injury, I have relearned the fine art of inline styles. I assume table layouts are next. To lurch back on topic: I’m doing this for AI-related stuff and yes, the AI pace of change is much worse, but they sure do make a nice feedback loop.

jFriedensreich 11mo ago

exactly what my experience is too. we focus all our energy on the parts that will not be solved by someone else in a few months.

sethev 11mo ago

I suspect you're right, but it's a bit discouraging to consider that an alternative way of framing this is that companies like OpenAI have a huge advantage in this landscape and anything that works will end up behind their API.

nrhrjrjrjtntbt 11mo ago

If you wait long enough in AI they may not need your agent they just use OpenAI directly.

DrewADesign 11mo ago
These days it seems like training yourself into a specialty that provides steadyish income for a year before someone obliterates your professional/corporate/field’s scaffolding with AI and you have to start over is kind of a win. Doesn’t it feel like a win? Look at the efficiency!

ramraj07 11mo ago

Vehement disagree. We implemented our own context editing features 4 months back. Claude released a very similar featureset we had all along last month. We were still glad we did it because (A) it took me half a day to do that work (B) our solution is still more powerful for our use case (C) our solution works on other models as well. It all comes down to trying to predict what will be your vendors' roadmap (or if youre savvy, get a peek into it) and whether the feature you want to create is fundamental to your applications behavior (I doubt encryption is unless youre a storage company).

popcorncowboy 11mo ago

This is the "Wait Calculation" and it's fiendish because there exists only some small, finite window in which it is indeed better to start before the tech is "better" in order to "win" (i.e. get "there" first, wherever "there" is in your scenario). Here's a nice article about it: https://www.oneusefulthing.org/p/the-lazy-tyranny-of-the-wait-calculation https://www.oneusefulthing.org/p/the-lazy-tyranny-of-the-wai...

wolttam 11mo ago

This has been my intuition with these models since close to the beginning. Any framework you build around the model is just behaviour that can be trained into the model itself