10 ms·

What I don't understand is, what benefit is there for 99% of companies to get in on the ground floor of LLMs? If you're not developing your own model, you're ef

What I don't understand is, what benefit is there for 99% of companies to get in on the ground floor of LLMs? If you're not developing your own model, you're effectively just beta testing someone else's model. And if the sales pitch of LLMs being able to do basically anything comes true, wouldn't most companies still get the same benefit if they just wait? It seems like a lot of companies are so terrified of missing the boat that they don't sit down and do actual risk analysis.

mrguyorama 2y ago
>What I don't understand is, what benefit is there for 99% of companies to get in on the ground floor of LLMs? Wallstreet. These companies are largely based on increasing the share price. That's what shareholders want, that's what the board wants, and that's what the C-suite, compensated mostly in stock, wants. Nothing really matters except increasing the stock price, and right now the AI bubble is so damn superheated that if you do not announce something AI, you might be punished in the stock market as investors pull their money and put it somewhere that is willing to bullshit in order to collect some of that sweet hype money. Because the vast majority of the stock market is literally a "greater fool" system. The point is not to make good companies that will enrich you with strong dividends forever, but rather to pump and dump. The entire point of the public market (for public companies) is that you can bullshit, pump up your stock price, and then dump those pumped up stocks onto a fool because you know the company isn't nearly as valuable as the stock says it is. Private investors are so goddamned wealthy nowadays that it makes zero sense to engage with the public stock market, and the regulation and demands that it requires of you, unless you are trying to dump on bagholders. Your average trader is so goddamned stupid you can literally run a company with no fundamentals and no plan and have a $13 billion market cap. Your entire business can be "sell shares repeatedly to these same losers who are blatantly illiterate, can't do basic math, and are outright gambling addicts" and retire on the free money that these people THANK YOU for taking. You can openly enrich yourself off these idiots, and they will vote for it willingly! The point of being on the public market is to attempt to take advantage of these people and take their money.
oorza 2y ago
I think you'll find most people in leadership positions at most companies are not that forward thinking, proactive, or frankly intelligent. I thought cost-benefit and risk was analyzed on most big company decisions, until I sat in rooms for a Fortune 500 where those decisions were getting made. If you assume that everyone everywhere is doing just barely the minimum to not get fired, you're right more often than not.
- this_user 2y ago
  Career risk is also a very real motivation. If you are an executive at a company whose competitors are jumping on the AI bandwagon, but you are not, you will have to justify that decision towards your superiors or the board and investors. They might decide that you are making a huge strategic blunder and need to be replaced. Being proven right years later doesn't do much for you when you no longer have a job. And if you were wrong, then things look even worse for you. On the other hand, if you do get on the bandwagon yourself, and things go sideways, you can always point to the fact that everyone else was making the same mistake.
atrus 2y ago
LLMs are a tool though, so it benefits getting in early and gaining experience using the tool. Companies need to either train up people using a tool, or buy that expertise with the tool. Not to mention all the LLM adjacent tools as well. It's a big, messy, wide field of new software things at this point.
- yifanl 2y ago
  Sure, but does it make sense that your cubicle row of data entry guys are on the cutting edge knowledge of LLM expertise? Because the way LLMs are being pushed in every spreedsheet software would imply that's something someone is asking for.
  - Sharlin 2y ago
    Cutting edge of LLM expertise? Just using someone's shitty LLM product doesn't require "cutting edge LLM expertise".
trillic 2y ago
Gotta check the “Does your product have AI” on the Gartner Magic Quadrant surgery.
llm_nerd 2y ago
I assume they believe in the advantages, to the company, of using these AI tools. Of adding them to the workflow, training their employees, and understanding quickly how it impacts their space. If you just wait and all of your competitors have a workforce enabled and equipped with the best tools, you are at a disadvantage. It's being the company that put off computerization or automation when everyone else went wild. And FWIW, I cannot imagine a programmer, in 2024, that remains dismissive of LLMs or the related tooling. While they are grossly oversold by the LinkedIn crowd, and are still a far ways from so-called "prompt engineers" replacing devs, they're a massive accelerator and "second set of eyes", especially if you're doing varied, novel work.
- nixosbestos 2y ago
  ChatGPT thinks 9.11 > 9.9. I'm in no hurry.
  - andiareso 2y ago
    It's weird, "is the following statement about floating point numbers true: 9.8 > 9.11" it works, but otherwise it has no ability to do it with "decimals"
  - guappa 2y ago
    ? My calculator does too. Unless you mean (9, 11) > (9, 9) which is an entirely different thing.
    - jdrek1 2y ago
      You should get your calculator checked. 9.11 is definitely less than 9.9
      - guappa 2y ago
        Lol, i'm an idiot :D
      - mathieuh 2y ago
addicted 2y ago
Talking about what “companies” are doing is misleading because it suggests thst the actions taken will be driven by the medium to long term interests of the company. In reality, the decisions are made by individual executives and managers with their own interests in mind who are being asked by the people they report to what they’re doing in AI. And this goes all the way to the top where the CEOs are basically required to tell shareholders how they’ve implemented AI and how it’s helping them a bunch. One of the nice things about being in AI right now is that your customers will advertise it and lie about how useful it has been.
mukunda_johnson 2y ago
I said the same thing to a previous company before I was let go. Confused why they were butchering their business strategy in favor of a gold rush. The main benefit of LLMs was already abundantly clear: literally just chat with it in day to day work when you can. Ask it questions about accounting, other domains it knows, etc. That's like up to 10-20% performance increase on tasks if you align OK. Still, they were in search of a unicorn, and it was really tiring to be asked regularly how AI could help my workflows. They were not even spending a real budget on discovering "groundbreaking" use cases, meanwhile hounding us to shove a RAG-bot into every product they owned. The only thing that made sense was that it was a marketing strategy to promote visibility, but they would not acknowledge that or tell us that directly (but still--it was not their business strategy to get NEW customers).
- bitcurious 2y ago
  > The main benefit of LLMs was already abundantly clear In my industry the main benefit (so far) is taking all of our human-legible unstructured data and translating it into computer-legible structured data. Loving it.
  - bronco21016 2y ago
    Are you able to talk more about that? I’m curious what costs are when you run this at scale. We paid a firm $60k to write a custom parser. We parse around 50,000 pages/month. The parser is 100% accurate and has near $0 continuing costs.
  - rcarmo 2y ago
    How do you do quality control?
- rsynnott 2y ago
  > Ask it questions about accounting, other domains it knows Be very careful here if you're using it for anything important! LLMs are quite good at answering questions about accounting in ways which are superficially convincing-looking, yet also complete nonsense. "But the magic robot told me it was okay" will not fly in a tax audit, say.
mentalgear 2y ago
Stock prices goes up for shareholders when the C-suite declares they are integrating "AI". This is a well made and short video about the strategy, but the short of it is: "AI integration is not for the sake of employees, but investor's stock price". https://www.youtube.com/watch?v=6Lxk9NMeWHg https://www.youtube.com/watch?v=6Lxk9NMeWHg
guappa 2y ago
I think for companies to use them is because employees want to use them and there's tons of blog posts (AI written?) that say it increases productivity and you'll be left behind if you don't adopt it. For startups that basically implement a frontend to chatgpt or similar… well they have no chance of ever being profitable but investors might not know that.
harrall 2y ago
I’ll tell you why this happens. You might use ChatGPT for a bit and your initial impressions will be great. It does what I ask of it! You might be aware that it makes mistakes sometimes, but when you use it, you don’t notice it because you’re using it interactively. Now if LLMs are just effective as your experience says, they are indeed extremely useful and you absolutely should see if they can help you. It’s only when you attempt to build a product — and it could be one person writing one Python script — that uses LLMs in an automated way with minimal human input that you really get insights into LLMs’ strengths and their limitations. You realize it could be useful, but you have to sometimes baby it a lot. How many people get to step two? That’s a select few. Most people are stuck in the dreamy phase of trying out interactive LLMs. This is a re-occurring issue with all new technology. Heck it happens with new software frameworks.
- hansonkd 2y ago
  The other problem I find is that LLMs are changing so fast, that what you evaluated 6-12 months ago, might be completely different now with newer models. So the strengths and weaknesses quickly can become outdated as the strengths grow and weaknesses diminish. When the first batch of LLMs people tried in 2023 had a lot of weaknesses. At the end of 2024, we can see increases in performance in speed and the complexity of output. People are creating frameworks on top of the LLMs that further increase their value. We went from thousands of tokens in context to millions of tokens pretty fast. I can see myself dividing problems up into 4 groups: 1. LLMs currently solve the problem 2. It doesn't solve it now, but we are within a couple iteration of next generation models or frameworks to be able to solve it 3. LLMs are still years off from being able to solve this effectively so wait and implement it when it can. 4. LLMs will never solve this. I think a lot of people building products are in group 2 right now.
  - rcarmo 2y ago
    Realism eventually sets in and they move to 3 and 4.
- tananan 2y ago
  This definitely resonates but I'm left wondering why there hasn't been a collective "sobering up" on this front. Not on a personal/team/company level, but just in terms of the general push to cram AI into everything. For how much longer will new s assault us in software where it ostensibly won't be that useful? It seems that the effort required to make an LLM work robustly within a single context (spreadsheet, worddoc, email, whatever) is so gargantuan (honestly) that the returns or even the initial manpower wouldn't be there. So any new feels more or less like bloat, and if not fully useless, then at least a bit anxiety inducing in that you have no clue how much you can rely on it.
oneeyedpigeon 2y ago
We're still in the buzzword stage where "AI" increases sales more than it decreases them, regardless of what your product is, whether it actually uses AI, or what it uses it for. So if your business sells pickles, better add "now with AI!" because more people will buy your pickles if they have AI than will stop buying it. (This comment is mainly facetiousness borne out of frustration, but the point stands)
- internet101010 2y ago
  Pure anecdote but in the meetings I have been in eyes glaze over when the SaaS salesman mentions AI in a generic way without providing any examples of how we can use it. The only way it gets any traction is when the technical wingman can answer questions about how it will account for required business logic and permissions.
blihp 2y ago
The thinking is that the trajectory of LLM's will get them an AI flywheel where they can pump money in and get unlimited amounts of intelligence either augmenting or replacing human labor for pennies on the dollar. The business 'thought leaders' on this view it as a largely zero-sum game: get there first or watch your business die as someone else beats you to it. This has a very late-90's vibe and is quite entertaining to watch.
moduspol 2y ago
Our competitors are getting in on the ground floor of LLMs. Do you want to be trying to raise money against them with nothing to tell potential investors about your company's usage of AI?
belfalas 2y ago
> What I don't understand is, what benefit is there for 99% of companies to get in on the ground floor of LLMs? "Mr. President - we must not allow an LLM gap!"
numpad0 2y ago
imwo(w: wrong), it's because just riding along fomo waves is a legit strat. It just doesn't matter if LLM or marijuana serves no purpose or whether it's detrimental or suggested to cause schizophrenia. It's more important for leviathan-class corporates to not be different where it doesn't pay to be different.
kube-system 2y ago
First-mover advantage. It doesn't matter if the product works or not, if it doesn't sell because everyone bought your competitor. The product will mature with time, what won't get better with time is your market position if you're seen as being behind the times.
deegles 2y ago
I think people are laying the groundwork for "true" AI to be plugged in. even if they know it's not currently that effective.
rchaud 2y ago
> what benefit is there for 99% of companies to get in on the ground floor of LLMs? Most LLM use at the corporate level is happening through Office 365 where Microsoft has put a Copilot button on everything including Word, Outlook and PowerPoint. Execs didn't necessarily ask for it, it protrudes very conspicuously in the UI now.

mrguyorama 2y ago

>What I don't understand is, what benefit is there for 99% of companies to get in on the ground floor of LLMs? Wallstreet. These companies are largely based on increasing the share price. That's what shareholders want, that's what the board wants, and that's what the C-suite, compensated mostly in stock, wants. Nothing really matters except increasing the stock price, and right now the AI bubble is so damn superheated that if you do not announce something AI, you might be punished in the stock market as investors pull their money and put it somewhere that is willing to bullshit in order to collect some of that sweet hype money. Because the vast majority of the stock market is literally a "greater fool" system. The point is not to make good companies that will enrich you with strong dividends forever, but rather to pump and dump. The entire point of the public market (for public companies) is that you can bullshit, pump up your stock price, and then dump those pumped up stocks onto a fool because you know the company isn't nearly as valuable as the stock says it is. Private investors are so goddamned wealthy nowadays that it makes zero sense to engage with the public stock market, and the regulation and demands that it requires of you, unless you are trying to dump on bagholders. Your average trader is so goddamned stupid you can literally run a company with no fundamentals and no plan and have a $13 billion market cap. Your entire business can be "sell shares repeatedly to these same losers who are blatantly illiterate, can't do basic math, and are outright gambling addicts" and retire on the free money that these people THANK YOU for taking. You can openly enrich yourself off these idiots, and they will vote for it willingly! The point of being on the public market is to attempt to take advantage of these people and take their money.

oorza 2y ago

I think you'll find most people in leadership positions at most companies are not that forward thinking, proactive, or frankly intelligent. I thought cost-benefit and risk was analyzed on most big company decisions, until I sat in rooms for a Fortune 500 where those decisions were getting made. If you assume that everyone everywhere is doing just barely the minimum to not get fired, you're right more often than not.

this_user 2y ago
Career risk is also a very real motivation. If you are an executive at a company whose competitors are jumping on the AI bandwagon, but you are not, you will have to justify that decision towards your superiors or the board and investors. They might decide that you are making a huge strategic blunder and need to be replaced. Being proven right years later doesn't do much for you when you no longer have a job. And if you were wrong, then things look even worse for you. On the other hand, if you do get on the bandwagon yourself, and things go sideways, you can always point to the fact that everyone else was making the same mistake.

atrus 2y ago

LLMs are a tool though, so it benefits getting in early and gaining experience using the tool. Companies need to either train up people using a tool, or buy that expertise with the tool. Not to mention all the LLM adjacent tools as well. It's a big, messy, wide field of new software things at this point.

yifanl 2y ago
Sure, but does it make sense that your cubicle row of data entry guys are on the cutting edge knowledge of LLM expertise? Because the way LLMs are being pushed in every spreedsheet software would imply that's something someone is asking for.
- Sharlin 2y ago
  Cutting edge of LLM expertise? Just using someone's shitty LLM product doesn't require "cutting edge LLM expertise".

trillic 2y ago

Gotta check the “Does your product have AI” on the Gartner Magic Quadrant surgery.

llm_nerd 2y ago

I assume they believe in the advantages, to the company, of using these AI tools. Of adding them to the workflow, training their employees, and understanding quickly how it impacts their space. If you just wait and all of your competitors have a workforce enabled and equipped with the best tools, you are at a disadvantage. It's being the company that put off computerization or automation when everyone else went wild. And FWIW, I cannot imagine a programmer, in 2024, that remains dismissive of LLMs or the related tooling. While they are grossly oversold by the LinkedIn crowd, and are still a far ways from so-called "prompt engineers" replacing devs, they're a massive accelerator and "second set of eyes", especially if you're doing varied, novel work.

nixosbestos 2y ago
ChatGPT thinks 9.11 > 9.9. I'm in no hurry.
- andiareso 2y ago
  It's weird, "is the following statement about floating point numbers true: 9.8 > 9.11" it works, but otherwise it has no ability to do it with "decimals"
- guappa 2y ago
  ? My calculator does too. Unless you mean (9, 11) > (9, 9) which is an entirely different thing.
  - jdrek1 2y ago
    You should get your calculator checked. 9.11 is definitely less than 9.9
    - guappa 2y ago
      Lol, i'm an idiot :D
    - mathieuh 2y ago

addicted 2y ago

Talking about what “companies” are doing is misleading because it suggests thst the actions taken will be driven by the medium to long term interests of the company. In reality, the decisions are made by individual executives and managers with their own interests in mind who are being asked by the people they report to what they’re doing in AI. And this goes all the way to the top where the CEOs are basically required to tell shareholders how they’ve implemented AI and how it’s helping them a bunch. One of the nice things about being in AI right now is that your customers will advertise it and lie about how useful it has been.

mukunda_johnson 2y ago

I said the same thing to a previous company before I was let go. Confused why they were butchering their business strategy in favor of a gold rush. The main benefit of LLMs was already abundantly clear: literally just chat with it in day to day work when you can. Ask it questions about accounting, other domains it knows, etc. That's like up to 10-20% performance increase on tasks if you align OK. Still, they were in search of a unicorn, and it was really tiring to be asked regularly how AI could help my workflows. They were not even spending a real budget on discovering "groundbreaking" use cases, meanwhile hounding us to shove a RAG-bot into every product they owned. The only thing that made sense was that it was a marketing strategy to promote visibility, but they would not acknowledge that or tell us that directly (but still--it was not their business strategy to get NEW customers).

bitcurious 2y ago
> The main benefit of LLMs was already abundantly clear In my industry the main benefit (so far) is taking all of our human-legible unstructured data and translating it into computer-legible structured data. Loving it.
- bronco21016 2y ago
  Are you able to talk more about that? I’m curious what costs are when you run this at scale. We paid a firm $60k to write a custom parser. We parse around 50,000 pages/month. The parser is 100% accurate and has near $0 continuing costs.
- rcarmo 2y ago
  How do you do quality control?
rsynnott 2y ago
> Ask it questions about accounting, other domains it knows Be very careful here if you're using it for anything important! LLMs are quite good at answering questions about accounting in ways which are superficially convincing-looking, yet also complete nonsense. "But the magic robot told me it was okay" will not fly in a tax audit, say.

mentalgear 2y ago

Stock prices goes up for shareholders when the C-suite declares they are integrating "AI". This is a well made and short video about the strategy, but the short of it is: "AI integration is not for the sake of employees, but investor's stock price". https://www.youtube.com/watch?v=6Lxk9NMeWHg https://www.youtube.com/watch?v=6Lxk9NMeWHg

guappa 2y ago

I think for companies to use them is because employees want to use them and there's tons of blog posts (AI written?) that say it increases productivity and you'll be left behind if you don't adopt it. For startups that basically implement a frontend to chatgpt or similar… well they have no chance of ever being profitable but investors might not know that.

harrall 2y ago

I’ll tell you why this happens. You might use ChatGPT for a bit and your initial impressions will be great. It does what I ask of it! You might be aware that it makes mistakes sometimes, but when you use it, you don’t notice it because you’re using it interactively. Now if LLMs are just effective as your experience says, they are indeed extremely useful and you absolutely should see if they can help you. It’s only when you attempt to build a product — and it could be one person writing one Python script — that uses LLMs in an automated way with minimal human input that you really get insights into LLMs’ strengths and their limitations. You realize it could be useful, but you have to sometimes baby it a lot. How many people get to step two? That’s a select few. Most people are stuck in the dreamy phase of trying out interactive LLMs. This is a re-occurring issue with all new technology. Heck it happens with new software frameworks.

hansonkd 2y ago
The other problem I find is that LLMs are changing so fast, that what you evaluated 6-12 months ago, might be completely different now with newer models. So the strengths and weaknesses quickly can become outdated as the strengths grow and weaknesses diminish. When the first batch of LLMs people tried in 2023 had a lot of weaknesses. At the end of 2024, we can see increases in performance in speed and the complexity of output. People are creating frameworks on top of the LLMs that further increase their value. We went from thousands of tokens in context to millions of tokens pretty fast. I can see myself dividing problems up into 4 groups: 1. LLMs currently solve the problem 2. It doesn't solve it now, but we are within a couple iteration of next generation models or frameworks to be able to solve it 3. LLMs are still years off from being able to solve this effectively so wait and implement it when it can. 4. LLMs will never solve this. I think a lot of people building products are in group 2 right now.
- rcarmo 2y ago
  Realism eventually sets in and they move to 3 and 4.
tananan 2y ago
This definitely resonates but I'm left wondering why there hasn't been a collective "sobering up" on this front. Not on a personal/team/company level, but just in terms of the general push to cram AI into everything. For how much longer will new s assault us in software where it ostensibly won't be that useful? It seems that the effort required to make an LLM work robustly within a single context (spreadsheet, worddoc, email, whatever) is so gargantuan (honestly) that the returns or even the initial manpower wouldn't be there. So any new feels more or less like bloat, and if not fully useless, then at least a bit anxiety inducing in that you have no clue how much you can rely on it.

oneeyedpigeon 2y ago

We're still in the buzzword stage where "AI" increases sales more than it decreases them, regardless of what your product is, whether it actually uses AI, or what it uses it for. So if your business sells pickles, better add "now with AI!" because more people will buy your pickles if they have AI than will stop buying it. (This comment is mainly facetiousness borne out of frustration, but the point stands)

internet101010 2y ago
Pure anecdote but in the meetings I have been in eyes glaze over when the SaaS salesman mentions AI in a generic way without providing any examples of how we can use it. The only way it gets any traction is when the technical wingman can answer questions about how it will account for required business logic and permissions.

blihp 2y ago

The thinking is that the trajectory of LLM's will get them an AI flywheel where they can pump money in and get unlimited amounts of intelligence either augmenting or replacing human labor for pennies on the dollar. The business 'thought leaders' on this view it as a largely zero-sum game: get there first or watch your business die as someone else beats you to it. This has a very late-90's vibe and is quite entertaining to watch.

moduspol 2y ago

Our competitors are getting in on the ground floor of LLMs. Do you want to be trying to raise money against them with nothing to tell potential investors about your company's usage of AI?

belfalas 2y ago

> What I don't understand is, what benefit is there for 99% of companies to get in on the ground floor of LLMs? "Mr. President - we must not allow an LLM gap!"

numpad0 2y ago

imwo(w: wrong), it's because just riding along fomo waves is a legit strat. It just doesn't matter if LLM or marijuana serves no purpose or whether it's detrimental or suggested to cause schizophrenia. It's more important for leviathan-class corporates to not be different where it doesn't pay to be different.

kube-system 2y ago

First-mover advantage. It doesn't matter if the product works or not, if it doesn't sell because everyone bought your competitor. The product will mature with time, what won't get better with time is your market position if you're seen as being behind the times.

deegles 2y ago

I think people are laying the groundwork for "true" AI to be plugged in. even if they know it's not currently that effective.

rchaud 2y ago

> what benefit is there for 99% of companies to get in on the ground floor of LLMs? Most LLM use at the corporate level is happening through Office 365 where Microsoft has put a Copilot button on everything including Word, Outlook and PowerPoint. Execs didn't necessarily ask for it, it protrudes very conspicuously in the UI now.