4 ms·

Software engineer here trying to start a startup. The amount of business advice GPT-4 has given me is immeasurable. I'd say I'm getting 10x more non technical

Software engineer here trying to start a startup. The amount of business advice GPT-4 has given me is immeasurable. I'd say I'm getting 10x more non technical business advice in multiple areas than I'm getting with the technical stuff. Kind of like an AI CEO advising me on what to do next.

Jensson 2y ago
> I'd say I'm getting 10x more non technical business advice in multiple areas than I'm getting with the technical stuff. Probably in large part because you can't verify when its wrong since that isn't your area of expertise, so those advice looks better to you. You should assume your org will be the same quality as a program written by a non-technical person using an LLM if you do it that way.
- goshx 2y ago
  The same can be said for a human CEO.
- j-bos 2y ago
  >You should assume your org will be the same quality as a program written by a non-technical person using an LLM if you do it that way. Only if OP is themself not somewhat business savvy
  - loloquwowndueo 2y ago
    Way to ignore the first paragraph of the one you’re replying to where he says exactly this: “ because you can't verify when its wrong since that isn't your area of expertise, so those advice looks better to you.”
    - j-bos 2y ago
      I've seen a non programmer navigate bad llm advice on writing code by double checking the info, both with the same llm and with google searches. It isn't their area of expertise, but their light familiarity is enough to build something decent. I can imagine OP doing similar with business advice.
- boolit 2y ago
  This is a critical insight. Whenever I ask an LLM about a topic I’m knowledgeable about, I can detect that it spews a load of nonsense. Talking to people in other fields, e.g., history and literature, they say it’s the same for them. LLMs are only useful if you know the topic you’re asking questions about. Otherwise, they generate so much nonsense that you cannot verify, that trusting it becomes a liability.
  - sebzim4500 2y ago
    Depends what the topic is. IME it hallucines rarely when asked practical programming questions but basically constantly when asked math questions beyond basic calculus.
    - mirsadm 2y ago
      It constantly invents functions that don't exist.
      - timnetworks 2y ago
        Deal with it like Wolfram, implement those functions! It's picking up on something. :D
    - TheNewsIsHere 2y ago
      The more abstract or nuanced something is, the less capable it seems to be unless you have a domain specific LLM trained on a corpus of quality data. As a business owner with both applied experience and formal education on both the tech and the business sides, I wouldn’t trust random LLM-generated advice on business. There are nuances and a certain humanity in good business that an LLM is unlikely to pick up. And the Internet is filled with horrible advice and useless insight on running a business that it can’t distinguish from good advice and valuable insight. My advice to anyone seriously considering a business is that a course or reading a good survey text on business will pay off in ways that ChatGPT can’t (yet?) give you.
    - dvfjsdhgfv 2y ago
- tossandthrow 2y ago
  I was speaking to a fellow software engineer the other day. She has a tech lead on their team. She reports that he is really nice, but some times he is insisting that he knows the working of some part of the system and is giving advice based on his understanding. Only thing is that he is wrong. The software engineer needs to "de-hallucinate" him in order to get a proper answer. I think we are at a time where we need not to categorically be critical about these things. Yes, preciseness is an issue. But there is a good chance that the LLM advice is on par with a random dude from a local accelerator.
  - oxfordmale 2y ago
    The current generation of ChatGPT hallucinates far more systematically than that. The problem is that it can regurgitate what the random dudes from local accelerators have written in their blog posts. We all know blog posts are often a sales pitch and don't include useful advice, because you can charge for the useful advice you give.
    - tossandthrow 2y ago
      > The problem is that it can regurgitate what the random dudes from local accelerators have written in their blog posts. This is not how modern training pipelines work. The models are trained on proprietary alignments that ensure factuality. Regardless, that random dude from a local accelerator probably regurgitates those blog articles himself.
      - yencabulator 2y ago
        "ensures", really?
        tossandthrow 2y ago
        Like with so many other thing, it is a process. Just like reviews "ensure" correctness on Wikipedia - which is actually a good case. In my childhood teachers would make us avoid Wikipedia. Now they advice students to use it.
- shmel 2y ago
  That's a good point, but I think it's important to critically analyze any information you find online. Anything I read online might be bullshit, and yet I've learned a lot from internet. And many other people have as well. I personally see ChatGPT as a better and faster search engine. If I need something basic, it's often on point and almost instant. If I need intricate advanced details about a niche topic, yes, it can be wrong. So are other people. While I lack expertise outside of my main areas to verify it, I usually have enough to judge whether it's a basic knowledge for experts in this area. At very least, ChatGPT doesn't intentionally try to mislead me to sell me something (yet).
  - deleted 2y ago
    [deleted]
  - deleted 2y ago
    [deleted]
- williamcotton 2y ago
  Business ideas can be tested so this epistemic haze isn’t as much of an issue as you’re making it out to be especially when error rates are taken into consideration. I have a friend who has business experience and has been getting verifiably helpful marketing advice from LLMs. Sales are up. That’s all that matters for any set of instructions: does it lead to the expected end result. These “do LLMs always hallucinate” questions are in the category of “do humans have a soul”. This is fine and dandy but the conversation doesn’t always need to go there as there is practical utility to discuss as well.
- latexr 2y ago
  > Probably in large part because you can't verify when its wrong since that isn't your area of expertise, so those advice looks better to you. Which is a kind of Gell-Mann Amnesia. Which, I just noticed, has “LLM” in the name. https://www.goodreads.com/quotes/65213-briefly-stated-the-gell-mann-amnesia-effect-is-as-follows-you https://www.goodreads.com/quotes/65213-briefly-stated-the-ge...
- greg_V 2y ago
  I had the "pleasure" of working for someone who would take the AI's word over mine in my domain of expertise. It was a hair-ripping experience to say the least and I'm while I'm not there anymore, from what I can see they hadn't improved in that area by an inch...
avidphantasm 2y ago
Please disclose your startup name so that we can stay far far away from whatever havoc you unleash.
- tossandthrow 2y ago
  Wow This is extremely rude and downright uncomfortable to read on a forum where people are supposed to be curious. Who tf do you think you are?
  - anileated 2y ago
    “Let me just use a glorified autocomplete instead of learning a subject domain or craft” seems like the opposite of hacker spirit and curiosity.
    - tossandthrow 2y ago
      I would love to understand this better. Why do you think it is anti-curiosity to use LLMs? The original commenter did not say they used it for producing content, so and auto complete is not really precise, is it? You could say that it is an advanced, stochastic knowledge-retrieval system. Does the workings of such merit your curiosity? Does the workings of compression algorithms merit your curiosity? Or are you merely and ignorant person already left behind due to your own inability to be curious?
      - anileated 2y ago
        It sounded like the author is not trying to learn, say, how to run a business, but instead just proxies output of an unthinking tool. After the former, you come out with new knowledge and skill. After the latter, you come out none the wiser. > Does the workings of such merit your curiosity? Not really, if the workings of such are well-understood (like in case of LLMs). > Or are you merely and ignorant person already left behind You think you are mounting a personal attack that would make me feel humiliated, but you need to check who would be ignorant after what I outlined in the beginning of my comment.
lelag 2y ago
The way I see the current crop of AI is that it has mastery of language and a very wide set of knowledge on many different areas but no deep understanding. Ask it about anything you are not very familiar with, and you will get something that looks like a really good answer. Yet, as soon as you try to poke it around subjects you do master, you quickly see its limits and shortcomings. A few months ago, I tested its knowledge on precision machining, and not knowing much about the subject, I got what felt like really solid advice: I felt like it knew everything. Yet, when I reviewed what it told me with my dad, who used to work in the industry, he was not impressed: the answers were missing important bits of information or were sometimes plain wrong. I do feel the current crop of AI is a game changer, but unless there is another breakthrough increasing its reasoning capabilities, I don't think human experts are at risk of losing their jobs anytime soon. The current crop is disrupting occupations that deal purely with language: copywriters, translators... Software engineers are probably very safe, but I believe that language specialists whose main skill is to deeply know a software stack will soon be hurting. I predict that software architecture and computer science skills mastery will become much more valuable than knowing a computer language syntax and associated API in depth. The AI will always be better than you at syntax now. The place I see transformer-based models changing everything is human-computer interaction: voice and multimodality will soon become the main way humans interact with computers rather than mouse and keyboard, and this is huge. However, I don't believe any GenAI model will be up to the task of running a company anytime soon, and I think that advice from a real CEO might prove immensely more useful than whatever GPT-4 is telling you. GPT-4 is probably telling you the obvious—things you could have read on blog spam sites by googling your question. An experienced CEO would probably have unobvious and valuable insights that I don't believe you can get from ChatGPT at the moment.
thih9 2y ago
Can you give a specific example? I’m curious if it was useful because of the AI’s skills, or perhaps it was the conversation itself that helped, like in rubber duck debugging[1] or journaling. In any case, interested to hear more details. [1]: https://en.m.wikipedia.org/wiki/Rubber_duck_debugging https://en.m.wikipedia.org/wiki/Rubber_duck_debugging
- shmel 2y ago
  I find ChatGPT incredibly useful to learn the vocabulary of an unfamiliar area. Often basic information in the area I know very little about can be found easily once you know the right keywords. Even if ChatGPT gets something wrong, it does pattern matching well enough to tell me something like "yes, there is a well-known concept that matches your vague noob description, it is called X". It works well in CS where I have an intuition there must be a mathematical abstraction/algorithm that solves my problem, but I don't know the name. Once I learn the keywords, I sometimes find an open-source library for that or at least a high-level description of the algorithm. It is also great at answering stupid questions. Sometimes I read a wikipedia article and don't understand enough to make sense of it. If I was in uni, I would have asked a prof: "Sorry, it is probably a very stupid question, but why X? and what is Y actually?" ChatGPT can give me ELI5 at pretty much anything and is faster than googling.
  - antihero 2y ago
    When learning a new thing it’s very good for compiling knowledge from lots of sources and laying it out in a good format. As long as it’s easy to verify when it screws up.
  - thih9 2y ago
    That is CS though; I’m not sure that would work with business advice, where problems can be less tangible or there can be no single or verifiable solution (nor library/algorithm). Unless this approach also helps in the context of business advice?
    - shmel 2y ago
deleted 2y ago
[deleted]
coldtea 2y ago
>Kind of like an AI CEO advising me on what to do next. Or like someone who never was a CEO, but has absorbed all kinds of articles, listicles, and books with business advice (most of them written by hustle hacks, non-CEO content creators, or failed small-time enterpreneurs) and serves advice based on that...
elicksaur 2y ago
I’m not doubting that you have found it helpful. I just wish people would be more judicious than making up numbers like “10x”. Everything can’t be “10x”.
- aussieguy1234 2y ago
  In my case, if you are talking about startups specifically, its true. I've been an engineer now for 15 years. About 5 years ago, long before LLMs became a thing, I got to the point where I stopped using stack overflow and google for most things, only using them for the hardest, most complex questions. Thats how I use LLMs nowdays for technical things, which is not often. Business however is new to me. As a solo founder, i've got to be the ceo, marketer, product manager, customer service rep, tech support person and a bunch of other things i've not previously done. GPT-4 is great at guiding me on these things. In a typical 2 person CEO/CTO founded early stage startup, the CEO might be doing these non technical things until the business gets enough funding to hire their first employees.
karasik-lawyer 2y ago
[dead]
darajava 2y ago
What sort of questions do you ask it?

Jensson 2y ago

> I'd say I'm getting 10x more non technical business advice in multiple areas than I'm getting with the technical stuff. Probably in large part because you can't verify when its wrong since that isn't your area of expertise, so those advice looks better to you. You should assume your org will be the same quality as a program written by a non-technical person using an LLM if you do it that way.

goshx 2y ago
The same can be said for a human CEO.
j-bos 2y ago
>You should assume your org will be the same quality as a program written by a non-technical person using an LLM if you do it that way. Only if OP is themself not somewhat business savvy
- loloquwowndueo 2y ago
  Way to ignore the first paragraph of the one you’re replying to where he says exactly this: “ because you can't verify when its wrong since that isn't your area of expertise, so those advice looks better to you.”
  - j-bos 2y ago
    I've seen a non programmer navigate bad llm advice on writing code by double checking the info, both with the same llm and with google searches. It isn't their area of expertise, but their light familiarity is enough to build something decent. I can imagine OP doing similar with business advice.
boolit 2y ago
This is a critical insight. Whenever I ask an LLM about a topic I’m knowledgeable about, I can detect that it spews a load of nonsense. Talking to people in other fields, e.g., history and literature, they say it’s the same for them. LLMs are only useful if you know the topic you’re asking questions about. Otherwise, they generate so much nonsense that you cannot verify, that trusting it becomes a liability.
- sebzim4500 2y ago
  Depends what the topic is. IME it hallucines rarely when asked practical programming questions but basically constantly when asked math questions beyond basic calculus.
  - mirsadm 2y ago
    It constantly invents functions that don't exist.
    - timnetworks 2y ago
      Deal with it like Wolfram, implement those functions! It's picking up on something. :D
  - TheNewsIsHere 2y ago
    The more abstract or nuanced something is, the less capable it seems to be unless you have a domain specific LLM trained on a corpus of quality data. As a business owner with both applied experience and formal education on both the tech and the business sides, I wouldn’t trust random LLM-generated advice on business. There are nuances and a certain humanity in good business that an LLM is unlikely to pick up. And the Internet is filled with horrible advice and useless insight on running a business that it can’t distinguish from good advice and valuable insight. My advice to anyone seriously considering a business is that a course or reading a good survey text on business will pay off in ways that ChatGPT can’t (yet?) give you.
  - dvfjsdhgfv 2y ago
tossandthrow 2y ago
I was speaking to a fellow software engineer the other day. She has a tech lead on their team. She reports that he is really nice, but some times he is insisting that he knows the working of some part of the system and is giving advice based on his understanding. Only thing is that he is wrong. The software engineer needs to "de-hallucinate" him in order to get a proper answer. I think we are at a time where we need not to categorically be critical about these things. Yes, preciseness is an issue. But there is a good chance that the LLM advice is on par with a random dude from a local accelerator.
- oxfordmale 2y ago
  The current generation of ChatGPT hallucinates far more systematically than that. The problem is that it can regurgitate what the random dudes from local accelerators have written in their blog posts. We all know blog posts are often a sales pitch and don't include useful advice, because you can charge for the useful advice you give.
  - tossandthrow 2y ago
    > The problem is that it can regurgitate what the random dudes from local accelerators have written in their blog posts. This is not how modern training pipelines work. The models are trained on proprietary alignments that ensure factuality. Regardless, that random dude from a local accelerator probably regurgitates those blog articles himself.
    - yencabulator 2y ago
      "ensures", really?
      - tossandthrow 2y ago
        Like with so many other thing, it is a process. Just like reviews "ensure" correctness on Wikipedia - which is actually a good case. In my childhood teachers would make us avoid Wikipedia. Now they advice students to use it.
shmel 2y ago
That's a good point, but I think it's important to critically analyze any information you find online. Anything I read online might be bullshit, and yet I've learned a lot from internet. And many other people have as well. I personally see ChatGPT as a better and faster search engine. If I need something basic, it's often on point and almost instant. If I need intricate advanced details about a niche topic, yes, it can be wrong. So are other people. While I lack expertise outside of my main areas to verify it, I usually have enough to judge whether it's a basic knowledge for experts in this area. At very least, ChatGPT doesn't intentionally try to mislead me to sell me something (yet).
- deleted 2y ago
  [deleted]
- deleted 2y ago
  [deleted]
williamcotton 2y ago
Business ideas can be tested so this epistemic haze isn’t as much of an issue as you’re making it out to be especially when error rates are taken into consideration. I have a friend who has business experience and has been getting verifiably helpful marketing advice from LLMs. Sales are up. That’s all that matters for any set of instructions: does it lead to the expected end result. These “do LLMs always hallucinate” questions are in the category of “do humans have a soul”. This is fine and dandy but the conversation doesn’t always need to go there as there is practical utility to discuss as well.
latexr 2y ago
> Probably in large part because you can't verify when its wrong since that isn't your area of expertise, so those advice looks better to you. Which is a kind of Gell-Mann Amnesia. Which, I just noticed, has “LLM” in the name. https://www.goodreads.com/quotes/65213-briefly-stated-the-gell-mann-amnesia-effect-is-as-follows-you https://www.goodreads.com/quotes/65213-briefly-stated-the-ge...
greg_V 2y ago
I had the "pleasure" of working for someone who would take the AI's word over mine in my domain of expertise. It was a hair-ripping experience to say the least and I'm while I'm not there anymore, from what I can see they hadn't improved in that area by an inch...

avidphantasm 2y ago

Please disclose your startup name so that we can stay far far away from whatever havoc you unleash.

tossandthrow 2y ago
Wow This is extremely rude and downright uncomfortable to read on a forum where people are supposed to be curious. Who tf do you think you are?
- anileated 2y ago
  “Let me just use a glorified autocomplete instead of learning a subject domain or craft” seems like the opposite of hacker spirit and curiosity.
  - tossandthrow 2y ago
    I would love to understand this better. Why do you think it is anti-curiosity to use LLMs? The original commenter did not say they used it for producing content, so and auto complete is not really precise, is it? You could say that it is an advanced, stochastic knowledge-retrieval system. Does the workings of such merit your curiosity? Does the workings of compression algorithms merit your curiosity? Or are you merely and ignorant person already left behind due to your own inability to be curious?
    - anileated 2y ago
      It sounded like the author is not trying to learn, say, how to run a business, but instead just proxies output of an unthinking tool. After the former, you come out with new knowledge and skill. After the latter, you come out none the wiser. > Does the workings of such merit your curiosity? Not really, if the workings of such are well-understood (like in case of LLMs). > Or are you merely and ignorant person already left behind You think you are mounting a personal attack that would make me feel humiliated, but you need to check who would be ignorant after what I outlined in the beginning of my comment.

lelag 2y ago

The way I see the current crop of AI is that it has mastery of language and a very wide set of knowledge on many different areas but no deep understanding. Ask it about anything you are not very familiar with, and you will get something that looks like a really good answer. Yet, as soon as you try to poke it around subjects you do master, you quickly see its limits and shortcomings. A few months ago, I tested its knowledge on precision machining, and not knowing much about the subject, I got what felt like really solid advice: I felt like it knew everything. Yet, when I reviewed what it told me with my dad, who used to work in the industry, he was not impressed: the answers were missing important bits of information or were sometimes plain wrong. I do feel the current crop of AI is a game changer, but unless there is another breakthrough increasing its reasoning capabilities, I don't think human experts are at risk of losing their jobs anytime soon. The current crop is disrupting occupations that deal purely with language: copywriters, translators... Software engineers are probably very safe, but I believe that language specialists whose main skill is to deeply know a software stack will soon be hurting. I predict that software architecture and computer science skills mastery will become much more valuable than knowing a computer language syntax and associated API in depth. The AI will always be better than you at syntax now. The place I see transformer-based models changing everything is human-computer interaction: voice and multimodality will soon become the main way humans interact with computers rather than mouse and keyboard, and this is huge. However, I don't believe any GenAI model will be up to the task of running a company anytime soon, and I think that advice from a real CEO might prove immensely more useful than whatever GPT-4 is telling you. GPT-4 is probably telling you the obvious—things you could have read on blog spam sites by googling your question. An experienced CEO would probably have unobvious and valuable insights that I don't believe you can get from ChatGPT at the moment.

thih9 2y ago

Can you give a specific example? I’m curious if it was useful because of the AI’s skills, or perhaps it was the conversation itself that helped, like in rubber duck debugging[1] or journaling. In any case, interested to hear more details. [1]: https://en.m.wikipedia.org/wiki/Rubber_duck_debugging https://en.m.wikipedia.org/wiki/Rubber_duck_debugging

shmel 2y ago
I find ChatGPT incredibly useful to learn the vocabulary of an unfamiliar area. Often basic information in the area I know very little about can be found easily once you know the right keywords. Even if ChatGPT gets something wrong, it does pattern matching well enough to tell me something like "yes, there is a well-known concept that matches your vague noob description, it is called X". It works well in CS where I have an intuition there must be a mathematical abstraction/algorithm that solves my problem, but I don't know the name. Once I learn the keywords, I sometimes find an open-source library for that or at least a high-level description of the algorithm. It is also great at answering stupid questions. Sometimes I read a wikipedia article and don't understand enough to make sense of it. If I was in uni, I would have asked a prof: "Sorry, it is probably a very stupid question, but why X? and what is Y actually?" ChatGPT can give me ELI5 at pretty much anything and is faster than googling.
- antihero 2y ago
  When learning a new thing it’s very good for compiling knowledge from lots of sources and laying it out in a good format. As long as it’s easy to verify when it screws up.
- thih9 2y ago
  That is CS though; I’m not sure that would work with business advice, where problems can be less tangible or there can be no single or verifiable solution (nor library/algorithm). Unless this approach also helps in the context of business advice?
  - shmel 2y ago

deleted 2y ago

[deleted]

coldtea 2y ago

>Kind of like an AI CEO advising me on what to do next. Or like someone who never was a CEO, but has absorbed all kinds of articles, listicles, and books with business advice (most of them written by hustle hacks, non-CEO content creators, or failed small-time enterpreneurs) and serves advice based on that...

elicksaur 2y ago

I’m not doubting that you have found it helpful. I just wish people would be more judicious than making up numbers like “10x”. Everything can’t be “10x”.

aussieguy1234 2y ago
In my case, if you are talking about startups specifically, its true. I've been an engineer now for 15 years. About 5 years ago, long before LLMs became a thing, I got to the point where I stopped using stack overflow and google for most things, only using them for the hardest, most complex questions. Thats how I use LLMs nowdays for technical things, which is not often. Business however is new to me. As a solo founder, i've got to be the ceo, marketer, product manager, customer service rep, tech support person and a bunch of other things i've not previously done. GPT-4 is great at guiding me on these things. In a typical 2 person CEO/CTO founded early stage startup, the CEO might be doing these non technical things until the business gets enough funding to hire their first employees.

karasik-lawyer 2y ago

[dead]

darajava 2y ago

What sort of questions do you ask it?