3 ms·

what do you trust it to do? the only example uses I see written about on HN appear to basically be Substack users asking o3 marketing questions and then writin

what do you trust it to do? the only example uses I see written about on HN appear to basically be Substack users asking o3 marketing questions and then writing substack posts about it, and a smattering of vague posts about debugging.

vessenes 1y ago
Long form research reporting. Example: Pull together a list of the top 20 startups funded in Germany this year, valuation, founder and business model. Estimate which is most likely to want to take on private equity investment from a lower mid market US PE fund, as well as which would be most suitable taking into consideration their business model, founders and market; write an approach letter in english and in german aimed at getting a meeting. make sure that it's culturally appropriate for german startup founders. I have no idea what the output of this query would be by the way, but it's one I would trust to get right on * the list of startups * the letter and its cultural sensitivity * broad strokes of what the startup is doing Stuff I'd "trust but verify" would be * Names of the founders * Size of company and target market Stuff I'd double check / keep my own counsel on * Suitability and why (note that o3 pro is def. better at this than o3 which is already not bad; it has some genuinely novel and good ideas, but often misses things.)
- leptons 1y ago
  This is all stuff I would expect an LLM to "hallucinate" about. Every bit of it.
  - steveklabnik 1y ago
    These tools can search the web to find this kind of data, and show you what they searched. Double checking is essential because hallucinations are still possible, but it's not like in the past where it would just try to make up the data from its training set. That said, it also may find bad data and give you a summary of that, which isn't a direct hallucination, but can still be inaccurate. This is why checking the sources is helpful too.
    - majormajor 1y ago
      I wouldn't expect it to hallucinate but how do you evaluate it's ability to distinguish spam from good info? I.e. the "the first four pages of google results is all crap nowdays" problem.
      - steveklabnik 1y ago
        By looking at the pages it looked at and deciding for yourself, just like you would with a web search you invoked yourself. I’ve generally found it to use trustworthy stuff like Stack Overflow, Wikipedia, and university websites. But I also haven’t used it in this way that much or for very serious things. I’d imagine more obscure questions are more likely to end up involving less trustworthy sites.
        leptons 1y ago
        >By looking at the pages it looked at and deciding for yourself, just like you would with a web search you invoked yourself. Or you could just cut out the middleman(bot) and just do the search yourself, since you're going to have to anyway to verify what the "AI" wrote. It's just all so stupid that society is rushing towards this iffy-at-best technology when we still need to do the same work anyway to verify it isn't bullshitting us. Ugh, I hate this timeline.
        steveklabnik 1y ago
        You could. What's correct depends on your needs. Search engines aren't useless because you have to make sure that the pages they return to you are quality, and so neither is using an agent in this way.
    - leptons 1y ago
      This is why I won't use these "AI" tools, because they obfuscate the bullshit. Since I have to check what it said anyway, it's easier if I just do a search and see where the info is coming from, and I can tell good from bad sources really easily. Putting an LLM in front off it is like polishing a turd - garbage in, garbage out.
  - thelock85 1y ago
    I recently tried a version of this landscape analysis within a space I understand very well (CA college access nonprofits) and was shocked at how few organizations were named, let alone described in detail. Even worse, the scope and reach of the named orgs were pretty off the mark. My best guess is that they were the SEO winners of the past.
  - vessenes 1y ago
    Well you’d be wrong in this case: Deep research will trigger a series of web searches first then reach out to tooling for follow ups as needed; most of the facts will be grounded in the sources it finds. With no deep research - agreed; too recent to believe info is accurately stored in the model weights.
- bananapub 1y ago
  why would you trust it to get any of that right? things like "top 20 startups in Germany" sound hard to determine. how do you validate all of that is actually correct?
  - jazzyjackson 1y ago
    A lot of stuff doesn't need to be accurate, it just needs to be enough information to act on. Like how there's a ton of psychics, tarot and palm readers around Wall St.
    - bananapub 1y ago
      That’s fine, but no one - not Sam Altman, not the fans on HN - are promoting them as $120/million token clairvoyants, they’re claiming they are srs bzns “iq maxxing” research tools. If OP had suggested that they were just medium-quality nonsense generators I would have just agreed and not replied.
lovich 1y ago
I’ve been using it in my job search by handing it stuff like the hn whose hiring threads, giving it a list of criteria i care about, and have it scour those posts for matching jobs, and then chase down all the companies posting and see if they have anything on their corporate site matching my descriptions. Then I have it take those matches and try and chase down the hiring manager based on public info. I did it at first just to see if it was possible, but I am getting direct emails that have been accurate a handful of times and I never would have gotten that on my own
- bananapub 1y ago
  This is a good data point - I guess another dimension is incompleteness-tolerance. An LLM is absolutely going to miss some but for your case that doesn’t matter very much. Thank you!
lukeschlather 1y ago
I don't think it's necessarily a question of trust, it's a question of cost/benefit, and I can apply this just as much to myself. I have been using a lot more SQL queries lately when I use ChatGPT, because I trust it pretty well to write gnarly queries with subqueries and CASE statements. Things that I wouldn't write myself because it's not worth the time to make the query correct, but ChatGPT can do it in seconds. I had an example where o1 really wowed me - something I don't want to post on the internet because I want to use it to test models. In that case I was thinking through a problem where I had made an incorrect mathematical assumption. I explained my reasoning to o1 and it was able to point out the flaw in my reasoning, along with some examples mathematical expressions that disproved my thinking. The funny thing in this case it basically functioned as a rubber duck. When it started producing a response I had deduced essentially what it told me - but it was pretty nice to see the detailed reasoning with examples that might've taken me a few more minutes to work out. And I never would've produced a little report explaining in detail why I was wrong, I would've just adjusted my thinking. Having the report was helpful.

vessenes 1y ago

Long form research reporting. Example: Pull together a list of the top 20 startups funded in Germany this year, valuation, founder and business model. Estimate which is most likely to want to take on private equity investment from a lower mid market US PE fund, as well as which would be most suitable taking into consideration their business model, founders and market; write an approach letter in english and in german aimed at getting a meeting. make sure that it's culturally appropriate for german startup founders. I have no idea what the output of this query would be by the way, but it's one I would trust to get right on * the list of startups * the letter and its cultural sensitivity * broad strokes of what the startup is doing Stuff I'd "trust but verify" would be * Names of the founders * Size of company and target market Stuff I'd double check / keep my own counsel on * Suitability and why (note that o3 pro is def. better at this than o3 which is already not bad; it has some genuinely novel and good ideas, but often misses things.)

leptons 1y ago
This is all stuff I would expect an LLM to "hallucinate" about. Every bit of it.
- steveklabnik 1y ago
  These tools can search the web to find this kind of data, and show you what they searched. Double checking is essential because hallucinations are still possible, but it's not like in the past where it would just try to make up the data from its training set. That said, it also may find bad data and give you a summary of that, which isn't a direct hallucination, but can still be inaccurate. This is why checking the sources is helpful too.
  - majormajor 1y ago
    I wouldn't expect it to hallucinate but how do you evaluate it's ability to distinguish spam from good info? I.e. the "the first four pages of google results is all crap nowdays" problem.
    - steveklabnik 1y ago
      By looking at the pages it looked at and deciding for yourself, just like you would with a web search you invoked yourself. I’ve generally found it to use trustworthy stuff like Stack Overflow, Wikipedia, and university websites. But I also haven’t used it in this way that much or for very serious things. I’d imagine more obscure questions are more likely to end up involving less trustworthy sites.
      - leptons 1y ago
        >By looking at the pages it looked at and deciding for yourself, just like you would with a web search you invoked yourself. Or you could just cut out the middleman(bot) and just do the search yourself, since you're going to have to anyway to verify what the "AI" wrote. It's just all so stupid that society is rushing towards this iffy-at-best technology when we still need to do the same work anyway to verify it isn't bullshitting us. Ugh, I hate this timeline.
        steveklabnik 1y ago
        You could. What's correct depends on your needs. Search engines aren't useless because you have to make sure that the pages they return to you are quality, and so neither is using an agent in this way.
  - leptons 1y ago
    This is why I won't use these "AI" tools, because they obfuscate the bullshit. Since I have to check what it said anyway, it's easier if I just do a search and see where the info is coming from, and I can tell good from bad sources really easily. Putting an LLM in front off it is like polishing a turd - garbage in, garbage out.
- thelock85 1y ago
  I recently tried a version of this landscape analysis within a space I understand very well (CA college access nonprofits) and was shocked at how few organizations were named, let alone described in detail. Even worse, the scope and reach of the named orgs were pretty off the mark. My best guess is that they were the SEO winners of the past.
- vessenes 1y ago
  Well you’d be wrong in this case: Deep research will trigger a series of web searches first then reach out to tooling for follow ups as needed; most of the facts will be grounded in the sources it finds. With no deep research - agreed; too recent to believe info is accurately stored in the model weights.
bananapub 1y ago
why would you trust it to get any of that right? things like "top 20 startups in Germany" sound hard to determine. how do you validate all of that is actually correct?
- jazzyjackson 1y ago
  A lot of stuff doesn't need to be accurate, it just needs to be enough information to act on. Like how there's a ton of psychics, tarot and palm readers around Wall St.
  - bananapub 1y ago
    That’s fine, but no one - not Sam Altman, not the fans on HN - are promoting them as $120/million token clairvoyants, they’re claiming they are srs bzns “iq maxxing” research tools. If OP had suggested that they were just medium-quality nonsense generators I would have just agreed and not replied.

lovich 1y ago

I’ve been using it in my job search by handing it stuff like the hn whose hiring threads, giving it a list of criteria i care about, and have it scour those posts for matching jobs, and then chase down all the companies posting and see if they have anything on their corporate site matching my descriptions. Then I have it take those matches and try and chase down the hiring manager based on public info. I did it at first just to see if it was possible, but I am getting direct emails that have been accurate a handful of times and I never would have gotten that on my own

bananapub 1y ago
This is a good data point - I guess another dimension is incompleteness-tolerance. An LLM is absolutely going to miss some but for your case that doesn’t matter very much. Thank you!

lukeschlather 1y ago

I don't think it's necessarily a question of trust, it's a question of cost/benefit, and I can apply this just as much to myself. I have been using a lot more SQL queries lately when I use ChatGPT, because I trust it pretty well to write gnarly queries with subqueries and CASE statements. Things that I wouldn't write myself because it's not worth the time to make the query correct, but ChatGPT can do it in seconds. I had an example where o1 really wowed me - something I don't want to post on the internet because I want to use it to test models. In that case I was thinking through a problem where I had made an incorrect mathematical assumption. I explained my reasoning to o1 and it was able to point out the flaw in my reasoning, along with some examples mathematical expressions that disproved my thinking. The funny thing in this case it basically functioned as a rubber duck. When it started producing a response I had deduced essentially what it told me - but it was pretty nice to see the detailed reasoning with examples that might've taken me a few more minutes to work out. And I never would've produced a little report explaining in detail why I was wrong, I would've just adjusted my thinking. Having the report was helpful.