5 ms·

Bingo. I've been beating this drum since the initial GPT-3 awe.. The future of AI is bespoke, purpose-driven models trained on a combination of public and (impo

Bingo. I've been beating this drum since the initial GPT-3 awe.. The future of AI is bespoke, purpose-driven models trained on a combination of public and (importantly) proprietary data. Data is still king.

JimtheCoder 3y ago
So, search isn't dead after all...
- stavros 3y ago
  Search has definitely been dead since before LLMs, we just don't have a replacement yet.
  - JimtheCoder 3y ago
    I mean more the concept of search, not the current implementation
    - stavros 3y ago
      Oh, I don't think we'll ever stop wanting to search for things. Maybe not everything, but some things.
    - imtringued 3y ago
      Search is about finding existing results. The opposite of search is to start from first principles and work your way up until you have created the desired result yourself. In the real world, search reduces information acquisition costs as you only have to spend time and resources on finding an existing result rather than recreating it.
  - Forgeties79 3y ago
    For a while my replacement was “use google, add ‘reddit’ at the end.” Not sure how much longer that will work given even just this limited blackout impacted how effective that was lol
    - edgyquant 3y ago
      That hasn’t worked since about three months after companies found out people do it. It’s all astroturfing now days anyway and if it applies to products (which it for sure does) you can be sure that government actors caught on as well.
      - mitchdoogle 3y ago
        I'm not really sure what you're saying. It seems to me the number of shills creating content is vastly outnumbered by normal people creating content, so the trick of adding "reddit" to the end of queries is still very much useful (blackout protests aside) since it's usefulness derives from getting information from normal people and then having normal people upvote the "best" comments. Just the other day I tried this with data recovery software very quickly found some free software that was highly praised. If I had just searched "data recovery software" on Google I get whichever data recovery place as the best SEO, including many of them who claim that a novice should be trying it at all and should use their services pronto. Using reddit in this case gave me the exact information I wanted, where Google alone was next to useless.
        Forgeties79 3y ago
        Yeah I don’t know their experience, but mine is when I add “Reddit” i often get 2-3 threads talking about the exact thing I’m looking up in the top 10 results.
        edgyquant 3y ago
        What I’m getting at is that those threads have a high likelihood of being gamed to make you think a certain way
      - quesera 3y ago
        I'm not sure if it would help or restore the kinds of results you were seeing previously, but instead of adding "reddit", you can add "site:reddit.com" to get only results from that site. (Originally a Google feature, but works on DuckDuckGo also. Not sure about others.) Unless you mean that Reddit is astroturfed with the SEO garbage you're trying to avoid, in which case this will definitely not help. Is search on Reddit itself still useless?
samstave 3y ago
It will be a terrific moment when we can browse a galaxy map or other lists of LLMs, and select "context" groups - so you can say from history, give me X
- thelittleone 3y ago
  That doesn't sound ideal at all to me. It also sounds like a bottleneck. I want to ask AI once and it recommends or selects the best "context" group.
  - samstave 3y ago
    It would bea good visual training tool for kids so they can get an understanding early on other than a block-box view.
danielvaughn 3y ago
I’d be really interested to see an AI model built off of Sci Hub data.
- fooker 3y ago
  Facebook announced this a month or so before chatgpt took off, and got lambasted everywhere because of hallucinations.
croes 3y ago
This would only work if thinks like GPT were actually intelligent. Train on dataset A to learn to think, use thinking on dataset B to become an export in B's field.
Animats 3y ago
You need sort of a "primary education" data set which gets the model up to roughly a high school education level. Then augment it with special-purpose models for specific areas. But until "I don't know" comes out, rather than hallucinations, we're in trouble.
- rco8786 3y ago
  > You need sort of a "primary education" data set which gets the model up to roughly a high school education level. Yea that was what I was getting at with the "combination" of data. The publicly available data provides the base/primary education, then you specialize it with your proprietary data and bam, you have an AI model that nobody else can produce...an actual product moat.

JimtheCoder 3y ago

So, search isn't dead after all...

stavros 3y ago
Search has definitely been dead since before LLMs, we just don't have a replacement yet.
- JimtheCoder 3y ago
  I mean more the concept of search, not the current implementation
  - stavros 3y ago
    Oh, I don't think we'll ever stop wanting to search for things. Maybe not everything, but some things.
  - imtringued 3y ago
    Search is about finding existing results. The opposite of search is to start from first principles and work your way up until you have created the desired result yourself. In the real world, search reduces information acquisition costs as you only have to spend time and resources on finding an existing result rather than recreating it.
- Forgeties79 3y ago
  For a while my replacement was “use google, add ‘reddit’ at the end.” Not sure how much longer that will work given even just this limited blackout impacted how effective that was lol
  - edgyquant 3y ago
    That hasn’t worked since about three months after companies found out people do it. It’s all astroturfing now days anyway and if it applies to products (which it for sure does) you can be sure that government actors caught on as well.
    - mitchdoogle 3y ago
      I'm not really sure what you're saying. It seems to me the number of shills creating content is vastly outnumbered by normal people creating content, so the trick of adding "reddit" to the end of queries is still very much useful (blackout protests aside) since it's usefulness derives from getting information from normal people and then having normal people upvote the "best" comments. Just the other day I tried this with data recovery software very quickly found some free software that was highly praised. If I had just searched "data recovery software" on Google I get whichever data recovery place as the best SEO, including many of them who claim that a novice should be trying it at all and should use their services pronto. Using reddit in this case gave me the exact information I wanted, where Google alone was next to useless.
      - Forgeties79 3y ago
        Yeah I don’t know their experience, but mine is when I add “Reddit” i often get 2-3 threads talking about the exact thing I’m looking up in the top 10 results.
      - edgyquant 3y ago
        What I’m getting at is that those threads have a high likelihood of being gamed to make you think a certain way
    - quesera 3y ago
      I'm not sure if it would help or restore the kinds of results you were seeing previously, but instead of adding "reddit", you can add "site:reddit.com" to get only results from that site. (Originally a Google feature, but works on DuckDuckGo also. Not sure about others.) Unless you mean that Reddit is astroturfed with the SEO garbage you're trying to avoid, in which case this will definitely not help. Is search on Reddit itself still useless?

samstave 3y ago

It will be a terrific moment when we can browse a galaxy map or other lists of LLMs, and select "context" groups - so you can say from history, give me X

thelittleone 3y ago
That doesn't sound ideal at all to me. It also sounds like a bottleneck. I want to ask AI once and it recommends or selects the best "context" group.
- samstave 3y ago
  It would bea good visual training tool for kids so they can get an understanding early on other than a block-box view.

danielvaughn 3y ago

I’d be really interested to see an AI model built off of Sci Hub data.

fooker 3y ago
Facebook announced this a month or so before chatgpt took off, and got lambasted everywhere because of hallucinations.

croes 3y ago

This would only work if thinks like GPT were actually intelligent. Train on dataset A to learn to think, use thinking on dataset B to become an export in B's field.

Animats 3y ago

You need sort of a "primary education" data set which gets the model up to roughly a high school education level. Then augment it with special-purpose models for specific areas. But until "I don't know" comes out, rather than hallucinations, we're in trouble.

rco8786 3y ago
> You need sort of a "primary education" data set which gets the model up to roughly a high school education level. Yea that was what I was getting at with the "combination" of data. The publicly available data provides the base/primary education, then you specialize it with your proprietary data and bam, you have an AI model that nobody else can produce...an actual product moat.