3 ms·

Let's see the products. I'm so tired of hearing Google talk a big game about AI without delivering.

by mvdtnz 3y ago

jstummbillig 3y ago
a) I think they have in a lot of ways, that are maybe not super flashy. Vertex AI offers more API/functionality/product than anyone else, by a mile. I think in a year or two this might be more important than the quality of model (not that the model won't have to be good, but all of them will be good enough to solve most peoples business problems) b) Gemini 1.5 Pro public preview with 1 mio tokens is out now (including API access). Has anyone benchmarked it?
- mvdtnz 3y ago
  > not that the model won't have to be good, but all of them will be good enough to solve most peoples business problems In a year or two? Mate I think you need to step back from the AI hype for a moment. We're not even close.
  - dudus 3y ago
    Having tried all 3 (Gemini, chatgpt and Claude) I'll agree with GP that 1-2 years seems conservative if anything. I don't think AI will be won on the model but the tooling. Google is arguably ahead. The models I'd say it's a toss. But it doesn't matter as before the year is up all of them will release new versions one-uping each other. It's a race to the bottom. They either collude on price or most will go bankrupt. Edit:typo
  - jstummbillig 3y ago
    When I write that tooling will be more important in the future, I am not making any crazy claims about the future of AI. My claim is: It does not require much more AI than we currently have to solve most business problems. But it will take better tooling. I don't think AGI is even remotely required to solve most peoples business problems. Better tooling is. Maybe we also have different ideas about how mundane most peoples business problems are.
- refulgentis 3y ago
  #1) That's absolutely, harrowingly, shockingly, untrue. They're just rolling out system messages! I actively integrate with Mistral, OpenAI, Gemini AI API, Vertex AI API, and Anthropic's API and Google's are the weakest. #2) It's Gemini 1.0 with a 1M context. It's not very good, it works like OpenAI long context for me (pulls out salient text, but it doesn't seem to be 'reasoning' over all of it)
gentleman11 3y ago
It’s almost like they hire exclusively for your ability to leetcode rather than trying to hire people who can produce software Maybe the ai teams were too busy keeping up to date with their tree reversals to read those textbooks on their nights off
xadhominemx 3y ago
Gemini is just as good if not better than GPT-4?
- belter 3y ago
  Extraordinary claims require extraordinary evidence... :-))
  - xadhominemx 3y ago
    You can use the product yourself or check one of the many benchmarking papers
    - ein0p 3y ago
      Benchmarking papers are borderline useless. Goodhart’s law is a thing when a measure becomes a target it ceases to be a good measure. That’s why I keep coming back to GPT4. It has the most “magic” and “mind reading ability”. Try to quantify that.

Let's see the products. I'm so tired of hearing Google talk a big game about AI without delivering.

by mvdtnz 3y ago

Let's see the products. I'm so tired of hearing Google talk a big game about AI without delivering.

a) I think they have in a lot of ways, that are maybe not super flashy. Vertex AI offers more API/functionality/product than anyone else, by a mile. I think in a year or two this might be more important than the quality of model (not that the model won't have to be good, but all of them will be good enough to solve most peoples business problems) b) Gemini 1.5 Pro public preview with 1 mio tokens is out now (including API access). Has anyone benchmarked it?

mvdtnz 3y ago
> not that the model won't have to be good, but all of them will be good enough to solve most peoples business problems In a year or two? Mate I think you need to step back from the AI hype for a moment. We're not even close.
- dudus 3y ago
  Having tried all 3 (Gemini, chatgpt and Claude) I'll agree with GP that 1-2 years seems conservative if anything. I don't think AI will be won on the model but the tooling. Google is arguably ahead. The models I'd say it's a toss. But it doesn't matter as before the year is up all of them will release new versions one-uping each other. It's a race to the bottom. They either collude on price or most will go bankrupt. Edit:typo
- jstummbillig 3y ago
  When I write that tooling will be more important in the future, I am not making any crazy claims about the future of AI. My claim is: It does not require much more AI than we currently have to solve most business problems. But it will take better tooling. I don't think AGI is even remotely required to solve most peoples business problems. Better tooling is. Maybe we also have different ideas about how mundane most peoples business problems are.
refulgentis 3y ago
#1) That's absolutely, harrowingly, shockingly, untrue. They're just rolling out system messages! I actively integrate with Mistral, OpenAI, Gemini AI API, Vertex AI API, and Anthropic's API and Google's are the weakest. #2) It's Gemini 1.0 with a 1M context. It's not very good, it works like OpenAI long context for me (pulls out salient text, but it doesn't seem to be 'reasoning' over all of it)

gentleman11 3y ago

It’s almost like they hire exclusively for your ability to leetcode rather than trying to hire people who can produce software Maybe the ai teams were too busy keeping up to date with their tree reversals to read those textbooks on their nights off

xadhominemx 3y ago

Gemini is just as good if not better than GPT-4?

belter 3y ago
Extraordinary claims require extraordinary evidence... :-))
- xadhominemx 3y ago
  You can use the product yourself or check one of the many benchmarking papers
  - ein0p 3y ago
    Benchmarking papers are borderline useless. Goodhart’s law is a thing when a measure becomes a target it ceases to be a good measure. That’s why I keep coming back to GPT4. It has the most “magic” and “mind reading ability”. Try to quantify that.