4 ms·
I'm super happy that these types of deep research applications are being released because it seems like such an obvious use case for LLMs. I ran Perplexity thr
by CSMastermind 2y ago
I'm super happy that these types of deep research applications are being released because it seems like such an obvious use case for LLMs.
I ran Perplexity through some of my test queries for these.
One query that it choked hard on was, "List the college majors of all of the Fortune 100 CEOs"
OpenAI and Gemini both handle this somewhat gracefully producing a table of results (though it takes a few follow ups to get a correct list). Perplexity just kind of rambles generally about the topic.
There are other examples I can give of similar failures.
Seems like generally it's good at summarizing a single question (Who are the current Fortune 100 CEOs) but as soon as you need to then look up a second list of data and marry the results it kind of falls apart.
- danielcampos93 2y agodoes it do the full 100? In my experience anything around many items that needs to be exhaustive (all states, all fortune 100) tends to miss a few.
- stagger87 2y agoHopefully the end user of these products know something about LLMs and why asking a question such as "List the college majors of all of the Fortune 100 CEOs" is not really suited well for them.
- iandanforth 2y agoPerhaps you can enlighten us as to why this isn't a good use case for an LLM during a deep research workflow.
- collinvandyck76 2y agoFor those that don't know, including myself, why would this question be particularly difficult for an LLM?
- stagger87 2y ago[flagged]
- prashp 2y agoPerplexity markets itself as a search tool. So even if LLMs are not search engines, Perplexity definitely is trying to be one.
- esafak 2y agoYou are a bit behind. All the "deep research" tools, and paid AI search tools in general, combine LLMs with search. When I do research on you.com it routinely searches a 100 sites. Even Google searches get Gemini'd now. I had to chuckle because your very link provides a demonstration.
- stagger87 2y ago> You are a bit behind. Quite the opposite. I'm familiar enough with these systems to know that asking the question "List the college majors of all Fortune 100 CEOs" is not going to get you a correct answer, Gemini and you.com included. I am happy to be proven wrong. :)
- brokencode 2y agoBut the whole point of these “deep research” models is to.. you know.. do research. LLMs by themselves have not been good at this, but the whole point is to find a way to make them good.
- CSMastermind 2y agoOpenAI and Gemini literally produce the correct results. It seems like you don't understand or haven't tried their deep research tools.
- rs186 2y agoIf "deep research" can't even handle this, I don't think I would trust it with even more complex tasks
- rchaud 2y agoHopefully my boss groks how special I am and won't assign me tasks I consider to be beneath my intelligence (and beyond my capabilities).