4 ms·

I was pretty cynical trying earlier models, but with Gemini Flash 2.0 I felt like there was a pretty significant boost in usability and capabilities. In partic

I was pretty cynical trying earlier models, but with Gemini Flash 2.0 I felt like there was a pretty significant boost in usability and capabilities. In particular I've found that these tools make it a lot easier to explore or get started with unfamiliar domains. One of my big issues has often been decision paralysis, so having a tool to help me narrow down the list of resources and make it more approachable has been a huge win. My general experience has been that getting AI tools to directly do stuff for you tends to produce pretty bad results, but you can use it as a force multiplier for your own capabilities. If I'm confused or uncertain about how to do something, AI tools are usually pretty good at clarifying what needs to be done.

chefandy 2y ago
Maybe there’s a particular cognitive profile that benefits most from LLM chat bots? I’ve tried multiple times to realize this force multiplier in my life for everything from day-to-day stuff to picking up new things, using the latest paid bots with the best models, and I’ve persistently found them to be awkward, inaccurate, difficult to pin down into giving useful info rather than a bunch of non-committal pablum without hallucinations, etc etc etc
- j7ake 2y ago
  Maybe you can give some specific examples
  - chefandy 2y ago
    For what — things that AI wasn’t helpful for when I gave it a shot?
- anon373839 2y ago
  They’re useful for tasks that don’t require correctness: brainstorming, exploratory research, sketching out the vague shape of a solution to a problem. They’re mediocre to awful at consistently following instructions, unless you have the fortune of having a task and domain that are well represented in the post-training data. Yesterday I needed to generate ~500 filenames given a (human written) summary of each document’s contents. This seemed to be the perfect task to throw an LLM into a for loop. Yet it took three hours of prompt engineering to get a passable set of results - yes, I could’ve done it by hand in that time. Each iteration on the prompt revealed new blind spots, new ways in which the models would latch onto irrelevant details or gloss over the main point.
  - chefandy 2y ago
    In my cognitive landscape, abstract reasoning is a huge spike for me while linear reasoning is closer to average. I just don’t think I need help with all of that exploratory stuff — I need something that’s going to root me in reality with hard facts, figures, and processes. I can see how folks that don’t exist primarily in “shower thoughts” land, like I do, and have better intuition with the concrete linear stuff might get more use out of it than me. Conversely, perhaps I get a bit more out of traditional search engine workflows than others? That would definitely account for the difference in my perception of their utility vs other peoples’.
    - TheAceOfHearts 2y ago
      My experience has been that converting abstract ideas into a set of actionable steps is one of the areas where AI tooling is really useful. So for example, if I want to generate some music programmatically for a small demo, it gives me concrete suggestions of the existing tool landscape and it helps me generate a plan of action. Then I fill in the remaining gaps myself in order to execute.
      - chefandy 2y ago
        Thanks— I’ll play around with that workflow.
  - Kye 2y ago
    Sometimes it helps to have it outline the task first with a prompt like "Suggest good guidelines for [task]" and then "Follow those guidelines for [task]" once you check to make sure the guidelines are good. LLMs aren't good at inventing processes on the fly, but they are good at writing plausible-sounding and often even correct processes, which they then follow. Applied to ChatGPT's 4o: > "Suggest good guidelines for generating ~500 filenames given a (human written) summary of each document’s contents" > "Pretend you have this data and create 10 example file names." > "Financial_Report_Q1_2024.xlsx (Summary: "Quarterly financial performance for Q1 2024, including revenue and expenses.")" > "Marketing_Strategy_SocialMedia_2024.docx (Summary: "Comprehensive marketing strategy focusing on social media growth for 2024.")" > "HR_Employee_Handbook_v2.1.pdf (Summary: "Updated version of the company employee handbook with revised policies.")" And so on. In general, you have to either create the process or have it create a process and refine it if it isn't quite there. You can even use standard processes and guidelines when they're available. Then it usually does a great job.

chefandy 2y ago

Maybe there’s a particular cognitive profile that benefits most from LLM chat bots? I’ve tried multiple times to realize this force multiplier in my life for everything from day-to-day stuff to picking up new things, using the latest paid bots with the best models, and I’ve persistently found them to be awkward, inaccurate, difficult to pin down into giving useful info rather than a bunch of non-committal pablum without hallucinations, etc etc etc

j7ake 2y ago
Maybe you can give some specific examples
- chefandy 2y ago
  For what — things that AI wasn’t helpful for when I gave it a shot?
anon373839 2y ago
They’re useful for tasks that don’t require correctness: brainstorming, exploratory research, sketching out the vague shape of a solution to a problem. They’re mediocre to awful at consistently following instructions, unless you have the fortune of having a task and domain that are well represented in the post-training data. Yesterday I needed to generate ~500 filenames given a (human written) summary of each document’s contents. This seemed to be the perfect task to throw an LLM into a for loop. Yet it took three hours of prompt engineering to get a passable set of results - yes, I could’ve done it by hand in that time. Each iteration on the prompt revealed new blind spots, new ways in which the models would latch onto irrelevant details or gloss over the main point.
- chefandy 2y ago
  In my cognitive landscape, abstract reasoning is a huge spike for me while linear reasoning is closer to average. I just don’t think I need help with all of that exploratory stuff — I need something that’s going to root me in reality with hard facts, figures, and processes. I can see how folks that don’t exist primarily in “shower thoughts” land, like I do, and have better intuition with the concrete linear stuff might get more use out of it than me. Conversely, perhaps I get a bit more out of traditional search engine workflows than others? That would definitely account for the difference in my perception of their utility vs other peoples’.
  - TheAceOfHearts 2y ago
    My experience has been that converting abstract ideas into a set of actionable steps is one of the areas where AI tooling is really useful. So for example, if I want to generate some music programmatically for a small demo, it gives me concrete suggestions of the existing tool landscape and it helps me generate a plan of action. Then I fill in the remaining gaps myself in order to execute.
    - chefandy 2y ago
      Thanks— I’ll play around with that workflow.
- Kye 2y ago
  Sometimes it helps to have it outline the task first with a prompt like "Suggest good guidelines for [task]" and then "Follow those guidelines for [task]" once you check to make sure the guidelines are good. LLMs aren't good at inventing processes on the fly, but they are good at writing plausible-sounding and often even correct processes, which they then follow. Applied to ChatGPT's 4o: > "Suggest good guidelines for generating ~500 filenames given a (human written) summary of each document’s contents" > "Pretend you have this data and create 10 example file names." > "Financial_Report_Q1_2024.xlsx (Summary: "Quarterly financial performance for Q1 2024, including revenue and expenses.")" > "Marketing_Strategy_SocialMedia_2024.docx (Summary: "Comprehensive marketing strategy focusing on social media growth for 2024.")" > "HR_Employee_Handbook_v2.1.pdf (Summary: "Updated version of the company employee handbook with revised policies.")" And so on. In general, you have to either create the process or have it create a process and refine it if it isn't quite there. You can even use standard processes and guidelines when they're available. Then it usually does a great job.