9 ms·

The problem is that you never really know whether the chat bot gets it right or terrifically wrong unless you already know how to do the task yourself. And in

by jawns 4y ago

The problem is that you never really know whether the chat bot gets it right or terrifically wrong unless you already know how to do the task yourself. And in some cases, paying an analyst $50/hr. for a higher degree of confidence than you can get from a $1 chat bot is still very much worth it. The stakes are higher, too. If the chat bot gets it wrong, what are you going to do, fire it? There goes a small trickle of revenue to OpenAI. Whereas if the analyst gets it wrong, there goes their livelihood. That said ... this will help the $50/hr. analyst improve their productivity!

mjburgess 4y ago
In my experience, in many cases, the so-called "first draft" (of code, content, etc.) is more just helpful for a mental warm-up. It isn't actually suitable. The time it takes to write/code/etc. isnt in the typing, that's trivial. It's in modelling the problem (, audience) and solving it. ChatGPT isnt in the game of modelling problems and solving them, it possesses only superficial semantic characteristics of the questions posed That its answers satisfy the requirements would require ChatGPT to ask you questions (and much else besides)
- ExxKA 4y ago
  This
- TillE 4y ago
  The similar thought I had yesterday was that ChatGPT is pretty good at generating sample code that doesn't otherwise exist. It can be really useful if you treat it like that. Of course I've also had it call functions and use constants which don't exist, and fail to correct itself even after repeated prompting. So even that only goes so far.
  - Shorel 4y ago
    Imagine if ChatGPT (or any other competitor) had access to a compiler and a set of tests for the program, a la HackerRank. You then give it the prompt and a lot of test cases to pass in order to consider the task complete. I can see this concept being implemented and working before the end of the year.
    - pja 4y ago
      Google / Deepmind has already done this last year. They published the paper at the end of 2022 I think. (searches) yep: https://www.deepmind.com/blog/competitive-programming-with-alphacode https://www.deepmind.com/blog/competitive-programming-with-a... It relies on test availability of good test cases to verify the code functionality: they generate many, many possible programs, throw out all the ones that aren't syntactically correct, compile the ones that are & then test them. This works reasonably well for programming competitions & the code generated will probably improve over time. The issue for "real-world" problems is that you can't verify the code without the test cases & writing good tests that ensure the code does what you want can be as much work as writing the code itself in the first place - the tests form a kind of mathematical co-domain to the code after all & only a small subset of problems are so simply defined that you can pin them down with just one or two tests.
- nerpderp82 4y ago
  > ChatGPT isnt in the game of modelling problems and solving them, it possesses only superficial semantic characteristics of the questions posed You can give it the datamodel and a semantic description and it will do a pretty good job of reprojecting your data. People talk about chatgpt as being a fixed thing, it isn't. ChatGPT is the Apple II of LLMs. 3-4 moore units from now every issue that people think is a "gotcha" for chatgpt will have evaporated.
  - ResearchCode 4y ago
    Still waiting for those autonomous cars.
    - burnished 4y ago
      They are already here. We can haggle over error rate and incidents per million miles or what have you but that is more an argument about safety and suitability.
      - ResearchCode 4y ago
        Those must be some big gotchas, because self-driving cars have not replaced a job here, yet. Large language models show more promise for replacing non-programming office workers.
        burnished 4y ago
        'Replaced all forms of relevant labor, or threaten to' isnt really what I am addressing or care to address. We have recognizably autonomous vehicles - that is the statement I am addressing and would ask you to, as well.
  - kietay 4y ago
    In your analogy to the Apple II, is there a fundamental problem people claimed for early computers, which didn’t just boil down to “they need to be faster”? There are fundamental limitations to what an LLM can achieve.
- frabcus 4y ago
  You can get it to ask more questions…
themodelplumber 4y ago
> $50/hr Is this somewhere's going rate for a good data analyst?
- lelandfe 4y ago
  That's about $100k salary. You can hire a rock solid data analyst at that price probably anywhere in the US. I know a couple talented analysts making ~$110k in NYC.
- blahblah1234567 4y ago
  In early 2018, I made $105k salary as an entry level "Business Systems Analyst" in San Jose, CA. Rented a studio in Milpitas for $1200 at the time. I built a data integration system with NodeJS, and marketing/customer service enterprise software REST APIs. It was my first professional programming job. No STEM degree, but a few IT Courses, plus digital marketing experience, plus self-taught full stack web app experience.
gremlinsinc 4y ago
Imagine you keep someone on staff at 50/hr, or you just consult based on need, i.e. 15 minute consults are 1/4th the cost, and you simply just ping them to double-check queries you've ran, that look right but maybe could be optimized. You've just cut back from maybe 20/hours to 8 per week needed for that freelancer.
spaceman_2020 4y ago
That's exactly what it is - a productivity booster. The point remains the same: it's going to theoretically put some SQL analysts out of work. If the $50/hr analyst is 10x faster at his job, you can replace 10 analysts with just 1.

by jawns 4y ago

mjburgess 4y ago

In my experience, in many cases, the so-called "first draft" (of code, content, etc.) is more just helpful for a mental warm-up. It isn't actually suitable. The time it takes to write/code/etc. isnt in the typing, that's trivial. It's in modelling the problem (, audience) and solving it. ChatGPT isnt in the game of modelling problems and solving them, it possesses only superficial semantic characteristics of the questions posed That its answers satisfy the requirements would require ChatGPT to ask you questions (and much else besides)

ExxKA 4y ago
This
TillE 4y ago
The similar thought I had yesterday was that ChatGPT is pretty good at generating sample code that doesn't otherwise exist. It can be really useful if you treat it like that. Of course I've also had it call functions and use constants which don't exist, and fail to correct itself even after repeated prompting. So even that only goes so far.
- Shorel 4y ago
  Imagine if ChatGPT (or any other competitor) had access to a compiler and a set of tests for the program, a la HackerRank. You then give it the prompt and a lot of test cases to pass in order to consider the task complete. I can see this concept being implemented and working before the end of the year.
  - pja 4y ago
    Google / Deepmind has already done this last year. They published the paper at the end of 2022 I think. (searches) yep: https://www.deepmind.com/blog/competitive-programming-with-alphacode https://www.deepmind.com/blog/competitive-programming-with-a... It relies on test availability of good test cases to verify the code functionality: they generate many, many possible programs, throw out all the ones that aren't syntactically correct, compile the ones that are & then test them. This works reasonably well for programming competitions & the code generated will probably improve over time. The issue for "real-world" problems is that you can't verify the code without the test cases & writing good tests that ensure the code does what you want can be as much work as writing the code itself in the first place - the tests form a kind of mathematical co-domain to the code after all & only a small subset of problems are so simply defined that you can pin them down with just one or two tests.
nerpderp82 4y ago
> ChatGPT isnt in the game of modelling problems and solving them, it possesses only superficial semantic characteristics of the questions posed You can give it the datamodel and a semantic description and it will do a pretty good job of reprojecting your data. People talk about chatgpt as being a fixed thing, it isn't. ChatGPT is the Apple II of LLMs. 3-4 moore units from now every issue that people think is a "gotcha" for chatgpt will have evaporated.
- ResearchCode 4y ago
  Still waiting for those autonomous cars.
  - burnished 4y ago
    They are already here. We can haggle over error rate and incidents per million miles or what have you but that is more an argument about safety and suitability.
    - ResearchCode 4y ago
      Those must be some big gotchas, because self-driving cars have not replaced a job here, yet. Large language models show more promise for replacing non-programming office workers.
      - burnished 4y ago
        'Replaced all forms of relevant labor, or threaten to' isnt really what I am addressing or care to address. We have recognizably autonomous vehicles - that is the statement I am addressing and would ask you to, as well.
- kietay 4y ago
  In your analogy to the Apple II, is there a fundamental problem people claimed for early computers, which didn’t just boil down to “they need to be faster”? There are fundamental limitations to what an LLM can achieve.
frabcus 4y ago
You can get it to ask more questions…

themodelplumber 4y ago

> $50/hr Is this somewhere's going rate for a good data analyst?

lelandfe 4y ago
That's about $100k salary. You can hire a rock solid data analyst at that price probably anywhere in the US. I know a couple talented analysts making ~$110k in NYC.
blahblah1234567 4y ago
In early 2018, I made $105k salary as an entry level "Business Systems Analyst" in San Jose, CA. Rented a studio in Milpitas for $1200 at the time. I built a data integration system with NodeJS, and marketing/customer service enterprise software REST APIs. It was my first professional programming job. No STEM degree, but a few IT Courses, plus digital marketing experience, plus self-taught full stack web app experience.

gremlinsinc 4y ago

Imagine you keep someone on staff at 50/hr, or you just consult based on need, i.e. 15 minute consults are 1/4th the cost, and you simply just ping them to double-check queries you've ran, that look right but maybe could be optimized. You've just cut back from maybe 20/hours to 8 per week needed for that freelancer.

spaceman_2020 4y ago

That's exactly what it is - a productivity booster. The point remains the same: it's going to theoretically put some SQL analysts out of work. If the $50/hr analyst is 10x faster at his job, you can replace 10 analysts with just 1.