6 ms·

This is exactly the kind of software that people should be making. Even if this one is bad, the concept is very sound. I want to have a specialist AI that is t

This is exactly the kind of software that people should be making. Even if this one is bad, the concept is very sound. I want to have a specialist AI that is trained to help me learn how to use the software. 100% what should be happening. General AI should know how to do stuff too but having an specialist AI implies that the company/group of people/person is making sure to tune the model. Just an IMO.

scarface_74 3y ago
Why? You can ask the same questions to ChatGPT and get the same or better answers. I also know from personal experience with ChatGPT, that you can use it to: - convert Python/boto3 to any language that has an AWS SDK - convert CloudFormation to Terraform or the CDK - write scripts that use the SDK You will get the occasional hallucination.
- politician 3y ago
  Ideally, specialist models are small enough to run locally.
- cle 3y ago
  Here's one example question that ChatGPT utterly fails at, but that this answers fine: "What is Amazon CodeCatalyst?" ChatGPT: "I'm sorry, but as of my knowledge cut-off in September 2021, there was no service, tool, or product known as Amazon CodeCatalyst offered by Amazon Web Services (AWS). [...]"
  - jacurtis 3y ago
    A lot of stuff in AWS has changed since Q3 2021.
    - joshspankit 3y ago
      It feels like the AWS vendor lock-in strategy is constant change.
- KRAKRISMOTT 3y ago
  ChatGPT is not up to date unless you start using the plugins. This sort of indexing is based on vector databases and various intermediate prompting. If you want to get technical, the academic term is "Retrieval Augmented Generation".
  - fzliu 3y ago
    Hallucination is unfortunately inevitable when it comes to any autoregressive model, even with RAG. You can minimize hallucination by prompting, but you'll still see some factually incorrect responses here and there (https://zilliz.com/blog/ChatGPT-VectorDB-Prompt-as-code https://zilliz.com/blog/ChatGPT-VectorDB-Prompt-as-code). I unfortunately don't think we'll be able to solve hallucination anytime soon. Maybe with the successor to the transformer architecture?
    - Simorgh 3y ago
      Hallucination is naturally a concern for anyone looking to depend upon LLM-generated answers. We’ve been testing LLM responses with a CLI, we’re using it to generate accuracy statistics, which is especially useful when the use-case Q/A is limited. If ‘confidence’ can be returned to the user, then at least they can have an indication if there is a higher quality-risk with a given response.
shaunpud 3y ago
I was looking at the CloudLinux[1] docs the other day and noticed an "Ask GPT?" feature which I found quite helpful. [1] https://docs.cloudlinux.com https://docs.cloudlinux.com
- deleted 3y ago
  [deleted]
raincole 3y ago
MDN implemented an "AI explains" feature, and the reception is extremely negative. https://github.com/mdn/yari/issues/9208 https://github.com/mdn/yari/issues/9208
- fragmede 3y ago
  Is it? It's true there are a lot of very loud messages on that linked issue, but only MDN knows how many people are using and engaging with their system compared to on that thread. We, as the general public, don't know anything other than there's a vocal group of dissidents, and what MDN tells us. Which I looked and didn't find stats on its usage.

scarface_74 3y ago

Why? You can ask the same questions to ChatGPT and get the same or better answers. I also know from personal experience with ChatGPT, that you can use it to: - convert Python/boto3 to any language that has an AWS SDK - convert CloudFormation to Terraform or the CDK - write scripts that use the SDK You will get the occasional hallucination.

politician 3y ago
Ideally, specialist models are small enough to run locally.
cle 3y ago
Here's one example question that ChatGPT utterly fails at, but that this answers fine: "What is Amazon CodeCatalyst?" ChatGPT: "I'm sorry, but as of my knowledge cut-off in September 2021, there was no service, tool, or product known as Amazon CodeCatalyst offered by Amazon Web Services (AWS). [...]"
- jacurtis 3y ago
  A lot of stuff in AWS has changed since Q3 2021.
  - joshspankit 3y ago
    It feels like the AWS vendor lock-in strategy is constant change.
KRAKRISMOTT 3y ago
ChatGPT is not up to date unless you start using the plugins. This sort of indexing is based on vector databases and various intermediate prompting. If you want to get technical, the academic term is "Retrieval Augmented Generation".
- fzliu 3y ago
  Hallucination is unfortunately inevitable when it comes to any autoregressive model, even with RAG. You can minimize hallucination by prompting, but you'll still see some factually incorrect responses here and there (https://zilliz.com/blog/ChatGPT-VectorDB-Prompt-as-code https://zilliz.com/blog/ChatGPT-VectorDB-Prompt-as-code). I unfortunately don't think we'll be able to solve hallucination anytime soon. Maybe with the successor to the transformer architecture?
  - Simorgh 3y ago
    Hallucination is naturally a concern for anyone looking to depend upon LLM-generated answers. We’ve been testing LLM responses with a CLI, we’re using it to generate accuracy statistics, which is especially useful when the use-case Q/A is limited. If ‘confidence’ can be returned to the user, then at least they can have an indication if there is a higher quality-risk with a given response.

shaunpud 3y ago

I was looking at the CloudLinux[1] docs the other day and noticed an "Ask GPT?" feature which I found quite helpful. [1] https://docs.cloudlinux.com https://docs.cloudlinux.com

deleted 3y ago
[deleted]

raincole 3y ago

MDN implemented an "AI explains" feature, and the reception is extremely negative. https://github.com/mdn/yari/issues/9208 https://github.com/mdn/yari/issues/9208

fragmede 3y ago
Is it? It's true there are a lot of very loud messages on that linked issue, but only MDN knows how many people are using and engaging with their system compared to on that thread. We, as the general public, don't know anything other than there's a vocal group of dissidents, and what MDN tells us. Which I looked and didn't find stats on its usage.