5 ms·

the customized GPTs are honestly a let down you can't do anything fun with them, and that's not because it isn't possible but because openai is afraid to step

the customized GPTs are honestly a let down you can't do anything fun with them, and that's not because it isn't possible but because openai is afraid to step outside of the safety zone. the response style really doesn't change much, default behavior leaks into everything so easily what were they expecting?

Tostino 3y ago
Honestly, I am excited for people to re-implement the API for this using open models because of this. Maybe let the API stabilize a little, and i'm sure some implementations will pop-up.
- nxobject 3y ago
  What's the state of open models, though? I know that OpenAI's secret sauce is paying $$$ for access to private corpora, like the AP's archive, and I wonder how much the different levels of access to training data are reflected in any divide between open and commercial models, especially in different use cases like fact retrieval and synthesis. (I've gotten too reliant on ChatGPT to satisfy my curiosity re: random questions – for example, "what are some French novels that depict post World War II Paris?" Honestly, if there's an open model that's beyond some "good enough" threshold – I don't know how to judge this – I'd download it on my computer ASAP.)
  - Tostino 3y ago
    Heh, so it totally depends on the use case. I use GPT4 constantly to chat through issues I am working on and get different perspectives. I cannot do that with local models. On the other hand, I have been processing a ton of text transcripts with a fine tuned llama2 13b model i've been working on, and for the tasks I have fine-tuned on, my local model is producing better results than GPT4, often taking a task that I had to do in multiple steps with GPT4, and being able to complete it in a single shot. I can run my local model through vLLM on my workstation at around the same tokens/sec as I can spend maxing out my API limits with GPT3.5-turbo (~$20/hr) while running on 2x 3090's. I'm hitting the vLLM (OpenAI clone) chat/completions endpoint. My model implements the HF chat_templates feature, and I worked on adding support for that to vLLM: https://github.com/vllm-project/vllm/pull/1756 https://github.com/vllm-project/vllm/pull/1756 (llama.cpp is talking about adding support for it too) so I could easily swap out my model in my data pipeline in place of GPT3.5/GPT4, and I wouldn't have to keep maintaining that code on my side. So, with these transcripts I've been: - Creating YAML knowledge graphs - Summaries - Fixing transcription issues - Creating QA pairs - Identifying sections related to promoting a product or brand - Grading the output that it had generated on accuracy So...it depends on your use case.

Tostino 3y ago

Honestly, I am excited for people to re-implement the API for this using open models because of this. Maybe let the API stabilize a little, and i'm sure some implementations will pop-up.

nxobject 3y ago
What's the state of open models, though? I know that OpenAI's secret sauce is paying $$$ for access to private corpora, like the AP's archive, and I wonder how much the different levels of access to training data are reflected in any divide between open and commercial models, especially in different use cases like fact retrieval and synthesis. (I've gotten too reliant on ChatGPT to satisfy my curiosity re: random questions – for example, "what are some French novels that depict post World War II Paris?" Honestly, if there's an open model that's beyond some "good enough" threshold – I don't know how to judge this – I'd download it on my computer ASAP.)
- Tostino 3y ago
  Heh, so it totally depends on the use case. I use GPT4 constantly to chat through issues I am working on and get different perspectives. I cannot do that with local models. On the other hand, I have been processing a ton of text transcripts with a fine tuned llama2 13b model i've been working on, and for the tasks I have fine-tuned on, my local model is producing better results than GPT4, often taking a task that I had to do in multiple steps with GPT4, and being able to complete it in a single shot. I can run my local model through vLLM on my workstation at around the same tokens/sec as I can spend maxing out my API limits with GPT3.5-turbo (~$20/hr) while running on 2x 3090's. I'm hitting the vLLM (OpenAI clone) chat/completions endpoint. My model implements the HF chat_templates feature, and I worked on adding support for that to vLLM: https://github.com/vllm-project/vllm/pull/1756 https://github.com/vllm-project/vllm/pull/1756 (llama.cpp is talking about adding support for it too) so I could easily swap out my model in my data pipeline in place of GPT3.5/GPT4, and I wouldn't have to keep maintaining that code on my side. So, with these transcripts I've been: - Creating YAML knowledge graphs - Summaries - Fixing transcription issues - Creating QA pairs - Identifying sections related to promoting a product or brand - Grading the output that it had generated on accuracy So...it depends on your use case.