3 ms·

I was initially enthusiastic about DS3, because of the price, but eventually I learned the following things: - function calling is broken (responding with exce

I was initially enthusiastic about DS3, because of the price, but eventually I learned the following things: - function calling is broken (responding with excessive number of duplicated FC, halucinated names and parameters) - response quality is poor (my use case is code generation) - support is not responding I will give a try to the reasoning model, but my expectations are low. ps. the positive side of this is that apparently it removed some traffic from anthropic APIs, and latency for sonnet/haikku improved significantly.

pmarreck 2y ago
I got some good code recommendations out of it. I usually give the same question to a few models and see what they say; they differ enough to be useful, and then I end up combining the different suggestions with my own to synthesize the best possible (by my personal metric, of course) code.
mtkd 2y ago
The company has just over 100 employees, built V3 with $5.5M of compute and is quietly releasing tangible product without any hyperbolic PR in advance They were fairly unknown until 26th Dec in west
- fullstackwife 2y ago
  you are right, but please don't forget they were publishing some benchmark results which were great
Gracana 2y ago
I was looking to see how you're supposed to configure v3, then realized you're probably using the API, and came across this: > The current version of the deepseek-chat model's Function Calling capabilitity is unstable, which may result in looped calls or empty responses. We are actively working on a fix, and it is expected to be resolved in the next version. https://api-docs.deepseek.com/guides/function_calling https://api-docs.deepseek.com/guides/function_calling That's disappointing.
coder543 2y ago
Maybe function calling using JSON blobs isn't even the optimal approach... I saw some stuff recently about having LLMs write Python code to execute what they want, and LLMs tend to be a lot better at Python without any additional function-calling training. Some of the functions exposed to the LLM can be calls into your own logic. Some relevant links: This shows how python-calling performance is supposedly better for a range of existing models than JSON-calling performance: https://huggingface.co/blog/andthattoo/dpab-a#initial-results https://huggingface.co/blog/andthattoo/dpab-a#initial-result... A little post about the concept: https://huggingface.co/blog/andthattoo/dria-agent-a https://huggingface.co/blog/andthattoo/dria-agent-a Huggingface has their own "smolagents" library that includes "CodeAgent", which operates by the same principle of generating and executing Python code for the purposes of function calling: https://huggingface.co/docs/smolagents/en/guided_tour https://huggingface.co/docs/smolagents/en/guided_tour smolagents can either use a local LLM or a remote LLM, and it can either run the code locally, or run the code on a remote code execution environment, so it seems fairly flexible.
rozap 2y ago
What are you using for structured output? Outlines, BAML, etc seem to vary a huge amount in quality. It was many moons ago, but outlines was unusable. BAML has been great.

pmarreck 2y ago

I got some good code recommendations out of it. I usually give the same question to a few models and see what they say; they differ enough to be useful, and then I end up combining the different suggestions with my own to synthesize the best possible (by my personal metric, of course) code.

mtkd 2y ago

The company has just over 100 employees, built V3 with $5.5M of compute and is quietly releasing tangible product without any hyperbolic PR in advance They were fairly unknown until 26th Dec in west

fullstackwife 2y ago
you are right, but please don't forget they were publishing some benchmark results which were great

Gracana 2y ago

I was looking to see how you're supposed to configure v3, then realized you're probably using the API, and came across this: > The current version of the deepseek-chat model's Function Calling capabilitity is unstable, which may result in looped calls or empty responses. We are actively working on a fix, and it is expected to be resolved in the next version. https://api-docs.deepseek.com/guides/function_calling https://api-docs.deepseek.com/guides/function_calling That's disappointing.

coder543 2y ago

Maybe function calling using JSON blobs isn't even the optimal approach... I saw some stuff recently about having LLMs write Python code to execute what they want, and LLMs tend to be a lot better at Python without any additional function-calling training. Some of the functions exposed to the LLM can be calls into your own logic. Some relevant links: This shows how python-calling performance is supposedly better for a range of existing models than JSON-calling performance: https://huggingface.co/blog/andthattoo/dpab-a#initial-results https://huggingface.co/blog/andthattoo/dpab-a#initial-result... A little post about the concept: https://huggingface.co/blog/andthattoo/dria-agent-a https://huggingface.co/blog/andthattoo/dria-agent-a Huggingface has their own "smolagents" library that includes "CodeAgent", which operates by the same principle of generating and executing Python code for the purposes of function calling: https://huggingface.co/docs/smolagents/en/guided_tour https://huggingface.co/docs/smolagents/en/guided_tour smolagents can either use a local LLM or a remote LLM, and it can either run the code locally, or run the code on a remote code execution environment, so it seems fairly flexible.

rozap 2y ago

What are you using for structured output? Outlines, BAML, etc seem to vary a huge amount in quality. It was many moons ago, but outlines was unusable. BAML has been great.