3 ms·

I'm sorry, but isn't Cursor just an editor? Maybe an editor shouldn't actually have garbage parts to avoid? Why not just use an editor that is focused on codin

I'm sorry, but isn't Cursor just an editor? Maybe an editor shouldn't actually have garbage parts to avoid? Why not just use an editor that is focused on coding, and then just not use an LLM at all? Less fighting the tooling, more getting your job done with less long term landmines. There are a lot of editors, and many of them even have native or semi-native LLM support now. Pick one. Edit: Also, side note, why are so many people running their LLMs in the cloud? All the cutting edge models are open weight licensed, and run locally. You don't need to depend on some corporation that will inevitably rug-pull you. Like, a 7900XTX runs you about $1000. You probably already own a GPU that cost more in your gaming rig.

krapht 2y ago
> Edit: Also, side note, why are so many people running their LLMs in the cloud? All the cutting edge models are open weight licensed, and run locally. You don't need to depend on some corporation that will inevitably rug-pull you. ??? Deepseek R1 doesn't run locally unless you program on a dual socket server with 1 TB of RAM. Or enough cash to have a cabinet of GPUs. The trend for state-of-the-art LLMs is to get bigger over time, not smaller. Look, I've played with llava and llama locally too, but the benchmarked performance is nowhere near what you can get from the larger cloud providers who can serve hundred-million+ parameter models without quantization.
- DiabloD3 2y ago
  You wouldn't use full fledged R1 for coding. There are distilled models using R1 for coding that get you most of the way there. R1 also doesn't take 1TB of RAM, go use read Unsloth's writeup on how to reduce model size without reducing quality (they got it to fit into 131GB): https://unsloth.ai/blog/deepseekr1-dynamic https://unsloth.ai/blog/deepseekr1-dynamic tl;dr parameter count is where the statistical model lives or dies, not weight precision; you can't blindly shrink every weight, and tooling is learning how to not butcher models. Also, performance between cloud-ran models and models I've ran locally with llama.cpp seem to be actually pretty similar. Are you sure your model didn't fit into your VRAM, or something else may have been misconfigured? Not fitting into VRAM slows everything to a halt. All the coder models that are worth looking at fit into 24GB cards in their full sized variants with the right quantization.
  - satvikpendem 2y ago
    Distilled "DeepSeek" models are not actually DeepSeek and should not be referred to as such.
    - DiabloD3 2y ago
      No one said they were. They're distilled using the original model and the same weights that match the ones in R1. Its ostensibly the original, but better. There are also fused models such as https://huggingface.co/FuseAI/FuseO1-DeepSeekR1-Qwen2.5-Coder-32B-Preview https://huggingface.co/FuseAI/FuseO1-DeepSeekR1-Qwen2.5-Code... that also seem to perform interestingly.
      - satvikpendem 2y ago
        They are not better, they are strictly worse in every way, and the performance characteristics show such degradation. There is a difference between distilling and quantizing, as while the latter does show some degradation too, it is not to the extent of distilled models and at least it's still the original model.
        DiabloD3 2y ago
        Depends how you define "worse in every way". With my own personal testing, DeepSeek's distillations have been able to do the task when the original upstream model either couldn't, or was marginally worse yet. You're preaching to the anti-choir on this, though: I do not think LLMs are ready for use yet. Maybe another few years, maybe another few decades, we'll find out I guess, but what we have today sure as hell isn't it.
deleted 2y ago
[deleted]
zild3d 2y ago
> All the cutting edge models are open weight licensed, and run locally. No? from https://lmarena.ai/ https://lmarena.ai/ coding: ... Rank* ... Model ... Score ... Org ... License 1 ... Grok-3-Preview-02-24 ... 1414 ... xAI ... Proprietary 1 ... GPT-4.5-Preview ... 1413 ... OpenAI ... Proprietary 3 ... Gemini-2.0-Pro-Exp-02-05 ... 1378 ... Google ... Proprietary 3 ... o3-mini-high ... 1369 ... OpenAI ... Proprietary 3 ... DeepSeek-R1 ... 1369 ... DeepSeek ... MIT 3 ... ChatGPT-4o-latest (2025-01-29) ... 1367 ... OpenAI ... Proprietary 3 ... Gemini-2.0-Flash-Thinking-Exp ... 1366 ... Google ... Proprietary 3 ... o1-2024-12-17 ... 1359 ... OpenAI ... Proprietary 3 ... o3-mini ... 1353 ... OpenAI ... Proprietary 4 ... o1-preview ... 1355 ... OpenAI ... Proprietary 4 ... Gemini-2.0-Flash-001 ... 1354 ... Google ... Proprietary 4 ... o1-mini ... 1353 ... OpenAI ... Proprietary 4 ... Claude 3.7 Sonnet ... 1350 ... Anthropic ... Proprietary
- DiabloD3 2y ago
  Yes, I'm aware of various rankings. Try all of those models on something that isn't commonly used on a benchmark, and you'll notice that a lot of the proprietary models have trouble actually producing statistically relevant results. The only one that I've come across that makes me think LLMs will maybe be useful someday is Deepseek R1 and the redistillations based on it. I've seen HN's fascination with OpenAI's products, and I can't understand why. Even O1 and O3, they're always too little too late, somebody else already is doing something better and throwing it into a HF repo. Must be the Silicon Valley RDF at work.

krapht 2y ago

> Edit: Also, side note, why are so many people running their LLMs in the cloud? All the cutting edge models are open weight licensed, and run locally. You don't need to depend on some corporation that will inevitably rug-pull you. ??? Deepseek R1 doesn't run locally unless you program on a dual socket server with 1 TB of RAM. Or enough cash to have a cabinet of GPUs. The trend for state-of-the-art LLMs is to get bigger over time, not smaller. Look, I've played with llava and llama locally too, but the benchmarked performance is nowhere near what you can get from the larger cloud providers who can serve hundred-million+ parameter models without quantization.

DiabloD3 2y ago
You wouldn't use full fledged R1 for coding. There are distilled models using R1 for coding that get you most of the way there. R1 also doesn't take 1TB of RAM, go use read Unsloth's writeup on how to reduce model size without reducing quality (they got it to fit into 131GB): https://unsloth.ai/blog/deepseekr1-dynamic https://unsloth.ai/blog/deepseekr1-dynamic tl;dr parameter count is where the statistical model lives or dies, not weight precision; you can't blindly shrink every weight, and tooling is learning how to not butcher models. Also, performance between cloud-ran models and models I've ran locally with llama.cpp seem to be actually pretty similar. Are you sure your model didn't fit into your VRAM, or something else may have been misconfigured? Not fitting into VRAM slows everything to a halt. All the coder models that are worth looking at fit into 24GB cards in their full sized variants with the right quantization.
- satvikpendem 2y ago
  Distilled "DeepSeek" models are not actually DeepSeek and should not be referred to as such.
  - DiabloD3 2y ago
    No one said they were. They're distilled using the original model and the same weights that match the ones in R1. Its ostensibly the original, but better. There are also fused models such as https://huggingface.co/FuseAI/FuseO1-DeepSeekR1-Qwen2.5-Coder-32B-Preview https://huggingface.co/FuseAI/FuseO1-DeepSeekR1-Qwen2.5-Code... that also seem to perform interestingly.
    - satvikpendem 2y ago
      They are not better, they are strictly worse in every way, and the performance characteristics show such degradation. There is a difference between distilling and quantizing, as while the latter does show some degradation too, it is not to the extent of distilled models and at least it's still the original model.
      - DiabloD3 2y ago
        Depends how you define "worse in every way". With my own personal testing, DeepSeek's distillations have been able to do the task when the original upstream model either couldn't, or was marginally worse yet. You're preaching to the anti-choir on this, though: I do not think LLMs are ready for use yet. Maybe another few years, maybe another few decades, we'll find out I guess, but what we have today sure as hell isn't it.

deleted 2y ago

[deleted]

zild3d 2y ago

> All the cutting edge models are open weight licensed, and run locally. No? from https://lmarena.ai/ https://lmarena.ai/ coding: ... Rank* ... Model ... Score ... Org ... License 1 ... Grok-3-Preview-02-24 ... 1414 ... xAI ... Proprietary 1 ... GPT-4.5-Preview ... 1413 ... OpenAI ... Proprietary 3 ... Gemini-2.0-Pro-Exp-02-05 ... 1378 ... Google ... Proprietary 3 ... o3-mini-high ... 1369 ... OpenAI ... Proprietary 3 ... DeepSeek-R1 ... 1369 ... DeepSeek ... MIT 3 ... ChatGPT-4o-latest (2025-01-29) ... 1367 ... OpenAI ... Proprietary 3 ... Gemini-2.0-Flash-Thinking-Exp ... 1366 ... Google ... Proprietary 3 ... o1-2024-12-17 ... 1359 ... OpenAI ... Proprietary 3 ... o3-mini ... 1353 ... OpenAI ... Proprietary 4 ... o1-preview ... 1355 ... OpenAI ... Proprietary 4 ... Gemini-2.0-Flash-001 ... 1354 ... Google ... Proprietary 4 ... o1-mini ... 1353 ... OpenAI ... Proprietary 4 ... Claude 3.7 Sonnet ... 1350 ... Anthropic ... Proprietary

DiabloD3 2y ago
Yes, I'm aware of various rankings. Try all of those models on something that isn't commonly used on a benchmark, and you'll notice that a lot of the proprietary models have trouble actually producing statistically relevant results. The only one that I've come across that makes me think LLMs will maybe be useful someday is Deepseek R1 and the redistillations based on it. I've seen HN's fascination with OpenAI's products, and I can't understand why. Even O1 and O3, they're always too little too late, somebody else already is doing something better and throwing it into a HF repo. Must be the Silicon Valley RDF at work.