3 ms·

> It took him two hours of passing errors to Claude for the endpoint to start working What? It's literally three actions and you're good: download llama.cpp, d

by stymaar 2mo ago

> It took him two hours of passing errors to Claude for the endpoint to start working What? It's literally three actions and you're good: download llama.cpp, download the model on Huggingface, and run it with. I have no idea how it's supposed to take two hours (unless you have a slow connection and the model download takes this much time, that is).

Schiendelman 2mo ago
Your mileage may vary. I tried this a couple months ago and spent a full day on it just not working before giving up. Anything I sent, it wouldn't run.
fxtentacle 2mo ago
My experience with Claude is that it suffers badly from “not invented here” syndrome. So probably it rebuilt something like llama from scratch and then 2 hours suddenly seems reasonable (if you don’t question the approach). And that’s the thing, someone with no experience isn’t going to question it.
- WhyNotHugo 2mo ago
  Not necessarily rebuild llama from scratch, build attempting to build it without cmake and manually invoking all the build commands would be quite in character.
- catlifeonmars 2mo ago
  I wonder if this is an artifact of RL, where the training heavily emphasizes codegen. It may be that the model is just better at generating code than reusing libraries, so it prefers the lowest cost approach. I also wonder if this manifests much less in contexts where the libraries/frameworks are a large part of the training set. It may be that the model doesn’t generalize well so it’s always better to use knowledge in its training set vs attempting to understand how to use a new, potentially never before seen (from the model perspective) api
raffraffraff 2mo ago
I'm not saying this is the case in the particular example, but these days a lot of people aren't bothering to read a 1 page README, and are instead letting claude "just fucking do it already". I see it at work. Claude has got better at "just fucking doing it" by asking if it's ok to go read the latest github issues and pull the README, which means that people will likely get lazier and lazier.
- funlang 2mo ago
  [flagged]
  - microtonal 2mo ago
    Add Just [...] And you have also fully adopted the writing patterns.
    - lelanthran 2mo ago
      Seems like he's self-aware, though - he's taking steps to move away from brain-atrophy.
      - bretpiatt 2mo ago
        My ability to detect sarcasm is not good. From looking at funlang's profile and other comments the profile looks like a LLM generated bot. Forums with full no verification pseudonyms seem like they have a real challenge ahead. How long until we need humanhackernews.com with public pseudonyms and a private trusted verification?
        funlang 1mo ago
        [flagged]
- enraged_camel 2mo ago
  To me laziness has nothing to do with it. I'd rather focus on my limited time on things that actually matter, e.g. the design of an important product feature or the root cause of a bug and the proper fix. I delegate everything else to Claude. It's really nice to be able to say to Fable "see if you can get this running locally" and then come back 10-30 mins later and read the results. I've discovered a lot of neat tooling this way that I otherwise would not have bothered tryin to set up because that can take up a lot of time. And even when it's fast, you're suffering from context-switching penalties. You framed it as "read the latest github issues and pull the README" but anyone who has worked in tech for an appreciable amount of time knows that that stuff can turn into a deep rabbit hole.
LeBit 2mo ago
In the README there is usually a section where it explains which llama.cpp parameters to configure (sampling, penalties, context size, etc) to get better results.

by stymaar 2mo ago

Schiendelman 2mo ago

Your mileage may vary. I tried this a couple months ago and spent a full day on it just not working before giving up. Anything I sent, it wouldn't run.

fxtentacle 2mo ago

My experience with Claude is that it suffers badly from “not invented here” syndrome. So probably it rebuilt something like llama from scratch and then 2 hours suddenly seems reasonable (if you don’t question the approach). And that’s the thing, someone with no experience isn’t going to question it.

WhyNotHugo 2mo ago
Not necessarily rebuild llama from scratch, build attempting to build it without cmake and manually invoking all the build commands would be quite in character.
catlifeonmars 2mo ago
I wonder if this is an artifact of RL, where the training heavily emphasizes codegen. It may be that the model is just better at generating code than reusing libraries, so it prefers the lowest cost approach. I also wonder if this manifests much less in contexts where the libraries/frameworks are a large part of the training set. It may be that the model doesn’t generalize well so it’s always better to use knowledge in its training set vs attempting to understand how to use a new, potentially never before seen (from the model perspective) api

raffraffraff 2mo ago

I'm not saying this is the case in the particular example, but these days a lot of people aren't bothering to read a 1 page README, and are instead letting claude "just fucking do it already". I see it at work. Claude has got better at "just fucking doing it" by asking if it's ok to go read the latest github issues and pull the README, which means that people will likely get lazier and lazier.

funlang 2mo ago
[flagged]
- microtonal 2mo ago
  Add Just [...] And you have also fully adopted the writing patterns.
  - lelanthran 2mo ago
    Seems like he's self-aware, though - he's taking steps to move away from brain-atrophy.
    - bretpiatt 2mo ago
      My ability to detect sarcasm is not good. From looking at funlang's profile and other comments the profile looks like a LLM generated bot. Forums with full no verification pseudonyms seem like they have a real challenge ahead. How long until we need humanhackernews.com with public pseudonyms and a private trusted verification?
      - funlang 1mo ago
        [flagged]
enraged_camel 2mo ago
To me laziness has nothing to do with it. I'd rather focus on my limited time on things that actually matter, e.g. the design of an important product feature or the root cause of a bug and the proper fix. I delegate everything else to Claude. It's really nice to be able to say to Fable "see if you can get this running locally" and then come back 10-30 mins later and read the results. I've discovered a lot of neat tooling this way that I otherwise would not have bothered tryin to set up because that can take up a lot of time. And even when it's fast, you're suffering from context-switching penalties. You framed it as "read the latest github issues and pull the README" but anyone who has worked in tech for an appreciable amount of time knows that that stuff can turn into a deep rabbit hole.

LeBit 2mo ago

In the README there is usually a section where it explains which llama.cpp parameters to configure (sampling, penalties, context size, etc) to get better results.