5 ms·

I just realized that Justine was the person responsible for the massive reduction in the memory footprint of the Llama models back in March.[1] Super impressive

• gajnadsgjoas 3y ago
  You wanted to say the one was banned by the author because of all the drama that followed
  • jimsimmons 3y ago
    What drama. Ooc
    • pcj-github 3y ago
      https://github.com/github-drama/github-drama/pull/46 https://github.com/github-drama/github-drama/pull/46
      • skeaker 3y ago
        I don't understand this. What's the contention that's actually getting people upset here?
        • nl 3y ago
          Having read through a bunch of outraged comments the issue seems to have been: * It changed the file format * For some people it was slower that the original version particularly on low end computers One person was particularly outraged and reverted the whole change. However, the current version does use a similar approach to that which was proposed: https://github.com/ggerganov/llama.cpp/commit/f963b63afa0e057cfb9eba4d88407c6a0850a0d8 https://github.com/ggerganov/llama.cpp/commit/f963b63afa0e05...
          • nicman23 3y ago
            Lol why oss people are so antisocial - I include myself in the subset
            • mlajtos 3y ago
              Just a wild guess – developers lack soft/managerial skills. (Overgeneralization.) In companies, this is accounted for because developers are shielded by managers. But in F/OSS you get to interact directly with the developers. As for the why devs lack soft skills – hold your hats – programming is basically commanding. Without "thank you", without "please" and with lot of swearing. No time for pleasant bullshit. (By this reasoning, devs using declarative/functional programming should be more polite than imperative programmers, right? :D)
              • jart 3y ago
                It was 4chan. You can read my comment history on Hacker News if you want to learn more.
          • deleted 3y ago
            [deleted]
• m00x 3y ago
  It's just using mmap, nothing too impressive. It's a nice contribution nonetheless.
  • dundarious 3y ago
    There was more. You can't just splat giant C structs with pointers into shared memory/a file, and expect another process to just mmap and be able to recreate valid state again. At the very least the pointers are going to be all wrong. There was necessary work to adjust the file format. Not rocket science, but not just turning while(fread()) into open();mmap(). Also, there were insights into how to minimize which models needed adjustment. The ideas and code were worked on by at least 2 people, and I'm an outsider on that project, but I didn't see anything untoward like "stealing credit". The magic change wasn't a perfect move, but is the kind of thing I do locally when I don't know the project/binary format well yet, so not exactly the megalomaniacal move it was painted as. Better that only the version number changed, but she's independent and doing good work, so you'd kind of hope she has a self-promotion streak! Changing the magic would be on the very very low end of letting that side go a bit too far, assuming that was the impetus.
• CaptainNegative 3y ago
  Why is it Justine posts and seemingly only Justine posts that always get this type of comments? Do people regularly comment on the authors of other content, for better or for worse, and I miss it?
  • Conscat 3y ago
    Faster Than Lime, too.