16 ms·

huh? Of all things what does 4chan have to do with anything here?

jart 4y ago
See https://rentry.org/Jarted https://rentry.org/Jarted I'm not sure if it's 4chan but it appears to be some chan. When I was working on solving the mmap() problem in https://github.com/ggerganov/llama.cpp/issues/91 https://github.com/ggerganov/llama.cpp/issues/91 (that issue tracks the full history) I originally wrote a malloc() hack to prove zero latency load times could work. But I thought we should hold off and wait until we could fix the file format before merging anything into master. A week or so later, an anonymous GitHub user called Slarin pointed out something really smart. He said the 7B model only has 1D tensors, so we could bring the benefits of mmap() to 7B users right away. My file format change would have solved it for all models. But it wasn't ready yet. So when Slarin asked me to abandon my mmap branch, start over, and use his change as the starting point, I said sure. Since that would have ensured users of the project get some benefit as soon as possible. And we could iterate. So Slarin and I collaborated on Discord. I had a really positive impression. As far as I knew, everything was fine. Slarin made a pull request, which he kept in draft mode. The work he was doing was basically to (1) call mmap() to load the full weights into memory, and (2) update the tensor->data instead of calling read() if the tensor was 1D. The issue is he Slarin got blocked on getting the WIN32 support to work. He had difficulties getting it to not crash. And by the time he was blocked, I had already finished rewriting the file format and data conversion tools. So I said, since I'm done with the full change, just push whatever you did to your PR branch, and I'll just rebase on that, I'll fix your WIN32 code, and then we'll merge our commits as part of the same PR. That PR was https://github.com/ggerganov/llama.cpp/pull/613 https://github.com/ggerganov/llama.cpp/pull/613 where I was very careful to recognize and document all of Slarin's contributions, which are now reflected in the master branch. Slarin agreed, and told me "it looks like you've got this situation handled" and then left. We haven't seen him on the Discord since. I honestly had no idea he was so unhappy about the way things went. It really bums me out, because I have no way of ensuring people stay happy if they don't tell me they're unhappy. It appears, based on the link above, that Slarin felt I had put him in charge of the mmap() contribution and that the change belonged to him because his work was going to be its genesis once merged in the master branch. After the contribution went viral, he probably felt I didn't do enough to give him credit (I'm not exactly sure how to credit an anonymous developer more than I did?) and as a result he went to an online forum to ask a troll brigade to give justice. The fallout of that "chan" troll brigade, is what you see here. They've also been going after my Wikipedia page among other things. I'm probably never going to work with an anonymous person again. I'm a public figure. I live with the sword of damocles hanging over my head. The power that gives anonymous people over me is really quite huge. The consequences here have damaged my reputation and career. And I don't even know who I'd reach out to if I wanted to mediate this. Quite a shame.
- xiphias2 4y ago
  Thanks, that clears it up somewhat (especially the trollings and downvotes). The cool thing is nobody cares about your wikipedia page, people care about your contributions. I still think you should hop onto a video chat with Greg and be humble (I hope he can be more understanding and humble as well). File formats don't matter, that's just technical detail (although important) that will be resolved. People relations do matter.
  - jart 4y ago
    I tried getting him on video chat before this troll brigade even happened. He hasn't responded to any of my messages over the past few days.
    - xiphias2 4y ago
      I see, I'm sorry about that.
      - jart 4y ago
        No reason to be sorry. GG has had more success than he can handle right now. It's a good problem to have. It takes quite a stressful toll the first time one of your community projects skyrockets into the big leagues. I'm sure things will be fine if we just wait a little bit for this to blow over.
        chuddington 4y ago
        You're Wondering Now, But All Results Well :)
- stevenhuang 4y ago
  Hey Jart, I read that rentry and suspected there was more to it, so I'm relieved to see your input and the full picture here. Such is the way with online drama. Though I think only working with known people might not be it, as the same could still happen (not being communicative on their part, silently brooding, then doing something regrettable). Perhaps it's less likely. I would venture those in the field understand that unfortunate misunderstandings of this sort happen all the time and find nuance in the situation. Please don't let this stop you from contributing, your work is amazing. Cheers.
washadjeffmad 4y ago
That's where the brunt of the coordination, outreach, and testing around these models is happening.

jart 4y ago

See https://rentry.org/Jarted https://rentry.org/Jarted I'm not sure if it's 4chan but it appears to be some chan. When I was working on solving the mmap() problem in https://github.com/ggerganov/llama.cpp/issues/91 https://github.com/ggerganov/llama.cpp/issues/91 (that issue tracks the full history) I originally wrote a malloc() hack to prove zero latency load times could work. But I thought we should hold off and wait until we could fix the file format before merging anything into master. A week or so later, an anonymous GitHub user called Slarin pointed out something really smart. He said the 7B model only has 1D tensors, so we could bring the benefits of mmap() to 7B users right away. My file format change would have solved it for all models. But it wasn't ready yet. So when Slarin asked me to abandon my mmap branch, start over, and use his change as the starting point, I said sure. Since that would have ensured users of the project get some benefit as soon as possible. And we could iterate. So Slarin and I collaborated on Discord. I had a really positive impression. As far as I knew, everything was fine. Slarin made a pull request, which he kept in draft mode. The work he was doing was basically to (1) call mmap() to load the full weights into memory, and (2) update the tensor->data instead of calling read() if the tensor was 1D. The issue is he Slarin got blocked on getting the WIN32 support to work. He had difficulties getting it to not crash. And by the time he was blocked, I had already finished rewriting the file format and data conversion tools. So I said, since I'm done with the full change, just push whatever you did to your PR branch, and I'll just rebase on that, I'll fix your WIN32 code, and then we'll merge our commits as part of the same PR. That PR was https://github.com/ggerganov/llama.cpp/pull/613 https://github.com/ggerganov/llama.cpp/pull/613 where I was very careful to recognize and document all of Slarin's contributions, which are now reflected in the master branch. Slarin agreed, and told me "it looks like you've got this situation handled" and then left. We haven't seen him on the Discord since. I honestly had no idea he was so unhappy about the way things went. It really bums me out, because I have no way of ensuring people stay happy if they don't tell me they're unhappy. It appears, based on the link above, that Slarin felt I had put him in charge of the mmap() contribution and that the change belonged to him because his work was going to be its genesis once merged in the master branch. After the contribution went viral, he probably felt I didn't do enough to give him credit (I'm not exactly sure how to credit an anonymous developer more than I did?) and as a result he went to an online forum to ask a troll brigade to give justice. The fallout of that "chan" troll brigade, is what you see here. They've also been going after my Wikipedia page among other things. I'm probably never going to work with an anonymous person again. I'm a public figure. I live with the sword of damocles hanging over my head. The power that gives anonymous people over me is really quite huge. The consequences here have damaged my reputation and career. And I don't even know who I'd reach out to if I wanted to mediate this. Quite a shame.

xiphias2 4y ago
Thanks, that clears it up somewhat (especially the trollings and downvotes). The cool thing is nobody cares about your wikipedia page, people care about your contributions. I still think you should hop onto a video chat with Greg and be humble (I hope he can be more understanding and humble as well). File formats don't matter, that's just technical detail (although important) that will be resolved. People relations do matter.
- jart 4y ago
  I tried getting him on video chat before this troll brigade even happened. He hasn't responded to any of my messages over the past few days.
  - xiphias2 4y ago
    I see, I'm sorry about that.
    - jart 4y ago
      No reason to be sorry. GG has had more success than he can handle right now. It's a good problem to have. It takes quite a stressful toll the first time one of your community projects skyrockets into the big leagues. I'm sure things will be fine if we just wait a little bit for this to blow over.
      - chuddington 4y ago
        You're Wondering Now, But All Results Well :)
stevenhuang 4y ago
Hey Jart, I read that rentry and suspected there was more to it, so I'm relieved to see your input and the full picture here. Such is the way with online drama. Though I think only working with known people might not be it, as the same could still happen (not being communicative on their part, silently brooding, then doing something regrettable). Perhaps it's less likely. I would venture those in the field understand that unfortunate misunderstandings of this sort happen all the time and find nuance in the situation. Please don't let this stop you from contributing, your work is amazing. Cheers.

washadjeffmad 4y ago

That's where the brunt of the coordination, outreach, and testing around these models is happening.