9 ms·

Hi all, I'm the main maintainer from the OpenLLM team here. I'm actively developing the fine-tuning feature and will release a PR soon enough. Stay tuned. In th

Hi all, I'm the main maintainer from the OpenLLM team here. I'm actively developing the fine-tuning feature and will release a PR soon enough. Stay tuned. In the meanwhile, the best way to track the development workflow is at our discord, so feel free to join!!

moffkalast 3y ago
Very cool, btw it's not mentioned in the readme so I assume it's only for running full precision models or do quantized GGML/GPTQ/etc. also work with it?
- aarnphm 3y ago
  Hi there, 8bit and 4bit is currently supported on main. GPTQ is working in progress, as well as GGML
  - nacs 3y ago
    GPTQ support would be amazing (AutoGPTQ is an easy way to integrate GPTQ support - it's basically just importing autogptq and switching out 1 line in the model loading code).
user432678 3y ago
Thanks for the great project! Any chance, your team might consider more open platform than Discord for posting updates? I personally find Discord hard to use, and there’s no way to have sensible subscription (like RSS). Discord is usually muted.
- nologic01 3y ago
  s/rd/urse/g
  - samstave 3y ago
    HAHA this was one of my panel interview questions at Goooog' Q: "How do you do a search and replace for a string in VI" Me: I cant recall right now, i'd just google it"
    - countspongebob 3y ago
      Alt proposed answer "I'd install emacs".
    - mdaniel 3y ago
      What an insulting interview question, I hope it was just in jest or at the end looking to pad the time However, it did make me realize hidden therein is an actual interesting interview question, similar to the "describe what happens when you type an address into the browser's URL bar and hit enter": describe what happens after you type `:s/foo/bar` and hit enter. Followup version: what about `:%s/foo/bar`? The kind of thing that can be interesting to watch them reason through even if they don't know the answer, or even know what those syntaxes do.
- 3np 3y ago
  Related: As an operator/mod/admin it's fairly straight-forward to bridge a Discord channel to Matrix (and, if one so desires, from there to IRC), allowing users not on Discord to participate. Conservative mods concerned about spam can start with an allowlist for which servers can join. https://github.com/matrix-org/matrix-appservice-discord https://github.com/matrix-org/matrix-appservice-discord
- pmoriarty 3y ago
  Discord is a black hole where information goes to die. Its search and scrollback is awful. It's awful at being an archive, as finding anything that was asked more than a day or two ago is impractical. To use Discord in good faith and with open eyes, you have to prioritize communication in the present, and give up hope of archiving anything that was said for people who might need the information in the future.
  - Kiro 3y ago
    I find their search amazing. What's your issue with it?
    - geysersam 3y ago
      The issue is it's not indexed by Google
      - micromacrofoot 3y ago
        have you used google lately? might as well not be indexed with all the seo spam you get as top results
        andsoitis 3y ago
        > have you used google lately? might as well not be indexed with all the seo spam you get as top results I just googled "how to use openllm" as an example to test your thesis, and the results look very relevant to me. https://www.google.com/search?client=safari&rls=en&q=how+to+use+openllm&ie=UTF-8&oe=UTF-8 https://www.google.com/search?client=safari&rls=en&q=how+to+...
        DragonStrength 3y ago
        You might want to glance again because all of those results are for a different product.
- ralusek 3y ago
  I know this isn't a great time for reddit, but I just made this on your behalf: https://www.reddit.com/r/OpenLLM/ https://www.reddit.com/r/OpenLLM/ I much prefer the HN/Reddit discussion format to Discord and even Stack Overflow.
- khimaros 3y ago
  plugging the open source and self hostable https://revolt.chat https://revolt.chat which i've found to have great UX and be very performant compared to discord.
  - shmolf 3y ago
    I'm liking revolt. Thanks for the suggestion.
- dluc 3y ago
  good alternative: https://www.linen.dev/ https://www.linen.dev/
manojlds 3y ago
How can we stay tuned if we can't do tuning? :P
- chaoyu 3y ago
  Fine-tuning is coming up in the next release! You can actually try it out on the main branch :P
ignoramous 3y ago
Discord: https://discord.com/invite/qc3RekjtuY https://discord.com/invite/qc3RekjtuY
bsaul 3y ago
Side question : why are people working on open source project communicating through discord a lot noawadays ? are discord conversations persisted and indexed on search engines ?
- ufo 3y ago
  1.People like talking to each other on discord. 2. No. :/
- cinntaile 3y ago
  Because it's easy, free and it just works. Very few people actually care about indexing the conversations.
  - deleted 3y ago
    [deleted]
  - pmontra 3y ago
    So all knowledge is lost and questions have to be asked and answered again and again?
    - numpad0 3y ago
      No, questions don't have to be asked and answered again and again, because all the knowledge is lost, full stop. No one would know anything.
    - michaelt 3y ago
      That didn't stop IRC being popular in the 1990s. There has long been a place in the ecosystem for ephemeral chat. Often alongside non-ephemeral things like written documentation.
      - nixass 3y ago
        Also monks being the only ones who can read and write didn't stop religion to be popular in middle ages. /s C'mon
photochemsyn 3y ago
What's the rationale for telemetry tracking? https://github.com/bentoml/OpenLLM/blob/main/src/openllm/utils/analytics.py https://github.com/bentoml/OpenLLM/blob/main/src/openllm/uti...
- nacs 3y ago
  They have a section about it in the README: https://github.com/bentoml/OpenLLM#-telemetry https://github.com/bentoml/OpenLLM#-telemetry
jasmer 3y ago
[dead]

moffkalast 3y ago

Very cool, btw it's not mentioned in the readme so I assume it's only for running full precision models or do quantized GGML/GPTQ/etc. also work with it?

aarnphm 3y ago
Hi there, 8bit and 4bit is currently supported on main. GPTQ is working in progress, as well as GGML
- nacs 3y ago
  GPTQ support would be amazing (AutoGPTQ is an easy way to integrate GPTQ support - it's basically just importing autogptq and switching out 1 line in the model loading code).

user432678 3y ago

Thanks for the great project! Any chance, your team might consider more open platform than Discord for posting updates? I personally find Discord hard to use, and there’s no way to have sensible subscription (like RSS). Discord is usually muted.

nologic01 3y ago
s/rd/urse/g
- samstave 3y ago
  HAHA this was one of my panel interview questions at Goooog' Q: "How do you do a search and replace for a string in VI" Me: I cant recall right now, i'd just google it"
  - countspongebob 3y ago
    Alt proposed answer "I'd install emacs".
  - mdaniel 3y ago
    What an insulting interview question, I hope it was just in jest or at the end looking to pad the time However, it did make me realize hidden therein is an actual interesting interview question, similar to the "describe what happens when you type an address into the browser's URL bar and hit enter": describe what happens after you type `:s/foo/bar` and hit enter. Followup version: what about `:%s/foo/bar`? The kind of thing that can be interesting to watch them reason through even if they don't know the answer, or even know what those syntaxes do.
3np 3y ago
Related: As an operator/mod/admin it's fairly straight-forward to bridge a Discord channel to Matrix (and, if one so desires, from there to IRC), allowing users not on Discord to participate. Conservative mods concerned about spam can start with an allowlist for which servers can join. https://github.com/matrix-org/matrix-appservice-discord https://github.com/matrix-org/matrix-appservice-discord
pmoriarty 3y ago
Discord is a black hole where information goes to die. Its search and scrollback is awful. It's awful at being an archive, as finding anything that was asked more than a day or two ago is impractical. To use Discord in good faith and with open eyes, you have to prioritize communication in the present, and give up hope of archiving anything that was said for people who might need the information in the future.
- Kiro 3y ago
  I find their search amazing. What's your issue with it?
  - geysersam 3y ago
    The issue is it's not indexed by Google
    - micromacrofoot 3y ago
      have you used google lately? might as well not be indexed with all the seo spam you get as top results
      - andsoitis 3y ago
        > have you used google lately? might as well not be indexed with all the seo spam you get as top results I just googled "how to use openllm" as an example to test your thesis, and the results look very relevant to me. https://www.google.com/search?client=safari&rls=en&q=how+to+use+openllm&ie=UTF-8&oe=UTF-8 https://www.google.com/search?client=safari&rls=en&q=how+to+...
        DragonStrength 3y ago
        You might want to glance again because all of those results are for a different product.
ralusek 3y ago
I know this isn't a great time for reddit, but I just made this on your behalf: https://www.reddit.com/r/OpenLLM/ https://www.reddit.com/r/OpenLLM/ I much prefer the HN/Reddit discussion format to Discord and even Stack Overflow.
khimaros 3y ago
plugging the open source and self hostable https://revolt.chat https://revolt.chat which i've found to have great UX and be very performant compared to discord.
- shmolf 3y ago
  I'm liking revolt. Thanks for the suggestion.
dluc 3y ago
good alternative: https://www.linen.dev/ https://www.linen.dev/

manojlds 3y ago

How can we stay tuned if we can't do tuning? :P

chaoyu 3y ago
Fine-tuning is coming up in the next release! You can actually try it out on the main branch :P

ignoramous 3y ago

Discord: https://discord.com/invite/qc3RekjtuY https://discord.com/invite/qc3RekjtuY

bsaul 3y ago

Side question : why are people working on open source project communicating through discord a lot noawadays ? are discord conversations persisted and indexed on search engines ?

ufo 3y ago
1.People like talking to each other on discord. 2. No. :/
cinntaile 3y ago
Because it's easy, free and it just works. Very few people actually care about indexing the conversations.
- deleted 3y ago
  [deleted]
- pmontra 3y ago
  So all knowledge is lost and questions have to be asked and answered again and again?
  - numpad0 3y ago
    No, questions don't have to be asked and answered again and again, because all the knowledge is lost, full stop. No one would know anything.
  - michaelt 3y ago
    That didn't stop IRC being popular in the 1990s. There has long been a place in the ecosystem for ephemeral chat. Often alongside non-ephemeral things like written documentation.
    - nixass 3y ago
      Also monks being the only ones who can read and write didn't stop religion to be popular in middle ages. /s C'mon

photochemsyn 3y ago

What's the rationale for telemetry tracking? https://github.com/bentoml/OpenLLM/blob/main/src/openllm/utils/analytics.py https://github.com/bentoml/OpenLLM/blob/main/src/openllm/uti...

nacs 3y ago
They have a section about it in the README: https://github.com/bentoml/OpenLLM#-telemetry https://github.com/bentoml/OpenLLM#-telemetry

jasmer 3y ago

[dead]