lxe

35 ms·

1. 571.▲
   by lxe 4y ago

   I think JS ecosystem needs a gradio-like pattern such as this one. I'm tired of how incredibly artificially low-level frontend development has become. I don't want to solve functional programming puzzles to make a button do someth
2. 572.▲
   by lxe 4y ago

   Ah yes that's right. Well they technically do use a visual transformer for CLIP text encoder as I understand.
3. 573.▲
   by lxe 4y ago

   Keep in mind that image transformer models like stable diffusion are generally smaller than language models, so they are easier to fit in wasm space. Also. you can finetune llama-7b on a 3090 for about $3 using LoRA.
4. 574.▲
   by lxe 4y ago

   Keep in mind, this tends to hallucinate a lot. I will attempt to debug this behavior in the future.
5. 575.▲
   Show HN: Cerebras-GPT-2.7B finetuned on Stanford Alpaca dataset(github.com)

   4 points by lxe 4y ago|1 comments
6. 576.▲
   by lxe 4y ago

   Just want to ad to this statement. Highest quality piece of software I've used in a while.
7. 577.▲
   by lxe 4y ago

   GPT3.5-trained LoRAs always sound like ChatGPT, and have nebulous use terms, especially for commercial usecases
8. 578.▲
   by lxe 4y ago

   Ah this is crazy how fast the space is moving. I'm literally trying to build a document embedding UI to do Q&A on. Great work!
9. 579.▲
   by lxe 4y ago

   For self-directed learning, my favorite has been actually using ChatGPT (GPT4, especailly), because you can just ask as you go along. Some questions I asked: I have a pytorch ml llm gpt-style model and it has many layers, called &quot
10. 580.▲
    by lxe 4y ago

    I'm betting many of the policymakers understand as much about EVs, the power grid, and global supply chains, as they do about TikTok.
11. 581.▲
    by lxe 4y ago

    This has the same energy as the Y2K hysteria.
12. 582.▲
    by lxe 4y ago

    with load_in_8bits ?
13. 583.▲
    by lxe 4y ago

    https://huggingface.co/OpenAssistant/oasst-sft-1-pythia-12b
14. 584.▲
    by lxe 4y ago

    Thank you! I'm looking into sentence-transformers also to create embeddings from documents.
15. 585.▲
    by lxe 4y ago

    Best way to mess around with FlexGen and LLMs on local hardware in general is https://github.com/oobabooga/text-generation-webui
16. 586.▲
    by lxe 4y ago

    Bing says: I could not find any specific information on how these sanctions affect academic research and cooperation between the US and Russia. Some sources suggest that some academics have canceled conferences, joint projects, and fu
17. 587.▲
    by lxe 4y ago

    Found the answer here: https://github.com/openai/chatgpt-retrieval-plugin#retrieval... The plugin uses OpenAI's text-embedding-ada-002 embeddings model to generate embeddings of document chunks, and then sto
18. 588.▲
    by lxe 4y ago

    How does the semantic search plugin work? Does it create an embedding for the model using the uploaded documents, or does it perform full-text search on the docs and then uses the model to answer based on the retrieved section of the docume
19. 589.▲
    by lxe 4y ago

    OpenChatKit is great tbh
20. 590.▲
    by lxe 4y ago

    Barebones llama is just a text completion model. You can give it a prompt like A conversation between a human and assistant Human: How old is the sun? Assistant: And it will complete it. Alpaca/Dalai are finetuned on
21. 591.▲
    by lxe 4y ago

    If there's context limitations (your document cant' fit in the context, or your session quickly loses the context), fine-tuning is a great option.
22. 592.▲
    by lxe 4y ago

    > What is the "output" when the input is just a (logically coherent) chunk of text? It probably won't change much if it's just a single sample. If you put in a large corpus of samples that repeat on the same theme, th
23. 593.▲
    by lxe 4y ago

    > With no real knowledge of LLM and only recently started to understand what LLM terms mean, such as 'model, inference, LLM model, intruction set, fine tuning' whatelse do you think is required to make a took like yours? This w
24. 594.▲
    by lxe 4y ago

    So hard I’ve been using lambda labs, vast.ai, and runpod to rent machines. A 3090 is about 30 cents an hour.
25. 595.▲
    by lxe 4y ago

    You can probably finetune a 13b one with that. Try these scripts: https://github.com/zphang/minimal-llama/#minimal-llama
26. 596.▲
    by lxe 4y ago

    > I can't help but question the actual value of such a tool. Is it just an attempt to dumb down the process for newcomers? Actually, you've hit the nail on the head here. I wanted something where I, a complete beginner, can qui
27. 597.▲
    by lxe 4y ago

    > How does the memory usage scale as the sequence length grows? That's a good question. I was under the assumption it's linearly proportional, but I can test it out I guess.
28. 598.▲
    by lxe 4y ago

    Lots of VRAM, but the method is the same. Here's someone who finetuned llama-30b on alpaca dataset for example: https://github.com/deep-diver/Alpaca-LoRA-Serve
29. 599.▲
    by lxe 4y ago

    Just added a screenshot of the inference UI. I just finished using it to tune on a subset of https://huggingface.co/datasets/Anthropic/hh-rlhf , and it seems to be working.
30. 600.▲
    by lxe 4y ago

    I haven't tried it myself, and I haven't actually heard anyone attempt doing it with a model this large.