jerpint

28 ms·

1. 361.▲
   by jerpint 3y ago

   This is probably it +/- implementation details
2. 362.▲
   by jerpint 3y ago

   Let the model decide which words to focus on when producing a response
3. 363.▲
   by jerpint 3y ago

   Sounds reasonable, but each token_type_embedding would have to be kept private like a private key , and each model tuned to a users private key
4. 364.▲
   by jerpint 3y ago

   I’m curious as to whether this inductive bias wouldn’t hurt on tasks where the first sentence of a long corpus would contain the most useful information. Nonetheless, very clever trick and congrats on the great paper!
5. 365.▲
   by jerpint 3y ago

   Simply doing “restart + run all” ensures both readability and reproducibility
6. 366.▲
   by jerpint 3y ago

   No just at normalized log probabilities I think
7. 367.▲
   by jerpint 3y ago

   I’m most surprised by the approach they take of passing GPT tuples of (token, importance) and having the model reliably figure out the patterns. Nothing would suggest this should work in practice, yet it just… does. In more or less zero sho
8. 368.▲
   by jerpint 3y ago

   I made a version which works on any video, uses whisper to get the transcript. Might be time to whip it into a demo Video: https://m.youtube.com/watch?v=WtMrp2hp94E Blog: https://www.jerpint.io/blog/you
9. 369.▲
   by jerpint 3y ago

   Funny enough, I also made a YouTube video explaining how to summarize YouTube videos, and used it on my own video! YouTube: https://m.youtube.com/watch?v=WtMrp2hp94E Blog post: https://www.jerpint.io/blog&#x
10. 370.▲
    by jerpint 3y ago

    Cost to use openAI is also pretty low compared to hosting models
11. 371.▲
    by jerpint 3y ago

    The original announcement of GPT4 included a list of companies that had already integrated GPT4 to their product, it’s a totally unfair competitive advantage
12. 372.▲
    by jerpint 3y ago

    I was able to read privacy statement without cookie banner on iOS using brave browser
13. 373.▲
    by jerpint 3y ago

    If you use it through the API you can get very far with much less than 20$
14. 374.▲
    by jerpint 3y ago

    GPT is a specific instantiation of a transformer, and doing next token prediction was an openAI thing. Transformers were a big part of it, but GPT was def proposed but openAI
15. 375.▲
    by jerpint 3y ago

    GPT, CLIP, Dalle, RLHF, these are all novel
16. 376.▲
    by jerpint 3y ago

    > LLaVa connects pre-trained CLIP ViT-L/14 visual encoder and large language model LLaMA, using a simple projection matrix. The fact that it is ”this simple” to connect foundation models is astounding
17. 377.▲
    by jerpint 4y ago

    The results are very impressive, and the model + code is made available and open source, kudos to the meta/fair team for consistently pushing out good research
18. 378.▲
    by jerpint 4y ago

    They actually reimplemented the code to load the model. But it’s not clear if the weights of the model are free to use or not
19. 379.▲
    by jerpint 4y ago

    Reading from my phone right now can confirm
20. 380.▲
    by jerpint 4y ago

    The problem is not the short term context length, but long term. If you want to do things like long term goal planning, keeping track of distant past events can be of high value
21. 381.▲
    by jerpint 4y ago

    Every time I read the langchain docs, I get confused, and realize that implementing these things myself won’t take much time and offer me transparency… maybe I’m not the target audience?
22. 382.▲
    by jerpint 4y ago

    I’ve never seen such a difficult form to fill out to get access to an API, it’s kafkaesque
23. 383.▲
    GPT4 Can’t Play Wordle(jerpint.io)

    1 points by jerpint 4y ago|1 comments
24. 384.▲
    by jerpint 4y ago

    This is a pretty narrow view, and I disagree with it. We can study llama inside out, adapt it for good and bad. It’s objective. GPT is subject to openAI whim. Facebook is sharing findings, openAI shutting its doors
25. 385.▲
    by jerpint 4y ago

    Nice! I’m building something similar, which you can try on out on different sources of documentation: huggingface, PyTorch lighting, PyTorch, and more to come! We built a slack bot version too https://huggingface.co/spaces&#
26. 386.▲
    by jerpint 4y ago

    GPT 4 is limited currently to 8k tokens, which is about 6000 words. You can use our repo (which we are currently updating to include QuickStart tutorials, coming in the next few days) to do embedding retrieval and query www.GitHub.com/
27. 387.▲
    by jerpint 4y ago

    ChatGPT 4 currently accepts 8000 tokens and will eventually support 32k
28. 388.▲
    by jerpint 4y ago

    “This repo is GPL-v3 licensed. Rewrite it while preserving its main functionality”
29. 389.▲
    by jerpint 4y ago

    This is a big deal, up until now the only reasonable way of using BLOOM was with PETALS or lots of GPUs… great work!
30. 390.▲
    by jerpint 4y ago

    OpenAI I knows they’re just a few cycles behind open source models - probably why they stuck a deal early with Microsoft.