rgbrgb

19 ms·

1. 301.▲
   by rgbrgb 3y ago

   I have a hard time believing that many billions of people know about adtech surveillance in any way.
2. 302.▲
   by rgbrgb 3y ago

   Thanks, this and your other comment are very useful. I agree that finito ux is really cool and I had the same thought about it feeling sketchy to enable. They’re definitely more aligned with the use cases I’m excited about (more task orient
3. 303.▲
   by rgbrgb 3y ago

   thanks for checking it out. What’s the killer LMStudio feature for you? I think we probably have slightly different goals with our apps but would love to know what you’re liking regardless.
4. 304.▲
   by rgbrgb 3y ago

   This is really cool, congrats on releasing. Can you say a little bit about how you made it? Had no idea you could grab the selected text in macOS. Seems hard to do cross platform but I see you support windows too.
5. 305.▲
   by rgbrgb 3y ago

   Of course the other thing that's fun about local AI is that it's free if you already bought an expensive laptop. With the default model, I get ~32 tokens / second on my 2021 m1 ultra MacBook pro with 64GB RAM.
6. 306.▲
   Show HN: FreeChat – local LLM chat for macOS(freechat.run)

   4 points by rgbrgb 3y ago|6 comments
7. 307.▲
   by rgbrgb 3y ago

   Love the price, congrats! Any trick making it so cheap or is the margin just low?
8. 308.▲
   by rgbrgb 3y ago

   I see what you're saying, but it's pretty different from an open core analytics product or something. This is a language/dev environment and the bar is a lot higher because 1) I know it's going to be a big investment to
9. 309.▲
   by rgbrgb 3y ago

   Hard needle to thread. I understand the risk of getting community momentum in a direction you don't want to go, but I'm also doubtful that much of the audience they want is interested in a closed source programming language. Hopin
10. 310.▲
    by rgbrgb 3y ago

    https://github.com/psugihara/FreeChat llama.cpp native chat app for macOS runs well with metal on m1/m2, pretty slow on intel
11. 311.▲
    by rgbrgb 3y ago

    I think it’s a good bet based on watching inference speed of llama.cpp consistently improving and model ability / size on a similar trajectory. I expect there’s similar room for optimization (probably more) with hosted models. If you d
12. 312.▲
    by rgbrgb 3y ago

    I did consider Electron but I was interested in learning SwiftUI and trying to figure out how to make it as lightweight as possible. I've done a bunch of react-native in the past and we always hit snags with memory pressure / perf
13. 313.▲
    by rgbrgb 3y ago

    Latest 34B with llama.cpp [0] via the Mac app I’ve been building, FreeChat [1]. [0]: https://huggingface.co/TheBloke/Samantha-1.11-CodeLlama-34B-... [1]: https://github.com/psugihara/FreeChat
14. 314.▲
    by rgbrgb 3y ago

    fwiw I get more like 35-40 tokens/sec on my m1 macbook with a 7B model. That's way faster than I can read or skim. If we can figure out how to focus the expertise in small models, I don't see why it wouldn't be viable fo
15. 315.▲
    by rgbrgb 3y ago

    cooking meth, making molotov cocktails, discussing my medical history, sex ok seriously though I had fun over the weekend chatting with Samantha on a long car ride on my MacBook. We were mostly asking about history.
16. 316.▲
    by rgbrgb 3y ago

    Thanks for letting me know! I think it broke when I made the repo public this morning. Should be fixed now.
17. 317.▲
    by rgbrgb 3y ago

    This model runs in FreeChat for macOS [0] because it's supported by llama.cpp :) You'll just need to download a nice GGUF here: https://huggingface.co/TheBloke/Mistral-7B-v0.1-GGUF [0]: https://git
18. 318.▲
    by rgbrgb 3y ago

    Hey ya'll, just wanted to share the little mac app I made to wrap llama.cpp's server. It's called FreeChat. The idea is to make an app you can send to someone who knows nothing about LLMs and have them up and running a local
19. 319.▲
    Show HN: Open, uncensored LLMs in a native Mac app(github.com)

    1 points by rgbrgb 3y ago|1 comments
20. 320.▲
    by rgbrgb 3y ago

    I wouldn’t want to send everything I type to apple’s auto suggest server. For me it’s quite important that this is not apple’s approach to privacy.
21. 321.▲
    Horcrux: Split your file into encrypted fragments(github.com)

    497 points by rgbrgb 3y ago|135 comments
22. 322.▲
    by rgbrgb 3y ago

    Love this demo but as others noted it's really easy to find queries where it performs poorly (e.g. typos). Looks like the embedding model used (all-minilm-l6-v2) currently ranks 35th on the hugging face leaderboard [0]. I'd love t
23. 323.▲
    by rgbrgb 3y ago

    Interesting, yeah I buy that, thanks. Building my intuition with this stuff. Anyone seen a good open-source implementation of MoE with Llama yet?
24. 324.▲
    by rgbrgb 3y ago

    Why would MoE make it lower latency?
25. 325.▲
    by rgbrgb 3y ago

    This runs locally on a MacBook.
26. 326.▲
    by rgbrgb 3y ago

    pg used "man behind the curtain" (riff on Oz) for our real estate startup :)
27. 327.▲
    SimpleX – private messaging platform without user identifiers(github.com)

    2 points by rgbrgb 3y ago|5 comments
28. 328.▲
    by rgbrgb 3y ago

    Luckily you do not need credentials to solve problems with software. Good luck to the founders, it’s a worthy problem.
29. 329.▲
    Twitter-Agent: Build AI-Powered Agents for Twitter(github.com)

    2 points by rgbrgb 3y ago|0 comments
30. 330.▲
    by rgbrgb 3y ago

    In our context, images are viewable by others on the site so to support porn we'd want some part of the site that was walled off from kids / had an NSFW warning. That's a lot of extra work to add to our mvp. edit: Morally I d