xyc

8 ms·

31.
by xyc 2y ago
Things like grab some markdown text and ask to make a pip/npm install one liner, or quick js scripts to paste in the console (which I didn't bother to open an editor), a fun use case was random drawing some lucky winners for the a
32.
by xyc 2y ago
Thanks! lmk when/if you wanna give it a spin as free trial hasn't been updated with the latest but I'll try to do it this week. I've actually been playing around with speech to text recently. Thank you for the pointer, d
33.
by xyc 2y ago
llama3.2 1b & 3b is really useful for quick tasks like creating some quick scripts from some text, then pasting them to execute as it's super fast & replaces a lot of temporary automation needs. If you don't feel like inve
34.
Fibonacci numbers form curves with line wrapping(twitter.com)
1 points by xyc 2y ago|0 comments
35.
by xyc 2y ago
They are not mutually exclusive though. WebSQL doesn't prevent anyone from loading a WASM blob. And while moving slowly, the browsers does deprecate old stuff and update implementation.
36.
by xyc 2y ago
dev of https://recurse.chat/ here, thanks for mentioning! rn we are focusing on features like shortcuts/floating window, but will look into support this in some time. to add to the llama.cpp support discussion, it&#x27
37.
by xyc 2y ago
i have a feeling that it set back the web by a decade https://x.com/chxy/status/1822858746307170640
38.
by xyc 2y ago
I built https://recurse.chat/ to solve this! Zero-setup is the goal. just starting and the app prompts you for downloading model.
39.
by xyc 2y ago
I wonder if it's possible for llamafile to distribute without the need for Xcode Command Line Tools, but perhaps it's necessary for the single cross-platform binary. Loved llamafile and used it to build the first version of https
40.
by xyc 2y ago
You can use llama.cpp server's tokenize endpoint to tokenize and count the tokens: https://github.com/ggerganov/llama.cpp/blob/master/examples/...
41.
by xyc 2y ago
If anyone is interested in trying local AI, you can give https://recurse.chat/ a spin. It lets you use local llama.cpp without setup, chat with PDF offline and provides chat history / nested folders chat organization,
42.
by xyc 2y ago
Privacy could be one reason. There are a lot of cases where people do not want to send data to a cloud service.
43.
by xyc 2y ago
Just realized I read your blog about Llava llamafile which got me interested in local AI and made the app :) What's your reservation about running it locally?
44.
by xyc 2y ago
Cloudflare has it https://developers.cloudflare.com/workers-ai/models/llava-1.... Locally it's actually quite easy to setup. I've made an app https://recurse.chat/ which supports Llava 1
45.
by xyc 2y ago
Not really. VS Code does have some performance optimizations where even the web browser optimization wouldn't suffice, for example it implements its own scroll bar instead of using the web native scroll bar. But for the most part the b
46.
by xyc 2y ago
VS Code vs XCode situation is an exact counter example of this. Non-optimized native apps could be that much slower than well optimized Electron apps.
47.
by xyc 2y ago
A user's personal data really does not have that much scale. Worst case they can cache everything locally. I've imported thousands of chat sessions into a local AI chat app's database, total storage is under 30MB. Full text s
48.
by xyc 2y ago
Check out https://recurse.chat (I'm the dev). You can import ChatGPT messages. It has almost instant full text search over thousands of chat sessions. Also supports llama.cpp, local embedding / RAG, and most recently b
49.
by xyc 2y ago
You can run llama.cpp and structured output with GNBF. There are tools to convert JSON schema to GNBF.
50.
Adding Hugging Face GGUF Model to RecurseChat(medium.com)
1 points by xyc 2y ago|0 comments
51.
by xyc 2y ago
I have been using local LLM as a daily driver. Built https://recurse.chat for it. I've used Llama 3, WizardLM 2, Mistral mostly, and sometimes just trying out models from hugging face (Recently added support for adding it f
52.
by xyc 2y ago
api access is text/vision for now https://x.com/mpopv/status/1790073021765505244
53.
by xyc 2y ago
will see :) heard video capability is rolling out later
54.
by xyc 2y ago
Seems that no client-side changes needed for gpt-4o chat completion Added a custom OpenAI endpoint to https://recurse.chat (i built it) and it just works: https://twitter.com/recursechat/status/17900744
55.
by xyc 2y ago
If you has suggestions for making RecurseChat more useful for you especially for RAG, I'd love to hear about it!
56.
by xyc 2y ago
Thank you, glad to hear that!
57.
by xyc 2y ago
have fun :) your users will probably let you know which way they want, or both
58.
by xyc 2y ago
Thank you! Yes the app has bundled built-in llama.cpp binary with it. The app just launches the executable - nothing fancy. We are a little llama.cpp wrapper :) It doesn't require Ollama, but if you have existing Ollama, it works with
59.
by xyc 2y ago
I use it as a daily driver (built https://recurse.chat/ ). Local RAG and chat with PDF is handy. Some of our users are using it to format transcripts (example: https://talk.macpowerusers.com/t/recursecha
60.
by xyc 2y ago
I don't think we need > 1 million vector yet. But we plan to target folder of local document such as obsidian vault or store web search results which could rack up a large number of vectors. Persistence on disk is also a desired fea

More ›