5 ms·
This is off-topic, but are there any communities or congregations (that aren't reddit) based around locally hosted LLMs? I'm asking because while I see a bunch
by srslack 3y ago
This is off-topic, but are there any communities or congregations (that aren't reddit) based around locally hosted LLMs? I'm asking because while I see a bunch of projects for exposing GGML/LLaMA to OpenAI compatible interfaces, some UIs, etc, I can't really find a good community or resources for the concept in general.
I'm working on a front-end for LLMs in general, having re-implemented a working version of OpenAI's code interpreter "plugin" already within the UI (and yes, I support file uploads), and support for the wealth of third-party OpenAI plugins that don't require auth (I've been testing with the first diagram plugin I found, it works well.) I'm planning to open source it once my breaking changes slow down.
This field moves very fast, I'm looking for feedback (and essentially testers/testing data) on what people want, and looking for prompts/chat logs/guidance templates (https://github.com/microsoft/guidance https://github.com/microsoft/guidance) for tasks they expect to "just work" with natural language.
Instead of being limited by the monetization for ChatGPT Plus (and limited number of messages every four hours) for extensibility within a chat interface, I want to open it and free it, with a Bring-Your-Own-(optionally local)-LLM/API key setup.
- dharma1 3y agoit's reddit, but /r/LocalLLaMA/
- anon178 3y agoThe /g/ board on 4chan has a /lmg/ general that focuses on running models locally. They regularly discuss fine tuning models, quantization tech, and building apps on text-generation-webui/kobold. You might get some interest but it's also 4chan...
- MarcoZavala 3y ago[dead]
- thorum 3y agoI'd love to hear more about your approach for getting LLMs to understand how to use plugin commands. My own experiments have not worked very well (even vanilla ChatGPT through the gpt-3.5-turbo API doesn't seem to get the concept, most of the time).
- srslack 3y agoI'll do a show HN probably at the beginning of next month after this hackathon, but basically look into langchain's "tools" and the different agents they have. You don't need langchain for this at all, but it gives you the groundwork. I saw their Code Interpreter demo on Twitter (converting an uploaded video file in a chat UI) and decided that I need that, without continuing to pay them money (because they still haven't given me access to it yet.) So, that, and after sam a went in front of congress for the regulatory capture play, was the motivation I needed to work towards commoditizing these fuckers. The secret sauce here with code interpreter is, well, literally a python code interpreter you can run in your browser, and it's not so secret.
- pizza 3y agoI imagine you might find your answer in the form of a discord community
- hnjst 3y agohttps://discord.gg/dwXDHMGY https://discord.gg/dwXDHMGY for example.
- MacsHeadroom 3y agoThe "lmg" (Local Models General) thread on 4chan's technology board /g/[0] is the premiere community communication spot for open source models, believe it or not. Everyone from the infamous "oobabooga" to llama.cpp's Georgi Gerganov regularly hangs out in the thread. If you have questions, you will get answers there. [0] https://boards.4channel.org/g/#s=lmg https://boards.4channel.org/g/#s=lmg
- ShamelessC 3y agoYou know HN has gotten lackluster when 4chan is more informed.
- Tepix 3y agoHN is hardly the right place for an ongoing discussion
- ShamelessC 3y agoClearly.
- ianpurton 3y agoYou might want to try some of the discord channels connected to some of the repos. i.e. GPT4All https://github.com/nomic-ai/gpt4all https://github.com/nomic-ai/gpt4all scroll down for the discord link.
- deleted 3y ago[deleted]