6 ms·
Show HN: AI companions stack – create and host your own AI companions
- Havoc 3y agoIs this a16z associate with the vc a16z?
- dmarcos 3y agoYeah. https://twitter.com/stuffyokodraws/status/1678799439471972352?s=46&t=2HptW4-5P7ffTQpwDgInRQ https://twitter.com/stuffyokodraws/status/167879943947197235... Smart. A VC firm that heavily invests in software platforms will make much better decisions if they have first hand experience using the products.
- rand1239 3y ago[flagged]
- arisAlexis 3y agoI don't get why I wouldn't just keep an open tab with chatgpt on specific topics. They will very soon have big enough context windows. Why build another UI, deployment pipes and all that jazz ? P.s nobody will *sms the companion
- rootusrootus 3y agoFor me, it would be cost and alignment. If I own the software, I can choose whatever alignment suits me, or none at all. And ChatGPT is $20/month (assuming you want GPT4, and I do). But there's still a good argument for a hybrid solution. Buy GPT4 access through the API and get a native UI to query it. Much cheaper to pay as you go, and someone else is still handling the heavy lifting. But if you want an uncensored model, you're out of luck.
- babyshake 3y agoIts roughly comparable to how you can use a spreadsheet to do a lot of different things, but it improves the UX (with some trade-offs) to have more custom designed UIs instead of directly using a spreadsheet.
- kimburgess 3y agoAs someone working in a large non-software org, that statement does not hold. Spreadsheets are the underlying infrastructure to a mildly terrifying amount of modern civilization.
- Karunamon 3y agoPrivacy is a big one. ChatGPT (the website) gives Openai the right to use your conversations for model training (unless you turn conversation history off but that feature is rather important to the UX). Anything going through the API on the other hand has a commitment to not do this and to purge the history after a month.
- eob 3y agoChatGPT is amazing but ultimately unwieldy for directed, long running relationships more nuanced than general themed chit chat. We’re in the nascent stages but I think there will probably always be a community of folks who want to add more nuance to the communication, whether it’s reveries that enact a mood or goal, tie-ins to other services, etc. Eg imagine wanting to have your ChatGPT DnD master also keep some kind of score. It may be ultimately easiest to put a wrapper around a themed GPT window that imposes a predictable way to do that rather than require everyone to figure out how to prompt it correctly.
- arisAlexis 3y agoBut they created a wrapper web page for a web page here
- Zetobal 3y agoIt's mostly horny teenagers, tweens and 4chan degenerates. They don't need a good product they just need a uncensored one.
- yonixw 3y agoIt's not host your own AI at all! You need like 5 saas api-keys to operate this app. But sure, you can host this UI on your own.
- causality0 3y agoIt makes me wonder how long it will be until open-source locally-run AI chatbots reach the GPT-4 level. Five years? Ten?
- swyx 3y agoits more a question of hardware progress than software probably. theres a minimum level for these behaviors to emerge
- crooked-v 3y agoThe real limiter is having enough GPU RAM to train and run a usefully large model. Everything else is just fiddly details.
- brucethemoose2 3y agoNot anymore. Llama.cpp and kobold.cpp run well with low vram and non Nvidia GPUs. Huggingface is full of finetunes now, and I believe a 33b model can be finetuned on a single 3090. Llama.cpp is developing some kind of training, but I have no idea what the requirements will be.
- causality0 3y agoTo me, GPT's real "secret sauce" is how it primes the model for interactive prompting instead of just text completion.
- ynniv 3y agoThere are "chat" and "instruct" variants of most models now.
- mcbuilder 3y agoI hosted my own AI using SillyTavern + text-generation backend (e.g. oogabooga / KoboldAI), just need the power needed to crank the LLM.
- brucethemoose2 3y agoKoboldcpp (with ngrok if you need it) is another excellent self hosting solution. 13b will work on 16GB RAM, and 33b on 32GB RAM, with pretty much any dGPU for a little acceleration and RAM offloading. Doubly so if you host it as an AI Horde node (so you have priority access to many models through the web browser).
- rand1239 3y ago[flagged]
- yding 3y agoGood job Yoko!
- nl 3y agoI love how honest the README is: > Shortcomings > Oh, there are so many.
- veaxvoid 3y agoyou mean kaggle? don’t you?
- asynchronous 3y agoTangentially related, how much would you guys be willing to pay for a company that could deliver a local model implementation that could run on high tier consumer grade hardware at a reduced ability? I feel like even if there was some severe restrictions (model wasn't open source, DRM, etc.) I’d still be willing to fork out for it.
- fallingmeat 3y agoIs this the promise of Amazon Sagemaker?
- FanaHOVA 3y agoMLC Chat lets you run RedPajama 3B on an iPhone. It's free. You'd need to specify what type of model you're thinking about I guess?
- idkyall 3y agoI've seen at least one startup around this idea posted on HN: https://codecomplete.ai/ https://codecomplete.ai/ Probably also tiny corp for self hosting: https://geohot.github.io/blog/jekyll/update/2023/05/24/the-tiny-corp-raised-5M.html https://geohot.github.io/blog/jekyll/update/2023/05/24/the-t... I think there's a tremendous concern about letting data exfiltrate to any AI SaaS offering among business execs. If you could offer an experience minus the cloud that's easy to use and has compliance/logging features, I think you'd find success from industries that are reticent sharing customer data(e.g. Banking, government) who benefit greatly from the NLP workflows that LLMs enable
- franchiser 3y ago[dead]