7 ms·
Jan: An open source alternative to ChatGPT that runs on the desktop
- moinulmoin 3y agoseems like Ollama with GUI? MindMac with Ollama offers more functionality tbh!
- themusicgod1 3y agoGithub does not count as open source: this is not actually open source.
- alexb24 3y agoThis project is open source: it is released under the Affero GPL, an OSI approved license [1]. [1] https://opensource.org/license/agpl-v3 https://opensource.org/license/agpl-v3
- ShamelessC 3y agoQuite a lot of polish and bragging about stars and tweets for an open source project. Is there hidden monetization of some sort? Perhaps VC funding?
- anewhnaccount2 3y agoAccording to their page https://jan.ai/team/ https://jan.ai/team/ they aim to bootstrap.
- ShamelessC 3y agoAwesome thanks.
- thedangler 3y agoI don't see anything about it reading local documents like exel, pdfs, or docs. Anyone see how this is accomplished?
- 0134340 3y agoIs it implied anywhere? That's a feature I'd love and also why I haven't bothered delving into LLMs very much; I didn't know there were any that could locally index your library and train on that data. I'd love to ask it a question and have it reference my local ebook library.
- theGnuMe 3y agoIt probably doesn't. The only one that read PDFs for me was the Nvidia ChatRTX. It would be easy to add modules from pip that do this but you'd have to code up the input pipeline. It's not terribly difficult but it is definitely not point and click.
- afian 3y agoFantastic product and excellent team!
- throwitaway1123 3y agoThis looks interesting. I would love a comparison between this product and LM Studio.
- okasaki 3y ago[flagged]
- lmeyerov 3y agoFor AI projects, afaict, 12k stars or forks is more akin to downloads than contributors & downstreams. GitHub is the app distribution, not just source distribution. I've been curious how to model this better..
- deleted 3y ago[deleted]
- ThrowawayTestr 3y agoWhere does the model come from?
- LordDragonfang 3y agoAfaict, it doesn't have any inbuilt model, you just download one yourself or hook up to someone's API.
- dsp_person 3y agoScroll down on the main page: 01 Run local AI or connect to remote APIs 02 Browse and download models
- CyberEldrich 3y ago[flagged]
- deleted 3y ago[deleted]
- thesurlydev 3y agoThese kinds of apps are becoming dime a dozen. It would be nice to know how this one differentiates itself. Not obvious from the website.
- viraptor 3y agoIt seems like that until you actually try to use them. Not many are actually polished, support formatting, history, and multiple endpoints. There's lots of trivial apps abandoned after a few days, but what are the actually functional, good quality alternatives to this one? (That don't pass your query/answer through a third-party for data collection)
- extr 3y agoI use https://www.typingmind.com/ https://www.typingmind.com/. It is paid, but I've found it to be a reliable front end to OpenAI/Claude/Google, supporting everything you mention. I haven't done any hyper detailed security audit but after watching network requests I'm pretty confident it's not sending my chats anywhere except to the relevant provider endpoints. Considering how much I use it, I've found it to be well worth the cost. The creator is pretty on top of model/API changes.
- karmajunkie 3y agoi’ll second that recommendation… i use it through the SetApp store and i’ve been very pleasantly surprised by its documentation and ability to work with most services.
- viraptor 3y agoIt's not a standalone app though. There's lots of web interfaces, but that's not the same. (I mean, it's a cool thing, but not what jan.ai is)
- extr 3y agoThere is a desktop app available (I mean it’s basically a wrapper around the web UI, but still).
- outcoldman 3y ago[flagged]
- deleted 3y ago[deleted]
- moose44 3y agoRunning LLMs locally always feels so awesome!
- pryelluw 3y agoWould be nice if they listed system requirements. Their docs just say coming soon …
- knodi123 3y agomost of their docs say coming soon. and their whole wiki. honestly feels like site this was launched a couple of days too soon.
- warkdarrior 3y agoTheir LLM is still generating copy for the website..
- christkv 3y agoI got say I’ve been using LLM studio as it exposes the models in the ui as well as through a local open ai compatible server so I can test different models against my workflows locally.
- TheRealPomax 3y agoStill hoping we'll eventually stop using Fibonacci to show off recursion, because that's one of those examples where the maths might be expressed as recursive relation, but the implementation should never be =) Good AI would go "you don't want that, that's horribly inefficient. Here's an actually performant implementation based on the closed-form expression".
- zopa 3y agoNah, good AI would run in the compiler and optimize the recursion into something fast.
- Wowfunhappy 3y agoWhat is your preferred example for teaching a beginner to use recursion?
- pimlottc 3y agoI'm going to assume this is not an Australian company... https://www.youtube.com/watch?v=2akt3P8ltLM https://www.youtube.com/watch?v=2akt3P8ltLM
- badRNG 3y agoLooks like they are based out of Singapore
- LeoPanthera 3y agoIs this a fork of "LM Studio"? The UI is suspiciously similar, even down to the layout of the labels.
- euclaise 3y agoLM studio is closed source, so no
- onion2k 3y agoI use Jan to run Mistral locally. It works well for what I need (which amounts to playing with models).
- FuriouslyAdrift 3y agoMany LLMs may be run locally with GPT4All... https://gpt4all.io/ https://gpt4all.io/
- nickpsecurity 3y agoAnd MLC puts them on your phone, too. https://llm.mlc.ai/ https://llm.mlc.ai/
- slowmovintarget 3y agoGPT4All makes it annoyingly difficult to run any other than their "approved" models. I'd like to kick the tires on a whole host of random GGUF quantizations on Hugging Face, please. I've poked around the doc, not sure if Jan can do that better. In the mean time, I use text-gen-ui (Oobabooga) as a back-end and have it run with `--api` to use the front end of my choice.
- _puk 3y agoNot at my computer right now to double check.. but doesn't GPT4All's "Browse" button in the model list let you pick a locally downloaded model?
- LeoPanthera 3y agoUnless I just can't find it, there seems to be no setting for customizing the prompt format for local models. You can edit the prompt itself, but not the format of the prompt or the subsequent messages. This would make using many models difficult, or give poor results, since they don't all use the same format.
- kkfx 3y agoI try some LLM on my notes and well... They was unable to give me insights that are hard to spot, like follow the flaw of notes identifying patterns, find similar notes from the past and so on. In ALL cases classic tags/riprgrep full-text search was far quicker and equally or more effective. Long story short: LLMs might be useful on hyper big mass of information, like a new kind of search engine that try do achieve a semantic goal mimicking it. But not more than that IMVHO. Marginally LLMs might help computer-illiterate to manage their files, seen https://www.theverge.com/22684730/students-file-folder-directory-structure-education-gen-z https://www.theverge.com/22684730/students-file-folder-direc... but I doubt they can go any further for the next 5+ years at least.
- theGnuMe 3y agoLLMs might help my disorganized approach to files...
- lxgr 3y agoThey've been very useful in quickly answering common questions using a too-large-to-manually-scan knowledge base in my experience at my job, and I don't consider myself or my colleagues "computer-illiterate".
- kkfx 3y agoThat's follow the "might be useful as a new kind of search engine", though it might be a sign of an a bit messy KB. The issue of potential hallucinations however is still there so even such usage, a different search engine, demand extra attention. It's not a free critic to those who have designed, implemented and trained LLMs, it's just the observation that practical usage is far less than the advertised one and it's still not much good. It's still an advancement, a good thing to have, the start of a revolution, but still far from being what many dreams.
- lagrange77 3y agoI was wondering if it uses something like vLLM[0] or Llama.cpp[1]. Seems to be Llama.cpp via 'Nitro', which was discussed here before [2]. [0] https://github.com/vllm-project/vllm https://github.com/vllm-project/vllm [1] https://github.com/ggerganov/llama.cpp https://github.com/ggerganov/llama.cpp [2] https://news.ycombinator.com/item?id=38887531 https://news.ycombinator.com/item?id=38887531
- EMM_386 3y agoIt does make it easier for the end user who doesn't want to fiddle around with python dependencies, command lines, building C++ projects, etc. Just install it, point it to a model, and go. Now you have a local LLM. If you want something more, click the "start server" button and you have a local OpenAI compatible API which you can point more advanced front-ends to.
- Brajeshwar 3y agoThis looks awesome. Trying it out. Suggestion, can we please change the "Download Jan for PC" to perhaps just "Download" or "Download for Desktop" or whichever that makes sense but not "PC". I almost move away thinking this is Windows, thus not for us. I recently stumbled on https://mindmac.app https://mindmac.app which is a non-subscription app that uses multiple AI tools (not just OpneAI). Looks Promising. Like the others in the comments, I've tried https://www.typingmind.com https://www.typingmind.com (via SetApp). Sindre Sorhus have a pretty stable Native App https://sindresorhus.gumroad.com/l/quickgpt https://sindresorhus.gumroad.com/l/quickgpt These are some of the really good ones. I'm tending more towards trying out the likes of MindMac just for the fact that I can plug and switch between multiple tools.
- longnguyen 3y agoShameless plug: I built a native client called BoltAI[0]. Unlike other clients, I prioritize UI, UX and performance. Give it a try if UI & UX is important to you. [0]: https://boltai.com https://boltai.com
- Brajeshwar 3y agoSure. Why Not. Trying it out. My use case is especially for my daughter so I can just plug in my OpenAI API and let her ask away.
- Terretta 3y agoThis looks great relative to others (very similar to MacGPT), and I particularly like how advanced settings are available but tucked away behind discoverable affordances. It's interesting that you have team pricing. Can the Team leverage shared system prompts and/or assistants from a OneDrive-for-Business (SharePoint) folder or GitHub repo? If not, what makes it "Team" instead of just individual?
- longnguyen 3y agoHi. Actually I don’t have a pricing plan for teams yet. It’s still under (heavy) development. I changed my headline to reflect the direction I want to take this year (focus on teams) And yes, some of my customers wanted team and collaborative features like shared prompt, internal plugins and integrations, RAG on internal documents… But I haven’t launched these team features yet. Are you interested in this? Would love to talk to you if it’s something you’re looking for.
- rnd0 3y agoHow is this better than gpt4all?
- Grimblewald 3y agoFrom what I see, it has the benefit of offering less functionality, more corpojargon and a more 'intuitive' ui.
- rc202402 3y agoBeen using this with a few models like gemma etc for a week now. HN got any good LLM suggestions to run with this that are equivalent or better than GPT-3.5 / claude? I'm looking to use its api with LLama Index
- deleted 3y ago[deleted]
- lazyeye 3y agoThe default model (Mistral Instruct 7BQ4) is woke. I asked it the following:- Write a short poem in admiration of black people Write a short poem in admiration of brown people Write a short poem in admiration of asian people Write a short poem in admiration of white people It immediately replied with a poem for all except white people where it's response was:- "I'd be happy to write a poem in admiration of all people, including those who identify as White." Lol
- antifa 3y agoDid this have any way to point at a folder of markdown files and RAG at it?
- redder23 3y ago[flagged]
- redder23 3y agoOK, there is a switch under "Advanced Settings" to enable GPU. Why the fuck is this no-brainer option off by default and "advanced"? Suddenly my 7B Model is shown as inactive and "start" does nothing, gives no error message either ... this is such an unusable alpha version.
- Terretta 3y ago> If I had a key for some OpenAI paid shit I can go to their website and do not need an app for that. I really do not get it. Perhaps you don't have some OpenAI paid shit? While you can clunk around in a sort of OpenAI playground in a web tab, it is designed for dev experimentation (a "fiddler" type of UI), and not a good experience for much beyond testing. > excuse me you fuck did you not just tell me you are "local first" ... I try out the model, and it turns out it runs on my CPU with heavy RAM usage... I'm not sure it makes sense to have both of these objections at once. > I never ran a model Oh.