jmorgan

9 ms·

91.
by jmorgan 3y ago
If you haven't yet checked out the Generative Agents project referenced by OP, definitely give it a look, it's open source: https://github.com/joonspk-research/generative_agents Over the weekend Lance Martin
92.
by jmorgan 3y ago
RE 2 - neat! What are some tasks you've been using smaller models (with perhaps larger context sizes) for?
93.
by jmorgan 3y ago
It's early, and this definitely isn't customer facing in the traditional sense, but a team member of mine set up a Discord bot running Llama 2 70B on a Mac studio and we've been quite impressed by its responses to folks who t
94.
by jmorgan 3y ago
This appears to be a web frontend with authentication for Azure's OpenAI API, which is a great choice if you can't use Chat GPT or its API at work. If you're looking to try the "open" models like Llama 2 (or it&#x27
95.
by jmorgan 3y ago
There is! While not easy to use yet, there's a sort-of-hidden way models can be listed with: curl https://ollama.ai/v2/_catalog | jq Then to list "tags" for a given model (e.g. llama2): curl ht
96.
by jmorgan 3y ago
I really enjoyed Anrej Kaparthy's llama2.c project ( https://github.com/karpathy/llama2.c ), which runs through creating and running a miniature Llama2 architecture model from scratch.
97.
by jmorgan 3y ago
An easy way to try many of the fine-tuned Llama 2 models is https://github.com/jmorganca/ollama . A maintainer of the project has been collecting a full list here (with different quantization levels), most of which are
98.
by jmorgan 3y ago
I do think this is an apt analogy. I've heard a counterpoint that there won't be enough "destinations" for this to work, but then it's not hard to imagine a single order of magnitude more LLM "destinations&quot
99.
by jmorgan 3y ago
Great. Let's chat!
100.
by jmorgan 3y ago
The idea of an LLM proxy is super compelling. There's a lot of powerful ideas baked into the proxy form factor – I think you've listed out quite a few of them. It reminds me a bit of what Cloudflare did for the web: both making it
101.
PrivateGPT with Llama 2 Uncensored(github.com)
6 points by jmorgan 3y ago|0 comments
102.
by jmorgan 3y ago
LangChain supports local LLMs like Llama 2 with Ollama ( https://github.com/jmorganca/ollama ) as of this morning, in both their Python and Javascript versions: https://python.langchain.com/docs/inte
103.
by jmorgan 3y ago
Indeed, many tools in this space don't maximize resource utilization at runtime. Even the quantized models are massive resource hogs.. so you need all the performance you can get! Ollama on macOS will use both the GPU and the Accelerat
104.
by jmorgan 3y ago
If you're looking to run Llama 2 locally via a CLI or REST API (vs the web ui this article highlights), there's an open-source project some folks and I have been working on over the last few weeks: https://github.com&#x
105.
by jmorgan 3y ago
Original HN discussion for Eric Hartford's post on Uncensored Models: https://news.ycombinator.com/item?id=35946060 . The article gives a great background on uncensored models, why they should exist and how to train&#x2
106.
Run Llama 2 uncensored locally(ollama.ai)
722 points by jmorgan 3y ago|212 comments
107.
by jmorgan 3y ago
How much memory do you have in your Macbook? The 7B models seem to work well with at least 16GB of unified memory, but I’ve seen Macs with 8GB really struggle.
108.
by jmorgan 3y ago
This is a super cool project. Congrats! If you’re looking at trying different models with one API check out an open-source project a few folks and I have been working on in July in case it’s helpful https://github.com/jmorga
109.
by jmorgan 3y ago
Looking into this! Since quite a few of the recommended prompts don't always work https://github.com/jmorganca/ollama/issues/217
110.
by jmorgan 3y ago
% ollama run llama2-uncensored >>> Generate a json object describing the most stunning part of the movie: "The Matrix" { "title": "The Matrix", "director": "The Wacho
111.
by jmorgan 3y ago
In case you might be trying this with Ollama (given the model:tag syntax) – I haven't added this to the readme yet (seeing some issues with the prompt template) but check out llama2-uncensored: ollama run llama2-uncensored
112.
by jmorgan 3y ago
> adjust your paths as necessary. It has a tendency to talk to itself. This is always a fun surprise. What I've seen help, especially with chat models, is to use a prompt template. Some tools (e.g. https://ollama.ai/
113.
by jmorgan 3y ago
Good suggestion! This is definitely on the radar, so that running `ollama` will start the server when it's needed (instead of erroring!): https://github.com/jmorganca/ollama/issues/47
114.
by jmorgan 3y ago
That's a great observation.
115.
by jmorgan 3y ago
Absolutely. Ideally it's one consistent interface/api/experience that interleaves text and images, sort of like https://github.com/microsoft/TaskMatrix , although there must be some more recent work done
116.
by jmorgan 3y ago
Not yet, but this totally makes sense – will look into it!
117.
by jmorgan 3y ago
It is! You can reach it from http://localhost:11434 . More documentation to come, but to generate text there's an /api/generate endpoint: https://github.com/jmorganca/ollama#rest-api
118.
by jmorgan 3y ago
Thanks for mentioning this!
119.
by jmorgan 3y ago
One fun use case of local models is creating "characters" or companions (à la character.ai and other tools). It's early but for those interested, this project has a way to do with the concept of a "Modelfile". Here&
120.
by jmorgan 3y ago
The llama.cpp project is absolutely amazing. Our goal was to build with/extend the project (vs try to be an alternative). Ollama was originally inspired by the "server" example: https://github.com/ggerganov&#x

More ›