jmorgan

25 ms·

121.
by jmorgan 3y ago
Definitely the latter!
122.
by jmorgan 3y ago
It adds a symlink for the "ollama" cli (you can see the source here: https://github.com/jmorganca/ollama/blob/main/app/src/instal... ) /usr/local/bin/ollama -&g
123.
by jmorgan 3y ago
To add: the easiest way to get up and running is to download the Mac app: https://ollama.ai/download Then running a model (e.g. Llama 2) can be done with ollama run llama2 For Macs with less memory (<= 8GB) you&#x
124.
by jmorgan 3y ago
Under the hood it does a few things I'll shed some light on (at a high level!): 1. Running the model: it's built on the open-source (and amazing) llama.cpp project for running quantized (i.e. compressed) models like Llama 2 (launc
125.
by jmorgan 3y ago
This is super helpful! Currently it runs 4-bit quantization models (with names often ending in q4_0). Sorry this isn't more obvious. Will fix that and soon update the hosted models to support different quantization levels, so you can c
126.
by jmorgan 3y ago
While we haven't built Linux binaries yet (waiting to make sure we have _fast_ GPU support before publishing a Linux and Windows binaries) it does run on Linux. For example it's used to create and publish the "library" i
127.
by jmorgan 3y ago
For disk size, you can see how big models are locally by running "ollama list" NAME SIZE MODIFIED llama2:latest 3.8 GB 3 hours ago Now, if you make new models that are "FROM llama2" for example,
128.
by jmorgan 3y ago
You can, although this is definitely "hard mode" when writing Modelfiles. Easier is to override SYSTEM which allows will customize the system prompt. SYSTEM "You are an expert in writing python." If you do want modify th
129.
Show HN: Ollama – Run LLMs on your Mac(github.com)
284 points by jmorgan 3y ago|94 comments
130.
by jmorgan 3y ago
That's a great idea! Will do. Generally, a 7B model requires at least a Mac with 16GB of shared memory. In practice I see it using about 6GB of RAM when loaded.
131.
by jmorgan 3y ago
If you want to try running Llama 2 locally, you can use https://github.com/jmorganca/ollama To run Llama 2 with it: ollama run llama2
132.
TaskMatrix(github.com)
2 points by jmorgan 3y ago|0 comments
133.
The Path to Self-Sovereign Identity(lifewithalacrity.com)
2 points by jmorgan 3y ago|0 comments
134.
The Linux Kernel Key Retention Service and why you should use it(blog.cloudflare.com)
20 points by jmorgan 4y ago|1 comments
135.
The Future design information(klim.co.nz)
1 points by jmorgan 4y ago|0 comments
136.
by jmorgan 4y ago
So many teams have told us similar stories: that access is a "tier 1" service (i.e. it can't go down!). Most tools involving a single point of failure end up trading off reliability for security, whereas reliability is of equ
137.
by jmorgan 4y ago
Thanks Nick! Brings back to our days together @ Docker :-)
138.
by jmorgan 4y ago
Great question! We looked heavily into Dex before creating Infra, and even spoke with their maintainers. Dex is a federated OIDC provider. Most managed Kubernetes services (e.g. Azure AKS) don't support using custom OIDC providers for
139.
by jmorgan 4y ago
Great question! Most managed Kubernetes services don't support OIDC (and for EKS, which does support custom OIDC providers, it requires restarting the entire control plane to edit the configuration). Infra runs a lightweight process in
140.
by jmorgan 4y ago
Tokens are verified by intercepting API server requests in-cluster against a central root of trust. This is similar to how OpenID tokens from identity providers such as Okta or Active Directory are verified by destination web applications.
141.
by jmorgan 4y ago
Hey there! No changes to the api server configuration are required. We've designed Infra around this since AKS/EKS/GKE don't expose the ability to edit api server parameters to users.
142.
by jmorgan 4y ago
Thanks! Curious, where did HTTP+JSON break down for you? Was it specifically around audit/event streaming? This would be helpful as we consider building out future updates to Infra, especially considering tools like Kubernetes have put
143.
by jmorgan 4y ago
Great point on GRPC having better support for event streaming! We originally built Infra to have a GRPC API, but many users we spoke to didn't yet have load balancers or ingress controllers that supported the GRPC protocol (e.g. one us
144.
by jmorgan 4y ago
Infra's designed to integrate with existing identity & access tooling (vs replace them). While we don't have a built-in integration for FreeIPA, Infra has a REST API and will integrate with any OpenID-compliant identity provid
145.
by jmorgan 4y ago
Thanks for the questions! Vault doesn't have a deep integration to generate credentials for Kubernetes, and Infra plugs in to users' tooling (e.g. kubectl and Kubeconfig) to keep credentials up to date automatically. Infra's
146.
Launch HN: Infra (YC W21) – Open-source access management for Kubernetes(github.com)
159 points by jmorgan 4y ago|58 comments
147.
Halide Mark II(halide.cam)
140 points by jmorgan 6y ago|96 comments
148.
Filecoin Mainnet Is Live(filecoin.io)
42 points by jmorgan 6y ago|22 comments
149.
U-2 Federal Lab achieves flight with Kubernetes(hill.af.mil)
3 points by jmorgan 6y ago|0 comments
150.
Headless UI(headlessui.dev)
1 points by jmorgan 6y ago|0 comments

More ›