mchiang

8 ms·

1. 91.▲
   Llama 2 based WizardMath model to solve math problems with examples(ollama.ai)

   7 points by mchiang 3y ago|0 comments
2. 92.▲
   by mchiang 3y ago

   Thanks for sharing this. How do you think the local LLM movement will evolve? Especially as in the post, you mentioned startups and VCs both hoarding GPUs to attract talent. There seems to be a good demand behind tools like llama.cpp or oll
3. 93.▲
   FastViT: A Fast Hybrid Vision Transformer Using Structural Reparameterization(github.com)

   2 points by mchiang 3y ago|0 comments
4. 94.▲
   Llama 2 uncensored vs. GPT3.5 benchmarking(promptfoo.dev)

   3 points by mchiang 3y ago|0 comments
5. 95.▲
   by mchiang 3y ago

   They used to have this: https://github.com/onnx/onnx-coreml
6. 96.▲
   by mchiang 3y ago

   This is usually because the Ollama server isn't running. To solve it either: - Start the Ollama app (which will run the Ollama server) - Open terminal: `ollama serve` to start the server. We'll fix this in the upcoming release
7. 97.▲
   by mchiang 3y ago

   Eric’s blog is a great read on how to create the uncensored models - link to the original blog here: https://erichartford.com/uncensored-models
8. 98.▲
   by mchiang 3y ago

   This is from us manually testing it on macbooks that we have available. It might run, but it's probably using swap.
9. 99.▲
   by mchiang 3y ago

   Local memory management will definitely get better in the future. For now: You should have at least 8 GB of RAM to run the 3B models, 16 GB to run the 7B models, and 32 GB to run the 13B models. My personal recommendation is to get as much
10. 100.▲
    by mchiang 3y ago

    Yeah! How much memory do you have? If by lower-end Macbook air, you mean with 8GB of memory, try the smaller models (Such as Orca Mini 3B). You can do this via LM Studio, Oogabooga/text-generation-webui, KoboldCPP, GPT4all, ctransforme
11. 101.▲
    by mchiang 3y ago

    Hi, By 'add documents', can I assume you are asking about embeddings? Ollama doesn't yet support embeddings. We are looking into how we can support this in the future.
12. 102.▲
    by mchiang 3y ago

    I used the 13B model locally on Ollama, and it gave the same answer. You can run the 13B model by using the 13b tag like: ollama run llama2:13b >>> tell me a joke about emacs Here's a joke about Emacs: Why did the Emacs user b
13. 103.▲
    by mchiang 3y ago

    Try running the orca model (default is 3B), and it requires much less memory. ``` ollama run orca ```
14. 104.▲
    by mchiang 3y ago

    This is most likely an out of memory problem without seeing the logs. We have a fix in the works that will be released soon. May I ask what mac & memory you're running this on?
15. 105.▲
    by mchiang 3y ago

    On the mac, we have enabled Metal support.
16. 106.▲
    by mchiang 3y ago

    Linux support is coming, you can build it right now by running: `CGO_ENABLED=1 go build . `
17. 107.▲
    by mchiang 3y ago

    By default the `llama2` model is the 7B model, and it's recommended you have at least 16GB of memory to run it. Regarding the disk space, the model itself is 3.8GB.
18. 108.▲
    by mchiang 4y ago

    regarding boundary, it's a great project but many times it requires too much to set up / manage. For dynamic credentials, you can leverage Vault. For discovery of your infrastructure, you can use Consul.
19. 109.▲
    by mchiang 4y ago

    Thanks for the note — would love to hear what infrastructure outside of Kubernetes you’d like to connect to, so I can target the answer more specifically. For expanding to infrastructure outside of Kubernetes, we will ultimately generating
20. 110.▲
    by mchiang 4y ago

    Yes, if you use Infra's local users, you can sign-in headless.
21. 111.▲
    by mchiang 4y ago

    Thanks for checking out Infra! Infra App is Jeff and I's passion project when we started. We still patch it for security / bugs. That being said, we've definitely been thinking about how we should maintain / let the rest
22. 112.▲
    by mchiang 4y ago

    thank you for this feedback. We've made several edits to the quickstart to get users started as quickly as possible as proof-of-concept installs either on a test cluster in the cloud or a local cluster. For the a longer setup: https:
23. 113.▲
    by mchiang 4y ago

    hey, thank you for the comments. We plan to build a managed service to provide a 'centralized experience'. This is where, we'd issue certificates/tokens for the users & machines. That being said, many of our users wa
24. 114.▲
    by mchiang 4y ago

    Dex doesn't support many managed Kubernetes services. It's because its OIDC support is not configurable. (ie. https://github.com/dexidp/dex/issues/1268 ) It is in EKS now, but you'd have to rest
25. 115.▲
    by mchiang 4y ago

    hey, I'm one of the co-founders of Infra. Under the hood, we support OIDC, and should be able to support custom IdPs. If you have specific requirements, definitely let me know.
26. 116.▲
    by mchiang 5y ago

    why use blockchain key pairs under the hood? what’s the actual use case for that?
27. 117.▲
    by mchiang 7y ago

    This looks like a repost. Comments are being made in: https://news.ycombinator.com/item?id=22485625
28. 118.▲
    by mchiang 7y ago

    I haven’t done cost calculations yet, but this might actually make me want to explore to EKS pricing. I do love the experience of GKE.
29. 119.▲
    by mchiang 7y ago

    Hey everyone, I’m one of the co-founders for Infra.app. I’m super excited to launch this in early access. Previous to this my co-founder and I were building Docker Desktop and Kitematic.
30. 120.▲
    by mchiang 10y ago

    Added. Please check your email.