3Sophons

9 ms·

91.
by 3Sophons 3y ago
A demo. Full and true portability achieved with wasm runtime
92.
Efficient and Portable AI / LLM Inference on the Edge Cloud(2024.wasmio.tech)
1 points by 3Sophons 3y ago|0 comments
93.
Create an LLM web service with LlamaEdge on MacBook, deploy it on Nvidia devices [video](youtube.com)
1 points by 3Sophons 3y ago|1 comments
94.
LlamaEdge 0.4.0 Released: Multimodal LLM support, RAG assistant API and more(github.com)
2 points by 3Sophons 3y ago|0 comments
95.
by 3Sophons 3y ago
A fully portable Rust+Wasm app
96.
Demo: Run LLava multimodal LLM with a 2M Rust app portable across GPU/CPU(twitter.com)
2 points by 3Sophons 3y ago|0 comments
97.
Run Alicloud's Qwen LLM with a 2MB cross GPU/CPU rust app(secondstate.io)
1 points by 3Sophons 3y ago|1 comments
98.
Self-Host Across GPU/CPUs Google's Gemma LLMs with a 2MB App(secondstate.io)
3 points by 3Sophons 3y ago|1 comments
99.
by 3Sophons 3y ago
Still need room for the models themselves
100.
Free Linux Foundation Training/ Certification for New Open Source Contributors(secondstate.io)
1 points by 3Sophons 3y ago|0 comments
101.
Complete tutorial: Run Qwen LLM across your local machines without Python/C++(secondstate.io)
3 points by 3Sophons 3y ago|0 comments
102.
Launched! WebAssembly System Interface WASI Preview 2 Discussion(youtube.com)
1 points by 3Sophons 3y ago|1 comments
103.
by 3Sophons 3y ago
WASI Preview 2 is a milestone for the Wasm community. It includes wasi-io, wasi-clocks, wasi-random, wasi-filesystem, wasi-sockets, wasi-cli, and wasi-http APIs.
104.
by 3Sophons 3y ago
LlamaEdge is not just a lightweight and portable runtime for chatbots. It is an app dev platform for portable LLM apps! RAG chatbots and AI agents can be developed on Mac or Windows, compiled to Wasm, and then deployed across Nvidia /
105.
Develop+Deploy RAG Bots with LlamaEdge: Across OSes, NPUs, GPUs Using Vector DB(github.com)
1 points by 3Sophons 3y ago|1 comments
106.
Self-host any GUFF LLMs on Hugging Face and run across devices(secondstate.io)
1 points by 3Sophons 3y ago|0 comments
107.
by 3Sophons 3y ago
LlamaEdge is not just a tool to run chatbots. It is a dev platform for LLM apps with custom models and actions. It is lightweight and portable - you can create an LLM app on a Mac, compile to Wasm, and then run the binary app on Nvidia devi
108.
Run the leaked Mistral Medium, miqu-1-70B across GPUs CPUs and OSes(secondstate.io)
3 points by 3Sophons 3y ago|3 comments
109.
Self-host StableLM-2-Zephyr-1.6B. Portable across GPUs CPUs OSes(secondstate.io)
2 points by 3Sophons 3y ago|1 comments
110.
by 3Sophons 3y ago
“Small” LLMs are the ones that have 1-2B parameters (instead of 7-200B). They are still trained with trillions of words. The idea is to push the envelope on “information compression” to develop models that can be much faster and much smalle
111.
by 3Sophons 3y ago
Full portability across CPUs GPUs and operating systems
112.
LFX Mentorship 2024 Spring LLM Projects: Build Open Source AI Inference Infra(secondstate.io)
1 points by 3Sophons 3y ago|1 comments
113.
by 3Sophons 3y ago
Get paid working on open source LLM infra
114.
LlamaEdge 0.2.9 released. Now works with Hugging Face's 3000 GGUF models(twitter.com)
1 points by 3Sophons 3y ago|1 comments
115.
by 3Sophons 3y ago
Excited to share my journey in creating a Telegram bot that learns Japanese and answers Q&As, all thanks to https://flows.network/ built in Rust
116.
Rust boosts LLM app development: Make a serverless Japanese Learning bot in mins(flows.network)
1 points by 3Sophons 3y ago|1 comments
117.
by 3Sophons 3y ago
The Rust+Wasm stack provides a strong alternative to Python in AI inference. * Lightweight. Total runtime size is 30MB as opposed 4GB for Python and 350MB for Ollama. * Fast. Full native speed on GPUs. * Portable. Single cross-platform bi
118.
by 3Sophons 3y ago
Please let us know if you have other concerns or bump into any issues? There is a different LLM based article for each model, like the Mixtral MoE https://www.secondstate.io/articles/mixtral-8-7b/
119.
by 3Sophons 3y ago
detailed steps: https://www.secondstate.io/articles/wasm-runtime-agi/ or run with a single command: https://www.secondstate.io/run-llm/
120.
by 3Sophons 3y ago
Please look at this separated steps as requested! https://www.secondstate.io/articles/wasm-runtime-agi/

More ›