abdullin

9 ms·

61.
by abdullin 3y ago
The main offer of Tenstorrent goes into server racks and is designed to form clusters. Standalone cards are more like dev kits. (I’ve been tracking Tenstorrent for 3+ years and currently have Grayskull in ML test rig together with 3090)
62.
by abdullin 3y ago
On my LLM benchmarks Claude 3 Opus beats only one flavor of GPT-4: GPT-4 Turbo v3/1106-preview All the other flavors are still better, with top winners: GPT-4 v1/0314 and GPT-4 Turbo v4/0125-preview. The benchmark is based on
63.
by abdullin 3y ago
I prefer to use local models when running data extraction or processing over 10k or more records. Hosted services would be slow and brittle at this point. Mistral 7B fine-tunes (OpenChat is my favorite) just chug through the data and get th
64.
by abdullin 3y ago
Thanks! Custom wake-word on a chip is a bit of a pain. So we are running two models. One on the chip and the second, more powerful, on the server. It filters out false positives.
65.
by abdullin 3y ago
I'm not aware of any detailed write-ups. Mostly gathered information bit by bit. On a high level here is how it is working for us: 0. When voice assistant device (ESP32) starts, it establishes web-socket connection to the server. 1. E
66.
by abdullin 3y ago
We are using inmp441. They work well with ESP IDF libraries shipped by Expressif.
67.
by abdullin 3y ago
Generally I'd suggest to pick the route the excites you the most. Because you'll naturally spend more time and energy there, allowing you to advance the most. Personally I've always been fascinated with AI/ML, because of
68.
by abdullin 3y ago
Great write-up! It is a pleasure to see more people explore this area. You can make it even more lean and frugal, if you want. Here is how we built a voice assistant box for Bashkir language. It is currently deployed at ~10 kindergartens&#x
69.
by abdullin 3y ago
Training, especially on large GPU clusters, is inherently non-deterministic. Even, if all seeds are fixed. This boils down to framework implementations, timing issues and extra cost of trying to ensure determinism (without guarantees).
70.
by abdullin 3y ago
Yes, that is the perfect spot! If you already have a coding background and want to focus on practical applications, there is surprisingly little to catch up with LLM/ML/ChatGPT. Here is my quickstart on LLM/ChatGPT for develo
71.
by abdullin 3y ago
I’ve been working in software since ~2004. Mostly working with startups as tech lead/team lead or full-stack. Half of these companies were about big data/data science. But applications of AI/ML always attracted me more :)
72.
by abdullin 3y ago
I work as head of ML & Innovation at my day job. On the side, as a technical consultant, I help teams to ship products with ML under the hood. I live in Vienna, so that makes it convenient for evening calls with USA/Canada, where m
73.
by abdullin 3y ago
LangChain is good for the demos and learning, but it is too complex and brittle for my taste. Using a bit of boilerplate code (a couple of python files) that I copy to new projects.
74.
by abdullin 3y ago
If I absolutely need to avoid hallucinations (e.g. when building marketing/sales assistants for the businesses), then I allow LLM to control and drive search for the relevant documents. On a high level: (1) give LLM ability and enough
75.
by abdullin 3y ago
One example - a local model (Mistral 7B OpenChat 3.5, which ranks high in my benchmarks) was generating additional search keywords for products at online marketplace. A second run of the model was cleaning up bad keywords. The fun part here
76.
by abdullin 3y ago
It doesn't need to be an exclusive choice. Hosted and local models can complement each other. On one of the projects I used Chat GPT-4 to write instructions/tutorials that were then executed by local model on a large chunk of data
77.
by abdullin 3y ago
Embeddings can be tricky. They are just an average semantic vector over a chunk of text. There is a high chance that a plain similarity search (dot product or cosine distance) will bring a lot of noise and junk into the request. And high no
78.
by abdullin 3y ago
Putting too much information in the context window is counter-productive in my experience. Low signal/noise ratio tends to increate the likelihood of model hallucinations, and we don't want that! What works in my experience - stru
79.
by abdullin 3y ago
I've been building workflow assistants that make existing employees more productive or enable entirely new business models. Some of these assistants use selected local models (due to cost or privacy factors) Currently the stack is grav
80.
by abdullin 3y ago
With a local SQLite database you don’t need to pay cost of network transfers. A dozen of local SQL calls could be way faster than a single API call within a region.
81.
by abdullin 3y ago
There also is a productivity boost within this approach. When you have a local DB as an integration point, you could just upload it to ChatGPT Code Interpreter for building extra reports and running analysis on it. Alternatively, upload the
82.
by abdullin 3y ago
On our benchmarks, Claude v1 beats GPT-3.5 (v0613) while v2 looses to it.
83.
by abdullin 3y ago
On our benchmarks, Claude v2 scores worse than v1 in categories “code”, “docs”, “integrate” and “marketing”. It also is more chatty than v1 (or GPT-3/4), even when asked to just pick one option out of three. These benchmarks are produc
84.
by abdullin 3y ago
Here is a test to assess quality of these assistants. (1) upload the bitcoin white-paper. (2) ask question “What is the contribution of R.C. Merkle to this reasearch?” The proper answer should mention “Merkle Trees”.
85.
by abdullin 3y ago
Not any time soon, I believe. AI hardware is useless without software ecosystem (AMD and Intel could tell a story about that). Latest marketing materials of Tenstorrent tell stories about great chips and licensing deals, but not a single w
86.
New ChatGPT models and ML product cases(abdullin.substack.com)
1 points by abdullin 3y ago|0 comments
87.
by abdullin 3y ago
Great question! 1. I've observed multiple products across customers. 1.1 Correcting or filling missing information in structured data. For example a system to suggest corrections to products in a company catalogue (each product categor
88.
by abdullin 4y ago
I can’t recommend Borg Backup enough (OSS). It does deduplication at the chunk level. This handles both duplicate files and large binaries that change slowly over time.
89.
by abdullin 5y ago
You need over 200 karma to create a poll. OP doesn't have that much, yet.
90.
by abdullin 5y ago
(lived in Russia until 2021, not authorative source of info) All ISP providers are required to have black boxes installed on-site. These boxes can do traffic shaping (e.g. Twitter is degraded) and basic filtering based on IP addresses. I th

More ›