Tostino

35 ms·

811.
by Tostino 3y ago
Heh, I've been working on...a good portion of the basics that this project / paper have tested out for the past few months as an idea (as I work more on other more material problems for my side project). I have a whole document of
812.
by Tostino 3y ago
He is also featured somewhat centrally in the latest Folding Ideas video on this whole situation: https://youtu.be/5pYeoZaoWrA
813.
by Tostino 3y ago
I cannot wait for the day where I don't have to actively drive my RV cross country on a trip.
814.
by Tostino 3y ago
I'd rather the knowledge be out there, it helps you not go down dead end paths that you would otherwise have explored.
815.
by Tostino 3y ago
So, llama.cpp already somewhat supports this: https://github.com/ggerganov/llama.cpp/issues/3440
816.
by Tostino 3y ago
They aren't fastest. I get over 1k tokens/sec on llama2 13b on my 2x 3090 with vLLM. They both need to fix their batch inference and it'll be more competitive. They are though.
817.
by Tostino 3y ago
Yup, and you can fine-tune an LLM to be a specialized classifier with relatively little data compared to building a classifier from scratch. I think that's the biggest benefit to LLMs, the data required to show it a behavior is pretty
818.
by Tostino 3y ago
Partially true...if you are looking for general purpose models. Instead of general purpose, I rather like more specialized models. I have some transcription correction needs, and I had a set of prompts that could get GPT4 to do it...but boy
819.
by Tostino 3y ago
The fact that people are still treating it like entirely raw text input is insane to me. If you have a document, have a separate input for the user to paste/upload data, and then another for the user's instruction. That allows you
820.
by Tostino 3y ago
If you want to do that, I have a model trained specifically on a dataset of building recursive summaries. Some of my training documents are 40-50k tokens. https://huggingface.co/Tostino/Inkbot-13B-8k-0.2 Just chunk you
821.
by Tostino 3y ago
Think of how differently the 20th century would have played out if we didn't have such a demand for oil.
822.
by Tostino 3y ago
I mean I was partially joking about the origin of the dataset, but they likely could work with DoD at this point to get a model that is acceptable to put in place, after as many leaks as they have had.
823.
by Tostino 3y ago
Train a ML model as a classified document classifier and scan everything through that... now they just need to dig through their archives for the training set from all the past leaks.
824.
by Tostino 3y ago
Your job will change in fundamental ways at least.
825.
by Tostino 3y ago
A ton on here like to pretend to take a moderate viewpoint when espousing strictly conservative views or talking points.
826.
by Tostino 3y ago
Especially when never bringing up what some of those more important things could be.
827.
by Tostino 3y ago
Why would they want that many available, or installed? Those are two different questions.
828.
by Tostino 3y ago
Maybe you should try looking at them as people?
829.
by Tostino 3y ago
Litigation is one thing, totally erasing it from the public internet if it were hosted centrally is something else.
830.
by Tostino 3y ago
The category is "high sugar". Not the shape or grain used in the cereal.
831.
by Tostino 3y ago
People seem to underestimate the scale you can get to on a single machine, and overestimate how easy it will be to go up from there. An in memory index is about as good as it gets for a single node performance, and fitting that many vectors
832.
by Tostino 3y ago
Re-read their post. It was a secondary system setup to just run these analytics which were loaded from the production system. No issues if the whole machine had to be rebuilt.
833.
by Tostino 3y ago
You also have fine-tuned models for specific tasks that may see very similar inputs for a variety of outputs. Think an LLM trained on pulling out specific types of information, no matter where it was stored within the file. E.g. "find
834.
by Tostino 3y ago
You can't FT 70b with 48gb of vram (2x 3090) either.
835.
by Tostino 3y ago
No. Great argument, right?
836.
by Tostino 3y ago
Well I've been thoroughly convinced by this argument.
837.
by Tostino 3y ago
I am looking for a model similar to this, but for text. I want to group text with different labels that apply to subsets of the text. Think of being able to quickly pull-out related segments from a large body of text. Let's take, for i
838.
by Tostino 3y ago
Not that I've tested it with this model, but I have for Llama2 models...it makes a minimal difference. I haven't found anything that was noticeable with ~30-70b models, and from what I can tell from the scaling charts, it matters
839.
by Tostino 3y ago
Thanks for the laugh. I really wish this type of thinking weren't so common in the population, but the fact is, it is pretty common. You need to account for it.
840.
by Tostino 3y ago
Pair it with a solution like zxcvbn and you can stop your users from their worst impulses as far as passwords.

More ›