Tostino

41 ms·

781.
by Tostino 3y ago
That may be true, and for some tasks the accuracy may be high enough. I have gotten much more consistency in my tasks by fine tuning though. Getting a consistently good result for one shape of input may not indicate that same performance fo
782.
by Tostino 3y ago
I'm not competing with OpenAI in any sense of the word.
783.
by Tostino 3y ago
So I described my approach to how I fine tune a specific task below to another user, but I'll copy it here: > Design your tasks to be repeatable and small steps, call the OpenAI API and log all requests/responses. > Filter o
784.
by Tostino 3y ago
It's entirely possible without OpenAI doing anything else. Design your tasks to be repeatable and small steps, call the OpenAI API and log all requests/responses. Filter out any bad responses and take a representative sample of th
785.
by Tostino 3y ago
The grid can be negotiated with if we put in the infrastructure.
786.
by Tostino 3y ago
When plugged into the grid, it makes sense to spend a few cents of energy a day to filter out unwanted solicitations, harassment that you may not want to see, scam emails or texts, etc. If I didn't have to worry about my grandparents g
787.
by Tostino 3y ago
You do realize how possible it is to fine tune a task like this (along with a hundred others in a similar vein) on a tiny model you can scale on your own hardware? I've run hundreds of millions (150m so far in a couple of weeks of non-
788.
by Tostino 3y ago
Wow that seems confusing. I know I would have been scratching my head at what a green left arrow with a circle around it meant different that a regular green left arrow. Would not have thought that it was equivalent of flashing yellow.
789.
by Tostino 3y ago
It depends on how you look at things. If all you care about is monetary gain, than burning through VC money and cashing out as much as you can along the way is likely to be personally more enriching. But if you care about advancing a techno
790.
by Tostino 3y ago
Uh, not sure how I missed that last line in your original post about this being a more real-time requirement. No, that for sure complicates things if you don't have a "base load" you can use your dedicated compute for when yo
791.
by Tostino 3y ago
You don't know how to run the infra for this then. For batch workloads, I can get slightly higher throughput with a 13b llama2 fine tune on my local workstation and 2x 3090s than I can get by saturating my OpenAI api key token limits f
792.
by Tostino 3y ago
I just assumed it was a joke about how they only started caring about quality in 2021.
793.
by Tostino 3y ago
The obvious answer would be for apple to choose the best search engine for the users needs
794.
by Tostino 3y ago
I clicked into the article because I was dumbfounded that the cereal was making that much a year before my brain kicked in.
795.
by Tostino 3y ago
Yes, it's a massive difference.
796.
by Tostino 3y ago
Then why did we feel the need to clear the land of them? cough genocide cough
797.
by Tostino 3y ago
Arguing we should spend more on housing the homeless, fine. But spending more resources on vanity projects like "the line"??? You lost me.
798.
by Tostino 3y ago
Is this a joke? Isn't their whole thing pretty much lacking rigor in everything they do? Why would you want benchmarks from them?
799.
by Tostino 3y ago
You don't know people who live near the margins then. Cost was pretty much the primary decider for me prior to making a decent living, so much so that I put off rotator cuff surgery for 10 years because I couldn't afford to have i
800.
by Tostino 3y ago
By having a brittle system held up by "norms" that were simply ignored.
801.
by Tostino 3y ago
I don't think you are reading that the same way most others here would.
802.
by Tostino 3y ago
Very little seems to cross their minds. This thread is full of some really "hot takes I've absolutely never heard before". I'm disappointed as hell with the level of discourse on this site.
803.
by Tostino 3y ago
Llama.cpp has faster single thread inference, but I actually haven't tried their batched inference yet, so I don't know what throughput I can get through it.
804.
by Tostino 3y ago
Nothing special, other than I am running my fork of vLLM which supports HF prompt templates: https://github.com/vllm-project/vllm/pull/1365 Using --tensor-parallel-size 2 since I have 2 GPUs on my machine. Ru
805.
by Tostino 3y ago
There are some pretty interesting cooling solutions in development. Will be interesting to see which ones become economical enough for mass market use.
806.
by Tostino 3y ago
The market failed to innovate on safety for years. And the issue with a market based solution is it can be much cheaper to just not include any of those safety features. Some things can be below your personal risk tolerance, but be well abo
807.
by Tostino 3y ago
While stopping journalists from using any footage of war regardless of context.
808.
by Tostino 3y ago
And that is where MoE comes in with some more advancements in routing
809.
by Tostino 3y ago
Temperature being set differently is one culprit. There are a few hyper parameters that can be tweaked to get some pretty different output.
810.
by Tostino 3y ago
I know we're chatting on Discord, but figured i'd leave the link to what I was working on a couple of months ago here if anyone else is interested: https://gist.github.com/Tostino/3f0b0887591ed06aa9f54ca2ddbd.

More ›