Tostino

62 ms·

871.
by Tostino 3y ago
That seems more in-line with my experience. I have been using GPT-3.5 and GPT-4 for data cleaning pipelines, and have tried to swap out LLaMA2 70B in a few of the "easier" tasks, and it hasn't performed well enough yet for an
872.
by Tostino 3y ago
Especially market niches where there is a lot of government money be spent. A bit ironic considering his more recent rhetoric.
873.
by Tostino 3y ago
Right, but there is a big difference between ”generally knows what JSON looks like and gets it right most of the time" and "generates perfect JSON every time".
874.
by Tostino 3y ago
This is important for "smaller" models, because you don't have to waste some of the potential "intelligence" (parameter space) on training it how to generate valid JSON or YAML or anything like that.
875.
by Tostino 3y ago
It's not though... it has been dead for over a decade. It was about price for silicon density. Prices have gone up significantly for the newer nodes without the density increase necessary to offset.
876.
by Tostino 3y ago
That sounds like an okay first level support, but in no way is it a replacement for a human support team.
877.
by Tostino 3y ago
There is pretty much a free flow of US guns going into Mexico.
878.
by Tostino 3y ago
Bing has been so thoroughly labotamized compared to launch it's not even worth using in my opinion. ChatGPT4 looks like a Rhoads scholar in comparison, even with its own nerfs.
879.
by Tostino 3y ago
Exllama is significantly faster if you can fit the whole model in VRAM.
880.
by Tostino 3y ago
At least Totoro was the latter movie in the pair so you didn't go home feeling terrible.
881.
by Tostino 3y ago
It sounds like you don't know how to use it effectively is all I can see from your post.
882.
by Tostino 3y ago
Steamdeck is running PC games though...
883.
by Tostino 3y ago
Oh man, something written in a language I actually know and use?! Will check it out.
884.
by Tostino 3y ago
I've tried to switch out prompts between LLM's and I've had to change every bit of code provided by laying chain with a different implementation between them. It is an entirely useless abstraction. The prompt template is not
885.
by Tostino 3y ago
When you actually get down to it, they're pointless and counterproductive. Templates are pretty useless and have inconsistently implemented features which make them non-compatible with different LLM backends without changing all your c
886.
by Tostino 3y ago
Yeah I've got a few thousand lines of langchain code now for a data cleaning pipeline... I've been fighting it every step of the way. Trying to replace sections of the pipeline to use a local LLM instead of OpenAI has had me have
887.
by Tostino 3y ago
If weights are even protected by copyright at all...which would be a departure from current law.
888.
by Tostino 3y ago
I'm sorry, I can't hear you over this Taylor Swift song I've got blasting...
889.
by Tostino 3y ago
I have zero money on the line for either. Please defend PPP loan forgiveness vs student loan forgiveness. I'd love to hear your take.
890.
by Tostino 3y ago
I haven't actually used it yet, it can be very efficient to compute right next to where your data is though. A lot of inefficiency comes into play when you're marshalling data around between servers. And you want to grab that fish
891.
by Tostino 3y ago
What are you talking about? 7b parameter models run insanely fast if you can offload to gpu, and are entirely reasonable speed if CPU only.
892.
by Tostino 3y ago
All things that others are doing that has between zero and very little impact on you. No one is forcing you into any of it. Why not attempt to keep your moral outrage to yourself and just live your life?
893.
by Tostino 3y ago
Looks to be an OpenAI problem due to how they host the API. People are speculating that different servers in their pool are ot always on the exact same version of the model, and you can get pointed to a different version...no proof I saw,
894.
by Tostino 3y ago
That isn't what I was replying to though. Factuality is different than determinism. They are not trained to retain facts. They are trained to predict the next word.
895.
by Tostino 3y ago
Possibly, but the perplexity has shown to decrease while fine-tuning a 2048 model on larger context sizes for outputs within it's original context limit...so, more research needed.
896.
by Tostino 3y ago
You can have an LLM be entirely deterministic. This is by design how they work. Having the randomness added to their response is a choice.
897.
by Tostino 3y ago
Just that it was a smaller number than would you are expected to sell your soul for with VC cash. Only raising what you need to grow "naturally" and retaining control is a good thing in my book, but that isn't what a lot of p
898.
by Tostino 3y ago
Yes, and then I iterate on it until it is clear and simple enough for weaker models to actually follow consistently.
899.
by Tostino 3y ago
https://arxiv.org/abs/2306.15595
900.
by Tostino 3y ago
You must have missed the memo... It's now super easy to extend the context of 2k llama models to 8k, 16k, or even 32k with just a small fine tune and a tweak to the code. You still need the memory to be able to go that high, but it&#x2

More ›