Tostino

41 ms·

841.
by Tostino 3y ago
Another way to do this is using the pgml extension. You can run huggingface embedding models, which have surpassed OpenAI's at this point. It's pretty fast if you run it on a machine with a gpu for acceleration. I've created
842.
by Tostino 3y ago
Generally good as long as you target all layers.
843.
by Tostino 3y ago
It's a play on the prior context extension method, RoPE for those who aren't getting the name.
844.
by Tostino 3y ago
Actually being able to get to the recipe for one. Just generally being competent at navigating the site is the main thing, she uses it to find ideas for crafting often.
845.
by Tostino 3y ago
Anything worse than alcohol?
846.
by Tostino 3y ago
My wife somehow knows how to navigate that site and can extract knowledge from it that I would only dream was available. I'm much closer to your side of the aisle. Burn it with fire.
847.
by Tostino 3y ago
A.k.a pretty much the entire state of FL.
848.
by Tostino 3y ago
Better reasoning and general performance than 13b by far (if llama1 was any indication), and like the other user said, can fit on a single 24gb vram gaming card, and can be peft fine-tuned with 2x 24gb cards.
849.
by Tostino 3y ago
1x 3090 IMO is about the minimum you'd want to waste time with. It can serve a 13b + 7b model at once if you want, you can qlora train a 13b with a ton of context length, and it's fast enough to iterate with for training. I have 2
850.
by Tostino 3y ago
I opted for lora (qlora), but I targeted all layers with it.
851.
by Tostino 3y ago
So, as somebody who has fine tuned llama2 (13b) on both a new prompt template / chat format, as well as instruction following, summarization, knowledge graph creation, traversing a knowledge graph for information, describing relationsh
852.
by Tostino 3y ago
Yeah... But for every new token you generate, you need to take that into account, along with all prior generated tokens and input provided by user for generating the next one.
853.
by Tostino 3y ago
You absolutely want to chunk them smaller than 8k. Have you tested different chunk strategies? It can make a huge difference for actually recalling useful information in small enough chunks to be usable.
854.
by Tostino 3y ago
I don't believe this is correct from my direct experience running local models. They generate slower when you start to fill up the context window compared to when it first starts responding.
855.
by Tostino 3y ago
Yes, quite successfully. And qlora, where things are further quantized while just sitting in VRAM.
856.
by Tostino 3y ago
If the consumer is prohibited from circumventing those limits on their own property, do they really own it?
857.
by Tostino 3y ago
There is an actual difference. You can make acutal, real-world, impacts with today's LLMs in the existing business world. They can actually make a difference for customers. That was never the case with a crypto currency.
858.
by Tostino 3y ago
"thanks Facebook" was not a term I have ever actually wanted to utter.
859.
by Tostino 3y ago
I've literally not seen any real (more than buying another card) improvement with NVlink on anything i've seen online over the past ~year. I'd honestly be a bit suspect.
860.
by Tostino 3y ago
That is pretty depressing. It's funny how easy it would be to not give yourself away too. It's ridiculously easy to spot some of the general ChatGPT tells and simply not recite them.
861.
by Tostino 3y ago
There are people in this thread bringing up that bullshit. I really dislike a lot of the recent discourse on HN.
862.
by Tostino 3y ago
I was sad to not see them move their tooling to multi-database. I had a couple calls with them ~5-6 years ago to talk about our needs and how they would be solved by a Postgres version of their tools (like SQL Compare), but apparently it ne
863.
by Tostino 3y ago
The term is used colloquially so often that I didn't even associate it with military service in this context in the least.
864.
by Tostino 3y ago
There was also a concerted effort from major tech employers to depress wages to control costs. They don't need to directly talk to each other to plan it, they state it in their earnings calls.
865.
by Tostino 3y ago
You don't know their situation or work history, why reply with something so negative?
866.
by Tostino 3y ago
I should be clear though, I didn't fine-tune on my specific tasks myself (as that is the end result of what I am doing with the pipelines outputs). I just tried the LLaMA2-chat model (which is fine-tuned), and the Freewilly 2 finetune.
867.
by Tostino 3y ago
No, I just used YAML to represent it. Here is an example: https://gist.github.com/Tostino/f6f19e88e39176452c1a765cb7c2... Here is the transcript that I created that knowledge graph from, and then annotated with the kno
868.
by Tostino 3y ago
I have not read that yet, thank you! Also, I haven't tried using Whisper for getting a transcription from the audio. I went the route of downloading the automatically generated transcripts from Youtube for a set of videos. An audio pro
869.
by Tostino 3y ago
That is with fine-tuning: https://stability.ai/blog/freewilly-large-instruction-fine-t...
870.
by Tostino 3y ago
Youtube transcripts. It only works with single-person channels at the moment, as I haven't worked on disambiguating multiple speakers. They are very messy if they are just an auto transcription from Google. Practically unusable in most

More ›