rolisz

23 ms·

301.
by rolisz 3y ago
https://rolisz.ro I've blogged for more than 10 years, from technical stuff that I've learned, to places I've traveled, to random musings I've had.
302.
by rolisz 3y ago
I've had very good results with even a 3.5 B model (Fastchat-T5) for retrieval augmented generation (aka putting information into the context window and letting the model rephrase it).
303.
by rolisz 3y ago
To debug stuff.
304.
by rolisz 3y ago
Where can you vote without being a citizen? I think across all of Europe you have to be a citizen to be able to vote.
305.
by rolisz 3y ago
I think that detour through the ocean and the clouds can make quite a difference in the outcome...
306.
by rolisz 3y ago
Flan-T5 is a 3B model that is of comparable quality to Llama 13B. Moreover, you can fine-tune model for your specific tasks and you need fewer resources to fine tune a smaller model.
307.
by rolisz 3y ago
And Flan-T5 models are awesome! Fastchat is a finetuned 3b parameter Flan-T5 model that can perform really well, comparably to llama 13b
308.
by rolisz 3y ago
Flan-T5 models perform really well, considering their size. In my experiments and in some recent papers, they are very close to 13b Llama for example: https://twitter.com/YiTayML/status/1668302949276356609
309.
by rolisz 3y ago
To quote someone: RASP is like Matlab, designed by Satan. There is an interpreter for a RASP like language if you want to try it out: https://srush.github.io/raspy/ And deepmind published a compiler from RASP to Transf
310.
by rolisz 3y ago
The EU doesn't pass laws, they pass some sort of recommendations that individual countries must implement as laws in a certain for another.
311.
by rolisz 3y ago
If they don't know how to call an API, they won't know how to run local models (at the moment it's quite a pain to set-up all the dependencies)
312.
by rolisz 3y ago
I'm also a huge fan of Kagi. I've been a paying user since they launched the paying subscription. Really happy with it!
313.
by rolisz 3y ago
BERT is not a generative model (you might be able to force it to do that, but that's not what it's meant for). It's good for text classification tasks, for named entity recognition and other tasks like this
314.
by rolisz 3y ago
BERT is 500x smaller than GPT3, it doesn't hallucinate (it might make wrong predictions, but it won't make unexpectedly wrong predictions), it can run on a CPU in 500ms. I really like BERT style models (there are a couple newer mo
315.
by rolisz 3y ago
Think Instagram or Twitter tags.
316.
by rolisz 3y ago
Contextual embeddings are not a strict improvement over static embeddings: if you don't have context, they tend to be worse. For example, if you have tags for example, which usually are just single words, word2vec style embeddings are
317.
by rolisz 3y ago
But then aren't pesticides and herbicides a problem for a plant based diet too?
318.
by rolisz 3y ago
How much of that is because of inflation though? 1 million in 1995 corresponds to 2 million in 2023 dollars.
319.
by rolisz 3y ago
I'm pretty sure money existed before fractional reserve banking....
320.
by rolisz 3y ago
What about "organic" milk? By that I mean milk that is grown by my small local farm, humane conditions for the cows, no antibiotics or supplements for them.
321.
by rolisz 3y ago
Was your vitamin D deficiency and allergy related to long COVID?
322.
by rolisz 3y ago
Even PrivateGPT does that, using Chroma as vector DB
323.
by rolisz 3y ago
Well you wouldn't input the whole Vault to the model, you would use embeddings to find the content that is most relevant to the question being asked.
324.
by rolisz 3y ago
For some reason, downloading the model they suggest keeps failing. I tried to download it in Firefox and Edge. I'm using Windows, if that matters. Anyone else seeing similar issues?
325.
by rolisz 3y ago
One of my use cases is to memorize function names that I use about once a month - not often enough to memorize them from usage only, but frequently enough that the gains from not having to Google their name every time are worth it.
326.
by rolisz 3y ago
Did you quit cold turkey? Or did you slowly taper it off? I'm taking Rybelsus (pill form of semaglutide) and there are 3, 7 and 14 mg versions. My plan is to take 14 mg for a couple more months, then switch to 7 mg for 1-2 months and t
327.
by rolisz 3y ago
Isn't that the case for every diet? You can lose weight on diets and then gain it back when you start eating the way you used to.
328.
by rolisz 3y ago
Yes, a 100 times this! Thank you for putting this so nicely into words! Too bad there's no HN gold or whatever for me to award you!
329.
by rolisz 3y ago
It's beaten by only two weights available models, it beats LLama 13B and Dolly. What do you mean it's training in a quasi-Transformer manner? Do you mean the language pretraining? That's been done before Transformers. RWKV do
330.
Context-Free Grammar Parsing with LLMs(matt-rickard.com)
5 points by rolisz 3y ago|0 comments

More ›