leobg

51 ms·

871.
by leobg 4y ago
hnswlib? Local. Single file. Allows pre-filtering.
872.
by leobg 4y ago
Question is, would an incompetent, overworked person take the time and humility to actually ask the AI? You did, because you cared. With incompetent or overworked people, the problem is often that they just don’t care, or have no capacity t
873.
by leobg 4y ago
What we now need is an AI that actually reads all that AI generated stuff. It’s like we solved one problem (creation), and created another (consumption).
874.
by leobg 4y ago
Thanks for the clarification. Does that mean then that when parallelization is not important, training an adapter might be just as good as or better than LoRA?
875.
by leobg 4y ago
Pirate the hell out of their stuff, I guess. After all, can’t borrow anymore, can we? :)
876.
LLaMA-Adapter: Fine-tuning LLaMA within 1 Hour and 1.2M Parameters(github.com)
2 points by leobg 4y ago|0 comments
877.
by leobg 4y ago
I’m wondering what the difference is between LoRA and adapters. Especially I’m wondering whether LoRA is being used for LLaMA because it is proven to be better or simply because it happened to be the first such technique that worked and now
878.
by leobg 4y ago
That’s probably what OpenAI does with their custom fine tuned models, no?
879.
by leobg 4y ago
What is the difference to training an adapter? Or to adding a new task specific layer [0]? Has it been demonstrated that LoRA works best out of all of these approaches? [0] https://towardsdatascience.com/adding-custom-layers
880.
by leobg 4y ago
Silly. Yes, there will be human language interfaces to existing APIs. But: The APIs themselves will still be in their own “non natural” language. Precisely because such a technical language defines boundaries. Even the author, in his exampl
881.
by leobg 4y ago
Exactly. After all, this is the point of AI. That you don’t write the code/rules/weights yourself.
882.
by leobg 4y ago
Why? Dolly had nothing to do with Llama or its weights. Besides: How would anyone ever know which model generated the output you are serving? AFAIK there is no fingerprint in any model’s output. And even if there was, it would probably be d
883.
by leobg 4y ago
Hilarious. These super tough dudes with their guns and military style badass moves. And now they sue for emotional distress.
884.
by leobg 4y ago
SimpleT5 Python library might be if help. Allows you to train a model basically on just a dataframe. Can probably run it on Google Colab for free, given that you just want to classify a few hundred megabytes of text data. Also, there are ze
885.
by leobg 4y ago
Isn’t it interesting how that “cloze deletion” technique described here in 1997 is what they use today to train powerful large language models? (Obviously it’s been described even earlier, for instance back in the 1960s by BF Skinner, for “
886.
by leobg 4y ago
I was wondering. They do give you an embeddings endpoint. Can’t that theoretically be used to reconstruct the model’s weights?
887.
by leobg 4y ago
Sounds a bit like Chomsky’s critique of Skinners “Verbal Behavior”, which kind of put him on the map back in the 60s. Though Skinner gave us teaching machines, ping pong playing pigeons, clicker training and, indirectly, ChatGPT.
888.
by leobg 4y ago
Ha. Thanks for clarifying. I had read your comment the other way around and thought you were trying to be rude. I guess if someone knows emacs then maybe they’ll just try those key bindings and discover that they work. I never worked with e
889.
by leobg 4y ago
I’m not embarrassed at all. Else I wouldn’t have dared to ask. And would never learn anything new.
890.
by leobg 4y ago
Wrong. I’m using it for over 15 years now! And I’ve never ever even just heard about this. And keyboard shortcuts are the first thing I look for in any app. Hence my surprise
891.
by leobg 4y ago
Ok. Interesting. Where on macOS can I use these for navigating inside text? If I’m inside Textmate, for instance, typing F just… well, types an F. And it would seem to me that that’s the case anywhere on macOS where there is a cursor, no?
892.
by leobg 4y ago
Which shortcuts? You have a link to a list? I only know Cmd-left/right to skip words. But anything more fancy?
893.
by leobg 4y ago
WTF. You should have your own TV show! I mean, if TV shows where still a thing. Cool trick!
894.
by leobg 4y ago
Ha! I used it in law school. Actually wrote my “cards” down on a palm pilot during classes using the funny stylus it had. Professor later told me he had wondered what I was doing with this odd device. Until it dawned on him that I was takin
895.
by leobg 4y ago
There is a GitHub repo / Python lib called com2fun which exploits this. Allows you to get results from functions that you only pretend exist. (Am on mobile and can’t link to it right now.)
896.
by leobg 4y ago
I guess the idea is to trick the model into generating pseudo code. Which really doesn’t do much more than to act as a “scratchpad“ to focus the attention of the model to reason through the problem. Besides, the Codex models are free right
897.
by leobg 4y ago
This actually has nothing to do with fine-tuning in the technical sense. You are actually using vector search and injecting the results of that into the prompt for GPT. It is a good approach, but to use the word “fine-tuning“ for that is co
898.
by leobg 4y ago
I don’t even have enough time to read about why I don’t have enough time
899.
by leobg 4y ago
I see you paid attention. :)
900.
by leobg 4y ago
Don’t even need human time. Just run it through davinci. Is this correct, yes or no? Or MTurk. Can cost less than $500.

More ›