p1esk

9 ms·

181.
by p1esk 9mo ago
perhaps because you are interested in optimizations or distillation or something Yes, my job is model compression: quantization, pruning, factorization, ops fusion/approximation/caching, in the context of hw/sw codesign. In
182.
by p1esk 9mo ago
No. Each projection is ~5% of total FLOPs/params. Not enough model capacity change to care. From what I remember, removing one of them was worse than other two, I think it was Q. But in all three cases, degradation (in both loss and pe
183.
by p1esk 9mo ago
1. The two papers you linked are about importance of attention weights, not QKV projections. This is orthogonal to our discussion. 2. I don't see how the transformations done in one attention block can be reversed in the next block (or
184.
by p1esk 9mo ago
You should compare the number of top AI scientists each company has. I think those numbers are comparable (I’m guessing each has a couple of dozen). Also how attractive each company is to the best young researchers.
185.
by p1esk 9mo ago
I glanced at these links and it seems that all these attention variants still use QKV projections. Do you see any issues with my interpretation of them?
186.
by p1esk 9mo ago
The way I think about QKV projections: Q defines sensitivity of token i features when computing similarity of this token to all other tokens. K defines visibility of token j features when it’s selected by all other tokens. V defines what fe
187.
by p1esk 10mo ago
But that's the problem - if you don't like doing anything, what will you do? What will you fill your life with? You will quickly get bored of anything you try. Your life will have no meaning, and you will probably turn to alcohol
188.
by p1esk 10mo ago
Financial freedom is about not having to worry about losing your job, or tolerating shitty work conditions. Why would you retire if you do what you love? I think the real problem might be if there's nothing you actually love doing (lon
189.
by p1esk 10mo ago
Almost nobody else in engineering did this. What you described is the job of a product manager. Are there no PMs at Google?
190.
by p1esk 10mo ago
Strange question. If you don’t know why you need this, you probably don’t. It will be the same as with the introductory AI course you did 20 years ago.
191.
by p1esk 10mo ago
I love to sit alone in a cafe - reading. Before smartphones I was reading newspapers or books. Now I read on my phone or tablet. While there, I don’t want to talk to anyone, I just want to sit and read quietly.
192.
by p1esk 10mo ago
What do you mean?
193.
by p1esk 10mo ago
It would need an educated populace How do you measure that?
194.
by p1esk 10mo ago
If someone is interested in low level tensor implementation details they could benefit from a course/book “let’s build numpy in C”. No need to complicate DL library design discussion with that stuff.
195.
by p1esk 10mo ago
Wanting power usually has little to do with making the world better.
196.
by p1esk 10mo ago
Sure, I get it - trying to understand a specific condition affecting someone close to you. I personally have very little trust in doctors. But, outside of this need, what actionable science have you learned and applied to your own life?
197.
by p1esk 10mo ago
My comment is also directed towards OP: you don’t need small talk to make friends.
198.
by p1esk 10mo ago
OK, I just read the abstract and conclusion of the NAC paper posted above. But then I saw a comment from Aurornis saying it’s not that good. Not sure who I should listen to.
199.
by p1esk 10mo ago
Would regular engineers like us understand molecular biology papers?
200.
by p1esk 10mo ago
Is Estonia a good country to immigrate to for an American?
201.
by p1esk 10mo ago
If something is not clear in the book, can I ask it to explain it to me?
202.
by p1esk 10mo ago
This doesn’t make sense to me. Why would you want something like this? What is it exactly that you expect from such a finetuned model that you cannot get from a frontier general purpose model?
203.
by p1esk 10mo ago
I hate small talk. Never needed it to make friends.
204.
by p1esk 10mo ago
Why limit to 2 agents? I typically use all 3.
205.
by p1esk 10mo ago
Please elaborate
206.
by p1esk 10mo ago
No China or Russia? Strange…
207.
by p1esk 10mo ago
they spending $20 billion dollars to handicap an inference company Inference hardware company
208.
by p1esk 10mo ago
The model might get loaded on every token - from GPU memory to GPU. This depends on how much of it is cached on GPU. Inputs to every layer must be loaded as well. Also, if your model doesn’t fit in GPU memory but fits in CPU memory, and you
209.
by p1esk 10mo ago
Depends on map_location arg in torch.load: might be loaded straight to GPU memory
210.
by p1esk 10mo ago
We’ve had “compute in flash” for a few years now: https://mythic.ai/product/

More ›