yalok

9 ms·

91.
by yalok 2y ago
So basically the idea is to pack 3 ternary weights (-1,0,1) into 5 bits instead of 6, but they compare the results with fp16 model which would use 48 bits for those 3 weights… And speed up comes from the memory io, compensated a bit by the
92.
by yalok 2y ago
how much memory do you need to run fp8 llama 3 70b - can it potentially fit 1 H100 GPU with 96GB RAM? In other words, if you wanted to run 8 separate 70b models on your cluster, each of which would fit into 1 GPU, how much larger your overa
93.
by yalok 2y ago
This quote summarizes the main secret sauce to me - once they generate a wrong token/phrase, the whole answer goes south - and it basically explains why the whole CoT approach works - prevent LLM from generating a wrong answer with 2 t
94.
by yalok 2y ago
It may be a local thing in Bay area, but usually there’s some way to get some discount when making a purchase with Apple - be it via education, or via a corporate discount (just show your badge from another company), or via a friend who wor
95.
by yalok 2y ago
that's great to hear! amazing performance of the model! for voice chat bots, however, shorter input utterances are a norm (anywhere from 1-10 sec), with lots of silence in between, so this limitation is a bit sad: > On the Gigaspeec
96.
by yalok 2y ago
I remember printing the whole K & R "The C programming language" book on my dot-matrix printer, with some roll-paper, and then drilling the pages through to staple it with some thick paper clips wires. Legendary devices.
97.
by yalok 2y ago
Removing a battery attached with previous type of adhesive is torture - the elastic tab frequently tears off, and I ended up a few times having to bend the old battery a lot, to get it out (very unsafe, it starts heating). So, to me, this i
98.
by yalok 2y ago
I have had suspicions for a while that the whole product is built & sold by just 1 person... if this is true, that's just amazing.
99.
by yalok 2y ago
One of the clearest examples of this to me is the requirement to show consent to store cookies. This just simply ruined web surfing experience, where now every site has to show a popup of all kinds of shapes and forms. Why not let the users
100.
by yalok 2y ago
I'm learning something new today. In some other languages these 2 are usually the same 1 word.
101.
by yalok 2y ago
is it just me, or 101 is 2-3x more dangerous than 280 per this map in Bay Area? definitely more density of pins on 101 (you have to click to the center in between these 2 highways to see pins on both). It makes sense, since 280 is a bit le
102.
by yalok 2y ago
When I try to journal during the actual live conversation, I feel like I'm not fully there in the moment, not as responsive, not as realtime... So far, only chat history and email history serve that purpose well. I wish all the journal
103.
Meta Hires Ruslan Salakhutdinov(threads.net)
1 points by yalok 2y ago|0 comments
104.
by yalok 2y ago
you may be double counting opus encoding/decoding delay - usually, you can run it with 20ms frame, and both encoder and decoder take less than 1ms of realtime for their operation - so it should be ~ 21ms, instead of 30+30ms for 1 direc
105.
by yalok 2y ago
I think these guys have a viable approach - https://xcimer.energy
106.
by yalok 2y ago
this codec is for RTC comms - it supports 20ms frame rate. They did mention it's launched in their calling products: "We have already fully launched MLow to all Instagram and Messenger calls and are actively rolling it out on What
107.
by yalok 2y ago
I would add one more - 4. Add an option to auto-backup all the data from the account to the outside backup service of users choice. This would help not just with these kind of accidents, but also any kind of data corruption/availabilit
108.
by yalok 2y ago
there needs to be some automated backup option to periodically dump your whole G.suite data to some non-Google service... I'd pay for it.
109.
by yalok 2y ago
It’d be very interesting to see the results for this metric for the existing audio and voice codecs (like AAC, AAC-LD, mp3, opus), and how it compares to the existing metrics for them? Couldn’t find it in their paper.
110.
by yalok 2y ago
I would really like to see a decaf option there.
111.
by yalok 2y ago
I'd add this to your list: - optimize the system to run in true realtime - most self checkouts I used have perceptible latency of a few seconds between one scan and the next one - usually lagging behind of how fast a person can scan al
112.
by yalok 2y ago
he said "Q4" - meaning 4-bit weights.
113.
by yalok 3y ago
I wonder if there’s some “critical mass” for ML compute on device, that may make this kind of devices significantly more useful & usable. E.g., running 70B or even larger model on device should allow to cut interactive response time, a
114.
by yalok 3y ago
exactly, that's been my concern as well - more and more parts now are very specialized and hard to reuse in other projects. Makes me sad for the kids - this limits creativity...
115.
by yalok 3y ago
There are research papers where even 1 bit (not floating point) was enough, with some quality loss. 4 bits is effectively 16 different float point numbers - 8 positive, 8 negative, no zero and no NaN/inf. 1 bit for sign and 3 bits for
116.
by yalok 3y ago
There’s no explicit documentation. It’s just that if a non-Chinese company/individual will want to publish something there, they need to obtain a license. This license can’t be obtained so easily, so you would have to go through some p
117.
by yalok 3y ago
If you try to publish your app in any of the Chinese app stores, you will be required to add an SDK to your app that basically allows them (Chinese government) to track user activity in your app. This basically means that not only TikTok, b
118.
by yalok 3y ago
because Billion is ambiguous - Quote: Billion is a word for a large number, and it has two distinct definitions: 1,000,000,000, i.e. one thousand million, or 10^9 (ten to the ninth power), as defined on the short scale. This is now the most
119.
by yalok 3y ago
start with something small - take some kernel function in C, and try to optimize it for your laptops assembly SIMD instruction set.
120.
by yalok 3y ago
I wonder if this applies to kidneys cystic fibrosis? I friend of mine passed away a few years ago from it, and it was painful to watch slow degradation in his health over the years.

More ›