rfoo

32 ms·

481.
by rfoo 3y ago
I'm not talking about the so-called quadratic memory requirement of the attention step, there NEVER WAS ONE. I'm talking about a simple fact - to efficiently (cost-wise) run LLM inference you have to have a KV "cache" an
482.
by rfoo 3y ago
idk For longer context to brute force it the problem is more on the memory side instead of the compute. Both bandwidth and capacity. We have more than enough compute for N^2 actually. The initial processing is dense, but is still largely bo
483.
by rfoo 3y ago
My bet is it's just brute force. I don't understand how they did 10M though, this isn't in the brute-force-with-nice-optimizations-on-systems-side-may-do-it ballpark, but they aren't going to release this to the public a
484.
by rfoo 3y ago
The problem is these processors are unstable if they are not properly throttled. Of course users, especially enthusiast motherboard consumers, hate throttling, hence the default.
485.
by rfoo 3y ago
For the keyboard I use a CH340+CH9329 cable, the former is a USB to UART converter and the latter is a UART to USB-HID converter. Feels unnecessarily complicated but hey at least it's only $3. It's effectively a USB "keyboard
486.
by rfoo 3y ago
> just remove the RFID radios and cameras from the cranes Because they require these to work. > or just not allow them to call home? I can't say for sure, but I believe they already don't call home: this is also why building
487.
by rfoo 3y ago
Why would I lose all data on these SSDs when I initiate a power off of the VM on console, then? I believe local SSDs are definitely attached to the host. They are just not exposed via NVMe ZNS hence the performance hit.
488.
by rfoo 3y ago
> it's all the libraries that are the problem And also header files which are also part of macOS SDK, as the project claims to support C codes.
489.
by rfoo 3y ago
GP's point is you can cache the state after the model processed the super long context but before it ingests your prompt. If you are going to ask "then why don't OpenAI do it now", the answer is it takes a lot of sto
490.
by rfoo 3y ago
But file(2) is already like that - my data files without headers are reported randomly as disk images, compressed archives or even executables for never-heard-of machines.
491.
by rfoo 3y ago
While I agree with what you said in general, Snowflake is a poor example. Data warehouses like Snowflake really can use the Tbps+ aggregated bandwidth between S3 and EC2 in the same region. There is no way for this to work over the Internet
492.
by rfoo 3y ago
It doesn't take that many hours, you are over-estimating by 2.5 order of magnitude
493.
by rfoo 3y ago
Building your code (I fixed a few typos, strings -> symbols, fprintf(" -> fprintf(stderr, ") with gcc -s -Os -fuse-ld=lld a.c && ls -al a.out leads to a 5496 bytes ELF though. Which is not much larger than j
494.
by rfoo 3y ago
$ gcc -Os a.c && ls -al a.out && size -rwxr-xr-x 1 user user 15952 Jan 24 17:49 a.out text data bss dec hex filename 1316 584 8 1908 774 a.out $ gcc -s -Os a.c &am
495.
by rfoo 3y ago
opt-level = "z" may be slightly slower than O2/O3. lto = true and lto-units = 1 makes the final linking unbearably slow for large programs. These are still okay, but then `panic = "abort"` shakes a lot of code off b
496.
by rfoo 3y ago
Ah, ok, I just tried and I'm wrong. That's correct, it provides full access.
497.
by rfoo 3y ago
This passcode is inherently limited to the service it bound to (IMAP or POP3), that's the whole point: don't expose your account password to something which only needs a finer-grained access. Edit: that's incorrect, see repli
498.
by rfoo 3y ago
Submitted title is misleading. Turns out, it's not "all but OAuth", it's "all but OAuth and app password". If you can't switch to OAuth, you can simply create an app password and continue using that as you
499.
by rfoo 3y ago
Sigh, I should have quoted both side, here is it: https://aws.amazon.com/blogs/aws/aws-free-tier-data-transfer... Cloudflare rant was posted in July 2021, new Cloudfront free tier was there Nov 2021. I consider ch
500.
by rfoo 3y ago
Unlikely. The "loophole" GCP patched was that you can use GCS to transfer data between regions on the same continent for free. This is already non-free on AWS. What OP mentioned is that transferring data between availability zones
501.
by rfoo 3y ago
Here's another one: You can download 1TB of data for free from AWS each month, as Cloudfront has a free tier [1] with 1TB monthly egress included. Point it to S3 or whatever HTTP server you want and voila. [1] It used to be 50GB per mo
502.
by rfoo 3y ago
> So overall it seems quite easy to force the matter and implement KYC and AML for msot legal Bitcoin transactions. At that point people would seriously have more interest in the so-called "L1 scaling solutions" which mining po
503.
by rfoo 3y ago
Because the reason why Python is one of the world's most popular language (a large set of scientific computing C extensions) is bound to every implementation details of the interpreter itself.
504.
by rfoo 3y ago
> how would you make it different? Make the cost of reusing software non-zero again. It doesn't have to be as painful as C++ without package managers, but should make every developer spend about 5~10 minutes labor work for adding ea
505.
by rfoo 3y ago
Hey, but at least we have rounded corners now, which is nowhere to be seen in your 2010s <table>-heavy borgmaster summary /s
506.
by rfoo 3y ago
... and I bet the top 1 engineer didn't actually add any bug to the code base, they just did a few repo-wide refactor.
507.
by rfoo 3y ago
The problem is a "free", proprietary service which is not explicitly opt-ed in. This is just Internet Explorer again, disguised.
508.
by rfoo 3y ago
> I never thought Ruby would have a shipping and production ready JIT before Python. This is entirely predictable - Ruby does not have a big scientific computing community which happened to depend on every implementation detail of the ho
509.
by rfoo 3y ago
Good one. - Why is Bard failing? - It's a chat app from Google.
510.
by rfoo 3y ago
> but I can't think of a single one for Apple Silicon. The post here is exactly one for Apple Silicon. It compared a naive implementation in PyTorch which may not even keep 4090 busy (for smaller/not-that-compute-intensive mode

More ›