andy_xor_andrew

11 ms·

61.
by andy_xor_andrew 2y ago
hoping an expert can answer a few Qs I have :) Is FlashAttention simply a drop-in replacement for the attention operation in an LLM? Can it be used anywhere that an "attention" operation is used? Or does a LLM need to be trained s
62.
by andy_xor_andrew 2y ago
Given the restrictions on GPUs to China, I'm curious what their training cluster looks like. (not saying this out of any support or non-support for such a GPU blockade; I'm just genuinely curious)
63.
by andy_xor_andrew 2y ago
When reading Hacker News you develop a signal/noise filter, where lots of headlines make bold claims but you filter them out as embellishment or exaggeration. My bullshit detector went off when I first saw Groq posted on HN - a startup
64.
by andy_xor_andrew 2y ago
in the screenshot you shared in the twitter link [2], the model appears to do everything correctly - it terminated its message with <|end|> which is correct according to the Phi-3 prompt format. It seems whatever environment you are h
65.
by andy_xor_andrew 2y ago
am i literally shaking with anger, pacing around my room, crying, cuddling my emotional support dog, because i'm so upset at this advertisement? no. is it a terrible, possibly even insulting ad? yeah, I think so. pretty clearly the ori
66.
by andy_xor_andrew 2y ago
> On macOS, for example, Zed makes direct use of Metal. We have our own shaders, our own renderer, and we put a lot of effort into understanding macOS APIs to get to 120FPS. So they are taking the exact opposite approach of Electron (VS
67.
by andy_xor_andrew 2y ago
The iPad Pros, though? I'm very curious how much iPad Pros sell. Out of all the products in Apple's lineup, the iPad Pro confuses me the most. You can tell what a PM inside Apple thinks the iPad Pro is for, based on the presentati
68.
by andy_xor_andrew 2y ago
One of the biggest struggles I have (and others have, judging from Stack Overflow) is how to generically handle accepting types of Box<T>, Rc<T>, T, Pin<T>, &T, &mut T, etc etc. Of course you can write a function t
69.
by andy_xor_andrew 2y ago
great, what are the odds they reverse the reversal next year in a hypothetical new administration?
70.
by andy_xor_andrew 3y ago
I'm confused about the fizzbuzz example they provide. ```js let array = new Array(100); let evil = { [Symbol.toPrimitive]() { array.length = 1; return 15; } }; array.push(evil); // At index 100, the @@toPrimitive callback of
71.
by andy_xor_andrew 3y ago
> But Transformers have one core problem. In a transformer, every token can look back at every previous token when making predictions. Lately I've been wondering... is this a problem, or a strength? It might be a fallacy to compare
72.
by andy_xor_andrew 3y ago
Interesting, and it kinda makes sense. You quantize, which invariably means you lose some precision, but then you can finetune post-quantization to recover at least some of it. Neat idea.
73.
by andy_xor_andrew 3y ago
Given his experience with AlphaGo/Star/Zero/etc, I'm sure if there's any way to apply tree search, or policy networks, to LLMs, Demis will find it. I've always found it strange that a LLM is basically an action
74.
by andy_xor_andrew 3y ago
I have some questions for you wizard shellcode-flinging hackers out there. with modern mitigations like ASLR and beyond, how are exploits like this even possible? I took a course in college which provided us with a dozen binaries, to be run
75.
by andy_xor_andrew 3y ago
> If you don’t want your device to be included in that tally of nearby contacts for others, you can toggle it off. Just remember that leaving it on won’t tell your friends—or Apple—anything about you personally. That sentence seems like
76.
by andy_xor_andrew 3y ago
on this topic, it's interesting how my mindset changes when I'm writing Rust vs Python, with regards to vtables, dispatching, and allocation. Writing Rust for a toy project: "I MUST avoid allocation, and the dyn keyword, at a
77.
by andy_xor_andrew 3y ago
This gave me lots of confidence in Unsloth when I first read it. I'll admit I was a little skeptical of Unsloth, since anything that boasts free perf improvement, just by dropping in some middleware, makes me suspicious. Especially f
78.
by andy_xor_andrew 3y ago
> Quite the cultural flame war in this thread. It's kind of the perfect storm, because both sides of the argument include a mixture of reasonable well-intentioned people, and crazy extremists. However you choose to be upset about th
79.
by andy_xor_andrew 3y ago
> That 7 second video consumed just 1,841 tokens out of my 1,048,576 token limit. is this simply an approximation done by Gemini in order to add some artificial limit on the amount of video? Or do video frames actually equate directly to
80.
by andy_xor_andrew 3y ago
> The training token count is tripled (6T vs. Llama2's 2T) Damn, 6T? That's a lot! Given that this model seems to roughly match Mistral (according to the numbers from Google), this makes me think we have saturated the 7B parame
81.
by andy_xor_andrew 3y ago
I'm way out of my depth in terms of the math here. But my 'software engineer brain' likes the ideal of using the prime factoring problem, because it's so simple to understand, and feels like some kind of universal primit
82.
by andy_xor_andrew 3y ago
On this topic (using local LLM for analyzing local text on an iDevice) - I highly suspect that the recent Journal app from Apple, which auto-installed via an iOS update, is intended to incentivize users to journal and write about their dail
83.
by andy_xor_andrew 3y ago
I found out recently there's a place on the web, right now, that has all the charm and personality of the "homepages" of the 90s and early 00s. A place where you can make your own little corner of the web, share it with frien
84.
by andy_xor_andrew 3y ago
are your accelerator chips designed in-house? or they're some specialized silicon or FPGPU or something that you wrote very optimized code for inference? it's really amazing! the first time I tried the demo, I had to try a few pro
85.
by andy_xor_andrew 3y ago
Every day - every single day - my mom gets in her car, and drives 2.5 miles (each way) to the nearest McDonald's, to buy a diet coke. That's the only item she gets. She has done this for over a decade. This used to drive me ins
86.
by andy_xor_andrew 3y ago
I just had a really stupid thought, after finishing reading the article. So, the electron is an elementary particle, right? Compared to the proton, the electron is "simple", yes? Despite this difference in complexity, an electron
87.
by andy_xor_andrew 3y ago
> but there are some things one can do with a mega context that you just can't get with RAG. Can you elaborate? In my mind, RAG and "mega context" are orthogonal - RAG is something done by adding documents to the context f
88.
by andy_xor_andrew 3y ago
This smells true to me, though I couldn't exactly explain why. (baseless speculation below) One thing I've noticed is how drastically my handwriting (and maybe hand dexterity in general) varies wildly from hour to hour, mostly dep
89.
by andy_xor_andrew 3y ago
so they branded this "Chat with RTX", using the RTX branding. Which, originally, meant "ray tracing". And the full title of your 2080 Ti is the "RTX 2080 Ti". So, reviewing this... - they are associating AI wit
90.
by andy_xor_andrew 3y ago
Living in Seattle, every election the ballet was filled with items like these, which sound great and have good-faith intentions that are totally in line with my own personal convictions. But I guess policy needs to be studied--really studie

More ›