3 ms·

Can you explain how you used 12 billion tokens to do useful work?

eru 2mo ago
(Not the original commenter.) You can rack up quite a lot of tokens if you ask it to try out a lot of things, eg for performance investigations and trying out optimisation ideas.
- fooblaster 2mo ago
  is there a standard pattern for this? Like spawn an agent for each technique to try?
  - lightbulbish 2mo ago
    maybe this? https://github.com/karpathy/autoresearch https://github.com/karpathy/autoresearch
  - eru 2mo ago
    Not sure. I usually tell the agent to spawn subagents at will. (And they are doing that on their own anyway.)
  - laichzeit0 2mo ago
    I regularly do a “go to DynaTrace, look at how this service gets used in production then use a profiler and see if there’s any low hanging optimisations we could make” on stuff. LLMs are really good at doing everything that was fun about software development.
    - eru 2mo ago
      Ha, I have a whole hobby work-stream going on about finding low-hanging fruit in various open source projects to turn into valuable contributions. Two premier sources: (1) look at good contributions someone already tried to make, but that got stuck in review or were otherwise abandoned. (2) look at user reported bugs and see if we can find a user reported bugs, and see if we can reproduce and fix cheaply. Both are explicitly scoped as best-effort affairs: move on, if you can't quickly make progress. Most of the work I have to do as a human is review and navigating the submission process: tokens are cheap these days, so you really need to make sure the contribution is actually worth someone's time to review.
monster_truck 2mo ago
Writing custom WMMA/MFMA kernels for an exact integer matrix multiplication library that uses RNS & CRT + Int8 GEMM to get ~90% of the theoretical i64 TOPS output from a 7900XTX (3.9 TOPS vs the 0.5 or so you get with naive hip-direct usage) I'm not sure if you think that's a lot, but that was barely even 8 hours. I've had 200B+ months lol

eru 2mo ago

(Not the original commenter.) You can rack up quite a lot of tokens if you ask it to try out a lot of things, eg for performance investigations and trying out optimisation ideas.

fooblaster 2mo ago
is there a standard pattern for this? Like spawn an agent for each technique to try?
- lightbulbish 2mo ago
  maybe this? https://github.com/karpathy/autoresearch https://github.com/karpathy/autoresearch
- eru 2mo ago
  Not sure. I usually tell the agent to spawn subagents at will. (And they are doing that on their own anyway.)
- laichzeit0 2mo ago
  I regularly do a “go to DynaTrace, look at how this service gets used in production then use a profiler and see if there’s any low hanging optimisations we could make” on stuff. LLMs are really good at doing everything that was fun about software development.
  - eru 2mo ago
    Ha, I have a whole hobby work-stream going on about finding low-hanging fruit in various open source projects to turn into valuable contributions. Two premier sources: (1) look at good contributions someone already tried to make, but that got stuck in review or were otherwise abandoned. (2) look at user reported bugs and see if we can find a user reported bugs, and see if we can reproduce and fix cheaply. Both are explicitly scoped as best-effort affairs: move on, if you can't quickly make progress. Most of the work I have to do as a human is review and navigating the submission process: tokens are cheap these days, so you really need to make sure the contribution is actually worth someone's time to review.

monster_truck 2mo ago

Writing custom WMMA/MFMA kernels for an exact integer matrix multiplication library that uses RNS & CRT + Int8 GEMM to get ~90% of the theoretical i64 TOPS output from a 7900XTX (3.9 TOPS vs the 0.5 or so you get with naive hip-direct usage) I'm not sure if you think that's a lot, but that was barely even 8 hours. I've had 200B+ months lol