bick_nyers

8 ms·

31.
by bick_nyers 2y ago
I disagree. I have to learn by committing to muscle memory. I always say I'm a slow learner, but a fast thinker. Yes, I can read/hear a concept and understand it in abstract and visualize it pretty easily, but it will leave my bra
32.
by bick_nyers 2y ago
I think this is often the reasoning of management, however, what they don't see is that developer skill often follows a power law and the flow state is a multiplier on top of that. The cost of a midday check-in is not the cost of how m
33.
by bick_nyers 2y ago
Objective Function: Optimize on a per-user basis. Constraints: Output generated must be considered legal in user's country. Both things can co-exist without being in conflict of each other. My (hot) take is I personally don't beli
34.
by bick_nyers 2y ago
Alternatively, you include information about the user of the model as part of the context to the inference query, so that the model can uniquely optimize its answer for that user. Imagine if you could give a model "how you think"
35.
by bick_nyers 2y ago
I had an idea similar to this for a model that allows you to parameterize a performance vs. accuracy ratio, essentially an imbalanced MoE-like approach where instead of the "quality score" in your example, you assign a score based
36.
by bick_nyers 2y ago
I didn't read the paper, but am familiar with: https://mpeg-miv.org/ One challenge here is dedicated hardware support, running any sort of video processing/decoding on the CPU is unfeasible (GPU less so), particul
37.
by bick_nyers 2y ago
To be fair, you need 2x 4090 to match the VRAM capacity of an RTX 6000 Ada. There is also the rest of the system you need to factor into the cost. When running 10-16x 4090s, you may also need to upgrade your electrical wiring to support tha
38.
by bick_nyers 2y ago
Agreed, but there are other costs associated with supporting 10-16x GPUs that may not necessarily happen with say 6 GPUs. Having to go from single socket (or Threadripper) to dual socket, PCIE bifurcation, PLX risers, etc. Not necessarily s
39.
by bick_nyers 2y ago
I'm curious what techniques they used to distill the 405B model down to 70B and 8B. I gave the paper they released a quick skim but couldn't find any details.
40.
by bick_nyers 2y ago
Would be interesting to see the performance on a dual-socket EPYC system with DDR5 running at maximum speed. Assuming NUMA doesn't give you headaches (which it will) you would be looking at nearly 1 TB/s
41.
by bick_nyers 2y ago
The best NVLINK you can reasonably purchase is for the 3090, which is capped somewhere around 100 Gbit/s. This is too slow. The 3090 has about 1 TB/s memory bandwidth, and the 4090 is even faster, and the 5090 will be even faster.
42.
by bick_nyers 2y ago
Energy costs are an important factor here too. While Quadro cards are much more expensive upfront (higher $/VRAM), they are cheaper over time (lower Watts/Token). Offsetting the energy expense of a 3090/4090/5090 build v
43.
by bick_nyers 2y ago
What if the LLM that a specific NPC utilizes was handcrafted/fine-tuned by a human?
44.
by bick_nyers 2y ago
Are you asking why pay extra for a CPU and RAM? Not everything can be done on a GPU, for example, .png decompression. If you really analyzed your training code and preprocessed your data substantially you could probably get away with very l
45.
by bick_nyers 2y ago
Great advice.
46.
by bick_nyers 2y ago
... you didn't have a UAT environment?
47.
by bick_nyers 2y ago
Depends on your definition of simplicity. Some view simplicity more as minimizing lines of code, e.g., less moving parts. I view simplicity more as increasing lines of code, the goal being to make the code very verbose. Sometimes this means
48.
by bick_nyers 2y ago
The issue I take with that is that it's another layer of abstraction that can be warped /inflated/etc. Similarly, mobile games often have a "game currency" that you purchase to obfuscate the true cost of a purchase.
49.
by bick_nyers 2y ago
M2 consumes about 2.5x the power of MicroSD, which may be fine for most use cases, but anything running on a battery will suffer. 5 watts just for the M2 drive is quite a lot.
50.
by bick_nyers 2y ago
I wonder how well dataset distillation would work between NN <-> KAN. Could significantly reduce training time by seeding the initial KAN model, thus making more of these experiments feasible to conduct and thus can experiment on larg
51.
by bick_nyers 2y ago
Depends on the pay. If more companies start doing this, then you can imagine having to slog through 10-20 of these as an applicant will increase the time to find a job significantly. Most people don't get hired off of their first succe
52.
by bick_nyers 2y ago
Are you using your Shield as an HTPC? In that case you can use the upscaler built into a TV. I prefer my LG C2 upscale (particularly the frame interpolation) compared to most Topaz AI upscales.
53.
by bick_nyers 2y ago
I know it's not an amazing safeguard, but providing that alternative would be a deterrent for some teachers just blindly feeding it into an AI and not reviewing the outputs critically. I don't think many teachers would do that, bu
54.
by bick_nyers 2y ago
I'm wondering if you could sort the two inputs, add the indicies for the multiplications together, then take the largest of that. In your 0.1 example, 1000 gets index 2, and 0.1 index 0, combines to 2. This will tie with the 1*8, but I
55.
by bick_nyers 3y ago
I've seen it done with a PLX Multiplexer as well, but they add quite a bit of cost: https://c-payne.com/products/pcie-gen4-switch-backplane-4-x1... Not sure if there exists an 8-way PCIE Gen 5 Multiplexer that doe
56.
by bick_nyers 3y ago
Interesting... 1 Zen 4 EPYC CPU yields a maximum of 128 PCIE lanes so it wouldn't be possible to put 8 full fat GPUs on while maintaining some lanes for storage and networking. Same deal with Threadripper Pro.
57.
by bick_nyers 3y ago
A comment I really liked on a previous post about ternary highlighted that what you are actually measuring with {-1, 0, 1} is inverse correlation, no correlation, and correlation.
58.
by bick_nyers 3y ago
I think they are moreso saying that GPUs are not optimized for those operations. CPU aren't "designed" for matrix multiplies yet we can still run them, albeit at a slower rate than on a GPU.
59.
by bick_nyers 3y ago
Gradient-directed evolutionary algorithm sounds kinda interesting.
60.
by bick_nyers 3y ago
It took a lot of human brain flops to get to this point in time though, I wonder how many orders of magnitude more than it took to train ChatGPT...

More ›