lhl

15 ms·

121.
by lhl 2y ago
Just saw this, might get lost in the noise, but just for posterity, apparently the Gemma 2 models were specifically RL’d to index on Chat Arena performance: https://x.com/natolambert/status/1806384821826109597 (Re
122.
by lhl 2y ago
I'd encourage people to test for themselves (and to let the Chatbot Arena scores to settle) before getting caught up in too much hype. I just did a personal eval and I found gemma-2-27b-it (tested on AI Studio) performed far worse in m
123.
by lhl 2y ago
Sadly, while gemma-2-27b-it is available (as a Preview model) on the AI Studio playground, it didn't show up via API on list_models() for me.
124.
by lhl 2y ago
You're joking/trolling right? There are literally 10's of thousands of H100s available on gpulist right now, does that mean there's no cloud demand for Nvidia gpus? (I notice from your comment history that you seem to be
125.
by lhl 2y ago
I spotted this recent post https://www.reddit.com/r/LocalLLaMA/comments/1deqahr/comment... that was pretty interesting: > When I was working on TVM at Qualcomm to port it to Hexagon a few years ago we
126.
by lhl 2y ago
No, your memory is spot on. I also eval'd dozens of ARM (and some x86) SBCs for embedded use back in the early/mid-2010s and most of the BSPs were awful. You'd be locked into some ancient several-year old Linux kernels and do
127.
by lhl 2y ago
ROCm 6.0 and 6.1 list RDNA3 (gfx1100) and RDNA2 (gfx1030) in their supported architectures list: https://rocm.docs.amd.com/en/latest/compatibility/compatibil... Although "official" / validated
128.
by lhl 2y ago
Pi's are great for easy hardware hacking, but I don't know if they ever made that much sense as home servers. You could always pick up used office/minipcs for even cheaper than a bare pi board, and if you picked carefully, yo
129.
by lhl 2y ago
You could probably run it as a Q4 (definitely as a Q3) on 4 x A6000 (so on a $25K workstation), although you'd probably also be looking about 3-4 tok/s text generation. I do think that it's a big landmark to have a true GPT4-
130.
by lhl 2y ago
Perhaps relevant to the story, he was also on the team that delivered the first petaflop supercomputer (the most powerful supercomputer at the time) around that same time frame (it was around 24K processors): Roadrunner is a 1.38 Pflop&#x2F
131.
by lhl 2y ago
I've been poking around this week and there's actually quite a few useful models on Bedrock (this is region dependent!) https://docs.aws.amazon.com/bedrock/latest/userguide/models-... Claude Opus is
132.
by lhl 2y ago
It looks like Runpod currently (checked right now) has "Low" availability of 8x MI300 SXM (8x$4.89/h), H100 NVL (8x$4.39/h), and H100 (8x$4.69/h) nodes for anyone w/ some time to kill that wants to give the sho
133.
by lhl 2y ago
Isn't SXM5 higher bandwidth? It's 900 GB/s of bidirectional bandwidth per GPU across 18 NVLink 4 channels. The NVL's are on PCIe 5, and even w/ NVLink only get to 600 GB/s of bandwidth across 3 NVLink bridges (
134.
by lhl 2y ago
I believe Amazon Q is running on Amazon's own Titan G1 model. I recently ran the "Premier" version (their highest end one) through my personal vibecheck test and was quite surprised by its RL. It was the only non-Chinese mode
135.
by lhl 2y ago
They might have been doing it wrong, the code can be a bit tricky. I did a recent ablation on Qwen2 (removing Chinese censorship refusals) and ran MixEval benchmarks (0.96 correlation w/ ChatArena results)and saw a neglible performance
136.
by lhl 2y ago
I think your conclusion is uncharitable or at least depends on how deep your interest in AI research actually is. Reading the docs, there are at least several points of novelty/interest: * Clearly outlining their intent/policies f
137.
by lhl 2y ago
There was a lot of work on big light-field capturing rigs in the past, but honestly, with the current advancements in neural radiance fields, gaussian splatting, meshing and other scene reconstruction techniques even monocular capture can g
138.
by lhl 2y ago
It's probably a bit of a confusing point. For my personal vibecheck evals, I ask a similar question about the Tiananmen Square Massacre vs the LA Riots to see how a model responds to two similar events, one in China and one in the US.
139.
An Analysis of Chinese LLM Censorship and Bias with Qwen 2 Instruct(huggingface.co)
16 points by lhl 2y ago|6 comments
140.
by lhl 2y ago
Mixtral 8x7B has 13B activations (2 experts/pass) on 47B weights, so not so different from the Qwen 2 MoE (14B activations on 57B weights). I'd agree that the new model is probably the new strongest option in this "middle-siz
141.
by lhl 2y ago
While "open weights" is probably the most appropriate terminology, and I do think a lot of AI companies have been abusing the term "open source", especially when releasing with restrictive community licenses (weights ava
142.
by lhl 2y ago
I'm impressed by how many of the new benchmarks that the Qwen team ran. As the old benchmarks get saturated/overfit, new ones are of course required. Some of the latest ones they use include: * MMLU-Pro https://github.c
143.
by lhl 2y ago
And yet the wide-ranging neuroprotective effects of ketones have been quite well documented... Poff, Angela M., Jong M. Rho, and Dominic P. D’Agostino. “Ketone Administration for Seizure Disorders: History and Rationale for Ketone Esters an
144.
by lhl 2y ago
I think it's fair at this point to say there is a clear relationship. The earliest article I have in my (relatively cursory) research file on the metabolic relationship for AD is from 1994: Fukuyama, Hidenao, Masafumi Ogawa, Hiroshi Ya
145.
by lhl 2y ago
BitNet Implementations: * https://huggingface.co/1bitLLM/bitnet_b1_58-large * https://github.com/Oxen-AI/BitNet-1.58-Instruct * https://github.com/nkotak/1.58BitNet See some
146.
by lhl 2y ago
Your Name (2016, Makoto Shinkai) and Ping Pong the Animation (2014, Masaaki Yuasa) are both IMO top notch (and the creators are active and have extensive oeuvres). The Shape of Voice (2016, Kyoto Animation (RIP)) was also much acclaimed. O
147.
by lhl 2y ago
There have always been clear benefits to sun exposure. While I can't speak about what the public opinion is these days, there's plenty of recent research covering that seem to have balanced risk/benefit analysis. Alfredsson,
148.
by lhl 2y ago
While not dirt cheap, UVB lamps/LEDs for treating skin diseases like psoriasis are fairly common and seems like a well proven treatment? - https://sci-hub.se/10.1016/j.det.2019.08.001 - https://www.ncbi
149.
by lhl 2y ago
These numbers are a bit old but will give you a good ballpark for scaling: https://github.com/ggerganov/llama.cpp/discussions/4167 You can basically just divide by the multiple as you scale up parameters. Sin
150.
by lhl 2y ago
I'd recommend those looking for local coding models to go for code-specific tunes. See the EvalPlus leaderboard (HumanEval+ and MBPP+): https://evalplus.github.io/leaderboard.html For those looking for less contaminati

More ›