woctordho

5 ms·

1.
by woctordho 21d ago
No, 10.1 is current
2.
by woctordho 24d ago
Let me put my two cents: In China we've got accustomed to the fact that every word we say will be seen by the surveillance, so it's not a big problem that Anthropic also see it. Also we know that they can see it but they can'
3.
by woctordho 26d ago
Reasoning works as long as there is a consistent latent space representation. Any kind of poison will just become part of the representation. There's evidence that even directly training on encrypted reasoning traces works, because the
4.
by woctordho 1mo ago
There are 'transfer stations' and that's how exactly I use GPT and Claude in China. OpenAI and Anthropic do not sell in China, so we use their AI with a much lower price like 1% of the official API price. The largest transfer
5.
by woctordho 1mo ago
All the RL data are exactly public. There are huge amount of distilled data freely available, and that amount is more than enough to train a ~10T model.
6.
by woctordho 2mo ago
MiniMax H3 is going to release weights. You can locally run it with definitely less than $10k (and possibly faster than Seedance's queue), and it's fun to train it for whatever you need.
7.
by woctordho 2mo ago
GGUF is at least better than bnb. From what I know, bnb does not yet find a way to quantize MoE with enough accuracy, and maintain the dequant-MoE kernels. In the age of Qwen 3.0, people tried to make some bnb '4-bit' quants of Mo
8.
by woctordho 2mo ago
Speaking of finetune, currently a common practice is LoRA over bnb 4-bit base model, but I think it's time to replace bnb with GGUF as the base model format. GGUF is actively supporting new model architectures and more aggressive quant
9.
by woctordho 2mo ago
Distributed training is much harder than distributed inference but not impossible. See the recent development of DiLoCo at Nous Research and Prime Intellect.
10.
by woctordho 3mo ago
Relevant: Why Switzerland has 25 Gbit internet and America doesn't https://news.ycombinator.com/item?id=47652400
11.
by woctordho 3mo ago
AI Horde has some measures to prevent Sybil attack that returns wrong results, but not enforce zero data retention. Prompts belong to the whole open source community. For example https://huggingface.co/datasets/la-ji&#x
12.
by woctordho 3mo ago
Petals is from 2022. Nowadays intelligence of smaller models, quantization techs, and optimizations to run models faster on consumer GPUs have improved a lot. For distributed inference of smaller LLMs and diffusion models that fits in one
13.
by woctordho 3mo ago
So is making a PR different from making the whole software. This is what an open source community is good for.
14.
by woctordho 3mo ago
See the recent development of DiLoCo at Nous Research and Prime Intellect.
15.
by woctordho 3mo ago
There's a lot of individual effort of improving the models. See how many finetuned models and LoRAs are there on Hugging Face.
16.
by woctordho 3mo ago
There is a forum named Zhihu. AI translation works mostly well to translate contents there into English.
17.
by woctordho 3mo ago
Yes in a mid-sized company. I'm exactly doing this, and what I'm competing against is the OpenAI API priced 0.2 CNY = 1 USD in China.
18.
by woctordho 3mo ago
There's nothing wrong to run CUDA on non-Nvidia hardware. CUDA has an interface that is reasonably well-designed, well-documented/reverse-engineered, and battle-tested for decades. What we need is not to invent another interface j
19.
by woctordho 3mo ago
And humans don't run on markets.
20.
by woctordho 3mo ago
Fun fact: Hacker News is canonically banned in China, but I'm still talking here. There are plenty of techs to work around region block. The incentive to report somebody is comically called '50w' (500k CNY) and no one gives a
21.
by woctordho 3mo ago
See the recent advance of DiLoCo at Nous Research and Prime Intellect.
22.
by woctordho 3mo ago
Don't trust US or China. Trust the open source community.
23.
by woctordho 3mo ago
There are lots of botnets providing home IPs.
24.
by woctordho 3mo ago
Lots of people have succeeded. Neither Anthropic nor OpenAI has any technical advantage in the field of subscription engineering.
25.
by woctordho 3mo ago
Actually nowadays LLMs are only trained with TBs rather than PBs of data, and it's not too hard to find GBs of agent traces online.
26.
by woctordho 4mo ago
Simple trick: Use an agentic tool like Pi or OpenCode that allows you to switch models. First do some chats with DeepSeek or GLM who shows full thinking traces, then switch to Claude or GPT and it's more likely to show full thinking tr
27.
by woctordho 4mo ago
There is already a lot of effort to collect agent traces including reasonings, e.g. see the recent discussion: https://old.reddit.com/r/LocalLLaMA/comments/1u795pb/donate_... We've been developing D
28.
by woctordho 4mo ago
There's `--filter=blob:none` and it allows to automatically fetch blobs when needed.
29.
by woctordho 4mo ago
It's 2026. Historically the way for large binaries in git was git LFS. Now the way for large binaries in git is just git.
30.
by woctordho 4mo ago
Users don't need to build wheels. Wheel builders need to first learn to build some mediocre wheels, then they can build better ones.

More ›