benob

9 ms·

91.
by benob 1y ago
Very nice. Are you planning a paper?
92.
by benob 1y ago
TIL that in python, 1--2==3
93.
by benob 1y ago
This is exactly what EU is discussing
94.
by benob 2y ago
A variant I have been thinking of: each parameter matrix (or block) is the sum of a random matrix (generated from a seed) and a low rank matrix (a LoRA). I'd like to experiment training from scratch in that setting.
95.
by benob 2y ago
To me natural language interfaces are like the mouse-driven menu vs terminal interpreter. They allow good discoverability in systems that we don't master at the cost of efficiency. As always, good UI allows for using multiple modalitie
96.
by benob 2y ago
Looks like a good candidate for implementing doom
97.
by benob 2y ago
I like that they give advice to hardware manufacturers: - offload communication to a dedicated co-proc - implement decent precision for accumulating fp8 operations - finer-grained quantization ...
98.
by benob 2y ago
I definitely though of the programming language at first. Although the speed of light is a bit presumptuous to qualify the speed of C, I now wonder why every benchmark is against C, and not some other language.
99.
by benob 2y ago
According to arxiv paper, code will be made available here: https://github.com/annavaughan/aardvark-weather-public (from 8 months ago)
100.
by benob 2y ago
Didn't read the paper, but it would be cool to apply distillation to these big physics-driven models, in order to simulate their outcome on much smaller models.
101.
by benob 2y ago
The benchmark only touches 8B-class models at 8-bit quantification. Would be interesting to see how it fares with models that use more of the card ram, and under varying quantization and context lengths.
102.
by benob 2y ago
I think the point is that there is opportunity for splitting the browser in multiple components with a defined interaction surface. Have a component deal with network interaction, another with playing media, another with text layout. This w
103.
by benob 2y ago
What software do you use to generate them?
104.
by benob 2y ago
Take it the other way. What is the point of a human q/a site when you can get good enough answers from an AI? It's their business model that is threatened, and they need to pivot in order to survive.
105.
by benob 2y ago
This calls for controlling post-training instruction data from the base model. Does it contain many instances of self-reflection? Also, has anyone tried non-instruct tuned base models?
106.
by benob 2y ago
You made me feel old ;)
107.
by benob 2y ago
What's the real business model behind it? Sell "custom" names? That's not going to get you very far.
108.
by benob 2y ago
How are you supposed to implement the concept of interface in non OOP languages?
109.
by benob 2y ago
> If you’ve ever talked to me in person, you’d know that I’m a disbeliever of AI replacing decompilers any time soon Decompilation, seen as a translation problem, is by any means a job that suits AI methods. Give time to researchers to g
110.
by benob 2y ago
Mine: bm25 I use for teaching (sorry for the French example) https://gist.github.com/benob/69d48421f88f5dcc2b26a204d3251d...
111.
by benob 2y ago
Yes, multi-GPU on the same machine is pretty straightforward. For example ollama uses all GPUs out of the box. If you are into training, the huggingface ecosystem supports it and you can always go the manual route to put tensors on their ow
112.
by benob 2y ago
> How do you people debug and test these meta programs? I couldn't find any other answer than using @compileLog to print-debug [1]. In lisp, apparently some implementations allow to trace macros [2]. Couln'd find anything about
113.
by benob 2y ago
Has anyone tried compiling a pdf processing suite to wasm, so that we not have to upload anything to servers?
114.
by benob 2y ago
A relatively detailed description of such systems: https://arxiv.org/abs/2410.00037
115.
by benob 2y ago
I can't stop thinking this is how we will save humanity from the internet. Who needs to use the internet when you can have an agent do it for you? And once it is filled with agents, what will be the point of using it?
116.
by benob 2y ago
This one is actually impressive: implements Adam training of RNNs, LSTMs, GRUs.
117.
by benob 2y ago
Should try with smollm2 which is one of the smallest instructed models.
118.
by benob 2y ago
Base models are useful in research to see the effect of instruction tuning
119.
by benob 2y ago
You might want to look into "task arithmetic" which aims at combining task-specific models post-training. For example: https://proceedings.neurips.cc/paper_files/paper/2023/file/d...
120.
by benob 2y ago
Firefox support is not great. Typing through the terminal eats some keystrokes.

More ›