robrenaud

8 ms·

61.
by robrenaud 11mo ago
The relevance here is pretty weak. https://sturdystatistics.com/deepdive?fast=0&q=reinforcement... I think only 1/10 of the articles is really on topic.
62.
by robrenaud 11mo ago
LLMs without a search engine attached suck for product reviews.
63.
by robrenaud 11mo ago
A lot of employed people like the status quo for the healthcare that they receive. "In contrast to their largely negative assessments of the quality and coverage of healthcare in the U.S., broad majorities of Americans continue to rate
64.
by robrenaud 1y ago
My biggest insight on my (self diagnosed, high functioning) Autism. For most people, the golden rule works. Treat other people as you want to be treated, and modulo rare interactions with asshole people, you get along well. For autistic pe
65.
by robrenaud 1y ago
But being wrong can cause downvotes.
66.
by robrenaud 1y ago
The Gemini IMO result used a specifically fine tuned model for math. Certainly they weren't training on the unreleased problems. Defining out of distribution gets tricky.
67.
by robrenaud 1y ago
The input size to output quality mapping is not linear. This is why we are in the regime of "build nuclear power plants to power datacenters". Fixed size improvements in loss require exponential increases in parameters/compu
68.
by robrenaud 1y ago
I think your core misunderstanding is that you are assuming K calls to generate 1 token is expensive as 1 call to generate K tokens. It is actually much more expensive to generate serially than even in small batches.
69.
by robrenaud 1y ago
I recently got to know someone with a resting heart rate of 45, who will pretty frequently do 8+ mile trail runs and 100 mile bike rides. He is also an amazing cook who makes decadent and delicious foods. He says he consumes 4,000 calories
70.
by robrenaud 1y ago
Why are 100s of millions of people using AI if it is providing no value?
71.
by robrenaud 1y ago
It's not even close.
72.
by robrenaud 1y ago
There is no real difference between fine-tuning with and without a lora. If you give me a model with a lora adapter, I can give you an updated model without the extra lora params that is functionally identical. Fitting a lora changes poten
73.
by robrenaud 1y ago
Meta synthetically generated lots of PHP from Python for Llama 3 for training purposes. Meta writes a crazy amount of PHP internally. Translation tends to be way easier than unconstrained generation for LLMs. But if you can translate an
74.
by robrenaud 1y ago
Human generated tokens contain so much more information per byte than random street view images.
75.
by robrenaud 1y ago
O3 is OpenAI. Street view is Google. I really doubt OpenAI is scraping enormous amounts of random street view images to train their model.
76.
by robrenaud 1y ago
> In this sparsely populated rural area, "I have at least two homes where I have to build a half-mile to get to one house," Mauch said, noting that it will cost "over $30,000 for each of those homes to get served." Do
77.
by robrenaud 1y ago
Find some nerdy social hobby, become part of a community. Board games, killer queen arcade, and indoor rock climbing have all been a bridge to some close friendships for me.
78.
by robrenaud 1y ago
They could provide verbatim snippets surrounded by explanations of relevance. Instead of the core of the answer coming from the LLM, it could piece together a few relevant contexts and just provide the glue.
79.
by robrenaud 2y ago
I am also interested in how to do eval on an open source corporate search system. Privacy and information security make this challenging, right?
80.
by robrenaud 2y ago
Deepseek is great. What fundamentally prevents an open source coalition from producing great AI systems for everyone?
81.
by robrenaud 2y ago
> "Note that this s1 dataset is distillation. Every example is a thought trace generated by another model, Qwen2.5" The traces are generated by Gemini Flash Thinking. 8 hours of H100 is probably more like $24 if you want any ki
82.
by robrenaud 2y ago
It's a joke about how Google has released/cancelled/renamed many messenging apps.
83.
by robrenaud 2y ago
Do you understand why RL is better than SFT for training on reasoning traces?
84.
by robrenaud 2y ago
Our genes are heavily evolved to live in calorie scarce environments. In those environments, high calorie foods are amazing. Our biology is built to find them incredibly rewarding. Science and capitalism have created incredibly delicious
85.
by robrenaud 2y ago
My best learning was difficult, whole mind encompassing, and incredibly fun. If you can get college students idle brains curiously contemplating the how and why of the subject, that's when the tuition is really worth it.
86.
by robrenaud 2y ago
Are the market dynamics such that effective small companies grow, and ineffective small companies shrink? Is this bad?
87.
by robrenaud 2y ago
I really do think hallucinated references are a thing of the past. Models will still make things up, but they won't make up references. ChatGPT with web search does a good job of summarizing content.
88.
by robrenaud 2y ago
I've been using self hosted langfuse via litellm in a juptyer notebook for a few weeks for some synthetic data experiments. It's been a nice/useful tool. I've liked having the traces and scores in a unified browser based
89.
by robrenaud 2y ago
Gemini is much worse as a product than 4o or Claude. I recommend using it from Google AI studio rather than the official consumer facing interface. But for tasks with large audio/visual input, it's better than 4o or Claude. Wheth
90.
by robrenaud 2y ago
Do you fear that some big company will just host your system for cheaper than you can if you catch a lot of success? That is, the same thing that Amazon did to Mongo will happen to you? Do you think working in the open enables you to spend

More ›