robrenaud

11 ms·

91.
by robrenaud 2y ago
> For example, combining a human-moderated knowledge graph with an LLM with RAG allows you to build "expert bots" that understand your business context / your codebase / your specific processes and act almost human-li
92.
by robrenaud 2y ago
Then those new models get distilled into smaller ones. Raising the max intelligence of the models tends to raise the intelligence of all the models via distillation.
93.
by robrenaud 2y ago
I just moved to SF a couple weeks ago. In city driving, I really like driving near Waymos as opposed to human cars. You quickly learn that the Waymos drive well and passively, which enables safe, assertive driving near them. Being able t
94.
by robrenaud 2y ago
> if advancing the SOTA AI is your dream, a product company may not be the right place. Does Meta get in the way of this? It's hard to compete with a company that is dead set on spending billions and seemingly wants to drive your SO
95.
by robrenaud 2y ago
Is the encoder style arch better for representing classification tasks at a given compute budget than a causal LM? Is this because the final represention in bert style models more globally focused, rather than being optimized for next token
96.
by robrenaud 2y ago
The physics professor AI wouldn't have to solve novel problems in physics, it just has to be able to remove the confusion of physics undergrads.
97.
by robrenaud 2y ago
As models improve, their representations converge. This even happens across modality for vision and text.
98.
The Platonic Representation hypothesis [video](youtube.com)
1 points by robrenaud 2y ago|1 comments
99.
by robrenaud 2y ago
The reason to fine tune is to get a model that performs well on a specific task. It could lose 90 percent of it's knowledge and beat the unturned model at the narrow task at hand. That's the point, no?
100.
by robrenaud 2y ago
Does this also apply to hiring as well as firing? Should companies hire more to make more money in times of growth? Is this a coherent position?
101.
by robrenaud 2y ago
A good, automatically run, privacy preserving search engine that uses electronic medical records might be a valuable resource for busy doctors.
102.
by robrenaud 2y ago
Newton.
103.
by robrenaud 2y ago
Do EMRs improve the standard of care?
104.
by robrenaud 2y ago
I got a couple patents while at Google. I sent a nice readable 4 page design doc that I wrote to a patent lawyer, and I got back 40 pages of nonsense that I basically didn't understand. I wish there was some kind of readability require
105.
by robrenaud 2y ago
It feels pretty good. I did some reading on some high quality posts about chess that I found through it. What was the biggest thing you learned while implementing this? Was anything surprisingly difficult? Was there anything that worked
106.
by robrenaud 2y ago
Maybe a combined approach beats either? Let some non-LLM reranker quickly spit out two results, and fill in the rest with the LLM.
107.
by robrenaud 2y ago
Finding ways to reward people for building great open source software is something that we need more of. It's not lame at all.
108.
by robrenaud 2y ago
Are there any strong LLMs trained without CUDA?
109.
by robrenaud 3y ago
Model quality matters a ton too. They aren't serving OpenAI or Anthropic models, which are state of the art.
110.
by robrenaud 3y ago
If you wanted to finetune a Mixtral 8x7B, what would you use?
111.
by robrenaud 3y ago
If you want something like Kolmogorev complexity for molecules, check out assembly theory. I am a CS person, but there are interesting, related ideas here. https://en.m.wikipedia.org/wiki/Assembly_theory
112.
by robrenaud 3y ago
It would have predictible structure. There would be groupwise, regular sparsity. You could exploit these patterns to infer properties like, "there is no connection from this vertex to this entire span of verticies". Just think a
113.
by robrenaud 3y ago
Click bait title, but there is a lot of interesting stuff here.
114.
by robrenaud 3y ago
What kind of queries do you handle well? Do you index podcast transcripts?
115.
by robrenaud 3y ago
Is there a sustainable business for a company that monetizes defenses against pig butchering? If you could make a honey pot LLM that engages the scammers, costing them time and learning their patterns, even misleading the scammers and figur
116.
by robrenaud 3y ago
Awesome! How are you doing the synthetic data generation? What is the best open source small LLM?
117.
by robrenaud 3y ago
Regarding your comment about how fast the research and industry is moving, would HN readers be interested in relevant one or two paragraph summaries that are basically "explain it like I am a machine learning engineer from 2020" b
118.
by robrenaud 3y ago
Only tangentially related, but I think the mid/late game is a much more interesting/useful/hard avenue for chess tools. Chess.com's opening explorer is pretty good, which it seems like this is kind of trying to compete a
119.
by robrenaud 3y ago
Diversity and quantity are important for LLM training. A search engine can index more than just "the best sources", and show results from the tail when no relevant matches are in the best sources. I would agree that with a softer
120.
by robrenaud 3y ago
Simulation to reality is a huge bottleneck in a lot of reinforcement learning work. Reality is just super messy and complicated. Tesla has an alternative. If you can get your devices widespread and can be recording observations and action

More ›