gaogao

6 ms·

31.
by gaogao 1y ago
Domains where fuzzing is useful are generally good candidates for formal verification, which I'm pretty bullish about in concert with LLMs. This is in part because you can just formal verify by exhaustiveness for many problems, but the
32.
by gaogao 1y ago
The order is a bit broad, but legal holds frequently interact with deletion commitments. In particular, the only purpose of data deleted under GDPR held up by a legal hold should be for that legal hold, so it would be a big no-no if OpenAI
33.
by gaogao 1y ago
The traditional way at the megacorp for something like that is unsafe_str_DO_NOT_USE_OR_YOU_WILL_BE_FIRED(template)
34.
by gaogao 2y ago
> Starlark is intended to be simple. There are no user-defined types, no inheritance, no reflection, no exceptions, no explicit memory management. Execution is finite. The language does not allow recursion or unbounded loops. That has so
35.
by gaogao 2y ago
> AMA is not the main culprit in supply of doctor's, that would be available spots for residents The AMA has actively lobbied to lower the number of residency spots. > In 1997, the AMA lobbied Congress to restrict the number of d
36.
by gaogao 2y ago
It's not just that they didn't care, but they had a better framing and RLHF helped quite a bit. Galactica got released at a similar time to ChatGPT and got castigated, in a large part due to a poor framing of it being for science
37.
by gaogao 2y ago
40+ is a bit weird of a timeline to give considering they invaded Vietnam 45 years ago, https://en.wikipedia.org/wiki/Sino-Vietnamese_War
38.
by gaogao 2y ago
Yeah, high bandwidth requirements still remaining. Over the past year, more research has looked from fully async to restrained cases that allow for geographically distributed compute. Async Local-SGD goes for a more standard training object
39.
by gaogao 2y ago
The intuition is that smaller models are figuring out things like grammar where the whole model comes into play, but larger models, especially in the back half of training, have localized knowledge updates that can merge easier in the AllRe
40.
by gaogao 2y ago
This approach likely scales to top models. There was a thought for a while among researchers that scaling up Hogwild / LocalSGD to more parameters would be less convergent, but it seems to be that things actually converge a bit better
41.
by gaogao 2y ago
Adding in a 100k iteration or so busyloop to the beginning of the food segment should do the trick and only add a few bytes. You could do something fancy with the clock interrupt, which is a bit more portable and stable, but might cost a fe
42.
by gaogao 2y ago
You can model types as a set and sets as types in many ways, so a number of the basic set theory axioms are pretty simple to express as lemmas from type axioms. IIRC you get constructive set theory easy, but do have to define additional axi
43.
by gaogao 2y ago
I don't think they work at Meta anymore.
44.
by gaogao 2y ago
One improvement though is using Starlark, instead of directly Python, since it offers a lot of advantages for a more lightweight runtime and parallelism.
45.
by gaogao 2y ago
There's an encylopedia you can't buy called Wikipedia. It also has its own structured auxillary database, WikiData. I wrote up roughly what this article is doing about a year and a half ago - https://friend.computer&#x2
46.
by gaogao 2y ago
From my read, it probably flows from 26 = 27 - 1 = 3^3 - 1, given the equation is looking at cubes.
47.
by gaogao 2y ago
Local recommendations is a big category of LLM questions that they're mostly bad at today that they can sell ads for in the future.
48.
by gaogao 2y ago
I've been wondering if you could just attach a chunk of vram over NVLink, since that's very roughly what FSDP is doing here anyways.
49.
by gaogao 2y ago
Engineers who are really, really good at CUDA are worth their weight in gold, so there's more projects for them than they have time. Worth their weight in gold isn't figurative here – the one I know has a ski house more expensive
50.
by gaogao 2y ago
Yeah, doing some preferred cloud Data Warehouse with an indexing layer seems fine for this sort of thing. That has an advantage over something specialized like this of still being able to easily do stream processing / Spark / etc,
51.
by gaogao 2y ago
Agreed. As an example of that, Meta's kept its research org, FAIR, still doing fundamental research. Research orgs are great at demos, but actual productionalization takes a different mindset.
52.
by gaogao 2y ago
Urban densification is definitely more eco-friendly than most suburbanization. The emissions of cars for the US is much higher than most people expect at 1.7 billion tons, while air conditioning is about 117 million tons. A/C also peak
53.
by gaogao 2y ago
Medicaid having a cliff, instead of progressive falloff, is a very regressive policy. There's a common misconception that earning too much will bump you over a tax bracket and cost you, but that's actually true for many safety net
54.
by gaogao 2y ago
Nah, it's one level up having basically everything as an Ent (ORM object) with an associated data access policy that does that.
55.
by gaogao 2y ago
> categorize what is PII and what is not and anonymize as needed That sounds like just de-identification / pseudoanonymization, if you're just targeting PII or not
56.
by gaogao 2y ago
Outperform in what way? There's some distinct things it already does better on like multi-hop and aggregate reasoning than a similarity context window dump. In general, tool-assisted, of which KG querying is one tool, does pretty good
57.
by gaogao 2y ago
Yeah, one of the specific things I'd love to do is collaboratively bulking up WikiData more. It's missing a ton of low hanging fruit that people using an ML augmented tool could really make some good progress on, similar to ML ass
58.
by gaogao 2y ago
Yup! I found Lua pretty lovely to use with Rust in my case.
59.
by gaogao 2y ago
I think there's a lot of cases where you don't want to just RAG it. If you're going for tool assisted, it's pretty neat to have agent write out queries for what it needs against the knowledge graph. There was an article
60.
by gaogao 2y ago
Yeah, I rather like the pattern of embedding a more rapid iteration scripting language on top for game stuff like Lua or stripped down Python variants, using Rust as the core and hot loop code. That said, there is a lot of folks wanting to

More ›