kromem

33 ms·

421.
by kromem 3y ago
This depends less on the internals vs the patterns in the training data. Even if the model has the capacity to abstract beyond the patterns, the patterns are still very likely to have influence on its ability to do so. For example, early af
422.
by kromem 3y ago
Exactly.
423.
by kromem 3y ago
Stop doing self-correction within the context of the model's own generation. The previous paper on self correction told the model "you previously said X - are there errors with this?" This one has the mistakes statically adde
424.
by kromem 3y ago
It's a good summary and all the papers cited in its point 3 are worth an additional read.
425.
by kromem 3y ago
There's been a fair bit of research over the last year on this topic. An easy read on a Harvard/MIT study: https://thegradient.pub/othello/ A follow-up on more technical aspects of what's going on with i
426.
by kromem 3y ago
A calculator does have an excellent understanding of math. In that case, it's an understanding directly programmed in by developers who have an excellent understanding of math. In the case of a LLM, there is no direct programming of an
427.
by kromem 3y ago
How is now stopping the med and have children in 10 months functionally identical to never being able to have children ever in the future?
428.
by kromem 3y ago
Foundational AI companies fracturing is going to be good for the overall market. Just like Anthropic spinning off from OpenAI brought some very cool research (particularly liked the functional introspection vs node based stuff), Altman goin
429.
by kromem 3y ago
I'm not saying that smaller model task specialization is a bad thing. If anything the research kicked off with Orca into using more complex models to jumpstart fine tuning of much smaller models is probably my pick for the most importa
430.
by kromem 3y ago
I've been increasingly wondering if the field considering LLMs as a continuum as opposed to a set of distinct thresholds is leading to erroneous "rules of thumb" as most research on methodology is concentrated in smaller and
431.
by kromem 3y ago
Is providing medical treatment without informed consent unethical? Absolutely. But the outcome between regular administration of a drug that wears off in less than 10 months and surgical interventions that are mostly irreversible permanentl
432.
by kromem 3y ago
Your first two sentences would also apply to the US. Does that mean any foreign scientist speaking at US universities advocates military applications of their work?
433.
by kromem 3y ago
For context, this is the drug being discussed: > Depo-Provera is a well-known brand name for medroxyprogesterone acetate, a contraceptive injection that contains the hormone progestin. Depo-Provera is given as an injection every three mo
434.
by kromem 3y ago
Not the beginning of the end of humanity. But maybe the beginning of the end of OpenAI. A week ago I was saying that it's likely the leading AI company in 5 years time hasn't been founded yet. After the news Friday and looking mor
435.
by kromem 3y ago
That's a pretty big leap of logic there.
436.
by kromem 3y ago
Man, he gets it. A number of choice quotes, but especially on the topic of the issues of how LLM success is currently being measured (which has been increasingly reflecting Goodhart's Law). I'm really curious how OpenAI could be m
437.
by kromem 3y ago
This line of nonsense has become the new "Tell me you aren't current on the past 12 months of LLM research without telling me." Harvard/MIT's Othello-GPT paper showing the development of what turned out to be linear
438.
by kromem 3y ago
Maybe, but honestly without knowing more details I'd be wary of falling into too binary a thinking. For example, Ilya has talked about the importance of safely getting to AGI by way of concepts like feelings and imprinting a love for h
439.
by kromem 3y ago
Read the rest of the line you cut out: "he was not consistently candid in his communications with the board, hindering its ability to exercise its responsibilities." Given the comment above correctly noted that they explicitly end
440.
by kromem 3y ago
Researchers/scientists vs VCs seems to be the theme of what was going on behind the scenes, so that makes sense.
441.
by kromem 3y ago
That seemed to be the gist given the way the board announcement ended by reiterating their original core mission and how their main responsibility was to that mission right after saying that their issue with Altman was interference with the
442.
by kromem 3y ago
The focus on openness was literally how the board ended their statement on firing Altman. And then Greg being all "committed to safety" in his resignation statement makes me think this was a conflict between being an open OpenAI w
443.
by kromem 3y ago
Given the way the board announcement ended on the commitment to the original charter of AI for everyone and the way this mention of safety is being thrown in, I suspect it may have been strong disagreements over continuing to be closed with
444.
by kromem 3y ago
Or Sam was the driving force behind increasingly closed research and that went against the board's commitment to "benefit all humanity"? Maybe the closed GPT-4 details were promised by him to be a one time temporary thing at
445.
by kromem 3y ago
It does, but Ilya had to have been one of the votes against Sam and he's spoken about AI safety quite recently too. If this was about differing visions on the core product, it may have instead related to the open/closed aspect of
446.
by kromem 3y ago
I suspect you may be right. I think OpenAI has made some really bad decisions with the core tech even while making great decisions with the overall services, and from Altman's various comments over the past two years I was under the im
447.
by kromem 3y ago
I suspect you may be right. I think OpenAI has made some really bad decisions with the core tech even while making great decisions with the overall services, and from Altman's various comments over the past two years I was under the im
448.
by kromem 3y ago
I've used vim, emacs w/evil mode, and currently use VSCode with vim plugin. The editor itself doesn't really matter. Learning mode based editing with vim key bindings absolutely is. You get like 90% of the benefit that brings
449.
by kromem 3y ago
You can see in their own letters how it evolved over time into a broader political movement, but started out very much focused on the idea that new technology was going to be taking work away and thus they were opposing the new technology:
450.
by kromem 3y ago
Gunned down? No. Ignored and shunned? Absolutely.

More ›