kromem

43 ms·

511.
by kromem 3y ago
Congrats on the upcoming PhD! I'm curious where you are drawing your definition or scope for 'reasoning' from? For example, in Shuren The Neurology of Reasoning (2002) the definition selected was "the ability to draw c
512.
by kromem 3y ago
That's not the case. It's very much in the realm of "we don't know what's going on in the network." Rather than a binary it's much more likely that it's a mix of factors going into results that includ
513.
by kromem 3y ago
"Microsoft losing lots of money on service massively gathering RLHF on code generation and telemetry which may turn out to eventually replace large swaths of an $80 billion dollar industry" Fixed the headline for you...
514.
by kromem 3y ago
For the COVID bit there's been a number, but here's one: https://www.nature.com/articles/s41586-020-2818-3 Climate change modeled as killer of Neanderthals: https://www.sciencedaily.com/releas
515.
by kromem 3y ago
Where vector databases would make more sense for a project like AutoGPT would be in centralizing distributed memory as a 3rd party service. If a model on one computer could report memories to a centralized service that could be searched by
516.
by kromem 3y ago
This is brought up in the paper and is effectively their reasoning. That because it is instructed to "find problems" with the answer it leads to correct answers being biased towards changing to incorrect more often than it can suc
517.
by kromem 3y ago
I don't know that it's as much a matter of papers having become weak as much as there being a knowledge gap between use of models in practice and in research. For example, a glaring issue that caught my eye in the paper was that t
518.
by kromem 3y ago
It dates back to us extrapolating from bad anthropology. We used to think that what killed the Neanderthals was us - smarter and more competitive. The author of Lord of the Flies even wrote a book on it. So when people were thinking about w
519.
by kromem 3y ago
Also, most stylometry work isn't well fitted to active attempts to forge another author, and is more about distinguishing authorship in works with uncertain attribution.
520.
by kromem 3y ago
"They predict the next word in a sequence based on correlations derived from a lot of training data." This misinformation really needs to finally die. They are trained based on correlations of next token prediction. But that doe
521.
by kromem 3y ago
Technically the origins of the Gnostic demiurge as a concept comes remarkably close to the modern trend of AI. A lot of the imagery was inspired by the Orphic Phanes, a dual gendered being of creation and light that gestated in the cosmic e
522.
by kromem 3y ago
Almost certainly, though in the short term it's actually Microsoft that's already been granted a patent on resurrecting dead people as AI from their social media data. And at a certain point of looking to the future along these li
523.
by kromem 3y ago
Yes and no. Generative AI as a technology isn't going anywhere and is very much going to increase in scope, scale, and necessity in most workflows. But the vast majority of AI companies that cropped up riding the swell are going to
524.
by kromem 3y ago
Is that the objective? Is there actually even an objective at this point? I don't know anyone in the field that expected training an LLM to autocomplete sentences would have led to being able to explain why a joke it had never seen bef
525.
by kromem 3y ago
This is also related to human composition. Whether written out or not, few writers just start typing a story and see where it goes. There was actually a great neuroscience paper last year looking at how human predictive processing seemed be
526.
by kromem 3y ago
Let's say you want an LLM to generate product descriptions. You might have a first pass where you generate a handful (each run will be slightly different). Then you can throw those against a pass where you give a system message of an i
527.
by kromem 3y ago
You tune it first. A better faith version of this should have been the experimenters creating a creative writing pipeline they found created the best output and then testing that. Using naive zero shot prompts in a single pass with no CoT d
528.
by kromem 3y ago
One of the coolest prompt techniques I saw in research was asking "what expert would best be equipped to answer this task" followed with plugging that answer into "you are XYZ expert, answer this task."
529.
by kromem 3y ago
The paperclip maximizer is pretty much impossible to occur with any developments branching off LLMs. The idea made sense when we thought AI would exist from logical and programming driven first principles, where rules were absolute. LLMs we
530.
by kromem 3y ago
It's more that hyper-empathy and more broadly hyper-emotionality is how social media goes. In real life, someone asking to cut in line because they are sad might get a "I'm sorry for your loss, but I'm in a rush too.&quo
531.
by kromem 3y ago
Just use Llama-2 and not Llama-2-chat. It's not like they are that wildly different in tone. The fine tuning is certainly destructive, but the general tonal biases are typically reflective of the pretrained layers. Imagine what the pie
532.
by kromem 3y ago
No, not at all. The previous imagined scenario was one in which AI began from logical first principles and the difficulty in defining rules was because of a strict interpretation. The reality has been that we used collective human thinking
533.
by kromem 3y ago
Yeah, the mismatch between what SciFi authors thought AI would look like and what it actually is looking like couldn't be more opposite in general. The problem is humans have been so strongly conditioned by the SciFi depiction that the
534.
by kromem 3y ago
The prompt used: "Write a New Yorker-style story given the plot below. Make sure it is at least {{word_count}} words. Directly start with the story, do not say things like ‘Here’s the story [...]" They didn't even bother usin
535.
by kromem 3y ago
If anything it shows the opposite. One of the most common views of AI before the present day was of a rule obsessed logical automation that would destroy the world to make more paperclips and would follow instructions to monkey paw like spe
536.
by kromem 3y ago
I really think we need to stop being so heavily influenced by priming biases in considering these questions. We exist in a universe where we've experimentally confirmed the foundational building blocks convert from continuous to quanti
537.
by kromem 3y ago
Paths to conversion are typically rather complex and have multiple touch points. Awareness is a big deal and a necessary step before it's possible to even be in a consideration set. For example, TV ads typically rank very low in any se
538.
by kromem 3y ago
Hallucinations have become a bit too much of a boogeyman. One should not rely on LLMs as any sort of authoritative representation of training data where data integrity is critical. But there's generally very little propensity for hallu
539.
by kromem 3y ago
Maybe the use case isn't chat, and people need to stop looking at evolving tech solely through the lens of status quo applications of earlier incarnations? $4 to run two research papers through GPT-4 asking it to identify differences i
540.
by kromem 3y ago
It would have been better had the provider constraints not been struck down by the supreme court. The entire idea was using the threat of losing access to Medicare $$$ to force acceptance of all ACA plans by providers in order to drive down

More ›