Imnimo

34 ms·

421.
by Imnimo 4y ago
>he’s a marketer who outsources the majority of his copy and content writing to me. So is this like SEO content marketing? Without understanding the nature of the job, it's hard to know what conclusions to draw. If the most importan
422.
by Imnimo 4y ago
The "higher level algorithm" of "how to do abstract thought" is unknown. Even if LLMs solve "how to do language", that was hardly the only missing piece of the puzzle. The fact that solving the language compone
423.
by Imnimo 4y ago
But it's not so simple to just "program the memory logic". The hand-holding offered here is sufficient to navigate this restricted simulated world, but what would be required to achieve increasingly complex behaviors? If a Ch
424.
by Imnimo 4y ago
It's interesting how much hand-holding the agents need to behave reasonably. Consider the prompt governing reflection: >What 5 high-level insights can you infer from the above statements? (example format: insight (because of 1, 5, 3
425.
by Imnimo 4y ago
The vector database would be good for retrieving vocabulary, but could it be expected to do things like retrieve sentences with similar syntax or tenses? It feels like it would be hard to successfully retrieve examples that were important f
426.
by Imnimo 4y ago
Setting aside the specific choice of tokenizer for GPT models, I'm curious how much difference in performance is made by the features of the human language used to represent the training data. Like if you kept the exact same training c
427.
by Imnimo 4y ago
Even in the case of a full size GPT, one could equivalently write out a lookup table for every possible permutation of tokens in the context window and the resulting output distribution. The table would of course be impossibly large.
428.
by Imnimo 4y ago
Sure, but some of the comparison cities (e.g. Memphis and Albuquerque) have huge areas of undeveloped land within their city limits. There's no good reason to compute a per square mile statistic given the vastly different definitions o
429.
by Imnimo 4y ago
That assumes that both your presence and the crime are equally distributed over area. If SF annexed a thousand square miles of crimeless, uninhabited wilderness, that would vastly decrease its crime per square mile, but would it affect how
430.
by Imnimo 4y ago
Are you using a reduced version of your prompt in production, and if not, why not?
431.
by Imnimo 4y ago
An important thing to keep in mind when reading articles about GPT's test-taking abilities is that for humans, the tests are merely proxies. The point of taking the bar exam isn't that we need people to be able to pass the bar exa
432.
by Imnimo 4y ago
>If the model's prediction is close to the actual next word, the neural network updates its parameters to reinforce the patterns that led to that prediction. What does "close to" mean here?
433.
by Imnimo 4y ago
Suppose that instead of a 100B+ parameter neural network language model, I had a website with a simple bigram language model that, after drawing enough samples, produced a similar output. Would anyone bat an eye? Would anyone bother to writ
434.
by Imnimo 4y ago
>China is currently about 2 years behind the US in AI. How can this be true if AI is not a race? What does it even mean to be behind in a technology that isn't a race?
435.
by Imnimo 4y ago
A more accurate title would be "decrease GPT4's context size by asking it to obfuscate your prompts". You're getting a prompt that consumes more tokens and isn't particularly faithful to the original.
436.
by Imnimo 4y ago
https://imgur.com/a/uahlB4H Here's exactly what I did. I used the prompts in the original tweet, but substituted in text that was published too recently to be in GPT-4's training data. I ran the compression a
437.
by Imnimo 4y ago
The text it compressed is from Wikipedia, so I'm not convinced it isn't primarily relying on having memorized it. I tried with text that was published in the past week (and so definitely isn't in GPT-4's training). GPT-4
438.
by Imnimo 4y ago
I don't understand what I'm expected to think of articles that are just "look at this variable name!" If you're not going to go through the trouble of finding out what the line of code actually does, what's the
439.
by Imnimo 4y ago
It's interesting that the dryer pops open and a few clothes spill out a second or so before we see the fire. Presumably it's the influx of oxygen that causes the fireball to erupt, but what causes the door to open?
440.
by Imnimo 4y ago
As if Elon has a clue what that feature is or is not being used for.
441.
by Imnimo 4y ago
Why are the example outputs in the Figures ungrammatical? >A text can describe the given image: a herd of giraffes and zebras grazing in a fields. In addition, there are five detected objects as giraffe with score 99.9%, zebra with score
442.
by Imnimo 4y ago
Bomb one. Tell my legion of LessWrong followers to do the same. The number I can personally bomb is irrelevant.
443.
by Imnimo 4y ago
Hmm, but if I'm reading this code correctly, it's also correct if the text of the correct answer appears anywhere in the output. Even if other incorrect answers also appear. https://github.com/manyoso/haltt4l
444.
by Imnimo 4y ago
I would be curious to see the log of correct answers for each model. It looks like the files in results/ are just incorrect answers? Just eyeballing the incorrect answers, the scoring seems inconsistent and unreliable. Consider this e
445.
by Imnimo 4y ago
I don't care at all about this from a copyright or data ownership perspective, but I am a little skeptical that it's a good idea to be this incestuous with training data in the long run. It's one thing to do fine tuning or kn
446.
by Imnimo 4y ago
The author's proposed solution is: Shut down all the large GPU clusters (the large computer farms where the most powerful AIs are refined). Shut down all the large training runs. Put a ceiling on how much computing power anyone is allo
447.
by Imnimo 4y ago
If Yudkowsky believes Israel's brand of preventative saboutage to be unethical, why does he advocate for exactly that style of measure to be taken against data centers in this very article?
448.
by Imnimo 4y ago
Israel believes that Iran having a nuclear weapon is an existential threat. What do they do? https://en.wikipedia.org/wiki/Assassination_of_Iranian_nucle... https://en.wikipedia.org/wiki/2020_Iran
449.
by Imnimo 4y ago
I think if I seriously thought all biological life was about to be destroyed, I could get over my distaste for bombing data centers.
450.
by Imnimo 4y ago
I feel like if I actually believed that the death of all biological life was imminent and that the only way to forestall it was to bomb data centers, I'd spend a lot more time planning how to bomb data centers and a lot less time curat

More ›