6 ms·

This sentence caught my attention: > You can’t just Google these kinds of things. That’s why they’re so interesting. So what Google did is dilute the informat

This sentence caught my attention: > You can’t just Google these kinds of things. That’s why they’re so interesting. So what Google did is dilute the information advantage. The more interconnected we are, the harder it becomes to maintain that advantage.

hawski 3y ago
That's also something I often encounter with ChatGPT. It often is very wrong about things I ask, because I often wonder and try things off the beaten path. That's our advantage against LLMs.
- from-nibly 3y ago
  No the advantage is that you have context. That context is HUGE. That's why you can't and won't be able to have chat gpt actually do your job for you. You still need to ask it questions. And to even know what questions you need to ask you need to have context.
  - Jensson 3y ago
    You don't just need a context, you need to finetune a model around that context since working memory isn't enough. That is what human workers do, they spend the first months finetuning their model around the company context before they can start to provide valuable code. Once you can pay to get your own fine tuned version of the big LLMs maybe we can start to do real work with these.
    - from-nibly 3y ago
      Agreed on the first part. However my point is that having the context and the ability to use it is sooooooo far out of reach for a computer it's unfathomable. In much the same way how life is unfathomable at some level. It's this self referencing infinity that is somehow collapsing into results while being uncollapseable. Computers can't do that not without something fundamentally different
- adtac 3y ago
  Purely probabilistically, trying things off the beaten path is just a matter of higher LLM temperature. Turning up GPT-4's temperature is basically an expensive /dev/urandom pipe today, but I don't see any fundamental reason why LLMs can't catch up. Maybe all it takes is tinkering with how temperature is calculated.
  - ben_w 3y ago
    You've missed the metaphor, I think. Higher temperatures will make it "more creative" for lack of a better term, but there's a lot of specialist knowledge it doesn't have and which you can't give it just by twiddling a dial. It has a massive speed advantage that lets it read the whole internet, but it's dumb enough that it also needs to, and when even that doesn't give it enough examples, it's like asking me to improvise Latin based on what I recognise of the loanwords in English.
  - jwilber 3y ago
    No - temperature is a sampling measurement from a distribution. If you adjust temperature, you still ultimately pull from the same distribution. Parent is saying the advantage is in pulling ideas from outside that encoded distribution.
cjohnson318 3y ago
The fact is that we're not that interconnected though. Finding people that have useful, actionable information, and want to share it with you, is extremely difficult.
- Jensson 3y ago
  That is a search issue not an interconnectivity issue.
- fxtentacle 3y ago
  Or maybe we have just conditioned people to not share useful information publicly anymore. Amazon selling what used to be open source projects has certainly had a chilling effect on people publishing source code under a permissive licence.
ssivark 3y ago
Google, ChatGPT, Gemini, Perplexity, etc are all retrieval services (with some rudimentary remixing capabilities thrown in, for the latter few). Some human has to actually have interesting insights and write them. Retrieval services help with distribution & discovery. The less we’re connected, the harder it is to find the right audience who value your (niche) insights. Of course attribution and linking is important.
- wordpad25 3y ago
  AI so much much more than retrieval since it knows how to correctly apply exist concepts to previously unexplored areas. You can ask ChatGPT how one might implement and enforce laws in a world where magic exists. Or time travel. Or whatever. Because it knows laws and it knows common fiction tropes and can reason well enough to surmise how to apply them together.
  - wordpad25 3y ago
    Wanted to give it a try. I didn't even consider many of these and doubt I would've come up with such a good list even if I had a lot of time. Much less than 1 second it took to generate the response. In a world where magic exists, laws and the legislative system would likely undergo significant adaptations to address the unique capabilities and risks presented by magic. Here are several ways in which these systems might be different, based on various assumptions about the nature and scope of magic: 1. *Regulation of Magical Practices*: Just as with professions and practices in our world, there would be regulations governing who can practice magic, how it can be learned, and where it can be used. Licenses or permits might be required for practicing certain types of magic, similar to medical licenses or driving licenses. 2. *Magic-Related Offenses*: New categories of crimes would exist, such as unauthorized enchantments, illegal curses, magical fraud (using illusions or mind control for deceit), and unlicensed necromancy. Laws would specify what constitutes a magical assault versus a non-magical one, with potentially different penalties. 3. *Enforcement and Jurisdiction*: Special law enforcement units trained in counter-magic or magical defense would be necessary to handle crimes involving magic. The jurisdiction might depend on the nature of the magic used; for example, cross-dimensional or time-altering magic could involve a higher authority or international magical council. 4. *Evidence and Investigation*: The collection of evidence might include magical means, such as truth spells, scrying (viewing distant or future events), or reviewing memories directly. However, there would also need to be laws protecting individuals against invasive magical investigations. 5. *Rights and Protections*: There would be laws to protect non-magical beings from discrimination or harm by magical individuals or entities. Similarly, sentient magical creatures or beings might have their own rights, leading to more complex legal considerations regarding interspecies interactions. 6. *Ethical Use of Magic*: Legislation would likely address the ethical implications of spellcasting, summoning, and other magical practices, including the use of magic on unwilling subjects or creating sentient beings. Regulations might prevent or restrict the use of magic that can alter someone's will, memories, or identity. 7. *Magical Contracts and Agreements*: The legal system would need to address the binding nature of magical contracts, which could have irrevocable consequences. Laws might specify how such contracts can be fairly negotiated and what remedies exist for breaking a magical contract. 8. *Intellectual Property*: In a world where spells, potions, and magical items are created, there would be intellectual property laws protecting the creators of magical inventions. This could include patents on spell formulas, copyright on magical texts, or trademarks on magical item designs. 9. *International Relations and Security*: On a larger scale, there might be international laws governing the use of magic, especially in conflicts or espionage. Treaties could limit the use of particularly destructive spells or ban the creation of magical weapons of mass destruction. 10. *Education and Public Safety*: Laws would govern the education and training of individuals with magical abilities, possibly requiring mandatory schooling to ensure that magic is used safely and responsibly. Public safety regulations would address the storage of magical materials and the construction of magically protected buildings. In this magical world, the legal and legislative system would need to be flexible and innovative, capable of addressing the ever-evolving nature of magic and its impact on society. It would be a blend of traditional legal principles and new rules designed to manage the unique challenges posed by the existence of magic.
chubot 3y ago
If you want to maintain an information advantage, you should definitely not rely solely on the web. Simply reading a book is often 10x better. Comment from a few months ago: https://news.ycombinator.com/item?id=38369583 https://news.ycombinator.com/item?id=38369583 One thing I've found comically underestimated is books. It's not whether the information is physical or electronic, but what actual corpus of information is available ... So yeah I'd say 3 main repos of knowledge are: the open Internet, printed books, and Scihub, and many people today only use the first one Though I'd also put "talking to the right people" and by extension "working at the right place" pretty high up there in getting novel information. I think for a certain generation, Google probably diluted their ability to access other knowledge bases, and act upon that info. And for the generation currently learning how to learn, LLMs may do the same thing.

hawski 3y ago

That's also something I often encounter with ChatGPT. It often is very wrong about things I ask, because I often wonder and try things off the beaten path. That's our advantage against LLMs.

from-nibly 3y ago
No the advantage is that you have context. That context is HUGE. That's why you can't and won't be able to have chat gpt actually do your job for you. You still need to ask it questions. And to even know what questions you need to ask you need to have context.
- Jensson 3y ago
  You don't just need a context, you need to finetune a model around that context since working memory isn't enough. That is what human workers do, they spend the first months finetuning their model around the company context before they can start to provide valuable code. Once you can pay to get your own fine tuned version of the big LLMs maybe we can start to do real work with these.
  - from-nibly 3y ago
    Agreed on the first part. However my point is that having the context and the ability to use it is sooooooo far out of reach for a computer it's unfathomable. In much the same way how life is unfathomable at some level. It's this self referencing infinity that is somehow collapsing into results while being uncollapseable. Computers can't do that not without something fundamentally different
adtac 3y ago
Purely probabilistically, trying things off the beaten path is just a matter of higher LLM temperature. Turning up GPT-4's temperature is basically an expensive /dev/urandom pipe today, but I don't see any fundamental reason why LLMs can't catch up. Maybe all it takes is tinkering with how temperature is calculated.
- ben_w 3y ago
  You've missed the metaphor, I think. Higher temperatures will make it "more creative" for lack of a better term, but there's a lot of specialist knowledge it doesn't have and which you can't give it just by twiddling a dial. It has a massive speed advantage that lets it read the whole internet, but it's dumb enough that it also needs to, and when even that doesn't give it enough examples, it's like asking me to improvise Latin based on what I recognise of the loanwords in English.
- jwilber 3y ago
  No - temperature is a sampling measurement from a distribution. If you adjust temperature, you still ultimately pull from the same distribution. Parent is saying the advantage is in pulling ideas from outside that encoded distribution.

cjohnson318 3y ago

The fact is that we're not that interconnected though. Finding people that have useful, actionable information, and want to share it with you, is extremely difficult.

Jensson 3y ago
That is a search issue not an interconnectivity issue.
fxtentacle 3y ago
Or maybe we have just conditioned people to not share useful information publicly anymore. Amazon selling what used to be open source projects has certainly had a chilling effect on people publishing source code under a permissive licence.

ssivark 3y ago

Google, ChatGPT, Gemini, Perplexity, etc are all retrieval services (with some rudimentary remixing capabilities thrown in, for the latter few). Some human has to actually have interesting insights and write them. Retrieval services help with distribution & discovery. The less we’re connected, the harder it is to find the right audience who value your (niche) insights. Of course attribution and linking is important.

wordpad25 3y ago
AI so much much more than retrieval since it knows how to correctly apply exist concepts to previously unexplored areas. You can ask ChatGPT how one might implement and enforce laws in a world where magic exists. Or time travel. Or whatever. Because it knows laws and it knows common fiction tropes and can reason well enough to surmise how to apply them together.
- wordpad25 3y ago
  Wanted to give it a try. I didn't even consider many of these and doubt I would've come up with such a good list even if I had a lot of time. Much less than 1 second it took to generate the response. In a world where magic exists, laws and the legislative system would likely undergo significant adaptations to address the unique capabilities and risks presented by magic. Here are several ways in which these systems might be different, based on various assumptions about the nature and scope of magic: 1. *Regulation of Magical Practices*: Just as with professions and practices in our world, there would be regulations governing who can practice magic, how it can be learned, and where it can be used. Licenses or permits might be required for practicing certain types of magic, similar to medical licenses or driving licenses. 2. *Magic-Related Offenses*: New categories of crimes would exist, such as unauthorized enchantments, illegal curses, magical fraud (using illusions or mind control for deceit), and unlicensed necromancy. Laws would specify what constitutes a magical assault versus a non-magical one, with potentially different penalties. 3. *Enforcement and Jurisdiction*: Special law enforcement units trained in counter-magic or magical defense would be necessary to handle crimes involving magic. The jurisdiction might depend on the nature of the magic used; for example, cross-dimensional or time-altering magic could involve a higher authority or international magical council. 4. *Evidence and Investigation*: The collection of evidence might include magical means, such as truth spells, scrying (viewing distant or future events), or reviewing memories directly. However, there would also need to be laws protecting individuals against invasive magical investigations. 5. *Rights and Protections*: There would be laws to protect non-magical beings from discrimination or harm by magical individuals or entities. Similarly, sentient magical creatures or beings might have their own rights, leading to more complex legal considerations regarding interspecies interactions. 6. *Ethical Use of Magic*: Legislation would likely address the ethical implications of spellcasting, summoning, and other magical practices, including the use of magic on unwilling subjects or creating sentient beings. Regulations might prevent or restrict the use of magic that can alter someone's will, memories, or identity. 7. *Magical Contracts and Agreements*: The legal system would need to address the binding nature of magical contracts, which could have irrevocable consequences. Laws might specify how such contracts can be fairly negotiated and what remedies exist for breaking a magical contract. 8. *Intellectual Property*: In a world where spells, potions, and magical items are created, there would be intellectual property laws protecting the creators of magical inventions. This could include patents on spell formulas, copyright on magical texts, or trademarks on magical item designs. 9. *International Relations and Security*: On a larger scale, there might be international laws governing the use of magic, especially in conflicts or espionage. Treaties could limit the use of particularly destructive spells or ban the creation of magical weapons of mass destruction. 10. *Education and Public Safety*: Laws would govern the education and training of individuals with magical abilities, possibly requiring mandatory schooling to ensure that magic is used safely and responsibly. Public safety regulations would address the storage of magical materials and the construction of magically protected buildings. In this magical world, the legal and legislative system would need to be flexible and innovative, capable of addressing the ever-evolving nature of magic and its impact on society. It would be a blend of traditional legal principles and new rules designed to manage the unique challenges posed by the existence of magic.

chubot 3y ago

If you want to maintain an information advantage, you should definitely not rely solely on the web. Simply reading a book is often 10x better. Comment from a few months ago: https://news.ycombinator.com/item?id=38369583 https://news.ycombinator.com/item?id=38369583 One thing I've found comically underestimated is books. It's not whether the information is physical or electronic, but what actual corpus of information is available ... So yeah I'd say 3 main repos of knowledge are: the open Internet, printed books, and Scihub, and many people today only use the first one Though I'd also put "talking to the right people" and by extension "working at the right place" pretty high up there in getting novel information. I think for a certain generation, Google probably diluted their ability to access other knowledge bases, and act upon that info. And for the generation currently learning how to learn, LLMs may do the same thing.