8 ms·

Cory Doctorow wants you to know what computers can and can’t do

166 points by cocacola1 4y ago

deleted 4y ago
[deleted]
amelius 4y ago
- throwup 4y ago
  The title is talking about end-users' control over their own hardware. Nothing to do with GPT-3.
  - cokeandpepsi 4y ago
- tdehnel 4y ago
  Not only is this not relevant to the article, it’s also wrong. Computers cannot create new explanatory knowledge like humans can. One day they may. But right now we have 0 clue how this happens in humans so we are not able to program it in computers.
  - CamperBob2 4y ago
    God of the Gaps, in other words. "At least computers can't do that. Only humans can do that." (GPT-3 demonstrably does that, or behaves in a manner indistinguishable from doing that) "Well, at least computers can't do this other thing ..." But right now we have 0 clue how this happens in humans so we are not able to program it in computers. The notion that humans must understand the programs that their machines are running is by no means a necessary one. Can AlphaGo's human programmers predict, debug, or even explain every move it made against Lee Sedol, for instance?
    - wizzwizz4 4y ago
      > God of the Gaps, in other words. No, not really. Make a sufficiently-powerful predictive text engine, and you get something like GPT-3. Make a sufficiently-powerful expert system, and you get a chess engine that can beat every grandmaster. These things were predicted in sci-fi novels with descriptions of their algorithms decades before we had computers powerful enough to execute them. For things like "intelligence" and "consciousness", the best we've got is on the order of Isaac Asimov's positronic brain: a total handwave, maybe with some technobabble or some oblique references to now-outdated models of psychology. The stuff humans can do that computers can't, are the things we don't have algorithms for, and – funnily enough – things that "just throw a bigger neural net at it" don't seem to be actually solving (however much they're getting better at fooling people into believing they're solving). "We might be able to solve this one day" doesn't mean we've already solved it.
      - CamperBob2 4y ago
        The stuff humans can do that computers can't, are the things we don't have algorithms for What are some examples? GPT-3 is already composing halfway-credible poetry and songs. (Edit: and it means nothing to say that "poetry and songs exist in its training data." The songs and poems it just gave me didn't.) I just asked it to write some code to walk a k-d tree, and it did that, too, complete with a unit test. Don't focus on the current state of the art; pay attention to its time derivative. That's all that matters. When the tech gets 10x better, it will be almost indistinguishable from human thought. Another 10x will tie up the remaining loose ends.
        wizzwizz4 4y ago
        > GPT-3 is already composing halfway-credible poetry and songs. Poetry and songs abound in its training data. > I just asked it to write some code to walk a k-d tree, and it did that, too, complete with a unit test. Such things abound in its training data. GPT-3 can't produce anything novel. It's a search engine over natural generalisations of its training data, but it can't think for itself. It couldn't solve any of the unsolved problems of our day. Give it a maths problem with a known solution, and it might give you a half-way decent proof. Give it a problem without a known solution – even an easier one – and it'll flounder.
  - JadeNB 4y ago
    > But right now we have 0 clue how this happens in humans so we are not able to program it in computers. Without taking a position on whether or not computers can create new exploratory knowledge (how could one begin to say authoritatively that they can't?), this "so" doesn't follow. The fact that we don't know how to do something doesn't mean we can't program it in computers.
- detaro 4y ago
  Could you elaborate how GPT-3 is relevant to the things discussed in the article?
deleted 4y ago
[deleted]
nonrandomstring 4y ago
Cory gets it, that computing is a battleground for the future. Code is war by other means. We will all be forced to pick a side. It would be wise to familiarise ourselves with the lay of the land.
- criddell 4y ago
  What are the sides?
  - mjh2539 4y ago
    The manufacturing of microprocessors and other ICs requires such an enormous amount of capital that all sides are practically indistinguishable. But more realistically, if you have access to the hardware, in principle, you can do whatever you want.
  - mlyle 4y ago
    Too many issues and too many sides. Too much grey within each debate, too. Computing appliances vs. general purpose machines. End-user control vs. vendor. Privacy vs. transparency. Freedom of expression vs. curated/"safer" experiences. Algorithms vs. humans in the loop. Governmental vs. corporatist vs. end-user freedoms. Cloud/timeshare/virtualization vs. on-premises/own your own machine/etc. Perpetual copyright and central ownership of culture vs. information wants to be free. Security vs. flexibility. Not to mention after you get past the whole ideological debates, you have active state vs. state cyberwarfare and information warfare and other proxy battles... and cybercrime. Computing and networking as mass-market phenomena is still new. As much as we pat ourselves on the back in tech for rapid progress, the social mechanisms and business models for all of this are still unstable.
    - sangnoir 4y ago
      > Freedom of expression vs. curated/"safer" experiences I would tweak this to user-controlled curation vs platform-controlled curation. By itself, curation is not opposed to freedom of expression, it can be transparent (incoming emails sent to your spam folder) or hidden from end-users view.
macintux 4y ago
Active discussion: https://news.ycombinator.com/item?id=33854184 https://news.ycombinator.com/item?id=33854184
californiadreem 4y ago
I have not read the article yet, but this is what I predict the contents will be, namely "Cory Doctorow Wants You to Know He Has a New Book Coming Out." I predict it will be an interview where Doctorow's trenchant commentary on how the emerging cyberdystopia can be meaningfully thwarted is by buying yet another novel by him about how a new technology is only being wielded incorrectly and if only the plucky hackers use it in transgressive ways, it will undermine the Institutional foundations of the technological order and not reinforce the status quo.
- bo1024 4y ago
  It's been an hour, did you read it yet? I would rate your prediction "mostly false". Interesting comments on Big Tech and politics, blockchain, development of cyberpunk, future of AI, etc.
- californiadreem 4y ago
  > "In your next novel, “Red Team Blues,” you focus on Martin Hench, a sixty-seven-year-old forensic accountant, who is tasked with recovering a set of stolen signing keys that, with some technical finagling, can permit one to rewrite a blockchain’s distributed ledger, swiping assets from one side to the other, as it were. Do you think blockchain tech is less secure than enthusiasts portray it to be?" Color me shocked.
  - deleted 4y ago
    [deleted]
  - culi 4y ago
    This is like every article/interview with authors ever. I'm not sure how much you listen to podcasts but most of the time when an author is being interviewed they have a new book coming out. Doing a round of interviews is one of the main ways authors raise awareness about their new books. Digital magazines, podcasts, talk shows, reddit AMAs, etc are all avenues for raising awareness that are actively encouraged by book publishers
    - 4y ago
photochemsyn 4y ago
Retro-computing is the most promising trend I've seen in the computing tech world. Open source hardware designs based on the RISC-V instruction set, returning to writing low-level firmware for RISC-V boards, using a low-level language with an open-source compiler, and making that approach the norm for all simple devices, seems to be a huge growth area. Built-in manual overrides, direct access to device internals, the return of knowledge that's on the verge of being lost to so many American consumers, it all looks good to me. Incidentally, this article uses the phrase 'cyberpunk' a lot but that genre is a lot more like 'cybernoir'. Look at for example Jean Pierre Melville films: https://www.currentaffairs.org/2021/06/the-noirs-of-melville https://www.currentaffairs.org/2021/06/the-noirs-of-melville
nuc1e0n 4y ago
Doctorow groks the issues of the technological world we live in today much better than most other commentators.
elgar1212 4y ago
> Well, Big Tech is not the only concentrated industry. A bunch of concentrated industries use Big Tech antitrust as a pretext for going after Big Tech—not to end monopoly but to redistribute the industry’s share of the monopoly themselves. So, cable operators, phone companies, entertainment companies. I think it’s fair to say that the big entertainment companies don’t want to kill Google; they just want to take it over. Some of the energy that comes up to break up or tame Big Tech is coming from other sectors that are every bit as much in need of taming and breakup as Big Tech is. This is the most notable bit of information in the article The whole big tech thing has three camps: big tech itself, other corporations like telecoms which want to buy out tech companies like verizon bought out yahoo, and politicians in DC who want to blackmail tech companies and control the flow of information[0] government regulation is a double edged sword, and wherever there's a bunch of money there's a bunch of people who want to take it [0] https://theintercept.com/2022/10/31/social-media-disinformation-dhs/ https://theintercept.com/2022/10/31/social-media-disinformat...
1vuio0pswjnm7 4y ago
"Crypto is weird, because, much more so than other technologies, if you don't like crypto, crypto people really want to convince you that you're wrong." For some, this is a difficult-to-ignore red flag.
- mwattsun 4y ago
  A fanatic is a person who consciously over-compensates a secret doubt. - Aldous Huxley
- nicbou 4y ago
  They tend to have skin in the game. It's not just an opinion, it's an investment.
leashless 4y ago
His stance on crypto is bizarre: it’s just time stamped digital signatures on statements of intent to pay. General purpose computers will always do cryptocurrency if people want them to.
notpushkin 4y ago
https://archive.ph/OBdL7 https://archive.ph/OBdL7
x-complexity 4y ago
...I can't ignore the technical inaccuracy here: > Ethereum is a project based around decentralized applications, which run on a scattered network of computers and don’t have a single owner who controls them. That would seem to be in line with what you want for the Internet, in the sense of more interoperability and more security. Or am I wrong? (Interviewer essentially asks "What do you think about Ethereum?") > I think distributed apps are a great idea. I am skeptical of smart contracts, which are the building block of distributed apps. Smart contracts are hard to get right. And this is not a thing that you can fix. *There’s this foundational idea in computer science called the halting problem, which says that, above a pretty minimal threshold, it’s impossible to know all the different ways that a program can behave.* One of the ways that computer scientists try to address this risk is by keeping Undo buttons around in our code. (The rest of the response goes towards the need for reversibility & how smart contracts don't allow for that, even through it's possible to implement tokens with reversible transactions (see ERC20R & ERC721R).) This was the point that I knew that Cory didn't even spend the bare minimum amount of effort to Google "Ethereum halting problem". https://jcrouser.github.io/CSC250/projects/etherium.html#:~:text=How%20does%20the%20Halting%20Problem%20relate%20to%20Ethereum%20and%20smart%20contracts%3F https://jcrouser.github.io/CSC250/projects/etherium.html#:~:... https://ethereum.org/en/developers/docs/gas/#why-do-gas-fees-exist https://ethereum.org/en/developers/docs/gas/#why-do-gas-fees... If he had even tried to do so, he would've known that Ethereum's solution to this problem is the gas limit, wherein: - A set limit on how much gas can be used is submitted in the transaction - If the limit is exceeded, the transaction fails & the state is reversed, as if there Most of what he otherwise says is passable, all except for that part. Instead of admitting a lack of technical knowledge about the subject matter at hand, he instead goes on to espouse how reversibility at the base layer is required in order to create a useful system, instead of trying to make a system that doesn't need reversibility at the base layer in the first place.

166 points by cocacola1 4y ago

deleted 4y ago

[deleted]

amelius 4y ago

throwup 4y ago
The title is talking about end-users' control over their own hardware. Nothing to do with GPT-3.
- cokeandpepsi 4y ago
tdehnel 4y ago
Not only is this not relevant to the article, it’s also wrong. Computers cannot create new explanatory knowledge like humans can. One day they may. But right now we have 0 clue how this happens in humans so we are not able to program it in computers.
- CamperBob2 4y ago
  God of the Gaps, in other words. "At least computers can't do that. Only humans can do that." (GPT-3 demonstrably does that, or behaves in a manner indistinguishable from doing that) "Well, at least computers can't do this other thing ..." But right now we have 0 clue how this happens in humans so we are not able to program it in computers. The notion that humans must understand the programs that their machines are running is by no means a necessary one. Can AlphaGo's human programmers predict, debug, or even explain every move it made against Lee Sedol, for instance?
  - wizzwizz4 4y ago
    > God of the Gaps, in other words. No, not really. Make a sufficiently-powerful predictive text engine, and you get something like GPT-3. Make a sufficiently-powerful expert system, and you get a chess engine that can beat every grandmaster. These things were predicted in sci-fi novels with descriptions of their algorithms decades before we had computers powerful enough to execute them. For things like "intelligence" and "consciousness", the best we've got is on the order of Isaac Asimov's positronic brain: a total handwave, maybe with some technobabble or some oblique references to now-outdated models of psychology. The stuff humans can do that computers can't, are the things we don't have algorithms for, and – funnily enough – things that "just throw a bigger neural net at it" don't seem to be actually solving (however much they're getting better at fooling people into believing they're solving). "We might be able to solve this one day" doesn't mean we've already solved it.
    - CamperBob2 4y ago
      The stuff humans can do that computers can't, are the things we don't have algorithms for What are some examples? GPT-3 is already composing halfway-credible poetry and songs. (Edit: and it means nothing to say that "poetry and songs exist in its training data." The songs and poems it just gave me didn't.) I just asked it to write some code to walk a k-d tree, and it did that, too, complete with a unit test. Don't focus on the current state of the art; pay attention to its time derivative. That's all that matters. When the tech gets 10x better, it will be almost indistinguishable from human thought. Another 10x will tie up the remaining loose ends.
      - wizzwizz4 4y ago
        > GPT-3 is already composing halfway-credible poetry and songs. Poetry and songs abound in its training data. > I just asked it to write some code to walk a k-d tree, and it did that, too, complete with a unit test. Such things abound in its training data. GPT-3 can't produce anything novel. It's a search engine over natural generalisations of its training data, but it can't think for itself. It couldn't solve any of the unsolved problems of our day. Give it a maths problem with a known solution, and it might give you a half-way decent proof. Give it a problem without a known solution – even an easier one – and it'll flounder.
- JadeNB 4y ago
  > But right now we have 0 clue how this happens in humans so we are not able to program it in computers. Without taking a position on whether or not computers can create new exploratory knowledge (how could one begin to say authoritatively that they can't?), this "so" doesn't follow. The fact that we don't know how to do something doesn't mean we can't program it in computers.
detaro 4y ago
Could you elaborate how GPT-3 is relevant to the things discussed in the article?

deleted 4y ago

[deleted]

nonrandomstring 4y ago

Cory gets it, that computing is a battleground for the future. Code is war by other means. We will all be forced to pick a side. It would be wise to familiarise ourselves with the lay of the land.

criddell 4y ago
What are the sides?
- mjh2539 4y ago
  The manufacturing of microprocessors and other ICs requires such an enormous amount of capital that all sides are practically indistinguishable. But more realistically, if you have access to the hardware, in principle, you can do whatever you want.
- mlyle 4y ago
  Too many issues and too many sides. Too much grey within each debate, too. Computing appliances vs. general purpose machines. End-user control vs. vendor. Privacy vs. transparency. Freedom of expression vs. curated/"safer" experiences. Algorithms vs. humans in the loop. Governmental vs. corporatist vs. end-user freedoms. Cloud/timeshare/virtualization vs. on-premises/own your own machine/etc. Perpetual copyright and central ownership of culture vs. information wants to be free. Security vs. flexibility. Not to mention after you get past the whole ideological debates, you have active state vs. state cyberwarfare and information warfare and other proxy battles... and cybercrime. Computing and networking as mass-market phenomena is still new. As much as we pat ourselves on the back in tech for rapid progress, the social mechanisms and business models for all of this are still unstable.
  - sangnoir 4y ago
    > Freedom of expression vs. curated/"safer" experiences I would tweak this to user-controlled curation vs platform-controlled curation. By itself, curation is not opposed to freedom of expression, it can be transparent (incoming emails sent to your spam folder) or hidden from end-users view.

macintux 4y ago

Active discussion: https://news.ycombinator.com/item?id=33854184 https://news.ycombinator.com/item?id=33854184

californiadreem 4y ago

I have not read the article yet, but this is what I predict the contents will be, namely "Cory Doctorow Wants You to Know He Has a New Book Coming Out." I predict it will be an interview where Doctorow's trenchant commentary on how the emerging cyberdystopia can be meaningfully thwarted is by buying yet another novel by him about how a new technology is only being wielded incorrectly and if only the plucky hackers use it in transgressive ways, it will undermine the Institutional foundations of the technological order and not reinforce the status quo.

bo1024 4y ago
It's been an hour, did you read it yet? I would rate your prediction "mostly false". Interesting comments on Big Tech and politics, blockchain, development of cyberpunk, future of AI, etc.
californiadreem 4y ago
> "In your next novel, “Red Team Blues,” you focus on Martin Hench, a sixty-seven-year-old forensic accountant, who is tasked with recovering a set of stolen signing keys that, with some technical finagling, can permit one to rewrite a blockchain’s distributed ledger, swiping assets from one side to the other, as it were. Do you think blockchain tech is less secure than enthusiasts portray it to be?" Color me shocked.
- deleted 4y ago
  [deleted]
- culi 4y ago
  This is like every article/interview with authors ever. I'm not sure how much you listen to podcasts but most of the time when an author is being interviewed they have a new book coming out. Doing a round of interviews is one of the main ways authors raise awareness about their new books. Digital magazines, podcasts, talk shows, reddit AMAs, etc are all avenues for raising awareness that are actively encouraged by book publishers
  - 4y ago

photochemsyn 4y ago

Retro-computing is the most promising trend I've seen in the computing tech world. Open source hardware designs based on the RISC-V instruction set, returning to writing low-level firmware for RISC-V boards, using a low-level language with an open-source compiler, and making that approach the norm for all simple devices, seems to be a huge growth area. Built-in manual overrides, direct access to device internals, the return of knowledge that's on the verge of being lost to so many American consumers, it all looks good to me. Incidentally, this article uses the phrase 'cyberpunk' a lot but that genre is a lot more like 'cybernoir'. Look at for example Jean Pierre Melville films: https://www.currentaffairs.org/2021/06/the-noirs-of-melville https://www.currentaffairs.org/2021/06/the-noirs-of-melville

nuc1e0n 4y ago

Doctorow groks the issues of the technological world we live in today much better than most other commentators.

elgar1212 4y ago

> Well, Big Tech is not the only concentrated industry. A bunch of concentrated industries use Big Tech antitrust as a pretext for going after Big Tech—not to end monopoly but to redistribute the industry’s share of the monopoly themselves. So, cable operators, phone companies, entertainment companies. I think it’s fair to say that the big entertainment companies don’t want to kill Google; they just want to take it over. Some of the energy that comes up to break up or tame Big Tech is coming from other sectors that are every bit as much in need of taming and breakup as Big Tech is. This is the most notable bit of information in the article The whole big tech thing has three camps: big tech itself, other corporations like telecoms which want to buy out tech companies like verizon bought out yahoo, and politicians in DC who want to blackmail tech companies and control the flow of information[0] government regulation is a double edged sword, and wherever there's a bunch of money there's a bunch of people who want to take it [0] https://theintercept.com/2022/10/31/social-media-disinformation-dhs/ https://theintercept.com/2022/10/31/social-media-disinformat...

1vuio0pswjnm7 4y ago

"Crypto is weird, because, much more so than other technologies, if you don't like crypto, crypto people really want to convince you that you're wrong." For some, this is a difficult-to-ignore red flag.

mwattsun 4y ago
A fanatic is a person who consciously over-compensates a secret doubt. - Aldous Huxley
nicbou 4y ago
They tend to have skin in the game. It's not just an opinion, it's an investment.

leashless 4y ago

His stance on crypto is bizarre: it’s just time stamped digital signatures on statements of intent to pay. General purpose computers will always do cryptocurrency if people want them to.

notpushkin 4y ago

https://archive.ph/OBdL7 https://archive.ph/OBdL7

x-complexity 4y ago

...I can't ignore the technical inaccuracy here: > Ethereum is a project based around decentralized applications, which run on a scattered network of computers and don’t have a single owner who controls them. That would seem to be in line with what you want for the Internet, in the sense of more interoperability and more security. Or am I wrong? (Interviewer essentially asks "What do you think about Ethereum?") > I think distributed apps are a great idea. I am skeptical of smart contracts, which are the building block of distributed apps. Smart contracts are hard to get right. And this is not a thing that you can fix. *There’s this foundational idea in computer science called the halting problem, which says that, above a pretty minimal threshold, it’s impossible to know all the different ways that a program can behave.* One of the ways that computer scientists try to address this risk is by keeping Undo buttons around in our code. (The rest of the response goes towards the need for reversibility & how smart contracts don't allow for that, even through it's possible to implement tokens with reversible transactions (see ERC20R & ERC721R).) This was the point that I knew that Cory didn't even spend the bare minimum amount of effort to Google "Ethereum halting problem". https://jcrouser.github.io/CSC250/projects/etherium.html#:~:text=How%20does%20the%20Halting%20Problem%20relate%20to%20Ethereum%20and%20smart%20contracts%3F https://jcrouser.github.io/CSC250/projects/etherium.html#:~:... https://ethereum.org/en/developers/docs/gas/#why-do-gas-fees-exist https://ethereum.org/en/developers/docs/gas/#why-do-gas-fees... If he had even tried to do so, he would've known that Ethereum's solution to this problem is the gas limit, wherein: - A set limit on how much gas can be used is submitted in the transaction - If the limit is exceeded, the transaction fails & the state is reversed, as if there Most of what he otherwise says is passable, all except for that part. Instead of admitting a lack of technical knowledge about the subject matter at hand, he instead goes on to espouse how reversibility at the base layer is required in order to create a useful system, instead of trying to make a system that doesn't need reversibility at the base layer in the first place.