15 ms·

A much more interesting set of books (I've never managed to finish Blindsight due to Watts writing) is Ian Banks' Culture series.

buildbot 3y ago
Love the culture series. Though I would say it’s on a different tangent, in that AIs will absolutely eclipse us but will hopefully self align on being morally good because we are cute pets ;) What’s interesting in the culture series in light of the AI alignment debates is that the culture aligns every mind to its task at “birth”. A picket ship, falling outside the normal moral constraints, is a very different set of personalities and morals than a GSV such as wisdom like silence
piaste 3y ago
[dead]
jiggawatts 3y ago
I rate Iain M Banks as my favourite author of all time, but the AIs in his book have a very different "flavour" compared to the intelligent-but-not-sentient aliens and AIs in Blindsight. An interesting aspect of the Culture Minds is that they're described as hybrids of human-like / neural-net / organic intelligence and also direct numerical compute and storage capacity. They can both "think" and "compute", something few if any current AI models attempt. A well-known limitation of current LLMs like ChatGPT is that they are atrocious at even basic arithmetic, whereas a $2 pocket calculator from the 1980s outperforms them in this respect! I'd love to see an attempt at a hybrid architecture where the neurons are also connected to a numeric compute component of the architecture. Think of parsing an expression such as: let foo = 5 + 10 ChatGPT parses this as: "let", " foo", " =", " 5", " +", " 10" Essentially, its lexer produces a series of token IDs that don't contain the numeric values of "5" and "10": [1309, 22944, 796, 642, 1343, 838] Programming language lexers would parse that as: [LET, ID("foo"), EQ, NUM(5), PLUS, NUM(10)] In other words, lexer tokens in programming languages can carry baggage such as numeric values. What if we lifted this restriction and made an AI embedding where every token had both an ID and a value? For literal text just set the ID to NaN or whatever. E.g.: assume that "integer constant" is ID 1000, then an LLM might be fed this input instead: [1309, 22944, 796, 1000, 1343, 1000] [ NaN, NaN, NaN, 5, NaN, 10] Then the model could be wired up such that the additional tensor dimension can be sent to both the neural net or a direct compute block that does various kinds of maths, such as adding all inputs, multiplying all inputs, etc... Could be interesting...
- jazzyjackson 3y ago
  Sounds like ChatGPT with the Wolfram plug-in, no?
gpderetta 3y ago
They are very different kind of authors. Watts writes hard sci-fi, Banks wrote (extremely good) space-opera.

buildbot 3y ago

Love the culture series. Though I would say it’s on a different tangent, in that AIs will absolutely eclipse us but will hopefully self align on being morally good because we are cute pets ;) What’s interesting in the culture series in light of the AI alignment debates is that the culture aligns every mind to its task at “birth”. A picket ship, falling outside the normal moral constraints, is a very different set of personalities and morals than a GSV such as wisdom like silence

piaste 3y ago

[dead]

jiggawatts 3y ago

I rate Iain M Banks as my favourite author of all time, but the AIs in his book have a very different "flavour" compared to the intelligent-but-not-sentient aliens and AIs in Blindsight. An interesting aspect of the Culture Minds is that they're described as hybrids of human-like / neural-net / organic intelligence and also direct numerical compute and storage capacity. They can both "think" and "compute", something few if any current AI models attempt. A well-known limitation of current LLMs like ChatGPT is that they are atrocious at even basic arithmetic, whereas a $2 pocket calculator from the 1980s outperforms them in this respect! I'd love to see an attempt at a hybrid architecture where the neurons are also connected to a numeric compute component of the architecture. Think of parsing an expression such as: let foo = 5 + 10 ChatGPT parses this as: "let", " foo", " =", " 5", " +", " 10" Essentially, its lexer produces a series of token IDs that don't contain the numeric values of "5" and "10": [1309, 22944, 796, 642, 1343, 838] Programming language lexers would parse that as: [LET, ID("foo"), EQ, NUM(5), PLUS, NUM(10)] In other words, lexer tokens in programming languages can carry baggage such as numeric values. What if we lifted this restriction and made an AI embedding where every token had both an ID and a value? For literal text just set the ID to NaN or whatever. E.g.: assume that "integer constant" is ID 1000, then an LLM might be fed this input instead: [1309, 22944, 796, 1000, 1343, 1000] [ NaN, NaN, NaN, 5, NaN, 10] Then the model could be wired up such that the additional tensor dimension can be sent to both the neural net or a direct compute block that does various kinds of maths, such as adding all inputs, multiplying all inputs, etc... Could be interesting...

jazzyjackson 3y ago
Sounds like ChatGPT with the Wolfram plug-in, no?

gpderetta 3y ago

They are very different kind of authors. Watts writes hard sci-fi, Banks wrote (extremely good) space-opera.