4 ms·

AI not being open is my own personal source of sadness. The mathematics involved aren't out of reach, but the computing power and _personal time_ necessary to b

AI not being open is my own personal source of sadness. The mathematics involved aren't out of reach, but the computing power and _personal time_ necessary to build a dataset and train a model are absolutely beyond my reach. It's unnerving that such a revolutionary new technology in computing is fundamentally tethered to large binary blobs, usually proprietary or of uncertain provenance. Which is not like computing advances that we have hitherto enjoyed; even technologies for the operations of large data centres and distributed systems could at least be deployed and toyed with on virtually all common consumer hardware. Perhaps it's because thus far the realm of Big Data wasn't alluring enough to draw in myself and the similarly minded. It's only now that a certain form of large data set has become tied to an interesting application that it's drawn my attention. Not to say that Big Data and AI are one and the same; only that they both deal with large-in-context data sets that are difficult to construct, acquire, and manipulate.

johnsutor 3y ago
I'm hopeful that this will get better as time goes on with advances in unsupervised and multimodal learning, that will enable us mere pions to achieve our niche use cases by fine-tuning a larger model with minimal compute.
seydor 3y ago
In economic terms, it feels like we the people won't have much of a chance at AI wealth. Software was infinitely scalable, easy to make and powerful, but AI is data with tiny amounts of code, and machine learning engineers are going to be a dime a dozen soon. So hardware companies will reap most of the benefits, and making it with a a garage startup is hard
fritzo 3y ago
This attitude is like the rugged forest hermit's aversion to computers because silicon wafer production is out of reach for individuals.
perryizgr8 3y ago
> fundamentally tethered to large binary blobs In many instances you don't even have the blob, you can just call an API that invokes it and runs the output thru multiple other hidden algorithms before sending it to you.
geor9e 3y ago
You're describing A.I. as a black box of proprietary workings, far beyond any mortal's personal budget to create, and too complex anyway, which internally runs in a language unintelligible to humans, and through layers and layers of abstractions, has a human-readable interface. To me, that exactly describes an x64 or ARM CPU. But since we are not children freshly learning about reality, we don't see these two black boxes in front of us in the same light. One is something we just got used to kinda-understanding-but-not-really decade after decade. One is something brand new to us.
- dleslie 3y ago
  x64 and ARM emulators exist, the code for some are quite legible. There's a whole community of folks who build RiscV implementations from verilog or vhdl. In my own experience at university, decades ago, we built functioning HC11 CPUs out of logic gates and wrote assembler for them. I'll admit that there's a definite and clear accessibility curve to building a CPU, but it's nowhere near as opaque as the binary blobs one can access for AI tooling.
  - entropicdrifter 3y ago
    Now I want to go and play NandGame again... https://nandgame.com/ https://nandgame.com/
    - paulmd 3y ago
      I hear really good things about turing complete https://store.steampowered.com/app/1444480/Turing_Complete/ https://store.steampowered.com/app/1444480/Turing_Complete/
  - deleted 3y ago
    [deleted]
- asadotzler 3y ago
  X86 and ARM are open in many many ways that for example ChatGPT isn't.
saltcured 3y ago
This is much how numerical computing has always been. We could individually approach the math and algorithms, but need major resources and input data to apply it to real-world problem like weather forecasts, geophysical simulations, digital wind tunnels, etc. I think what's different right now is the sudden hyper focus. It's a chaotic gold rush driven by speculators and prospectors as much as by technologists or scientists. I wonder what we'll see on the other side when the fog starts to clear. I expect lots of "losers" i.e. frustrated prospectors who never found their gold mine. And probably some proprietary wins locked up by big investors. But, maybe we'll at least benefit from some new commodity products reaching out to a broader market after they initially targeted the rush itself...
Shorel 3y ago
It is not fundamentally different from for example MS Office being a large binary blob. We don't have the source code, it is proprietary, and a company controls it. At the same time, a group of guys trained a model that plays go, it can beat AlphaGo, and it probably is the strongest go player ever. No one from the group of people could have created LeelaZero by themselves alone, but collaboration is possible.

johnsutor 3y ago

I'm hopeful that this will get better as time goes on with advances in unsupervised and multimodal learning, that will enable us mere pions to achieve our niche use cases by fine-tuning a larger model with minimal compute.

seydor 3y ago

In economic terms, it feels like we the people won't have much of a chance at AI wealth. Software was infinitely scalable, easy to make and powerful, but AI is data with tiny amounts of code, and machine learning engineers are going to be a dime a dozen soon. So hardware companies will reap most of the benefits, and making it with a a garage startup is hard

fritzo 3y ago

This attitude is like the rugged forest hermit's aversion to computers because silicon wafer production is out of reach for individuals.

perryizgr8 3y ago

> fundamentally tethered to large binary blobs In many instances you don't even have the blob, you can just call an API that invokes it and runs the output thru multiple other hidden algorithms before sending it to you.

geor9e 3y ago

You're describing A.I. as a black box of proprietary workings, far beyond any mortal's personal budget to create, and too complex anyway, which internally runs in a language unintelligible to humans, and through layers and layers of abstractions, has a human-readable interface. To me, that exactly describes an x64 or ARM CPU. But since we are not children freshly learning about reality, we don't see these two black boxes in front of us in the same light. One is something we just got used to kinda-understanding-but-not-really decade after decade. One is something brand new to us.

dleslie 3y ago
x64 and ARM emulators exist, the code for some are quite legible. There's a whole community of folks who build RiscV implementations from verilog or vhdl. In my own experience at university, decades ago, we built functioning HC11 CPUs out of logic gates and wrote assembler for them. I'll admit that there's a definite and clear accessibility curve to building a CPU, but it's nowhere near as opaque as the binary blobs one can access for AI tooling.
- entropicdrifter 3y ago
  Now I want to go and play NandGame again... https://nandgame.com/ https://nandgame.com/
  - paulmd 3y ago
    I hear really good things about turing complete https://store.steampowered.com/app/1444480/Turing_Complete/ https://store.steampowered.com/app/1444480/Turing_Complete/
- deleted 3y ago
  [deleted]
asadotzler 3y ago
X86 and ARM are open in many many ways that for example ChatGPT isn't.

saltcured 3y ago

This is much how numerical computing has always been. We could individually approach the math and algorithms, but need major resources and input data to apply it to real-world problem like weather forecasts, geophysical simulations, digital wind tunnels, etc. I think what's different right now is the sudden hyper focus. It's a chaotic gold rush driven by speculators and prospectors as much as by technologists or scientists. I wonder what we'll see on the other side when the fog starts to clear. I expect lots of "losers" i.e. frustrated prospectors who never found their gold mine. And probably some proprietary wins locked up by big investors. But, maybe we'll at least benefit from some new commodity products reaching out to a broader market after they initially targeted the rush itself...

Shorel 3y ago

It is not fundamentally different from for example MS Office being a large binary blob. We don't have the source code, it is proprietary, and a company controls it. At the same time, a group of guys trained a model that plays go, it can beat AlphaGo, and it probably is the strongest go player ever. No one from the group of people could have created LeelaZero by themselves alone, but collaboration is possible.