5 ms·

> What will we have in 20 years? Hopefully something that doesn't require the user to be a math wizard. I'm not into data science by any means, and I first le

by imiric 4y ago

> What will we have in 20 years? Hopefully something that doesn't require the user to be a math wizard. I'm not into data science by any means, and I first learned about tensors with Tensorflow. While I would like to experiment with neural nets, the current frameworks require me to learn the foundations of how they work. I'm sure I'll get arguments that this is a good thing, but I just want to do some cool stuff with computers, not take math courses to learn how the cake is made. High-level programming languages are abstractions to machine code that, for the most part, don't require the user to know how they work internally. I think we're missing similar abstractions for ML, or it's possible they already exist and I haven't looked hard enough.

cm2187 4y ago
The right analogy is cryptography. You don't need to understand the math behind cryptography to calculate a hash, or encrypt something. You just need to understand at a very high level what is asymmetric encryption, IV, etc.
- netheril96 4y ago
  That’s actually a bad analogy in my opinion. People who don’t learn cryptography make many mistakes calling cryptography libraries and those mistakes can’t be uncovered by testing.
  - bborud 4y ago
    Oh, it'll all be great when ChatGPT writes all that code for us and we can be off somewhere having a beer and a cry instead.
    - liotier 4y ago
      You'll cry but, to listen to your sorrow and offer friendly support, ChatGPT will be there for you !
      - bborud 4y ago
        That makes me think of a chapter in a Daniel Suarez novel where an AI tortures a human for a prolonged period of time.
        liotier 4y ago
        Or the Fifth Element's robot bartender, who will do great with a ChatGPT upgrade - https://youtu.be/sFAPW3W2yMk?t=114 https://youtu.be/sFAPW3W2yMk?t=114
  - vintermann 4y ago
    Learning how to use cryptography responsibly is different from learning to design and attack cryptographic algorithms, and fortunately a lot easier.
  - charcircuit 4y ago
    People can make ML mistakes if they don't learn ML. Some of the mistakes can be uncovered by testing, but only if you know enough about ML to know how to test it. Some mistakes just you with a bad model that could have been better.
- im3w1l 4y ago
  I think what we all want is some kind of blackbox where data goes in and insight comes out. Ideally we would have something that takes a bunch of data and finds the shortest program that generates it. Unfortunately such a black box cannot exist, it is uncomputable. So then we need to ask for something less than that and we don't know exactly what that is.
  - optimalsolver 4y ago
    Maybe not the shortest program, but perhaps a list that contains the shortest program: https://arxiv.org/abs/1212.6104 https://arxiv.org/abs/1212.6104 https://arxiv.org/abs/1301.1547 https://arxiv.org/abs/1301.1547
    - im3w1l 4y ago
      Interesting but how would one use such a list to make predictions?
      - renonce 4y ago
        If the data was generated from a generator of constant complexity, its kolmogorov complexity cannot exceed the complexity of the generator. As the amount of data increases, the kolmogorov complexity will increase until it converges after a finite amount of data. At this point we effectively have the generator so we can generate infinite data.
        KRAKRISMOTT 4y ago
        Would you mind elaborating on the proof?
laci37 4y ago
I don't think that you need to know any of the underlying maths to make a neural network in TF/Keras. You need to know (not understand!) the building blocks, and what are they good for, but learning these feels like learning the difference between lists and dicts. You can do programming without it, but it will be ugly.
jiggawatts 4y ago
They don't really exist. Ideally, this should all be integrated at the compiler level. Programming languages and their associated standard libraries should expose differentiation as a first-class abstraction, along with everything else. The problem is that the whole space is really immature, and huge strides are being made... but by many disparate groups, all working in different spaces and languages. Just in the last few months I read about several Python ML frameworks, a Julia differential compute capability, and that Haskell has a library where even the type system can express differentiation of general algebraic data types! The whole ecosystem needs to... congeal a little bit. This whole space reminds me of Object Oriented programming before C++ and Java.
- consilient 4y ago
  > Haskell has a library where even the type system can express differentiation of general algebraic data types! It's not really the same sort of differentiation. Derivatives of algebraic data types (as implementable in Haskell) only generalize formal derivatives of polynomials. You need a lot more machinery before you can talk about smoothness.
proto-n 4y ago
I understand what you mean, and I think we are unfortunately not there yet. Right now you still mostly need to understand the details to make something work. In many areas, it's not even just a leaky abstraction [1] where it works 90% of the time, but more like a series of magic incantations and tricks you need in a specific order to make it work. But as far as I can tell its getting better and more robust in general. [1] https://news.ycombinator.com/item?id=13215590 https://news.ycombinator.com/item?id=13215590
ansgri 4y ago
I'd rather hope that basic linear algebra and the notion of vectorized function application becomes universal math knowledge like addition and multiplication, as basic programming slowly becomes. We need languages and compilers with first-class optimized differential programming support, sure, but increasing baseline education would benefit wider population.
- imiric 4y ago
  I agree with your last point, but most people have no practical need to learn high-level math. We struggle globally with even basic education rates, and programming definitely isn't for everyone. What is needed are higher-level abstractions over ML principles that make ML more accessible for people who are interested in programming, but lack the math background to be productive. This doesn't necessarily mean that the lower-level details should be entirely hidden, but that the tools are friendly enough for beginners to easily pick up, and flexible enough for advanced users to tweak to their needs.
  - beepbooptheory 4y ago
    Why though? Aren't everyday users more interested in finished models themselves? Why would they want to make one, especially if they, in this example, don't really know how they work anyway? What is the value here?
    - imiric 4y ago
      "Everyday users" aren't programmers, but people who would use the apps powered by ML. Think ChatGPT. There are programmers, though, who are not fluent in the math that makes ML possible, but would be empowered by friendlier tools.
  - mananaysiempre 4y ago
    Note that “high-level math” by your definition covers things that were basically understood by 1900 (a good part even by 1850) and polished by 1940. It’s an incredible failure of the education system that any of this still counts as “advanced” in any way in 2023[1]. So abstract away, by all means, ensuring the user doesn’t need to know the Biot-Savart law to use a doorbell is the engineer’s job after all, and as semiconductor electronics show, users can include other engineers. (Hell, even the humble resistor is a fairly advanced piece of technology dependent a good chunk of 20th-century physics.) But sooner or later we’re going to have to pay back this “ed debt”. (“Soc debt”?) [1] See my recent comment at https://news.ycombinator.com/item?id=34389684 https://news.ycombinator.com/item?id=34389684 for a starting point for a literature dive.
visarga 4y ago
What you need is HuggingFace, a higher level library that comes with a model zoo, ready to plug and play.
odshoifsdhfs 4y ago
Try fast.ai (https://http://fast.ai https://http://fast.ai) They have a course that goes exactly like that (Practical Machine Learning or similar). You start high level with libraries and just a few lines of code, and gives you the high level tools to use ML in your projects, and sometimes goes a bit lower level when needed.
- EthicalSimilar 4y ago
  Link is broken.
- palotasb 4y ago
  https://www.fast.ai/ https://www.fast.ai/
- tluyben2 4y ago
  Also the book [0] is nice as a companion or when you don't want to watch videos (I don't). [0] https://www.amazon.com/Deep-Learning-Coders-fastai-PyTorch/dp/1492045527 https://www.amazon.com/Deep-Learning-Coders-fastai-PyTorch/d...
- imiric 4y ago
  Thanks! I've had this bookmarked for years now, but haven't gotten around to checking it out. It's on my list to look into next time I get the ML itch. :)
Bombthecat 4y ago
My wild guess? We will get AI driven UI Frameworks as a tool. You drag and drop stuff in there, it suggests already the basic stuff. Preselects sane filters and normalizes your data etc. Will handcrafted, math used me better. Very likely.but a VW is good enough for 99 percent of use cases :)
WastingMyTime89 4y ago
> I'm sure I'll get arguments that this is a good thing, but I just want to do some cool stuff with computers, not take math courses to learn how the cake is made. It’s not so much how the cake is made but rather what you are actually doing. You can already make something cool today by using end products and stringing their api together which might be what you are looking for. If you want to make something new however and wish to use ML tools, well, at some point, you will have to learn what these tools actually are.
niemandhier 4y ago
That funny, I looked into the whole neural net stuff the other day and thought: "Wow you basically don't need any math". I guess it depends a lot on perspective, to me the whole field looks like alchemy more than science. The results cannot be denied, but we understand so little.
- deepGem 4y ago
  This is true. In the current scheme of things, you don't need to know any math to get things going. At some level, if you really need to write a custom model from scratch, probably you need to - but not really. Now you have the likes of HuggingFace which is another abstraction on top of Pytorch. Heck you don't even need to code a transformer from scratch. They have done it for you. from transformers import AutoTokenizer, AutoModelForMaskedLM tokenizer = AutoTokenizer.from_pretrained("bert-base-uncased") model = AutoModelForMaskedLM.from_pretrained("bert-base-uncased") I was like wow, this level of abstraction is totally breaking barriers to entry and we will see many many flavours of huggingface in the years to come.
- calebkaiser 4y ago
  I think this is more to do with the way software ecosystems develop in general. You don't need to know much about the actual implementation of databases or low-level programming to build a data-heavy application, and you don't need to know the complexities of graphics programming to render complex visuals to a screen. If you want to do research in any branch of ML, you are almost certainly going to need some strong mathematical foundations. At the same time, if you're more interested in applying existing architectures/models, the ecosystem has a lot of really amazing tools that will take you a very long way without requiring you to study statistical learning theory.
eru 4y ago
> I'm not into data science by any means, and I first learned about tensors with Tensorflow. Tensorflow doesn't actually have any tensors. At least not in the sense that tensor used to be used before Tensorflow. Tensorflow just has arrays. See https://en.wikipedia.org/wiki/Tensor https://en.wikipedia.org/wiki/Tensor for the old, original meaning.
awesomegoat_com 4y ago
I am highly suspicious of people who are not math wizards. And guess what? It is good life strategy.
adw 4y ago
The ML is made of, and expressed in terms of, math. There’s no way round that if you want to train models; you need first-year undergrad statistics and linear algebra. In other words, just as you do with any language, you need to understand the grammar and vocabulary. You don’t have to know how the operations involved doing what they’re doing, in the sense that you don’t need to know how matrix multiplication is implemented any more than you need to understand Timsort to call sort() in Python, but you do need to know matrix arithmetic. That isn’t being a math wizard, it’s the price of entry.
zackmorris 4y ago
Came here to say this. Tensors are the wrong level of abstraction, because they're just matrices, which are just arrays. The best tool I've used for vector/matrix processing is MATLAB (or GNU Octave, which is free but without as many libraries). To say I'm disappointed that these never gained the same traction as Python would be an understatement. Unfortunately, these tools all suffer from a lack of insight from programmers. We abstract whenever possible, but for whatever reason, academics have a certain expectation that the user learn the entire system before starting. I believe that's why functional programming never quite caught on. That egocentric view infects everything from Mathematica/WolframAlpha to UNIX. I prefer other models. For example, I don't feel that anyone has explored genetic algorithms under the Actor model yet. Which severely limits how fast the user can explore a problem space. Part of that is due to hardware. A GPU may present itself as having X number of teraflops or whatever, but only have something like 64 cores and no API to access them directly like a CPU. Which limits it to a paltry number of environments and/or automatons to evolve. A better architecture would be something like 1024+ symmetric cores with a unified copy-on-write, content-addressable memory accessible through ordinary C desktop programming and orchestrated with something like Docker. Then we could explore auto-parallelization with languages like Julia and Clojure, possibly even Rust. I've written about that extensively throughout the years, but there's still been no change to the status quo, so I've all but given up on that. I do see a small hope with RISC-V on FPGA though. The somewhat ironic thing is that AI is evolving so quickly that it's passed the Turing test now for the masses. We only have maybe 5-20 years before it solves all problems better than us. So it's an odd feeling for me to have waited 20 years with no progress on the hardware that I needed to do what I wanted to do, only to make the thing I wanted to do irrelevant anyway. That's the challenge we all face now as play is automated by AI while we still have to go to work and pay rent within a 20th century Matrix reality. I hope we find a way to reverse that trend before we're too old to care anyway.
mikewarot 4y ago
You don't have to know how an internal combustion engine works to use one. If you need to troubleshoot or repair it, you have to know the basics. The same is true of neural networks. Using Tensorflow doesn't require you to know all the details, you build a model of the network, and it actually executes that model. The low level stuff is handled. You still have to know how to tell when it is working, or not. Properly rationing your training/testing/validation ports of data is surprisingly important for not ending up with a model that seems to work great until you use it in the real world.

by imiric 4y ago

cm2187 4y ago

The right analogy is cryptography. You don't need to understand the math behind cryptography to calculate a hash, or encrypt something. You just need to understand at a very high level what is asymmetric encryption, IV, etc.

netheril96 4y ago
That’s actually a bad analogy in my opinion. People who don’t learn cryptography make many mistakes calling cryptography libraries and those mistakes can’t be uncovered by testing.
- bborud 4y ago
  Oh, it'll all be great when ChatGPT writes all that code for us and we can be off somewhere having a beer and a cry instead.
  - liotier 4y ago
    You'll cry but, to listen to your sorrow and offer friendly support, ChatGPT will be there for you !
    - bborud 4y ago
      That makes me think of a chapter in a Daniel Suarez novel where an AI tortures a human for a prolonged period of time.
      - liotier 4y ago
        Or the Fifth Element's robot bartender, who will do great with a ChatGPT upgrade - https://youtu.be/sFAPW3W2yMk?t=114 https://youtu.be/sFAPW3W2yMk?t=114
- vintermann 4y ago
  Learning how to use cryptography responsibly is different from learning to design and attack cryptographic algorithms, and fortunately a lot easier.
- charcircuit 4y ago
  People can make ML mistakes if they don't learn ML. Some of the mistakes can be uncovered by testing, but only if you know enough about ML to know how to test it. Some mistakes just you with a bad model that could have been better.
im3w1l 4y ago
I think what we all want is some kind of blackbox where data goes in and insight comes out. Ideally we would have something that takes a bunch of data and finds the shortest program that generates it. Unfortunately such a black box cannot exist, it is uncomputable. So then we need to ask for something less than that and we don't know exactly what that is.
- optimalsolver 4y ago
  Maybe not the shortest program, but perhaps a list that contains the shortest program: https://arxiv.org/abs/1212.6104 https://arxiv.org/abs/1212.6104 https://arxiv.org/abs/1301.1547 https://arxiv.org/abs/1301.1547
  - im3w1l 4y ago
    Interesting but how would one use such a list to make predictions?
    - renonce 4y ago
      If the data was generated from a generator of constant complexity, its kolmogorov complexity cannot exceed the complexity of the generator. As the amount of data increases, the kolmogorov complexity will increase until it converges after a finite amount of data. At this point we effectively have the generator so we can generate infinite data.
      - KRAKRISMOTT 4y ago
        Would you mind elaborating on the proof?

laci37 4y ago

I don't think that you need to know any of the underlying maths to make a neural network in TF/Keras. You need to know (not understand!) the building blocks, and what are they good for, but learning these feels like learning the difference between lists and dicts. You can do programming without it, but it will be ugly.

jiggawatts 4y ago

They don't really exist. Ideally, this should all be integrated at the compiler level. Programming languages and their associated standard libraries should expose differentiation as a first-class abstraction, along with everything else. The problem is that the whole space is really immature, and huge strides are being made... but by many disparate groups, all working in different spaces and languages. Just in the last few months I read about several Python ML frameworks, a Julia differential compute capability, and that Haskell has a library where even the type system can express differentiation of general algebraic data types! The whole ecosystem needs to... congeal a little bit. This whole space reminds me of Object Oriented programming before C++ and Java.

consilient 4y ago
> Haskell has a library where even the type system can express differentiation of general algebraic data types! It's not really the same sort of differentiation. Derivatives of algebraic data types (as implementable in Haskell) only generalize formal derivatives of polynomials. You need a lot more machinery before you can talk about smoothness.

proto-n 4y ago

I understand what you mean, and I think we are unfortunately not there yet. Right now you still mostly need to understand the details to make something work. In many areas, it's not even just a leaky abstraction [1] where it works 90% of the time, but more like a series of magic incantations and tricks you need in a specific order to make it work. But as far as I can tell its getting better and more robust in general. [1] https://news.ycombinator.com/item?id=13215590 https://news.ycombinator.com/item?id=13215590

ansgri 4y ago

I'd rather hope that basic linear algebra and the notion of vectorized function application becomes universal math knowledge like addition and multiplication, as basic programming slowly becomes. We need languages and compilers with first-class optimized differential programming support, sure, but increasing baseline education would benefit wider population.

imiric 4y ago
I agree with your last point, but most people have no practical need to learn high-level math. We struggle globally with even basic education rates, and programming definitely isn't for everyone. What is needed are higher-level abstractions over ML principles that make ML more accessible for people who are interested in programming, but lack the math background to be productive. This doesn't necessarily mean that the lower-level details should be entirely hidden, but that the tools are friendly enough for beginners to easily pick up, and flexible enough for advanced users to tweak to their needs.
- beepbooptheory 4y ago
  Why though? Aren't everyday users more interested in finished models themselves? Why would they want to make one, especially if they, in this example, don't really know how they work anyway? What is the value here?
  - imiric 4y ago
    "Everyday users" aren't programmers, but people who would use the apps powered by ML. Think ChatGPT. There are programmers, though, who are not fluent in the math that makes ML possible, but would be empowered by friendlier tools.
- mananaysiempre 4y ago
  Note that “high-level math” by your definition covers things that were basically understood by 1900 (a good part even by 1850) and polished by 1940. It’s an incredible failure of the education system that any of this still counts as “advanced” in any way in 2023[1]. So abstract away, by all means, ensuring the user doesn’t need to know the Biot-Savart law to use a doorbell is the engineer’s job after all, and as semiconductor electronics show, users can include other engineers. (Hell, even the humble resistor is a fairly advanced piece of technology dependent a good chunk of 20th-century physics.) But sooner or later we’re going to have to pay back this “ed debt”. (“Soc debt”?) [1] See my recent comment at https://news.ycombinator.com/item?id=34389684 https://news.ycombinator.com/item?id=34389684 for a starting point for a literature dive.

visarga 4y ago

What you need is HuggingFace, a higher level library that comes with a model zoo, ready to plug and play.

odshoifsdhfs 4y ago

Try fast.ai (https://http://fast.ai https://http://fast.ai) They have a course that goes exactly like that (Practical Machine Learning or similar). You start high level with libraries and just a few lines of code, and gives you the high level tools to use ML in your projects, and sometimes goes a bit lower level when needed.

EthicalSimilar 4y ago
Link is broken.
palotasb 4y ago
https://www.fast.ai/ https://www.fast.ai/
tluyben2 4y ago
Also the book [0] is nice as a companion or when you don't want to watch videos (I don't). [0] https://www.amazon.com/Deep-Learning-Coders-fastai-PyTorch/dp/1492045527 https://www.amazon.com/Deep-Learning-Coders-fastai-PyTorch/d...
imiric 4y ago
Thanks! I've had this bookmarked for years now, but haven't gotten around to checking it out. It's on my list to look into next time I get the ML itch. :)

Bombthecat 4y ago

My wild guess? We will get AI driven UI Frameworks as a tool. You drag and drop stuff in there, it suggests already the basic stuff. Preselects sane filters and normalizes your data etc. Will handcrafted, math used me better. Very likely.but a VW is good enough for 99 percent of use cases :)

WastingMyTime89 4y ago

> I'm sure I'll get arguments that this is a good thing, but I just want to do some cool stuff with computers, not take math courses to learn how the cake is made. It’s not so much how the cake is made but rather what you are actually doing. You can already make something cool today by using end products and stringing their api together which might be what you are looking for. If you want to make something new however and wish to use ML tools, well, at some point, you will have to learn what these tools actually are.

niemandhier 4y ago

That funny, I looked into the whole neural net stuff the other day and thought: "Wow you basically don't need any math". I guess it depends a lot on perspective, to me the whole field looks like alchemy more than science. The results cannot be denied, but we understand so little.

deepGem 4y ago
This is true. In the current scheme of things, you don't need to know any math to get things going. At some level, if you really need to write a custom model from scratch, probably you need to - but not really. Now you have the likes of HuggingFace which is another abstraction on top of Pytorch. Heck you don't even need to code a transformer from scratch. They have done it for you. from transformers import AutoTokenizer, AutoModelForMaskedLM tokenizer = AutoTokenizer.from_pretrained("bert-base-uncased") model = AutoModelForMaskedLM.from_pretrained("bert-base-uncased") I was like wow, this level of abstraction is totally breaking barriers to entry and we will see many many flavours of huggingface in the years to come.
calebkaiser 4y ago
I think this is more to do with the way software ecosystems develop in general. You don't need to know much about the actual implementation of databases or low-level programming to build a data-heavy application, and you don't need to know the complexities of graphics programming to render complex visuals to a screen. If you want to do research in any branch of ML, you are almost certainly going to need some strong mathematical foundations. At the same time, if you're more interested in applying existing architectures/models, the ecosystem has a lot of really amazing tools that will take you a very long way without requiring you to study statistical learning theory.

eru 4y ago

> I'm not into data science by any means, and I first learned about tensors with Tensorflow. Tensorflow doesn't actually have any tensors. At least not in the sense that tensor used to be used before Tensorflow. Tensorflow just has arrays. See https://en.wikipedia.org/wiki/Tensor https://en.wikipedia.org/wiki/Tensor for the old, original meaning.

awesomegoat_com 4y ago

I am highly suspicious of people who are not math wizards. And guess what? It is good life strategy.

adw 4y ago

The ML is made of, and expressed in terms of, math. There’s no way round that if you want to train models; you need first-year undergrad statistics and linear algebra. In other words, just as you do with any language, you need to understand the grammar and vocabulary. You don’t have to know how the operations involved doing what they’re doing, in the sense that you don’t need to know how matrix multiplication is implemented any more than you need to understand Timsort to call sort() in Python, but you do need to know matrix arithmetic. That isn’t being a math wizard, it’s the price of entry.

zackmorris 4y ago

Came here to say this. Tensors are the wrong level of abstraction, because they're just matrices, which are just arrays. The best tool I've used for vector/matrix processing is MATLAB (or GNU Octave, which is free but without as many libraries). To say I'm disappointed that these never gained the same traction as Python would be an understatement. Unfortunately, these tools all suffer from a lack of insight from programmers. We abstract whenever possible, but for whatever reason, academics have a certain expectation that the user learn the entire system before starting. I believe that's why functional programming never quite caught on. That egocentric view infects everything from Mathematica/WolframAlpha to UNIX. I prefer other models. For example, I don't feel that anyone has explored genetic algorithms under the Actor model yet. Which severely limits how fast the user can explore a problem space. Part of that is due to hardware. A GPU may present itself as having X number of teraflops or whatever, but only have something like 64 cores and no API to access them directly like a CPU. Which limits it to a paltry number of environments and/or automatons to evolve. A better architecture would be something like 1024+ symmetric cores with a unified copy-on-write, content-addressable memory accessible through ordinary C desktop programming and orchestrated with something like Docker. Then we could explore auto-parallelization with languages like Julia and Clojure, possibly even Rust. I've written about that extensively throughout the years, but there's still been no change to the status quo, so I've all but given up on that. I do see a small hope with RISC-V on FPGA though. The somewhat ironic thing is that AI is evolving so quickly that it's passed the Turing test now for the masses. We only have maybe 5-20 years before it solves all problems better than us. So it's an odd feeling for me to have waited 20 years with no progress on the hardware that I needed to do what I wanted to do, only to make the thing I wanted to do irrelevant anyway. That's the challenge we all face now as play is automated by AI while we still have to go to work and pay rent within a 20th century Matrix reality. I hope we find a way to reverse that trend before we're too old to care anyway.

mikewarot 4y ago

You don't have to know how an internal combustion engine works to use one. If you need to troubleshoot or repair it, you have to know the basics. The same is true of neural networks. Using Tensorflow doesn't require you to know all the details, you build a model of the network, and it actually executes that model. The low level stuff is handled. You still have to know how to tell when it is working, or not. Properly rationing your training/testing/validation ports of data is surprisingly important for not ending up with a model that seems to work great until you use it in the real world.