4 ms·

So, way back in the last millenium, I did my Master's thesis (way smaller deal than a Ph.D. thesis) on neural networks. Since then, I have looked in on it ever

So, way back in the last millenium, I did my Master's thesis (way smaller deal than a Ph.D. thesis) on neural networks. Since then, I have looked in on it every few years. I think they're cool, I like using them, and writing multi-level backpropagation neural networks used to be one of the first things I'd do in a new language, just to get a feel for how it worked (until pytorch came along and I decided for the first time that using their library was easier than writing my own). So, it's not like I dislike ML. But, saying an investment is an "AI" startup, ought to be like saying it's a python startup, or saying it's a postgres startup. That ought not to be something you tell people as a defining characteristic of what you do, not because it's a secret but rather because it's not that important to your odds of success. If you used a different language and database, you would probably have about the same odds of success, because it depends more on how well you understand the problem space, and how well you architect your software. Linear models or other more traditional statistical models can often perform just as well as DL or any other neural network, for the same reason that when you look at a kaggle leaderboard, the difference between the leaders is usually not that big after a while. The limiting factor is in the data, and how well you have transformed/categorized that data, and all the different methods of ML that get thrown at it all end up with similar looking levels of accuracy. There used to be a saying: "If you don't know how to do it, you don't know how to do it with a computer." AI boosters sometimes sound as if they are suggesting that this is no longer true. They're incorrect. ML is, absolutely, a technique that a good programmer should know about, and may sometimes wish to use, kind of like knowing how a state machine works. It makes no great deal of difference to how likely a business is to succeed.

7532yahoogmail 7y ago
Thank you for the perspective. Now when we talk machine learning are we talking: L. Pachter and B. Sturmfels. Algebraic Statistics for Computational Biology. Cambridge University Press 2005. G. Pistone, E. Riccomango, H. P. Wynn. Algebraic Statistics. CRC Press, 2001. Drton, Mathias, Sturmfels, Bernd, Sullivant, Seth. Lectures on Algebraic Statistics, Springer 2009. Or more like: Watanabe, Sumio. Algebraic Geometry and Statistical Learning Theory, Cambridge University Press 2009. My understanding (I do not do AI or machine learning) that AI is distinct from these more mathematical analytic perspectives. Finally, might we argue that generally AI/ML is more easily suited to data that's already high quality eg. CERN data, trade data, drug trial data as opposed to unconstrained data eg. Find the buses in these 1MM jpegs?
- kk58 7y ago
  Pure CS based AI approaches are primarily for Image, Text, and maybe graphs and control. The domains are called computer vision, natural language processing, graph learning and reinforcement learning Structured Data like tables, time series etc the techniques are still from statistics. Regression for example is the workhorse for numerical prediction problems I think a lot of people are missing the point about leaps AI has made because they aren't aware of NLP or CV or reinforcement learning. So "AI" mentioned above is stunningly good for buses in 1MM image and reasonably good drug trial, cern data. The business models required for making AI business successful haven't been invented yet. Good AI model will be Deep stack : example would be something like precision agriculture where you'd use AI for designing rice then use iot and earth observation to locate right acreages and monitor growth and adjust nutrient at crop level and get dramatically great output with least wastage and highest nutritional content. Most AI companies are still started by ex CS folks who in general arent aware of deep technical opportunities in other disciplines. I think this will change soon very fast due to ubiquity of deep learning training material, libraries and research papers.
Tostino 7y ago
Well the term Postgres or Python startup may not make sense, but a Pytorch or TensorFlow startup may not either. A database startup though, tells me the company is likely going to be in the database field, and most likely is going to try and sell me something I don't need. An AI startup, similarly, is going to either be utilizing existing techniques on industry problems to sell me something I don't need, or making some novel improvement to the training or inference to sell me something else I don't need. So...yeah.
jedberg 7y ago
Saying that you're going to "use AI" is more akin to saying "we're going to have a web application" back in 1998. Back then a lot of startups didn't have websites, because they were making other products (hardware, boxed software, etc). If they had a website it was just a marketing page. So saying that you were going to make a "web application" did in fact differentiate you, in that it showed your approach was very different from the boxed software folks, but it didn't tell you much beyond that.
- all_blue_chucks 7y ago
  "Web application" came later. In the nineties it was called a "cgi web page" by your webmaster.
  - jedberg 7y ago
    I know, but I'm writing to a modern audience. :)
  - perl4ever 7y ago
    In the nineties, there was a huge difference between 1995 and 1998. It wasn't all that apparent to some of us until later, but things moved really fast in that timeframe. The years leading up to 2000 were almost like the imagining of approaching an event horizon or asymptote.
    - edw 7y ago
      What you’re describing is so hard to convey to people. In 1994 the we were building raised-floor data centers with halon for suppressors and marveling at our 2GB behemoth UNIX boxes. And writing our own web application framework using CGI. In ‘99 we were renting a suite at a colo and putting our own hardware there, running ColdFusion web apps. In ‘04 we were renting half a rack at the same colo and trying not to write three tier Java servlet based apps with 1,000 line web.xml files. And then AWS happened.
      - rbinv 7y ago
justinmeiners 7y ago
> If you don't know how to do it, you don't know how to do it with a computer. This is so true. We spent decades educating non-technical people that understanding a problem well is a prerequisite to programming it. Take something easy to understand like driving a car, doing it in a computer is now harder. AI is undoing all that. People reach a vague problem they can't describe and assume computers will magically fix it.
phreeza 7y ago
> There used to be a saying: "If you don't know how to do it, you don't know how to do it with a computer." This is a tautology in the narrow sense, but in the broader sense I think there surely exist things that humans don't "know" how to do without a computer, but know how to do with a computer. And the space of solveable problems is expanding, though AI is only a narrow slice of that.
- _laiq 7y ago
  I don't know. I think what we all do, we know how to do it without a computer. Computer just automate stuff for us. It's a very practical saying because it forces you to ask the right question about the problem you're trying to solve. (We all know how to do AirBNB by hand, or Uber by hand, but the mobile app is hyper efficient w/ GPS & 4G, that's all).

7532yahoogmail 7y ago

Thank you for the perspective. Now when we talk machine learning are we talking: L. Pachter and B. Sturmfels. Algebraic Statistics for Computational Biology. Cambridge University Press 2005. G. Pistone, E. Riccomango, H. P. Wynn. Algebraic Statistics. CRC Press, 2001. Drton, Mathias, Sturmfels, Bernd, Sullivant, Seth. Lectures on Algebraic Statistics, Springer 2009. Or more like: Watanabe, Sumio. Algebraic Geometry and Statistical Learning Theory, Cambridge University Press 2009. My understanding (I do not do AI or machine learning) that AI is distinct from these more mathematical analytic perspectives. Finally, might we argue that generally AI/ML is more easily suited to data that's already high quality eg. CERN data, trade data, drug trial data as opposed to unconstrained data eg. Find the buses in these 1MM jpegs?

kk58 7y ago
Pure CS based AI approaches are primarily for Image, Text, and maybe graphs and control. The domains are called computer vision, natural language processing, graph learning and reinforcement learning Structured Data like tables, time series etc the techniques are still from statistics. Regression for example is the workhorse for numerical prediction problems I think a lot of people are missing the point about leaps AI has made because they aren't aware of NLP or CV or reinforcement learning. So "AI" mentioned above is stunningly good for buses in 1MM image and reasonably good drug trial, cern data. The business models required for making AI business successful haven't been invented yet. Good AI model will be Deep stack : example would be something like precision agriculture where you'd use AI for designing rice then use iot and earth observation to locate right acreages and monitor growth and adjust nutrient at crop level and get dramatically great output with least wastage and highest nutritional content. Most AI companies are still started by ex CS folks who in general arent aware of deep technical opportunities in other disciplines. I think this will change soon very fast due to ubiquity of deep learning training material, libraries and research papers.

Tostino 7y ago

Well the term Postgres or Python startup may not make sense, but a Pytorch or TensorFlow startup may not either. A database startup though, tells me the company is likely going to be in the database field, and most likely is going to try and sell me something I don't need. An AI startup, similarly, is going to either be utilizing existing techniques on industry problems to sell me something I don't need, or making some novel improvement to the training or inference to sell me something else I don't need. So...yeah.

jedberg 7y ago

Saying that you're going to "use AI" is more akin to saying "we're going to have a web application" back in 1998. Back then a lot of startups didn't have websites, because they were making other products (hardware, boxed software, etc). If they had a website it was just a marketing page. So saying that you were going to make a "web application" did in fact differentiate you, in that it showed your approach was very different from the boxed software folks, but it didn't tell you much beyond that.

all_blue_chucks 7y ago
"Web application" came later. In the nineties it was called a "cgi web page" by your webmaster.
- jedberg 7y ago
  I know, but I'm writing to a modern audience. :)
- perl4ever 7y ago
  In the nineties, there was a huge difference between 1995 and 1998. It wasn't all that apparent to some of us until later, but things moved really fast in that timeframe. The years leading up to 2000 were almost like the imagining of approaching an event horizon or asymptote.
  - edw 7y ago
    What you’re describing is so hard to convey to people. In 1994 the we were building raised-floor data centers with halon for suppressors and marveling at our 2GB behemoth UNIX boxes. And writing our own web application framework using CGI. In ‘99 we were renting a suite at a colo and putting our own hardware there, running ColdFusion web apps. In ‘04 we were renting half a rack at the same colo and trying not to write three tier Java servlet based apps with 1,000 line web.xml files. And then AWS happened.
    - rbinv 7y ago

justinmeiners 7y ago

> If you don't know how to do it, you don't know how to do it with a computer. This is so true. We spent decades educating non-technical people that understanding a problem well is a prerequisite to programming it. Take something easy to understand like driving a car, doing it in a computer is now harder. AI is undoing all that. People reach a vague problem they can't describe and assume computers will magically fix it.

phreeza 7y ago

> There used to be a saying: "If you don't know how to do it, you don't know how to do it with a computer." This is a tautology in the narrow sense, but in the broader sense I think there surely exist things that humans don't "know" how to do without a computer, but know how to do with a computer. And the space of solveable problems is expanding, though AI is only a narrow slice of that.

_laiq 7y ago
I don't know. I think what we all do, we know how to do it without a computer. Computer just automate stuff for us. It's a very practical saying because it forces you to ask the right question about the problem you're trying to solve. (We all know how to do AirBNB by hand, or Uber by hand, but the mobile app is hyper efficient w/ GPS & 4G, that's all).