27 ms·

Andreessen-Horowitz craps on “AI” startups from a great height

695 points by dostoevsky 7y ago

lazzlazzlazz 7y ago
Is the misspelling of "Andreessen-Horowitz" and use of "A19H" instead of "a16z" intentional?
- khazhoux 7y ago
  You mean the fact that they left out an "s" in Andreessen?
  - dang 7y ago
    We've squeezed another s above.
- scottlocklin 7y ago
  I suck at spelling. If I was one of the cool kids I'd claim to be dyslexic.
  - yubozhao 7y ago
    hi OP. We built an open-source library called, BentoML(https://github.com/bentoml/bentoml https://github.com/bentoml/bentoml) to make model inferencing/serving a lot easier for Data scientists in various serving scenarios. Love to hear your thoughts on our library
    - EamonnMR 7y ago
      I was really hoping that you where about to offer an ML framework to improve spelling.
leetrout 7y ago
That is a great write up and very accurate description of both the costs and human intervention based on my experience with “AI” tools.
harias 7y ago
>That’s right; that’s why a lone wolf like me, or a small team can do as good or better a job than some firm with 100x the head count and 100m in VC backing. goes on to say >I agree, but the hockey stick required for VC backing, and the army of Ph.D.s required to make it work doesn’t really mix well with those limited domains, which have a limited market. Choose one? Also assumes running your own data center to be easy. Some people don't want to be up 24x7 monitoring their data center or to buy hardware to accommodate the rare 10 minute peaks in usage.
- icheishvili 7y ago
  I don't think these are necessarily contradictory. With pytorch-transformers, you can use a full-blown BERT model like the best in the world. And yet, to make this novel and defensible, you would need to build on top of it and innovate significantly, which would require significant capital to achieve.
- jjeaff 7y ago
  >rare 10 minute peaks But is that really the use case here? I haven't worked in ML. But I'm not seeing where you are going to need to handle a 10 minute spike that requires a whole datacenter. A month's worth of a quad gpu instance on AWS could pay for a server with similar capacity in a few months of usage. And hardware is pretty resilient these days. Especially if you co-locate it in a datacenter that handles all the internet and power up time for you. And when something does go wrong, they offer "magic hands" service to go swap out hardware for you. Colocation is surprisingly cheap. As is leasing 'managed' equipment.
- detaro 7y ago
  > Some people don't want to be up 24x7 monitoring their data center or to buy hardware to accommodate the rare 10 minute peaks in usage. Do you need that for training workloads, and what percentage of a startups workload is training?
- bsenftner 7y ago
  I ran a small data cluster for years, the horsepower behind my startup. Other than the Chinese DDoS attacks, running the cluster was absolutely elementary. The idea that running a server or a band of servers is difficult is a bold faced lie. People have got to stop repeating the cloud propaganda.
raiyu 7y ago
The number of places where machine learning can be used effectively from both a cost perspective and a return perspective are small. They are usually tremendously large datasets at gigantic companies, and they probably have to build in house expertise because it's hard to package this up into a product and resell it for various industries, datasets, etc. Certainly something like autonomous driving needs machine learning to function, but again, these are going to be owned by large corporations, and even when a startup is successful, it's really about the layered technology on-top of machine learning that makes it interesting. It's kind of like what Kelsey Hightower said about Kubernetes. It's interesting and great, but what will really matter is what service you put on top of it, so much so that whether you use Kubernetes becomes irrelevant. So I think companies that are focusing on a specific problem, providing that value added service, building it through machine learning, can be successful. While just broadly deploying machine learning as a platform in and of itself can be very challenging. And I think the autonomous driving space is a great example of that. They are building a value added service in a particular vertical, with tremendous investment, progress, and potentially life changing tech down the road. But as a consumer it's really the autonomous driving that is interesting, not whether they are using AI/machine learning to get there.
- Q6T46nT668w6i3m 7y ago
  How would you explain the rise (and success) of machine learning in science? A lab that uses some learning-based method will likely be limited to just one or two people (responsible for data acquisition, feature engineering, evaluation, etc.) and extremely finite data.
  - semi-extrinsic 7y ago
    How do you define success? Adoption? Because right now, writing "we will use machine learning to solve X" in a grant proposal is an easy way to increase chances of getting funding.
  - ska 7y ago
    It's not clear there has been any deep impact actually, but there has been a lot of discussion (and grant proposals) I've seen a lot of cross pollination of ML and AI techniques into various disciplines. A large percentage just didn't work at all, most of the rest were more "kind of interesting, but". Nothing earthshaking happened although pop sci press likes to talk about it a lot. If you have more digital data than you used to, using modern free frameworks and toolkits to do basic (i.e. older, boring, but understood) ML stuff to understand it seems to have a reasonable return. Mostly I think this is because it becomes accessible to someone without much background in the area, and you can do reasonable things without having to put 6 months of reading and implementing together before starting.
seibelj 7y ago
I wrote an article I published a week ago about how AI is the biggest misnomer in tech history https://medium.com/@seibelj/the-artificial-intelligence-scam-is-imploding-34b156c3537e https://medium.com/@seibelj/the-artificial-intelligence-scam... I wrote it to be tongue-in-cheek in a ranting style, but essentially "AI" businesses and the technology underpinning it are not the silver bullet the media and marketing hype has made it out to be. The linked article about a16z shows how AI is the same story everywhere - enormous capital to get the data and engineers to automate, but even the "good" AI still gets it wrong much of the time, necessitating endless edge-cases, human intervention, and eventually it's a giant ball of poorly-understand and impossible to maintain pipelines that don't even provide a better result than a few humans with a spreadsheet.
- scottlocklin 7y ago
  Coming from a fellow masshole: that's a great rant. There was this meme in the 70s about "self driving cars" following magnetic strips in the road in restricted highways. I remember at the time, being, like 8 and thinking "sure seems like an overly complicated train."
  - seibelj 7y ago
    Thanks man! Lifelong masshole here. Your post was much better than mine, but I appreciate the comment.
m0zg 7y ago
"Huge compute bills" usually come from training, or to be more precise, hyperparameter search that's required before you find a model that works well. You could also fail to find such a model, but that's another discussion. So yeah, you could spend one or two FTE salaries' (or one deep learning PhD's) worth of cash on finding such models for your startup if you insist on helping Jeff Bezos to wipe his tears with crisp hundred dollar bills. That's if you know what you're doing of course. Literally unlimited amounts could be spent if you don't. Or you could do the same for a fraction of the cost by stuffing a rack in your office with consumer grade 2080ti's. Just don't call it a "datacenter" or NVIDIA will have a stroke. Is that too much money? Not in most typical cases, I'd think. If the competitive advantage of what you're doing with DL does not offset the cost of 2 meatspace FTEs, you're doing it wrong. That, once again, assumes that you know what you're doing, and aren't doing deep learning for the sake of deep learning. Also, if your startup is venture funded, AWS will give you $100K in credit, hoping that you waste it by misconfiguring your instances and not paying attention to their extremely opaque billing (which is what most of their startup customers proceed to doing pretty much straight away). If you do not make these mistakes, that $100K will last for some time, after which you could build out the aforementioned rack full of 2080ti's on prem.
- avip 7y ago
  For balance, all big cloud providers - aws, gcp, azure, oracle [0] have pretty similar startup plans. Y$$MV (I'm in full agreement with everything you've written + it's well-phrased and funny. gj!) [0] that's not a typo - there is such thing as "Oracle cloud"
- alephnan 7y ago
  > Just don't call it a "datacenter" or NVIDIA will have a stroke. Context please :) ?
  - mereel 7y ago
    Just a guess but maybe it's some licensing issue? https://www.nvidia.com/en-us/drivers/geforce-license/ https://www.nvidia.com/en-us/drivers/geforce-license/ No Datacenter Deployment. The SOFTWARE is not licensed for datacenter deployment, except that blockchain processing in a datacenter is permitted.
marmaduke 7y ago
I sometimes contribute to methodology projects in neuroscience ("AI" for scientists). The most tiring part of it is explaining essentially these things over and over. Very interesting to see the sentiment vindicated in Startupistan.
atulkum 7y ago
On the other hand some of the startup is doing absolutely fraud on the name of AI.I went to a self checkout store (AIFI.io). I did not touch anything but they charge me $35.10. According to the receipt I took 17 packs of snacks :) These guys are doing fraud on the name of AI. They have no technology no software just put up some camera and open a store so that they can defraud the investor. Anyone can try if intersted https://www.aifi.io/loop-case-study https://www.aifi.io/loop-case-study
mtkd 7y ago
AI on the algo side is only half the story -- it has to sit in a domain specific framework to be most effective I see a lot of 'bolt-on' tech emerging -- it looks mostly snake oil -- there is no obvious way to be competitive against teams that baked it in to the bare metal design Also most commercial use-cases I've seen need effective ML more than anything else
dang 7y ago
A thread about the original article, from a few days ago: https://news.ycombinator.com/item?id=22352750 https://news.ycombinator.com/item?id=22352750
correlator 7y ago
No need to look at AZ for this. If you're building "AI" I wish you a speedy road to being acquired by a company that can put it to use. You've become a high priced recruiting firm. If you're solving a real problem and use ML in service of solving that problem, then you've got a great moat....happy trusting customers. It's not complicated
- motohagiography 7y ago
  Sssh! Valuations are a function of projected market size and opacity of the problem. Clarity like this collapses the uncertainty and destroys value. If you pour enough capital into rooms full of PhD's something's gotta hit. My way of saying, you're very, very right.
ativzzz 7y ago
I agree with the author's opinion about > I’ll go out on a limb and assert that most of the up front data pipelining and organizational changes which allow for it are probably more valuable than the actual machine learning piece. Especially at non-tech companies with outdated internal technology. I've consulted at one of these and the biggest wins from the project (I left before the whole thing finished unfortunately) were overall improvements to the internal data pipeline, such as standardization and consolidation of similar or identical data from different business units.
- noelsusman 7y ago
  I do data science at a non-tech company with outdated internal technology and I've seen this over and over again. Honestly though, it's worth every penny because often the only way to get the resources to truly solve data pipeline issues is to get an executive to buy some crap from a vendor and force everyone to make it work.
- jotakami 7y ago
  I was a consultant at one of the giant outsourcers and nod my head vigorously at this comment. The least sexy projects were MDM (master data management) but they were absolutely essential to the success of any other fancy analytics/BI/ML project.
  - 2sk21 7y ago
    Interestingly I too worked on MDM systems about ten years ago, when I was at IBM Research. Ironically, one of my first ideas for applying machine learning was in de-duplication of data in an MDM server. However the technology was a bit too primitive back in 2010 and the project was a hard sell so it was abandoned.
allovernow 7y ago
All of this might be true currently, but that's because this current first generation "AI" (technically should just be called ML) is mostly bullshit. To clarify, I don't mean anyone is lying or selling snake oil - what I mean by bullshit is that the vast majority of these services are cooked up by software developers without any background in mathematics, selling adtechy services in domains like product recommendation and sentiment analysis. They are single discipline applications accessable to devs without science backgrounds and do not rely on substantial expertise from other fields. That makes them narrow in technical scope and easy to rip off (hence no moat, lots of competition, and human reliance and lack of actual software). The next generation of Machine Learning is just emerging, and looks nothing like this. Funds are being raised, patents are being filed, and everything is in early stage development, so you probably haven't heard much yet - but these ML startups are going after real problems in industry: cross disciplinary applications leveraging the power of heuristic learning to make cross disciplinary designs and decisions currently still limited to the human domain. I'm talking about the kind of heuristics which currently exist only as human intuition expressed most compactly as concept graphs and, especially, mathematical relationships - e.g. component design with stress and materials constraints, geologic model building, treatment recommendation from a corpus of patient data, etc. ML solutions for problems like these cannot be developed without an intimate understanding of the problem domain. This is a generalist's game. I predict that the most successful ML engineers of the next decade will be those with hard STEM backgrounds, MS and PhD level, who have transitioned to ML. [Un]Fortunately for us, the current buzzwordy types of ML services give the rest of us a bad name, but looking at these upcoming applications the answers to the article tl;dr look different: >Deep learning costs a lot in compute, for marginal payoffs The payoffs here are far greater. Designs are in the pipeline which augment industry roles - accelerate design by replacing finite methods with vastly quicker ML for unprecedented iteration. Produce meaningful suggestions during the development of 3D designs. Fetch related technical documents in real time by scanning the progressive design as the engineer works, parsing and probabilistically suggesting alternative paths to research progression. Think Bonzi Buddy on steroids...this is a place for recurring software licenses, not SaaS. >Machine learning startups generally have no moat or meaningful special sauce For solving specific, technical problems, neural network design requires a certain degree of intuition with respect to the flow of information through the network, which both optimizes and limits the kind of patterns that a given net can learn. Thus designing NN for hard-industry applications is predicated upon an intimate understanding of domain knowledge, and these highly specialized neural nets become patentable secret sauces. That's half of the most - the other comes from competition for the software developers with first-hand experience in these fields, or a general enough math heavy background to capture the relationships that are being distilled into nets. >Machine learning startups are mostly services businesses, not software businesses Again only true because most current applications are NLP adtechy bullshit. Imagine coding in an IDE powered by an AI (multiple interacting neural nets) which guides the structure of your code at a high level and flags bugs as you write. This, at a more practical level, is the type of software that will eventually change every technical discipline, and you can sell licenses! >Machine learning will be most productive inside large organizations that have data and process inefficiencies This next generation goes far past simply optimizing production lines or counting missed pennies or extracting a couple extra percent of value from analytics data. This style of applied ML operates at a deeper level of design which will change everything.
whoisjuan 7y ago
An many times all these AI computations go into solving mundane problems like "What's the likelihood of this Ad to perform well". AI is so shiny that makes people want to jump as fast as they can into that boat but a reasonable objective analysis shows that a huge and not insignificant amount of software problems can still be solved without relying on the "AI black box".
_bxg1 7y ago
> Training a single AI model can cost hundreds of thousands of dollars (or more) in compute resources Why don't they buy their own hardware for this part? The training process doesn't need to be auto-scalable or failure-resistant or distributed across the world. The value proposition of cloud hosting doesn't seem to make sense here. Surely at this price the answer isn't just "it's more convenient"?
- KaiserPro 7y ago
  because you are trading speed for cash. Say you have $8M in funding, and you need to train a model to do x You can either: a) gain access to a system that scale ondemand and allows instant, actionable results. b) hire a infrastructure person, someone to write a K8s deployment system. Another person to come in a throw that all away. Another person to negotiate and buy the hardware, and another to install it. Option b is can be the cheapest in the long term, but it carries the most risk of failing before you've even trained a single model. It also costs time, and if speed to market is your thing, then you're shit out of luck.
  - _bxg1 7y ago
    Why in the world do you need a Kubernetes deployment system to run a single, manual, one-time (or a handful of times), high-compute job?
    - PeterisP 7y ago
      Because that high-compute job needs to be distributed on many, many machines, and if you're using cheap preemptible instances you have to handle machines dropping off and joining in while you're running that single job. It's definitely not something that you can launch manually - perhaps Kubernetes is not the best solution, but you definitely need some automation.
    - dsl 7y ago
      Because when all you have is a hammer, everything looks like a nail. We have become so DevOps and cloud dependent that everyone has forgotten how to just run big systems cheaply and efficiently.
bryanrasmussen 7y ago
Generally the use of the phrase from a great height implies the height is one of morality, intellect, or valor (each of these decreasing in usage), I'm not exactly sure what the great height Andreessen-Horowitz craps from is composed of - maybe money? I think they may just be crapping on them from a reasonable vantage point.
- KaiserPro 7y ago
  The height is not really about morals. Its more about the blast radius of the shit.
  - darwingr 7y ago
    Or like “nuked from orbit”
rossdavidh 7y ago
So, way back in the last millenium, I did my Master's thesis (way smaller deal than a Ph.D. thesis) on neural networks. Since then, I have looked in on it every few years. I think they're cool, I like using them, and writing multi-level backpropagation neural networks used to be one of the first things I'd do in a new language, just to get a feel for how it worked (until pytorch came along and I decided for the first time that using their library was easier than writing my own). So, it's not like I dislike ML. But, saying an investment is an "AI" startup, ought to be like saying it's a python startup, or saying it's a postgres startup. That ought not to be something you tell people as a defining characteristic of what you do, not because it's a secret but rather because it's not that important to your odds of success. If you used a different language and database, you would probably have about the same odds of success, because it depends more on how well you understand the problem space, and how well you architect your software. Linear models or other more traditional statistical models can often perform just as well as DL or any other neural network, for the same reason that when you look at a kaggle leaderboard, the difference between the leaders is usually not that big after a while. The limiting factor is in the data, and how well you have transformed/categorized that data, and all the different methods of ML that get thrown at it all end up with similar looking levels of accuracy. There used to be a saying: "If you don't know how to do it, you don't know how to do it with a computer." AI boosters sometimes sound as if they are suggesting that this is no longer true. They're incorrect. ML is, absolutely, a technique that a good programmer should know about, and may sometimes wish to use, kind of like knowing how a state machine works. It makes no great deal of difference to how likely a business is to succeed.
- 7532yahoogmail 7y ago
  Thank you for the perspective. Now when we talk machine learning are we talking: L. Pachter and B. Sturmfels. Algebraic Statistics for Computational Biology. Cambridge University Press 2005. G. Pistone, E. Riccomango, H. P. Wynn. Algebraic Statistics. CRC Press, 2001. Drton, Mathias, Sturmfels, Bernd, Sullivant, Seth. Lectures on Algebraic Statistics, Springer 2009. Or more like: Watanabe, Sumio. Algebraic Geometry and Statistical Learning Theory, Cambridge University Press 2009. My understanding (I do not do AI or machine learning) that AI is distinct from these more mathematical analytic perspectives. Finally, might we argue that generally AI/ML is more easily suited to data that's already high quality eg. CERN data, trade data, drug trial data as opposed to unconstrained data eg. Find the buses in these 1MM jpegs?
aj7 7y ago
“ Embrace services. There are huge opportunities to meet the market where it stands. That may mean offering a full-stack translation service rather than translation software or running a taxi service rather than selling self-driving cars. Building hybrid businesses is harder than pure software, but this approach can provide deep insight into customer needs and yield fast-growing, market-defining companies. Services can also be a great tool to kickstart a company’s go-to-market engine – see this post for more on this – especially when selling complex and/or brand new technology. The key is pursue one strategy in a committed way, rather than supporting both software and services customers.” Exactly wrong and contradicts most of the thesis of the article - that AI often fails to achieve acceptable models because of the individuality, finickiness, edge cases, and human involvement needed to process customer data sets. The key to profitability is for AI to be a component in a proprietary software package, where the VENDOR studies, determines, and limits the data sets and PRESCRIBES this to the customer, choosing applications many customers agree upon. Edge cases and cat-guacamole situations are detected and ejected, and the AI forms a smaller, but critical efficiency enhancing component of a larger system.
- TheOtherHobbes 7y ago
  The thesis of the article is that this is going to be called consultancy. Single-focus disruptors bad. Generic consultancy good - with ML secret sauce, possibly helped by hired specialist human insight. Companies that can make this work will kill it. Companies that can't will be killed. It's going to be IBM, Oracle, SAP, etc all over again. Within 10 years there will be a dominant monopolistic player in the ML space. It will be selling corporate ML-as-a-service, doing all of that hard data wrangling and model building etc and setting it up for clients as a packaged service using its own economies of scale and "top sales talent" (it says here). That's where the big big big big money will be. Not in individual specialist "We ML'd your pizza order/pet food/music choices/bicycle route to work" startups. Amazon, Google, MS, and maybe the twitching remnants of IBM will be fighting it out in this space. But it's possible they'll get their lunch money stolen by a hungry startup, perhaps in collaboration with someone like McKinsey, or an investment bank, or a quant house with ambitions. 5-10 years after that customisable industrial-grade ML will start trickling down to the personal level. But it will probably have been superseded by primitive AGI by then, which makes prediction difficult - especially about that future.
joshuaellinger 7y ago
I just spent $50K on coloc hardware. I'm taking a $10K/mo Azure spend down to a $1K/mo hosting cost. But the real kicker is that I get x5 the cores, x20 RAM, x10 storage, and a couple of GPUs. I'm running last-generation Infiniband (56gb/sec) and modern U.2 SSDs (say 500MB/sec per device). I figure it is going to take me about $10K in labor to move and then $1K/mo to maintain and pay for services that are bundled in the cloud. And because I have all this dedicated hardware, I don't have to mess around with docker/k8s/etc. It's not really a big data problem but it shows the ROI on owning your own hardware. If you need 100 servers for one day per month, the cloud is amazing. But I do a bunch of resampling, simple models, and interactive BI type stuff, so co-loc wins easily.
- wpietri 7y ago
  I'm sure your right for your case. But I'd add one caveat for those less experienced: if you own the hardware, you need to be prepared to go to the colo when something breaks. The various clouds are a much nicer experience when hardware fails. At the very least people should have enough spare capacity that a hardware failure means going sometime in the next couple of weeks, rather than getting up at 3 am and fixing things under pressure.
  - foobiekr 7y ago
    Operations teams deal with both. You design your system with enough spare capacity that you can live somewhat degraded for a time - you must if only due to the lead time. Software failures are far far far more common than hardware failures so once you combine these, the occasional midnight trip to the colo is both rare and oddly satisfying for hero types.
  - latch 7y ago
    Or take the middle road and just get rent the hardware (aka, dedicated hosting). You pay more than colo but still way less than cloud, get the same level of hardware support as a cloud provider but the same performance as colo.
    - 7y ago
fxtentacle 7y ago
I predict a great future for startups that sell pickaxes, err, tools for AI. AI is like the new gold rush. And just like back then, it's not the gold diggers that will get rich. "Most people in AI forget that the hardest part of building a new AI solution or product is not the AI or algorithms — it’s the data collection and labeling." https://medium.com/startup-grind/fueling-the-ai-gold-rush-7ae438505bc2 https://medium.com/startup-grind/fueling-the-ai-gold-rush-7a... (from 2017)
- moksly 7y ago
  Is it the new gold rush though. I work in a large organisation that has a lot of data and inefficient processes, and we haven’t bought anything. It hasn’t been for a lack of trying. We’ve had everyone from IBM and Microsoft to small local AI startup try to sell us their magic, but no one has come up with anything meaningful to do with our data that our analysis department isn’t already doing without ML/AI. I guess we could replace some of our analysis department with ML/AI, but working with data is only part of what they do, explaining the data and helping our leadership make sound decisions is their primary function, and it’s kind of hard for ML/AI to do that (trust me). What we have learned though, is that even though we have a truck load of data, we can’t actually use it unless we have someone on deck who actually understands it. IBM had a run at it, and they couldn’t get their algorithms to understand anything, not even when we tried to help them. I mean, they did come up with some basic models that their machine spotted/learned by itself by trawling through our data, but nothing we didn’t already have. Because even though we have a lot of data, the quality of it is absolute shite. Which is anecdotal, but it’s terrible because it was generated by thousand of human employees over 40 years, and even though I’m guessing, I doubt we’re unique in that aspect. We’ll continue to do various proof of concepts and listen to what suppliers have to say, but I fully expect most of it to go the way Blockchain did which is where we never actually find a use for it. With a gold rush, you kind of need the nuggets of gold to sell, and I’m just not seeing that with ML/AI. At least no yet.
- hooande 7y ago
  AI != gold. The market for selling tools to people who are essentially chasing buzz words is much smaller than that of selling tools to people extracting scarce metals from the ground. Ultimately the value of selling tools is dependent on the riches being mined actually existing. The value of AI/big data to the average business has yet to be determined
rotrux 7y ago
This is a terrific article. Two thumbs up.
moab 7y ago
I found it fun to read this after reading this other post that made the rounds today about AI automating most programming work and making program optimization irrelevant: https://bartoszmilewski.com/2020/02/24/math-is-your-insurance-policy/ https://bartoszmilewski.com/2020/02/24/math-is-your-insuranc...
blueyes 7y ago
The A16Z piece makes all these points quite clearly. This editorial is trying to put a finer point on a sharp knife.
deleted 7y ago
[deleted]
orasis 7y ago
Nice article. The flip side of the coin is that all these “problems” are potential moats for a well tuned ML company to use to defend market share.
DrNuke 7y ago
You all know a GTX 1070 with 8GB on a gaming laptop with 32GB is still doing wonders and covering 90%+ business cases when coupled with smart & batch techniques the likes of you learn from fast.ai or under direct pytorch implementation, right??
yogrish 7y ago
Now a days DL models are becoming commodities very fast. By the time you train NN to solve a particular problem, a new efficient model is out somewhere and is available public. So you need to go through the process entirely or else you risk losing business. Unless your NN is so unique like you are handcrafting your own in which case you take lot of time to arrive at a best model and you need more PhDs.
- jeremysalwen 7y ago
  Props to the ML community for being so open.
  - fncypants 7y ago
    Open does not mean patent-free.
Proven 7y ago
what's the point of this post? i suggest to simply read the original AH blog post and skip this rehash
inthewoods 7y ago
Having briefly worked for an AI company, I agree with the conclusion that AI companies are more like services businesses than software companies. I would add only one other thing: to me going forward there likely won't be "AI companies" - AI exists to power applications. And in my experience, unless the output is truly differentiated, customers aren't willing to spend more for something "powered by AI" - they just expect that software has evolved to provide the kind of insights that AI sometimes deliver.
- mapgrep 7y ago
  Aren’t software businesses increasingly like service businesses though? They deliver now often with backend cloud storage, update near continuously, integrate frequently with outside services, sometimes open source major components iteratively, typically have an evolving API and developer ecosystem to educate, and are sold as subscriptions. It’s not as “human in the loop” as some of the AI described in this article but it’s clearly moving toward services in terms of margins. Nothing is like the old shrink wrapped software business, basically.
  - inthewoods 7y ago
    Not from what I see - what I see is software companies using services as a way to shorten time-to-value for the customer. They do this either themselves or via professional services firms. To me, the services you describe are software-as-a-service - they scale well without adding more humans to the mix. Services businesses, in contrast, generally need more humans to do more work. I do think you are right that we are entering an age where the margin pressures will continue to increase. As the Amazon quote goes "your margin is my opportunity." In that world, strength accrues to the largest players - which is why AWS is so strong. I like to joke that AWS should refund money to the startup that buy booths at re:Invent only to find out AWS is rolling out a competing service (with the acknowledgement that AWS entering a space doesn't necessarily mean the end of the competing company.)
- shoo 7y ago
  For an example of a genuine software company vaguely in this ecosystem, consider companies that build the tools that some AI/ML/optimisation systems use as building blocks. Eg optimisation algorithms. If you need to solve gnarly industrial scale mixed integer combinatorial optimisation problems in the guts of your ML / optimisation engine, the commercial MIP solvers (gurobi , CPLEX ) or non-MIP based alternative combinatorial optimisation systems (localsolver ) can often give more optimal results in exponentially less running time than free open source alternatives. 1% more optimal solutions might translate into 1% more net profit for the entire org if you've gone whole hog and are trying to systematically profit optimise the entire business, so depending on the scale of the org it might be an easy business case to invest a few million dollars to set this system in place. Annual server licenses for this commerical MIP solver software was 0(100k) / yr per server & the companies that build these products bake a lot of clever tricks from academia into these products that you can exploit by paying the license fee. ( my knowledge of pricing is out of date by about 7 years ) .
jotakami 7y ago
> Better user interfaces are sorely underappreciated. This is why I’m much more excited by AR and VR than AI. Human brains are fucking amazing at certain kinds of data processing and inference and pretty mediocre at others. We should be focusing more on creating interfaces and data visualizations that unlock that superpower for wider applications.
shoo 7y ago
> most people haven’t figured out that ML oriented processes almost never scale like a simpler application would. You will be confronted with the same problem as using SAP; there is a ton of work done up front; all of it custom. I’ll go out on a limb and assert that most of the up front data pipelining and organizational changes which allow for [ML to be used operationally by an org] are probably more valuable than the actual machine learning piece. Strong agreement from me: I've never worked on deploying ML models, but have worked on deploying operations-research type automated decision systems that have somewhat similar data requirements. Most of the work is client org specific in terms of setting up the human & machine processes to define a data pipeline to provide input and consume output of the clever little black box. A lot of this is super idiosyncratic & non repeatable between different client deployments.
- izendejas 7y ago
  That's because, ML and operations-research problems can be simplified to set of optimization problems and the underlying math and statistics are all very similar if not identical in some cases. And the input matters, a lot. So the differentiating factor isn't the models, it's the data and companies like Google figured it out a long time ago. In short, find interesting problems, then the solutions -- not the other way around.
  - divbzero 7y ago
    This is spot on. Hence the open sourcing of ML code while keeping an iron grip on data.
  - killjoywashere 7y ago
    "The data" means more than pure computer science people want to admit. In any "advanced" application, that means annotators. Radiologists drawing circles around cancer, attorneys labeling contract clauses as unacceptable, drivers labeling stop signs, etc. ML is a mining problem. Digitizers are the miners. Annotators are the refiners.
    - joe_the_user 7y ago
NickKampe 7y ago
I guess I won't mention Kubeflow here.....
angry_octet 7y ago
There are many problems which are simply impossible to do with traditional optimisation or human analysis, that ML can do really well at. But I get the sense that this is not the type of problem that these "AI" startups referred to are addressing. Instead its like 'here is a problem I can charge for, with some ML magic it will be easy'. This is classic snake oil. Being able to sift/classify/analyse data with ML really can be a 'moat', an extreme competitive advantage. But using "AI" doesn't automatically get you there. Separately, AWS is an expensive luxury, which is worth it if for some reason you can't manage your own computers. I really annoys me when analysts like this guy mangle together things which are obvious and then comes up with an unsupported conclusion, like "second AI winter is coming man".
dcl 7y ago
I'm not terribly convinced of point 4. > Machine learning will be most productive inside large organizations that have data and process inefficiencies. I strongly believe ML is at worst dangerous and at best pointless here. Data and Process inefficiencies => garbage in, garbage out. ML is NOT a silver bullet in large organisations that have these issues*, I've seen managers try to adopt ML to solve issues, but the results are almost always suspect and/or marginally better than simple if else rules but require a multiple people or teams to get all the data and models right.
MacsHeadroom 7y ago
Well, duh. Unless you invent AGI you're always going to be fitting new models for new clients. The best case scenario is getting bought by a client and becoming their full-time ML tailor. For a pure ML company to IPO they'd have to both solve intelligence and manufacture their own hardware. FOMO screwed a lot of investors who would've been better off buying Google stock.
moandcompany 7y ago
Related to the topic of marginal benefits of AI models versus their costs: Green AI (Roy Schwartz, Jesse Dodge, Noah A. Smith, Oren Etzioni - 2019) https://arxiv.org/abs/1907.10597 https://arxiv.org/abs/1907.10597
tzm 7y ago
I view AI as the application of ML and ML as the implement (tool). Therefor, tooling efficiency is a competitive advantage of good ML projects.
dvfjsdhgfv 7y ago
> In the old days of on-premise software, delivering a product meant stamping out and shipping physical media – the cost of running the software, whether on servers or desktops, was borne by the buyer. Today, with the dominance of SaaS, that cost has been pushed back to the vendor. Most software companies pay big AWS or Azure bills every month – the more demanding the software, the higher the bill. This irrational sheep mentality amuses me. Yes, tehre are some very specific cases where AWS & ca. is clearly a better choice, but for the most cases I saw the TCO with hosting it on premises or renting servers is much lower, sometimes by an order of magnitude (in some cases even more). But people insist on doing it because others do it. We'll soon have an entire generation of engineers completely hooked on AWS & co. and not even realizing other solutions are possible, not to mention lower TCO.
etrk 7y ago
I interviewed at some AI companies a year or two back. They all had teams of people dedicated to support each client: to clean their data, train their models, integrate the domain-specific requirements, customize UIs, etc. They sold themselves as the next AI-powered mega-unicorns, but they were more like boutique consultancies with no obvious path to scale up.
- auxten 7y ago
  "Boutique Consultancy" is quite recapitulative for most AI companies for now. But this may be the only way to empower their clients. One of these startups will find the path to scale up eventually.
laktak 7y ago
> “AI coming for your jobs” meme; AI actually stands for “Alien (or) Immigrant” in this context. Finally a correct use of "AI".
pandascore 7y ago
Agree mostly but he only talk about some AI start-ups that have a 1 to 1 model or at best a 1 to few. There is some AI startups like ours which have a 1 to many model. We use Computer Vision to collect data from video streams and sell data and transformed data through our API. The output of our models is the same for everyone. Cost wise though it's clearly being not knowledgeable about how it works or at least think all AI startups have huge training set. For many companies owning your hardware for training is a very easy step to rationalise cost. It feels like an article written about all AI companies but actually (very) true only for some AI companies.
amai 7y ago
"(my personal bete-noir; the term “AI” when they mean “machine learning”)" This is so right. Using a term "artificial intelligence" for machine learning is like using "artificial horses" to describe cars. It is even worse, since we cannot even define what "natural intelligence" actually is. Stop talking about "artificial intelligence".
- DonHopkins 7y ago
  Or "artificial swans" that "appear even more lifelike". https://www.louwmanmuseum.nl/ontdekken/ontdek-de-collectie/brooke-25-30-hp-swan-car-1910 https://www.louwmanmuseum.nl/ontdekken/ontdek-de-collectie/b... >The bodywork represents a swan gliding through water. The rear is decorated with a lotus flower design finished in gold leaf, an ancient symbol for divine wisdom. Apart from the normal lights, there are electric bulbs in the swan’s eyes that glow eerily in the dark. The car has an exhaust-driven, eight-tone Gabriel horn that can be operated by means of a keyboard at the back of the car. A ship’s telegraph was used to issue commands to the driver. Brushes were fitted to sweep off the elephant dung collected by the tyres. The swan’s beak is linked to the engine’s cooling system and opens wide to allow the driver to spray steam to clear a passage in the streets. Whitewash could be dumped onto the road through a valve at the back of the car to make the swan appear even more lifelike. >The car caused panic and chaos in the streets on its first outing and the police had to intervene.
Zanneth 7y ago
I wonder how much of the formidable amount of computing resources required for deep learning can be attributed to wasteful and inefficient programming practices. A lot of the ML libraries that I see are written in Python with very little attention paid to aspects such as memory usage, cache coherency, concurrency, etc. If we focused on writing more efficient software instead of demanding bigger and faster machines with more and more GPUs, would the cost of ML become more practical? More importantly, as the author pointed out, would smaller companies have a better chance at making advancements in the field?
magwa101 7y ago
Here's what cloud gives you that is very costly to implement internally, cost accountability. Analysts running the same queries over and over would peg internal hardware all the time. When we went to the cloud, we made a budget for each division, problem solved. Same with DS. Give them a blank check, they'll spend it, manage to a budget, they'll do it.
mengibar10 7y ago
I wonder if there are any crips guidelines to lower cloud deployment costs.

695 points by dostoevsky 7y ago

lazzlazzlazz 7y ago

Is the misspelling of "Andreessen-Horowitz" and use of "A19H" instead of "a16z" intentional?

khazhoux 7y ago
You mean the fact that they left out an "s" in Andreessen?
- dang 7y ago
  We've squeezed another s above.
scottlocklin 7y ago
I suck at spelling. If I was one of the cool kids I'd claim to be dyslexic.
- yubozhao 7y ago
  hi OP. We built an open-source library called, BentoML(https://github.com/bentoml/bentoml https://github.com/bentoml/bentoml) to make model inferencing/serving a lot easier for Data scientists in various serving scenarios. Love to hear your thoughts on our library
  - EamonnMR 7y ago
    I was really hoping that you where about to offer an ML framework to improve spelling.

leetrout 7y ago

That is a great write up and very accurate description of both the costs and human intervention based on my experience with “AI” tools.

harias 7y ago

>That’s right; that’s why a lone wolf like me, or a small team can do as good or better a job than some firm with 100x the head count and 100m in VC backing. goes on to say >I agree, but the hockey stick required for VC backing, and the army of Ph.D.s required to make it work doesn’t really mix well with those limited domains, which have a limited market. Choose one? Also assumes running your own data center to be easy. Some people don't want to be up 24x7 monitoring their data center or to buy hardware to accommodate the rare 10 minute peaks in usage.

icheishvili 7y ago
I don't think these are necessarily contradictory. With pytorch-transformers, you can use a full-blown BERT model like the best in the world. And yet, to make this novel and defensible, you would need to build on top of it and innovate significantly, which would require significant capital to achieve.
jjeaff 7y ago
>rare 10 minute peaks But is that really the use case here? I haven't worked in ML. But I'm not seeing where you are going to need to handle a 10 minute spike that requires a whole datacenter. A month's worth of a quad gpu instance on AWS could pay for a server with similar capacity in a few months of usage. And hardware is pretty resilient these days. Especially if you co-locate it in a datacenter that handles all the internet and power up time for you. And when something does go wrong, they offer "magic hands" service to go swap out hardware for you. Colocation is surprisingly cheap. As is leasing 'managed' equipment.
detaro 7y ago
> Some people don't want to be up 24x7 monitoring their data center or to buy hardware to accommodate the rare 10 minute peaks in usage. Do you need that for training workloads, and what percentage of a startups workload is training?
bsenftner 7y ago
I ran a small data cluster for years, the horsepower behind my startup. Other than the Chinese DDoS attacks, running the cluster was absolutely elementary. The idea that running a server or a band of servers is difficult is a bold faced lie. People have got to stop repeating the cloud propaganda.

raiyu 7y ago

The number of places where machine learning can be used effectively from both a cost perspective and a return perspective are small. They are usually tremendously large datasets at gigantic companies, and they probably have to build in house expertise because it's hard to package this up into a product and resell it for various industries, datasets, etc. Certainly something like autonomous driving needs machine learning to function, but again, these are going to be owned by large corporations, and even when a startup is successful, it's really about the layered technology on-top of machine learning that makes it interesting. It's kind of like what Kelsey Hightower said about Kubernetes. It's interesting and great, but what will really matter is what service you put on top of it, so much so that whether you use Kubernetes becomes irrelevant. So I think companies that are focusing on a specific problem, providing that value added service, building it through machine learning, can be successful. While just broadly deploying machine learning as a platform in and of itself can be very challenging. And I think the autonomous driving space is a great example of that. They are building a value added service in a particular vertical, with tremendous investment, progress, and potentially life changing tech down the road. But as a consumer it's really the autonomous driving that is interesting, not whether they are using AI/machine learning to get there.

Q6T46nT668w6i3m 7y ago
How would you explain the rise (and success) of machine learning in science? A lab that uses some learning-based method will likely be limited to just one or two people (responsible for data acquisition, feature engineering, evaluation, etc.) and extremely finite data.
- semi-extrinsic 7y ago
  How do you define success? Adoption? Because right now, writing "we will use machine learning to solve X" in a grant proposal is an easy way to increase chances of getting funding.
- ska 7y ago
  It's not clear there has been any deep impact actually, but there has been a lot of discussion (and grant proposals) I've seen a lot of cross pollination of ML and AI techniques into various disciplines. A large percentage just didn't work at all, most of the rest were more "kind of interesting, but". Nothing earthshaking happened although pop sci press likes to talk about it a lot. If you have more digital data than you used to, using modern free frameworks and toolkits to do basic (i.e. older, boring, but understood) ML stuff to understand it seems to have a reasonable return. Mostly I think this is because it becomes accessible to someone without much background in the area, and you can do reasonable things without having to put 6 months of reading and implementing together before starting.

seibelj 7y ago

I wrote an article I published a week ago about how AI is the biggest misnomer in tech history https://medium.com/@seibelj/the-artificial-intelligence-scam-is-imploding-34b156c3537e https://medium.com/@seibelj/the-artificial-intelligence-scam... I wrote it to be tongue-in-cheek in a ranting style, but essentially "AI" businesses and the technology underpinning it are not the silver bullet the media and marketing hype has made it out to be. The linked article about a16z shows how AI is the same story everywhere - enormous capital to get the data and engineers to automate, but even the "good" AI still gets it wrong much of the time, necessitating endless edge-cases, human intervention, and eventually it's a giant ball of poorly-understand and impossible to maintain pipelines that don't even provide a better result than a few humans with a spreadsheet.

scottlocklin 7y ago
Coming from a fellow masshole: that's a great rant. There was this meme in the 70s about "self driving cars" following magnetic strips in the road in restricted highways. I remember at the time, being, like 8 and thinking "sure seems like an overly complicated train."
- seibelj 7y ago
  Thanks man! Lifelong masshole here. Your post was much better than mine, but I appreciate the comment.

m0zg 7y ago

"Huge compute bills" usually come from training, or to be more precise, hyperparameter search that's required before you find a model that works well. You could also fail to find such a model, but that's another discussion. So yeah, you could spend one or two FTE salaries' (or one deep learning PhD's) worth of cash on finding such models for your startup if you insist on helping Jeff Bezos to wipe his tears with crisp hundred dollar bills. That's if you know what you're doing of course. Literally unlimited amounts could be spent if you don't. Or you could do the same for a fraction of the cost by stuffing a rack in your office with consumer grade 2080ti's. Just don't call it a "datacenter" or NVIDIA will have a stroke. Is that too much money? Not in most typical cases, I'd think. If the competitive advantage of what you're doing with DL does not offset the cost of 2 meatspace FTEs, you're doing it wrong. That, once again, assumes that you know what you're doing, and aren't doing deep learning for the sake of deep learning. Also, if your startup is venture funded, AWS will give you $100K in credit, hoping that you waste it by misconfiguring your instances and not paying attention to their extremely opaque billing (which is what most of their startup customers proceed to doing pretty much straight away). If you do not make these mistakes, that $100K will last for some time, after which you could build out the aforementioned rack full of 2080ti's on prem.

avip 7y ago
For balance, all big cloud providers - aws, gcp, azure, oracle [0] have pretty similar startup plans. Y$$MV (I'm in full agreement with everything you've written + it's well-phrased and funny. gj!) [0] that's not a typo - there is such thing as "Oracle cloud"
alephnan 7y ago
> Just don't call it a "datacenter" or NVIDIA will have a stroke. Context please :) ?
- mereel 7y ago
  Just a guess but maybe it's some licensing issue? https://www.nvidia.com/en-us/drivers/geforce-license/ https://www.nvidia.com/en-us/drivers/geforce-license/ No Datacenter Deployment. The SOFTWARE is not licensed for datacenter deployment, except that blockchain processing in a datacenter is permitted.

marmaduke 7y ago

I sometimes contribute to methodology projects in neuroscience ("AI" for scientists). The most tiring part of it is explaining essentially these things over and over. Very interesting to see the sentiment vindicated in Startupistan.

atulkum 7y ago

On the other hand some of the startup is doing absolutely fraud on the name of AI.I went to a self checkout store (AIFI.io). I did not touch anything but they charge me $35.10. According to the receipt I took 17 packs of snacks :) These guys are doing fraud on the name of AI. They have no technology no software just put up some camera and open a store so that they can defraud the investor. Anyone can try if intersted https://www.aifi.io/loop-case-study https://www.aifi.io/loop-case-study

mtkd 7y ago

AI on the algo side is only half the story -- it has to sit in a domain specific framework to be most effective I see a lot of 'bolt-on' tech emerging -- it looks mostly snake oil -- there is no obvious way to be competitive against teams that baked it in to the bare metal design Also most commercial use-cases I've seen need effective ML more than anything else

dang 7y ago

A thread about the original article, from a few days ago: https://news.ycombinator.com/item?id=22352750 https://news.ycombinator.com/item?id=22352750

correlator 7y ago

No need to look at AZ for this. If you're building "AI" I wish you a speedy road to being acquired by a company that can put it to use. You've become a high priced recruiting firm. If you're solving a real problem and use ML in service of solving that problem, then you've got a great moat....happy trusting customers. It's not complicated

motohagiography 7y ago
Sssh! Valuations are a function of projected market size and opacity of the problem. Clarity like this collapses the uncertainty and destroys value. If you pour enough capital into rooms full of PhD's something's gotta hit. My way of saying, you're very, very right.

ativzzz 7y ago

I agree with the author's opinion about > I’ll go out on a limb and assert that most of the up front data pipelining and organizational changes which allow for it are probably more valuable than the actual machine learning piece. Especially at non-tech companies with outdated internal technology. I've consulted at one of these and the biggest wins from the project (I left before the whole thing finished unfortunately) were overall improvements to the internal data pipeline, such as standardization and consolidation of similar or identical data from different business units.

noelsusman 7y ago
I do data science at a non-tech company with outdated internal technology and I've seen this over and over again. Honestly though, it's worth every penny because often the only way to get the resources to truly solve data pipeline issues is to get an executive to buy some crap from a vendor and force everyone to make it work.
jotakami 7y ago
I was a consultant at one of the giant outsourcers and nod my head vigorously at this comment. The least sexy projects were MDM (master data management) but they were absolutely essential to the success of any other fancy analytics/BI/ML project.
- 2sk21 7y ago
  Interestingly I too worked on MDM systems about ten years ago, when I was at IBM Research. Ironically, one of my first ideas for applying machine learning was in de-duplication of data in an MDM server. However the technology was a bit too primitive back in 2010 and the project was a hard sell so it was abandoned.

allovernow 7y ago

All of this might be true currently, but that's because this current first generation "AI" (technically should just be called ML) is mostly bullshit. To clarify, I don't mean anyone is lying or selling snake oil - what I mean by bullshit is that the vast majority of these services are cooked up by software developers without any background in mathematics, selling adtechy services in domains like product recommendation and sentiment analysis. They are single discipline applications accessable to devs without science backgrounds and do not rely on substantial expertise from other fields. That makes them narrow in technical scope and easy to rip off (hence no moat, lots of competition, and human reliance and lack of actual software). The next generation of Machine Learning is just emerging, and looks nothing like this. Funds are being raised, patents are being filed, and everything is in early stage development, so you probably haven't heard much yet - but these ML startups are going after real problems in industry: cross disciplinary applications leveraging the power of heuristic learning to make cross disciplinary designs and decisions currently still limited to the human domain. I'm talking about the kind of heuristics which currently exist only as human intuition expressed most compactly as concept graphs and, especially, mathematical relationships - e.g. component design with stress and materials constraints, geologic model building, treatment recommendation from a corpus of patient data, etc. ML solutions for problems like these cannot be developed without an intimate understanding of the problem domain. This is a generalist's game. I predict that the most successful ML engineers of the next decade will be those with hard STEM backgrounds, MS and PhD level, who have transitioned to ML. [Un]Fortunately for us, the current buzzwordy types of ML services give the rest of us a bad name, but looking at these upcoming applications the answers to the article tl;dr look different: >Deep learning costs a lot in compute, for marginal payoffs The payoffs here are far greater. Designs are in the pipeline which augment industry roles - accelerate design by replacing finite methods with vastly quicker ML for unprecedented iteration. Produce meaningful suggestions during the development of 3D designs. Fetch related technical documents in real time by scanning the progressive design as the engineer works, parsing and probabilistically suggesting alternative paths to research progression. Think Bonzi Buddy on steroids...this is a place for recurring software licenses, not SaaS. >Machine learning startups generally have no moat or meaningful special sauce For solving specific, technical problems, neural network design requires a certain degree of intuition with respect to the flow of information through the network, which both optimizes and limits the kind of patterns that a given net can learn. Thus designing NN for hard-industry applications is predicated upon an intimate understanding of domain knowledge, and these highly specialized neural nets become patentable secret sauces. That's half of the most - the other comes from competition for the software developers with first-hand experience in these fields, or a general enough math heavy background to capture the relationships that are being distilled into nets. >Machine learning startups are mostly services businesses, not software businesses Again only true because most current applications are NLP adtechy bullshit. Imagine coding in an IDE powered by an AI (multiple interacting neural nets) which guides the structure of your code at a high level and flags bugs as you write. This, at a more practical level, is the type of software that will eventually change every technical discipline, and you can sell licenses! >Machine learning will be most productive inside large organizations that have data and process inefficiencies This next generation goes far past simply optimizing production lines or counting missed pennies or extracting a couple extra percent of value from analytics data. This style of applied ML operates at a deeper level of design which will change everything.

whoisjuan 7y ago

An many times all these AI computations go into solving mundane problems like "What's the likelihood of this Ad to perform well". AI is so shiny that makes people want to jump as fast as they can into that boat but a reasonable objective analysis shows that a huge and not insignificant amount of software problems can still be solved without relying on the "AI black box".

_bxg1 7y ago

> Training a single AI model can cost hundreds of thousands of dollars (or more) in compute resources Why don't they buy their own hardware for this part? The training process doesn't need to be auto-scalable or failure-resistant or distributed across the world. The value proposition of cloud hosting doesn't seem to make sense here. Surely at this price the answer isn't just "it's more convenient"?

KaiserPro 7y ago
because you are trading speed for cash. Say you have $8M in funding, and you need to train a model to do x You can either: a) gain access to a system that scale ondemand and allows instant, actionable results. b) hire a infrastructure person, someone to write a K8s deployment system. Another person to come in a throw that all away. Another person to negotiate and buy the hardware, and another to install it. Option b is can be the cheapest in the long term, but it carries the most risk of failing before you've even trained a single model. It also costs time, and if speed to market is your thing, then you're shit out of luck.
- _bxg1 7y ago
  Why in the world do you need a Kubernetes deployment system to run a single, manual, one-time (or a handful of times), high-compute job?
  - PeterisP 7y ago
    Because that high-compute job needs to be distributed on many, many machines, and if you're using cheap preemptible instances you have to handle machines dropping off and joining in while you're running that single job. It's definitely not something that you can launch manually - perhaps Kubernetes is not the best solution, but you definitely need some automation.
  - dsl 7y ago
    Because when all you have is a hammer, everything looks like a nail. We have become so DevOps and cloud dependent that everyone has forgotten how to just run big systems cheaply and efficiently.

bryanrasmussen 7y ago

Generally the use of the phrase from a great height implies the height is one of morality, intellect, or valor (each of these decreasing in usage), I'm not exactly sure what the great height Andreessen-Horowitz craps from is composed of - maybe money? I think they may just be crapping on them from a reasonable vantage point.

KaiserPro 7y ago
The height is not really about morals. Its more about the blast radius of the shit.
- darwingr 7y ago
  Or like “nuked from orbit”

rossdavidh 7y ago

So, way back in the last millenium, I did my Master's thesis (way smaller deal than a Ph.D. thesis) on neural networks. Since then, I have looked in on it every few years. I think they're cool, I like using them, and writing multi-level backpropagation neural networks used to be one of the first things I'd do in a new language, just to get a feel for how it worked (until pytorch came along and I decided for the first time that using their library was easier than writing my own). So, it's not like I dislike ML. But, saying an investment is an "AI" startup, ought to be like saying it's a python startup, or saying it's a postgres startup. That ought not to be something you tell people as a defining characteristic of what you do, not because it's a secret but rather because it's not that important to your odds of success. If you used a different language and database, you would probably have about the same odds of success, because it depends more on how well you understand the problem space, and how well you architect your software. Linear models or other more traditional statistical models can often perform just as well as DL or any other neural network, for the same reason that when you look at a kaggle leaderboard, the difference between the leaders is usually not that big after a while. The limiting factor is in the data, and how well you have transformed/categorized that data, and all the different methods of ML that get thrown at it all end up with similar looking levels of accuracy. There used to be a saying: "If you don't know how to do it, you don't know how to do it with a computer." AI boosters sometimes sound as if they are suggesting that this is no longer true. They're incorrect. ML is, absolutely, a technique that a good programmer should know about, and may sometimes wish to use, kind of like knowing how a state machine works. It makes no great deal of difference to how likely a business is to succeed.

7532yahoogmail 7y ago
Thank you for the perspective. Now when we talk machine learning are we talking: L. Pachter and B. Sturmfels. Algebraic Statistics for Computational Biology. Cambridge University Press 2005. G. Pistone, E. Riccomango, H. P. Wynn. Algebraic Statistics. CRC Press, 2001. Drton, Mathias, Sturmfels, Bernd, Sullivant, Seth. Lectures on Algebraic Statistics, Springer 2009. Or more like: Watanabe, Sumio. Algebraic Geometry and Statistical Learning Theory, Cambridge University Press 2009. My understanding (I do not do AI or machine learning) that AI is distinct from these more mathematical analytic perspectives. Finally, might we argue that generally AI/ML is more easily suited to data that's already high quality eg. CERN data, trade data, drug trial data as opposed to unconstrained data eg. Find the buses in these 1MM jpegs?

aj7 7y ago

“ Embrace services. There are huge opportunities to meet the market where it stands. That may mean offering a full-stack translation service rather than translation software or running a taxi service rather than selling self-driving cars. Building hybrid businesses is harder than pure software, but this approach can provide deep insight into customer needs and yield fast-growing, market-defining companies. Services can also be a great tool to kickstart a company’s go-to-market engine – see this post for more on this – especially when selling complex and/or brand new technology. The key is pursue one strategy in a committed way, rather than supporting both software and services customers.” Exactly wrong and contradicts most of the thesis of the article - that AI often fails to achieve acceptable models because of the individuality, finickiness, edge cases, and human involvement needed to process customer data sets. The key to profitability is for AI to be a component in a proprietary software package, where the VENDOR studies, determines, and limits the data sets and PRESCRIBES this to the customer, choosing applications many customers agree upon. Edge cases and cat-guacamole situations are detected and ejected, and the AI forms a smaller, but critical efficiency enhancing component of a larger system.

TheOtherHobbes 7y ago
The thesis of the article is that this is going to be called consultancy. Single-focus disruptors bad. Generic consultancy good - with ML secret sauce, possibly helped by hired specialist human insight. Companies that can make this work will kill it. Companies that can't will be killed. It's going to be IBM, Oracle, SAP, etc all over again. Within 10 years there will be a dominant monopolistic player in the ML space. It will be selling corporate ML-as-a-service, doing all of that hard data wrangling and model building etc and setting it up for clients as a packaged service using its own economies of scale and "top sales talent" (it says here). That's where the big big big big money will be. Not in individual specialist "We ML'd your pizza order/pet food/music choices/bicycle route to work" startups. Amazon, Google, MS, and maybe the twitching remnants of IBM will be fighting it out in this space. But it's possible they'll get their lunch money stolen by a hungry startup, perhaps in collaboration with someone like McKinsey, or an investment bank, or a quant house with ambitions. 5-10 years after that customisable industrial-grade ML will start trickling down to the personal level. But it will probably have been superseded by primitive AGI by then, which makes prediction difficult - especially about that future.

joshuaellinger 7y ago

I just spent $50K on coloc hardware. I'm taking a $10K/mo Azure spend down to a $1K/mo hosting cost. But the real kicker is that I get x5 the cores, x20 RAM, x10 storage, and a couple of GPUs. I'm running last-generation Infiniband (56gb/sec) and modern U.2 SSDs (say 500MB/sec per device). I figure it is going to take me about $10K in labor to move and then $1K/mo to maintain and pay for services that are bundled in the cloud. And because I have all this dedicated hardware, I don't have to mess around with docker/k8s/etc. It's not really a big data problem but it shows the ROI on owning your own hardware. If you need 100 servers for one day per month, the cloud is amazing. But I do a bunch of resampling, simple models, and interactive BI type stuff, so co-loc wins easily.

wpietri 7y ago
I'm sure your right for your case. But I'd add one caveat for those less experienced: if you own the hardware, you need to be prepared to go to the colo when something breaks. The various clouds are a much nicer experience when hardware fails. At the very least people should have enough spare capacity that a hardware failure means going sometime in the next couple of weeks, rather than getting up at 3 am and fixing things under pressure.
- foobiekr 7y ago
  Operations teams deal with both. You design your system with enough spare capacity that you can live somewhat degraded for a time - you must if only due to the lead time. Software failures are far far far more common than hardware failures so once you combine these, the occasional midnight trip to the colo is both rare and oddly satisfying for hero types.
- latch 7y ago
  Or take the middle road and just get rent the hardware (aka, dedicated hosting). You pay more than colo but still way less than cloud, get the same level of hardware support as a cloud provider but the same performance as colo.
  - 7y ago

fxtentacle 7y ago

I predict a great future for startups that sell pickaxes, err, tools for AI. AI is like the new gold rush. And just like back then, it's not the gold diggers that will get rich. "Most people in AI forget that the hardest part of building a new AI solution or product is not the AI or algorithms — it’s the data collection and labeling." https://medium.com/startup-grind/fueling-the-ai-gold-rush-7ae438505bc2 https://medium.com/startup-grind/fueling-the-ai-gold-rush-7a... (from 2017)

moksly 7y ago
Is it the new gold rush though. I work in a large organisation that has a lot of data and inefficient processes, and we haven’t bought anything. It hasn’t been for a lack of trying. We’ve had everyone from IBM and Microsoft to small local AI startup try to sell us their magic, but no one has come up with anything meaningful to do with our data that our analysis department isn’t already doing without ML/AI. I guess we could replace some of our analysis department with ML/AI, but working with data is only part of what they do, explaining the data and helping our leadership make sound decisions is their primary function, and it’s kind of hard for ML/AI to do that (trust me). What we have learned though, is that even though we have a truck load of data, we can’t actually use it unless we have someone on deck who actually understands it. IBM had a run at it, and they couldn’t get their algorithms to understand anything, not even when we tried to help them. I mean, they did come up with some basic models that their machine spotted/learned by itself by trawling through our data, but nothing we didn’t already have. Because even though we have a lot of data, the quality of it is absolute shite. Which is anecdotal, but it’s terrible because it was generated by thousand of human employees over 40 years, and even though I’m guessing, I doubt we’re unique in that aspect. We’ll continue to do various proof of concepts and listen to what suppliers have to say, but I fully expect most of it to go the way Blockchain did which is where we never actually find a use for it. With a gold rush, you kind of need the nuggets of gold to sell, and I’m just not seeing that with ML/AI. At least no yet.
hooande 7y ago
AI != gold. The market for selling tools to people who are essentially chasing buzz words is much smaller than that of selling tools to people extracting scarce metals from the ground. Ultimately the value of selling tools is dependent on the riches being mined actually existing. The value of AI/big data to the average business has yet to be determined

rotrux 7y ago

This is a terrific article. Two thumbs up.

moab 7y ago

I found it fun to read this after reading this other post that made the rounds today about AI automating most programming work and making program optimization irrelevant: https://bartoszmilewski.com/2020/02/24/math-is-your-insurance-policy/ https://bartoszmilewski.com/2020/02/24/math-is-your-insuranc...

blueyes 7y ago

The A16Z piece makes all these points quite clearly. This editorial is trying to put a finer point on a sharp knife.

deleted 7y ago

[deleted]

orasis 7y ago

Nice article. The flip side of the coin is that all these “problems” are potential moats for a well tuned ML company to use to defend market share.

DrNuke 7y ago

You all know a GTX 1070 with 8GB on a gaming laptop with 32GB is still doing wonders and covering 90%+ business cases when coupled with smart & batch techniques the likes of you learn from fast.ai or under direct pytorch implementation, right??

yogrish 7y ago

Now a days DL models are becoming commodities very fast. By the time you train NN to solve a particular problem, a new efficient model is out somewhere and is available public. So you need to go through the process entirely or else you risk losing business. Unless your NN is so unique like you are handcrafting your own in which case you take lot of time to arrive at a best model and you need more PhDs.

jeremysalwen 7y ago
Props to the ML community for being so open.
- fncypants 7y ago
  Open does not mean patent-free.

Proven 7y ago

what's the point of this post? i suggest to simply read the original AH blog post and skip this rehash

inthewoods 7y ago

Having briefly worked for an AI company, I agree with the conclusion that AI companies are more like services businesses than software companies. I would add only one other thing: to me going forward there likely won't be "AI companies" - AI exists to power applications. And in my experience, unless the output is truly differentiated, customers aren't willing to spend more for something "powered by AI" - they just expect that software has evolved to provide the kind of insights that AI sometimes deliver.

mapgrep 7y ago
Aren’t software businesses increasingly like service businesses though? They deliver now often with backend cloud storage, update near continuously, integrate frequently with outside services, sometimes open source major components iteratively, typically have an evolving API and developer ecosystem to educate, and are sold as subscriptions. It’s not as “human in the loop” as some of the AI described in this article but it’s clearly moving toward services in terms of margins. Nothing is like the old shrink wrapped software business, basically.
- inthewoods 7y ago
  Not from what I see - what I see is software companies using services as a way to shorten time-to-value for the customer. They do this either themselves or via professional services firms. To me, the services you describe are software-as-a-service - they scale well without adding more humans to the mix. Services businesses, in contrast, generally need more humans to do more work. I do think you are right that we are entering an age where the margin pressures will continue to increase. As the Amazon quote goes "your margin is my opportunity." In that world, strength accrues to the largest players - which is why AWS is so strong. I like to joke that AWS should refund money to the startup that buy booths at re:Invent only to find out AWS is rolling out a competing service (with the acknowledgement that AWS entering a space doesn't necessarily mean the end of the competing company.)
shoo 7y ago
For an example of a genuine software company vaguely in this ecosystem, consider companies that build the tools that some AI/ML/optimisation systems use as building blocks. Eg optimisation algorithms. If you need to solve gnarly industrial scale mixed integer combinatorial optimisation problems in the guts of your ML / optimisation engine, the commercial MIP solvers (gurobi , CPLEX ) or non-MIP based alternative combinatorial optimisation systems (localsolver ) can often give more optimal results in exponentially less running time than free open source alternatives. 1% more optimal solutions might translate into 1% more net profit for the entire org if you've gone whole hog and are trying to systematically profit optimise the entire business, so depending on the scale of the org it might be an easy business case to invest a few million dollars to set this system in place. Annual server licenses for this commerical MIP solver software was 0(100k) / yr per server & the companies that build these products bake a lot of clever tricks from academia into these products that you can exploit by paying the license fee. ( my knowledge of pricing is out of date by about 7 years ) .

jotakami 7y ago

> Better user interfaces are sorely underappreciated. This is why I’m much more excited by AR and VR than AI. Human brains are fucking amazing at certain kinds of data processing and inference and pretty mediocre at others. We should be focusing more on creating interfaces and data visualizations that unlock that superpower for wider applications.

shoo 7y ago

> most people haven’t figured out that ML oriented processes almost never scale like a simpler application would. You will be confronted with the same problem as using SAP; there is a ton of work done up front; all of it custom. I’ll go out on a limb and assert that most of the up front data pipelining and organizational changes which allow for [ML to be used operationally by an org] are probably more valuable than the actual machine learning piece. Strong agreement from me: I've never worked on deploying ML models, but have worked on deploying operations-research type automated decision systems that have somewhat similar data requirements. Most of the work is client org specific in terms of setting up the human & machine processes to define a data pipeline to provide input and consume output of the clever little black box. A lot of this is super idiosyncratic & non repeatable between different client deployments.

izendejas 7y ago
That's because, ML and operations-research problems can be simplified to set of optimization problems and the underlying math and statistics are all very similar if not identical in some cases. And the input matters, a lot. So the differentiating factor isn't the models, it's the data and companies like Google figured it out a long time ago. In short, find interesting problems, then the solutions -- not the other way around.
- divbzero 7y ago
  This is spot on. Hence the open sourcing of ML code while keeping an iron grip on data.
- killjoywashere 7y ago
  "The data" means more than pure computer science people want to admit. In any "advanced" application, that means annotators. Radiologists drawing circles around cancer, attorneys labeling contract clauses as unacceptable, drivers labeling stop signs, etc. ML is a mining problem. Digitizers are the miners. Annotators are the refiners.
  - joe_the_user 7y ago

NickKampe 7y ago

I guess I won't mention Kubeflow here.....

angry_octet 7y ago

There are many problems which are simply impossible to do with traditional optimisation or human analysis, that ML can do really well at. But I get the sense that this is not the type of problem that these "AI" startups referred to are addressing. Instead its like 'here is a problem I can charge for, with some ML magic it will be easy'. This is classic snake oil. Being able to sift/classify/analyse data with ML really can be a 'moat', an extreme competitive advantage. But using "AI" doesn't automatically get you there. Separately, AWS is an expensive luxury, which is worth it if for some reason you can't manage your own computers. I really annoys me when analysts like this guy mangle together things which are obvious and then comes up with an unsupported conclusion, like "second AI winter is coming man".

dcl 7y ago

I'm not terribly convinced of point 4. > Machine learning will be most productive inside large organizations that have data and process inefficiencies. I strongly believe ML is at worst dangerous and at best pointless here. Data and Process inefficiencies => garbage in, garbage out. ML is NOT a silver bullet in large organisations that have these issues*, I've seen managers try to adopt ML to solve issues, but the results are almost always suspect and/or marginally better than simple if else rules but require a multiple people or teams to get all the data and models right.

MacsHeadroom 7y ago

Well, duh. Unless you invent AGI you're always going to be fitting new models for new clients. The best case scenario is getting bought by a client and becoming their full-time ML tailor. For a pure ML company to IPO they'd have to both solve intelligence and manufacture their own hardware. FOMO screwed a lot of investors who would've been better off buying Google stock.

moandcompany 7y ago

Related to the topic of marginal benefits of AI models versus their costs: Green AI (Roy Schwartz, Jesse Dodge, Noah A. Smith, Oren Etzioni - 2019) https://arxiv.org/abs/1907.10597 https://arxiv.org/abs/1907.10597

tzm 7y ago

I view AI as the application of ML and ML as the implement (tool). Therefor, tooling efficiency is a competitive advantage of good ML projects.

dvfjsdhgfv 7y ago

> In the old days of on-premise software, delivering a product meant stamping out and shipping physical media – the cost of running the software, whether on servers or desktops, was borne by the buyer. Today, with the dominance of SaaS, that cost has been pushed back to the vendor. Most software companies pay big AWS or Azure bills every month – the more demanding the software, the higher the bill. This irrational sheep mentality amuses me. Yes, tehre are some very specific cases where AWS & ca. is clearly a better choice, but for the most cases I saw the TCO with hosting it on premises or renting servers is much lower, sometimes by an order of magnitude (in some cases even more). But people insist on doing it because others do it. We'll soon have an entire generation of engineers completely hooked on AWS & co. and not even realizing other solutions are possible, not to mention lower TCO.

etrk 7y ago

I interviewed at some AI companies a year or two back. They all had teams of people dedicated to support each client: to clean their data, train their models, integrate the domain-specific requirements, customize UIs, etc. They sold themselves as the next AI-powered mega-unicorns, but they were more like boutique consultancies with no obvious path to scale up.

auxten 7y ago
"Boutique Consultancy" is quite recapitulative for most AI companies for now. But this may be the only way to empower their clients. One of these startups will find the path to scale up eventually.

laktak 7y ago

> “AI coming for your jobs” meme; AI actually stands for “Alien (or) Immigrant” in this context. Finally a correct use of "AI".

pandascore 7y ago

Agree mostly but he only talk about some AI start-ups that have a 1 to 1 model or at best a 1 to few. There is some AI startups like ours which have a 1 to many model. We use Computer Vision to collect data from video streams and sell data and transformed data through our API. The output of our models is the same for everyone. Cost wise though it's clearly being not knowledgeable about how it works or at least think all AI startups have huge training set. For many companies owning your hardware for training is a very easy step to rationalise cost. It feels like an article written about all AI companies but actually (very) true only for some AI companies.

amai 7y ago

"(my personal bete-noir; the term “AI” when they mean “machine learning”)" This is so right. Using a term "artificial intelligence" for machine learning is like using "artificial horses" to describe cars. It is even worse, since we cannot even define what "natural intelligence" actually is. Stop talking about "artificial intelligence".

DonHopkins 7y ago
Or "artificial swans" that "appear even more lifelike". https://www.louwmanmuseum.nl/ontdekken/ontdek-de-collectie/brooke-25-30-hp-swan-car-1910 https://www.louwmanmuseum.nl/ontdekken/ontdek-de-collectie/b... >The bodywork represents a swan gliding through water. The rear is decorated with a lotus flower design finished in gold leaf, an ancient symbol for divine wisdom. Apart from the normal lights, there are electric bulbs in the swan’s eyes that glow eerily in the dark. The car has an exhaust-driven, eight-tone Gabriel horn that can be operated by means of a keyboard at the back of the car. A ship’s telegraph was used to issue commands to the driver. Brushes were fitted to sweep off the elephant dung collected by the tyres. The swan’s beak is linked to the engine’s cooling system and opens wide to allow the driver to spray steam to clear a passage in the streets. Whitewash could be dumped onto the road through a valve at the back of the car to make the swan appear even more lifelike. >The car caused panic and chaos in the streets on its first outing and the police had to intervene.

Zanneth 7y ago

I wonder how much of the formidable amount of computing resources required for deep learning can be attributed to wasteful and inefficient programming practices. A lot of the ML libraries that I see are written in Python with very little attention paid to aspects such as memory usage, cache coherency, concurrency, etc. If we focused on writing more efficient software instead of demanding bigger and faster machines with more and more GPUs, would the cost of ML become more practical? More importantly, as the author pointed out, would smaller companies have a better chance at making advancements in the field?

magwa101 7y ago

Here's what cloud gives you that is very costly to implement internally, cost accountability. Analysts running the same queries over and over would peg internal hardware all the time. When we went to the cloud, we made a budget for each division, problem solved. Same with DS. Give them a blank check, they'll spend it, manage to a budget, they'll do it.

mengibar10 7y ago

I wonder if there are any crips guidelines to lower cloud deployment costs.