10 ms·

Machine learning in UK financial services

139 points by goatinaboat 7y ago

goatinaboat 7y ago
The key takeaway from this for me is that there is no shortage of machine learning practitioners in the UK despite popular opinion that it is a good career move
- onion2k 7y ago
  I don't quite see how you can come to that conclusion. The report is only about companies that responded to the survey: "The survey asked firms to provide information on two ML case studies within their organisation." Any company that doesn't have two ML case studies to talk about isn't included. That might be 10% of finance companies or it might be 90%. We can't know from what's in the report.
  - goatinaboat 7y ago
    Charts 11 and 12 show that lack of talent is a minor consideration. I would be willing to bet that most people would have expected it to be much higher.
    - chillydawg 7y ago
      Self selected for firms who have big enough teams to respond to the study. Bias.
- bogle 7y ago
  Where I am at the moment, a UK bank's retail data analysis service, any ML projects are set up on a "learn as you go" basis whilst still doing your main job. No specialised practitioners are deemed required. Whether or not this is a good idea is moot: this is in the bowels of the bank and not the bleeding edge of commerce.
  - faceplanted 7y ago
    We don't really need specific ML practitioners for most jobs in the same way we don't need Genetic Algorithm practitioners in most jobs either, unless you're a researcher you can just read a book about it and start trying it out.
    - goatinaboat 7y ago
      Indeed. If you already understand the data and the business and you already know Python or R then you can download Keras in the morning and have something useful to the business running in the afternoon. Then you can add ML to your CV. It’s not clear where an ML expert who isn’t already familiar with the data and the business fits in or adds value, or even what an expert really means in industry.
      - mellosouls 7y ago
        Adding ML to your CV with the Mickey Mouse experience and understanding you describe would be a good way to get yourself drummed out of a job where you are actually expected to have the expertise you claim. Agree with your point about domain knowledge being important.
      - ska 7y ago
        Domain knowledge is really important, you can't overstate that. But you've also succinctly described why a huge percentage of ML as practiced in industry under performs or flat out fails. To a first approximation the person who reads a few tutorials and plays with example data sets, then sets out to apply it to their own domain, has no real idea what they are doing - and it shows.
    - atupis 7y ago
      It is more like do you specialist x, eg if you have fullstack engineer do you really need frontend developer or DevOps engineer, generally no but there is case where specialist might make big differences.
      - 7y ago
- spratzt 7y ago
  Absolutely true. I have several friends who are machine learning practitioners. Many are actively looking to get out. When you strip away the hype and BS the demand in London is very thin.
  - tixocloud 7y ago
    There are likely very specific areas where ML can have a real impact and other areas that are nice, interesting but not really priority. From my experience, a large part of the demand also needs to be driven by the AI and ML execs' ability to sell to the business.
    - goatinaboat 7y ago
      If you have deep domain knowledge already then ML skills are the icing on the cake. But opportunities for those with just the ML part are minimal.
      - tixocloud 7y ago
        Agreed - many folks discount deep domain knowledge which is, my opinion, a mistake.
    - deleted 7y ago
      [deleted]
  - shrikant 7y ago
    In my experience, this is usually because they realise that applying sexy algorithms and generating magic earth-shattering insights is about 2% of the job, and the rest is "boring" stuff like automation, data cleansing, performance at scale, and building and maintaining discipline around sound development practices. Edited to add: this isn't a diss on them or this phenomenon in general -- it keeps me employed :)
- deleted 7y ago
  [deleted]
- m-i-l 7y ago
  Indeed. The Bank of England's report was summarised by one news source yesterday as "Banks say there's no shortage of machine learning talent"[0]. [0] https://news.efinancialcareers.com/uk-en/3002418/machine-learning-jobs-banking https://news.efinancialcareers.com/uk-en/3002418/machine-lea...
- angry_octet 7y ago
  It just tells me that senior banking people in the UK have such a shortage of senior ML people that all they see is mickey mouse ML stuff by shysters and snake oilers. They would have to do real work on the antique back end to make it work, and that is hard.
hnarn 7y ago
My understanding of the history of the stock market is that it was nothing short of a revolution in the sense that "normal people" suddenly could invest in large companies, and that these new types of publicly traded companies through the power of what we today would call "crowdsourcing" managed to bypass the old giants through their accountability to their shareholders and their ability to raise capital from the masses. So far, so (pretty) good. Next, I'm reminded of when Elon Musk talked about integrated AI, and how we may one day end up in a situation where any type of intellectual or physical competition between humans, in the labor market or elsewhere, may just come down to how much money they have to spend on cybernetic, biological enhancement products. My synthesis here is that from my perspective, the stock market has since the advent of digital trading tools, now vastly accelerated by expanded usage of machine learning in financial markets, completely undermined the original benefit of the stock market, where a pretty true democratization of the economy, almost an incredibly true one considering it was born in a capitalist society, has now been rolled back and undermined by "big capital" finding a tool that allows them to rig the game to their favor. So if the assumption is correct that machine learning in financial markets essentially is just an interface for the already-rich to beat the never-rich by throwing more money at the problem, doesn't it stand to reason that these types of tools should be heavily regulated, if we want a stock market that has any resemblance of fairness and democratic influence on the economy? I understand very well we are far away from this today, and I understand that rich people will not allow regulations that punish them, but if we go beyond the knee-jerk reaction of "cash rules baby", is this really the way things should be?
- bogle 7y ago
  Tim Harford in "The Undercover Economist" has a neat description of how the market isn't rational and really can't be predictable. So I'm not worried about machines in the market as they can't really do any better, I believe. As for the super-rich, they have asset managers who pool family funds. Even a small family fund will have millions invested and a small asset manager will have billions under management (pick your own major currency). Every millionaire will be doing the same thing so none of them get an advantage and going against the herd would be unlikely to pay off, short to mid-term (again, my reading of "The Undercover Economist").
- 7y ago
osullivj 7y ago
Interesting to see that insurance is ahead in adoption, compared to capital markets, where I work. Suspect that the tree based models reported as popular are used for underwriting decision making. We sales & trading capital markets people tend to see insurance as the sleepy backwater of wholesale capital; just cash cow real money funds to be gouged by sharp market makers to put it bluntly.
- w0rkaccount 7y ago
  Why is this? Insurance providers actually provide a valuable service (not necessarily implying that much of the enormous UK financial industry does not...) and is profitable and sustainable; I am surprised to hear it described as something to be "gouged". How would such a gouging occur?
  - disgruntledphd2 7y ago
    It's worth noting that traditionally, actual insurance runs at a loss or break-even, most of the profits tend to come from investment. This has changed recently, as investment returns have dropped, and may also be driving the adoption of better statistical techniques for modelling risk (as they can't rely on investment income to make up for insurance losses). Also, having read this report, I'm very very sceptical about whether or not companies are using "ML". I suspect that most of them are just doing linear/logistic regression on larger data sizes, which isn't really the same.
    - mlthoughts2018 7y ago
      If the linear / logistic regression stuff is improved through Bayesian methods and hierarchical modeling, and if it grows large enough that there is a need to use MCMC sampling techniques for approximate posterior inference, then even though the model specification itself would be very simple, I’d absolutely say this is “ML”.
      - disgruntledphd2 7y ago
mlmbinaryplan 7y ago
The concept is on air, thanks for sharing informative information.
tixocloud 7y ago
Thanks for sharing. Was literally just reading it a few minutes ago with great intrigue. Would be interesting to see what is classified as real ML and how much of it is being used for actual decision making - I lead an ML team at a bank and everyone tries to sell it more than it is. Fraud detection is one of the areas I've come across that are actually doing something interesting within retail banking. But agreed - plenty of opportunities for ML to make things more efficient.
- bigred100 7y ago
  A a paraphrase of a kind of self-deprecating but insightful comment from a ML prof I heard: There’s a lot of hype around machine learning, but it can’t really do that much. If you want to see what it’s capable of, check your Amazon recommendations. Sort of works some of the time, and other times offers you complete nonsense. Amazon has very smart engineers and the best algorithms and tons of data. This is about the limit of what you can currently do.
BenoitEssiambre 7y ago
It's hard not to be cynical reading these discussions. Avoiding plastic or recycling it does near zero for the environment. Reducing travelling, reducing goods consumption, reducing construction, reusing things and protecting land are the real effective solutions but require sacrifice. Yet people refuse to go beyond performing ridiculous plastic straw theatre.
- anovikov 7y ago
  Well this is not the way. We don't want or need to improve environment by 'reducing travel, consumption or construction' i.e. destroying the economy and going back to stone age. Goal is to optimize resource use to improve environment while INCREASING all of that steadily.
  - ben_w 7y ago
    Hypothetically, if you had to choose, which would you choose — the economy or the environment?
    - coherentpony 7y ago
      Those aren't mutually exclusive.
      - hn_temp_10_1_19 7y ago
        Learn to read.
    - anovikov 7y ago
      Of course economy. Environment is just one of the externalities. Also, environmental issues are mostly just a pretext for geopolitics right? Say, real deal about global warming isn't saving some turtles or some Pacific islands from sinking, but crippling bad guys such as Russians and Arabs who export oil, right? It's a way of putting liberals, unknowable to them, in the same directions as conservatives are.
ThetaOneOne 7y ago
Maybe before we rush to adopt machine learning in financial services we should consider the consequences of blithely giving this technology such a central position in our lives.
hogFeast 7y ago
Watching the UK try to adopt ML is like watching a toddler try to drive a Ferrari. The BoE prides itself on being "up" with the latest talking points amongst the chattering classes. But take a look at how the BoE is actually run and you will get an idea why British society struggles with change. Incompetence and knowledge of irrelevant information is pride of place in British society. Example: Carney has been the most accident prone BoE governor since...the last one, who was trying to raise rates in 2008...the Chief Economist was head of Financial Stability in 2006...yes, seriously (incidentally, he recovered his reputation by being able to hit headlines by talking about new trendy topics like ML). Where ML has been implemented in the UK it is often by managers who have almost no competence in any real-world skills. They have heard about ML, think it is magic, and want to look like they know what they are doing. This has resulted in very bad outcomes (a recent story - https://www.theguardian.com/society/2019/oct/15/councils-using-algorithms-make-welfare-decisions-benefits https://www.theguardian.com/society/2019/oct/15/councils-usi...). I don't think this effect is at all well understood (economists, for example, tend to assume that the person in charge actually knows what they are doing...despite all the research suggesting the contrary in the UK). Insurance has done well because there is a good level of statistical knowledge amongst managers. But I suspect the UK will continue to lag the world (as it does in almost everything else) as the hype disappears, and the typical British suspicion of new technology and change returns.
- 7952 7y ago
  I work in a corporate and am constantly amazed how much currency buzz words seem to have. We got a new CEO who is into "tech" and different directors are fighting to deploy poorly though out AI and VR projects. It has little benefit to the business but it helps people sell themselves within the company.
  - bigred100 7y ago
    After working on a completely failed project in a non-software department of a large organization, I’ve realized that (depending where you are) the main deliverable for a lot of things is what your manager can put in his power point presentation to his manager. Of course if the project lasts long enough the software being reasonably good starts to matter as you start finding out things you promised people don’t work anymore, your huge rats nest of code makes it too hard to add features like, etc. But until it hits the “power point bottom line” a non technically educated manager will not care at all.
ArtWomb 7y ago
Thanks for sharing ;) Of interest as well may be London Fintech week July 4 2020 https://www.fintechweek.com/ https://www.fintechweek.com/
- deleted 7y ago
  [deleted]
bigred100 7y ago
My frank opinion as someone ~2 years into grad school in something to do with computers and statistical inference (taken a bunch of numerical classes, ML courses, maybe ~1 year of grad level stats coursework): if you actually want to do statistical analysis for a living, you need get a statistics degree. CS people are irrelevant except for computational issues. As a CS guy probably you can be useful for scaling a system to do automated inference, or as an advanced database guy, or writing shell scripts or something. If you go to grad school in CS and work hard in the right subfield you might develop the understanding of someone with part of a stats BS. Again, the part you would actually know something about is computation, which as far as I can tell isn’t a real bottleneck for anything except very large automated systems. Otherwise some guy can just use R and his laptop. I mentioned my status above because this is solely based on me evaluating things in an academic setting and not job experience, but this seems very clear to me once laid out like above. ML is “hot” but it seems to me like an extraordinarily specialized area of expertise with little general applicability.
- mistrial9 7y ago
  agree but -- previous eras of statistics used important and non-obvious ways to infer from subsets of data.. now, whole sets are used, and the subset part is more like a representation problem. Second, the things that are made to be important in school, are not always important outside of school. You do acknowledge that tacitly there.. The point of view in the post here is from an individual, regarding the work of another (hypothetical) individual, while the paper linked spends quite a lot of effort to characterize in the real world, real activity by large and very large, organizations, mainly business, and mainly the business of money. You conclude that the field has "little general applicability" and that might be true in some sense.. but large organizations, and particularly large finance organizations, are not general at all, but rather composed of a very large number of specialized and repetitive operations.. exactly what ML and AI perform, no?

139 points by goatinaboat 7y ago

The key takeaway from this for me is that there is no shortage of machine learning practitioners in the UK despite popular opinion that it is a good career move

onion2k 7y ago
I don't quite see how you can come to that conclusion. The report is only about companies that responded to the survey: "The survey asked firms to provide information on two ML case studies within their organisation." Any company that doesn't have two ML case studies to talk about isn't included. That might be 10% of finance companies or it might be 90%. We can't know from what's in the report.
- goatinaboat 7y ago
  Charts 11 and 12 show that lack of talent is a minor consideration. I would be willing to bet that most people would have expected it to be much higher.
  - chillydawg 7y ago
    Self selected for firms who have big enough teams to respond to the study. Bias.
bogle 7y ago
Where I am at the moment, a UK bank's retail data analysis service, any ML projects are set up on a "learn as you go" basis whilst still doing your main job. No specialised practitioners are deemed required. Whether or not this is a good idea is moot: this is in the bowels of the bank and not the bleeding edge of commerce.
- faceplanted 7y ago
  We don't really need specific ML practitioners for most jobs in the same way we don't need Genetic Algorithm practitioners in most jobs either, unless you're a researcher you can just read a book about it and start trying it out.
  - goatinaboat 7y ago
    Indeed. If you already understand the data and the business and you already know Python or R then you can download Keras in the morning and have something useful to the business running in the afternoon. Then you can add ML to your CV. It’s not clear where an ML expert who isn’t already familiar with the data and the business fits in or adds value, or even what an expert really means in industry.
    - mellosouls 7y ago
      Adding ML to your CV with the Mickey Mouse experience and understanding you describe would be a good way to get yourself drummed out of a job where you are actually expected to have the expertise you claim. Agree with your point about domain knowledge being important.
    - ska 7y ago
      Domain knowledge is really important, you can't overstate that. But you've also succinctly described why a huge percentage of ML as practiced in industry under performs or flat out fails. To a first approximation the person who reads a few tutorials and plays with example data sets, then sets out to apply it to their own domain, has no real idea what they are doing - and it shows.
  - atupis 7y ago
    It is more like do you specialist x, eg if you have fullstack engineer do you really need frontend developer or DevOps engineer, generally no but there is case where specialist might make big differences.
    - 7y ago
spratzt 7y ago
Absolutely true. I have several friends who are machine learning practitioners. Many are actively looking to get out. When you strip away the hype and BS the demand in London is very thin.
- tixocloud 7y ago
  There are likely very specific areas where ML can have a real impact and other areas that are nice, interesting but not really priority. From my experience, a large part of the demand also needs to be driven by the AI and ML execs' ability to sell to the business.
  - goatinaboat 7y ago
    If you have deep domain knowledge already then ML skills are the icing on the cake. But opportunities for those with just the ML part are minimal.
    - tixocloud 7y ago
      Agreed - many folks discount deep domain knowledge which is, my opinion, a mistake.
  - deleted 7y ago
    [deleted]
- shrikant 7y ago
  In my experience, this is usually because they realise that applying sexy algorithms and generating magic earth-shattering insights is about 2% of the job, and the rest is "boring" stuff like automation, data cleansing, performance at scale, and building and maintaining discipline around sound development practices. Edited to add: this isn't a diss on them or this phenomenon in general -- it keeps me employed :)
deleted 7y ago
[deleted]
m-i-l 7y ago
Indeed. The Bank of England's report was summarised by one news source yesterday as "Banks say there's no shortage of machine learning talent"[0]. [0] https://news.efinancialcareers.com/uk-en/3002418/machine-learning-jobs-banking https://news.efinancialcareers.com/uk-en/3002418/machine-lea...
angry_octet 7y ago
It just tells me that senior banking people in the UK have such a shortage of senior ML people that all they see is mickey mouse ML stuff by shysters and snake oilers. They would have to do real work on the antique back end to make it work, and that is hard.

hnarn 7y ago

My understanding of the history of the stock market is that it was nothing short of a revolution in the sense that "normal people" suddenly could invest in large companies, and that these new types of publicly traded companies through the power of what we today would call "crowdsourcing" managed to bypass the old giants through their accountability to their shareholders and their ability to raise capital from the masses. So far, so (pretty) good. Next, I'm reminded of when Elon Musk talked about integrated AI, and how we may one day end up in a situation where any type of intellectual or physical competition between humans, in the labor market or elsewhere, may just come down to how much money they have to spend on cybernetic, biological enhancement products. My synthesis here is that from my perspective, the stock market has since the advent of digital trading tools, now vastly accelerated by expanded usage of machine learning in financial markets, completely undermined the original benefit of the stock market, where a pretty true democratization of the economy, almost an incredibly true one considering it was born in a capitalist society, has now been rolled back and undermined by "big capital" finding a tool that allows them to rig the game to their favor. So if the assumption is correct that machine learning in financial markets essentially is just an interface for the already-rich to beat the never-rich by throwing more money at the problem, doesn't it stand to reason that these types of tools should be heavily regulated, if we want a stock market that has any resemblance of fairness and democratic influence on the economy? I understand very well we are far away from this today, and I understand that rich people will not allow regulations that punish them, but if we go beyond the knee-jerk reaction of "cash rules baby", is this really the way things should be?

bogle 7y ago
Tim Harford in "The Undercover Economist" has a neat description of how the market isn't rational and really can't be predictable. So I'm not worried about machines in the market as they can't really do any better, I believe. As for the super-rich, they have asset managers who pool family funds. Even a small family fund will have millions invested and a small asset manager will have billions under management (pick your own major currency). Every millionaire will be doing the same thing so none of them get an advantage and going against the herd would be unlikely to pay off, short to mid-term (again, my reading of "The Undercover Economist").
7y ago

osullivj 7y ago

Interesting to see that insurance is ahead in adoption, compared to capital markets, where I work. Suspect that the tree based models reported as popular are used for underwriting decision making. We sales & trading capital markets people tend to see insurance as the sleepy backwater of wholesale capital; just cash cow real money funds to be gouged by sharp market makers to put it bluntly.

w0rkaccount 7y ago
Why is this? Insurance providers actually provide a valuable service (not necessarily implying that much of the enormous UK financial industry does not...) and is profitable and sustainable; I am surprised to hear it described as something to be "gouged". How would such a gouging occur?
- disgruntledphd2 7y ago
  It's worth noting that traditionally, actual insurance runs at a loss or break-even, most of the profits tend to come from investment. This has changed recently, as investment returns have dropped, and may also be driving the adoption of better statistical techniques for modelling risk (as they can't rely on investment income to make up for insurance losses). Also, having read this report, I'm very very sceptical about whether or not companies are using "ML". I suspect that most of them are just doing linear/logistic regression on larger data sizes, which isn't really the same.
  - mlthoughts2018 7y ago
    If the linear / logistic regression stuff is improved through Bayesian methods and hierarchical modeling, and if it grows large enough that there is a need to use MCMC sampling techniques for approximate posterior inference, then even though the model specification itself would be very simple, I’d absolutely say this is “ML”.
    - disgruntledphd2 7y ago

mlmbinaryplan 7y ago

The concept is on air, thanks for sharing informative information.

tixocloud 7y ago

Thanks for sharing. Was literally just reading it a few minutes ago with great intrigue. Would be interesting to see what is classified as real ML and how much of it is being used for actual decision making - I lead an ML team at a bank and everyone tries to sell it more than it is. Fraud detection is one of the areas I've come across that are actually doing something interesting within retail banking. But agreed - plenty of opportunities for ML to make things more efficient.

bigred100 7y ago
A a paraphrase of a kind of self-deprecating but insightful comment from a ML prof I heard: There’s a lot of hype around machine learning, but it can’t really do that much. If you want to see what it’s capable of, check your Amazon recommendations. Sort of works some of the time, and other times offers you complete nonsense. Amazon has very smart engineers and the best algorithms and tons of data. This is about the limit of what you can currently do.

BenoitEssiambre 7y ago

It's hard not to be cynical reading these discussions. Avoiding plastic or recycling it does near zero for the environment. Reducing travelling, reducing goods consumption, reducing construction, reusing things and protecting land are the real effective solutions but require sacrifice. Yet people refuse to go beyond performing ridiculous plastic straw theatre.

anovikov 7y ago
Well this is not the way. We don't want or need to improve environment by 'reducing travel, consumption or construction' i.e. destroying the economy and going back to stone age. Goal is to optimize resource use to improve environment while INCREASING all of that steadily.
- ben_w 7y ago
  Hypothetically, if you had to choose, which would you choose — the economy or the environment?
  - coherentpony 7y ago
    Those aren't mutually exclusive.
    - hn_temp_10_1_19 7y ago
      Learn to read.
  - anovikov 7y ago
    Of course economy. Environment is just one of the externalities. Also, environmental issues are mostly just a pretext for geopolitics right? Say, real deal about global warming isn't saving some turtles or some Pacific islands from sinking, but crippling bad guys such as Russians and Arabs who export oil, right? It's a way of putting liberals, unknowable to them, in the same directions as conservatives are.

ThetaOneOne 7y ago

Maybe before we rush to adopt machine learning in financial services we should consider the consequences of blithely giving this technology such a central position in our lives.

hogFeast 7y ago

Watching the UK try to adopt ML is like watching a toddler try to drive a Ferrari. The BoE prides itself on being "up" with the latest talking points amongst the chattering classes. But take a look at how the BoE is actually run and you will get an idea why British society struggles with change. Incompetence and knowledge of irrelevant information is pride of place in British society. Example: Carney has been the most accident prone BoE governor since...the last one, who was trying to raise rates in 2008...the Chief Economist was head of Financial Stability in 2006...yes, seriously (incidentally, he recovered his reputation by being able to hit headlines by talking about new trendy topics like ML). Where ML has been implemented in the UK it is often by managers who have almost no competence in any real-world skills. They have heard about ML, think it is magic, and want to look like they know what they are doing. This has resulted in very bad outcomes (a recent story - https://www.theguardian.com/society/2019/oct/15/councils-using-algorithms-make-welfare-decisions-benefits https://www.theguardian.com/society/2019/oct/15/councils-usi...). I don't think this effect is at all well understood (economists, for example, tend to assume that the person in charge actually knows what they are doing...despite all the research suggesting the contrary in the UK). Insurance has done well because there is a good level of statistical knowledge amongst managers. But I suspect the UK will continue to lag the world (as it does in almost everything else) as the hype disappears, and the typical British suspicion of new technology and change returns.

7952 7y ago
I work in a corporate and am constantly amazed how much currency buzz words seem to have. We got a new CEO who is into "tech" and different directors are fighting to deploy poorly though out AI and VR projects. It has little benefit to the business but it helps people sell themselves within the company.
- bigred100 7y ago
  After working on a completely failed project in a non-software department of a large organization, I’ve realized that (depending where you are) the main deliverable for a lot of things is what your manager can put in his power point presentation to his manager. Of course if the project lasts long enough the software being reasonably good starts to matter as you start finding out things you promised people don’t work anymore, your huge rats nest of code makes it too hard to add features like, etc. But until it hits the “power point bottom line” a non technically educated manager will not care at all.

ArtWomb 7y ago

Thanks for sharing ;) Of interest as well may be London Fintech week July 4 2020 https://www.fintechweek.com/ https://www.fintechweek.com/

deleted 7y ago
[deleted]

bigred100 7y ago

My frank opinion as someone ~2 years into grad school in something to do with computers and statistical inference (taken a bunch of numerical classes, ML courses, maybe ~1 year of grad level stats coursework): if you actually want to do statistical analysis for a living, you need get a statistics degree. CS people are irrelevant except for computational issues. As a CS guy probably you can be useful for scaling a system to do automated inference, or as an advanced database guy, or writing shell scripts or something. If you go to grad school in CS and work hard in the right subfield you might develop the understanding of someone with part of a stats BS. Again, the part you would actually know something about is computation, which as far as I can tell isn’t a real bottleneck for anything except very large automated systems. Otherwise some guy can just use R and his laptop. I mentioned my status above because this is solely based on me evaluating things in an academic setting and not job experience, but this seems very clear to me once laid out like above. ML is “hot” but it seems to me like an extraordinarily specialized area of expertise with little general applicability.

mistrial9 7y ago
agree but -- previous eras of statistics used important and non-obvious ways to infer from subsets of data.. now, whole sets are used, and the subset part is more like a representation problem. Second, the things that are made to be important in school, are not always important outside of school. You do acknowledge that tacitly there.. The point of view in the post here is from an individual, regarding the work of another (hypothetical) individual, while the paper linked spends quite a lot of effort to characterize in the real world, real activity by large and very large, organizations, mainly business, and mainly the business of money. You conclude that the field has "little general applicability" and that might be true in some sense.. but large organizations, and particularly large finance organizations, are not general at all, but rather composed of a very large number of specialized and repetitive operations.. exactly what ML and AI perform, no?