9 ms·

A couple weeks ago I was asked to meet with two actuaries. Their company (a big one, I'm working for them on contract) is trying to promote the use of Machine L

A couple weeks ago I was asked to meet with two actuaries. Their company (a big one, I'm working for them on contract) is trying to promote the use of Machine Learning and when they heard I had just completed post-grad in ML, they were eager to pick my brains. In a 60 minute meeting, we spent 5 minutes discussing how ML worked and 55 minutes going in circles. They had lots of data, but no problems to solve. And there are only so many ways to explain that a large dataset is not equivalent to a business problem. I'll be sending this on to them.

ggg9990 8y ago
If actuaries don’t have business problems that could use ML what the heck do they do all day? Insurance is a great target for ML.
- HenryTheHorse 8y ago
  That's a really good point. Discovering patterns and correlations is an actuary's only competency.
  - sulam 8y ago
    Which may mean that all the statistical techniques ML would use are fairly rudimentary for them.
- lordnacho 8y ago
  As with many lines of work, the cool part may be a very small part of their day. With insurance, there's probably an already entrenched way to think about the data that's been there for decades, as well as regulations that help to entrench it. It might also be the case that the business problem isn't risk analysis at all, as one might expect. It may be finding investments for the float that provide a sensible return within the regulatory remit is harder than figuring out how much needs to be charged for dinging someone's car.
  - nerdponx 8y ago
    Correct on all points. There is also the issue that the pricing models themselves need to be able to stand up to some form of regulatory scrutiny (depending on the state and the nature of the insurance product).
jcims 8y ago
I’ve been noodling a little with ML for infosec and if I spent an hour with a pro I might wind up doing the same thing. The problem is likely to be one of a cacophony of choice. It’s not that they don’t have any problems, it’s that they have a million of them and have no solid footing from which to frame their answer in the form of a question. I bet if you picked one field in one dataset and started drilling down into how they could possibly use it, the use cases will start to melt out of their frozen brains. Get ready.
- Kalium 8y ago
  I can think of some really useful infosec applications. "Is this binary malicious?" springs quickly to mind.
  - jcims 8y ago
    There are a bunch. Just about any will-instrumented platform presents many opportunities to do basically side-channel detection of compromise (e.g. cpu utilization, network utilziation, network flows, syslog data, process tree, etc)
- robertk 8y ago
  Maybe they could use an ML model to narrow down which one of their million use cases would benefit most from an ML model. ;)
ssivark 8y ago
While I get what you're saying, isn't that a very "supervised ML" way of thinking? In a world where unsupervised learning was easier to apply, it might be possible to start with data, compress it to a low-dimensional manifold, and use that understanding as inspiration for better directions/questions to pose. So I guess the real problem is that humans expect anything branded as "intelligence" to have the ability to learn in an unsupervised manner, while breakthroughs are closer to the regime where the goals are supervised while the solutions are discovered. Of course, the amount of supervision probably lies on a continuum, between everything being completely specified (classical programming) and everything unsupervised ("true" "human" intelligence).
- taeric 8y ago
  Lots of data does nothing to help when the person you are talking to is not given a) knobs to turn, and b) a goal to achieve. To that end, just having loads of data doesn't necessarily help. You also have to have loads of control over some part of the system. And even then, you can't expect ML to help, all on its own. So, the worst is folks that think just because you have full trace logs of your system, it should be trivial to apply anomaly detection. At face value, yes. Yes you can. But also at face value, your trace logs aren't free and it isn't like folks are keen on letting their detection system cut off anomalous transactions without some sort of kill switch.
  - ssivark 8y ago
    I think my point may not have gotten across clearly, so let me clarify. I agree with most of what you said, which IMHO comes under the spirit of supervised machine learning paradigm, which is the most advanced tool we have pragmatically available to use today. Looking beyond, if we find a model+learner which can discover a low dimensional latent space (through certain indirectly specified biases) then that low dimensional formulation of the domain can guide us towards interesting questions worth asking. To have a factorized low-dimensional formulation is roughly what it means to "understand" a subject, so such a toolkit would be enormously useful. Many people (including some experts, rightly or wrongly) believe that neural networks might be that model class, and (clever tweaks of) gradient descent might be an acceptable learner. All I'm saying is that the current hype about AI fails to separate the potential of the latter class from the currently available successful tools of the former class.

ggg9990 8y ago

If actuaries don’t have business problems that could use ML what the heck do they do all day? Insurance is a great target for ML.

HenryTheHorse 8y ago
That's a really good point. Discovering patterns and correlations is an actuary's only competency.
- sulam 8y ago
  Which may mean that all the statistical techniques ML would use are fairly rudimentary for them.
lordnacho 8y ago
As with many lines of work, the cool part may be a very small part of their day. With insurance, there's probably an already entrenched way to think about the data that's been there for decades, as well as regulations that help to entrench it. It might also be the case that the business problem isn't risk analysis at all, as one might expect. It may be finding investments for the float that provide a sensible return within the regulatory remit is harder than figuring out how much needs to be charged for dinging someone's car.
- nerdponx 8y ago
  Correct on all points. There is also the issue that the pricing models themselves need to be able to stand up to some form of regulatory scrutiny (depending on the state and the nature of the insurance product).

jcims 8y ago

I’ve been noodling a little with ML for infosec and if I spent an hour with a pro I might wind up doing the same thing. The problem is likely to be one of a cacophony of choice. It’s not that they don’t have any problems, it’s that they have a million of them and have no solid footing from which to frame their answer in the form of a question. I bet if you picked one field in one dataset and started drilling down into how they could possibly use it, the use cases will start to melt out of their frozen brains. Get ready.

Kalium 8y ago
I can think of some really useful infosec applications. "Is this binary malicious?" springs quickly to mind.
- jcims 8y ago
  There are a bunch. Just about any will-instrumented platform presents many opportunities to do basically side-channel detection of compromise (e.g. cpu utilization, network utilziation, network flows, syslog data, process tree, etc)
robertk 8y ago
Maybe they could use an ML model to narrow down which one of their million use cases would benefit most from an ML model. ;)

ssivark 8y ago

While I get what you're saying, isn't that a very "supervised ML" way of thinking? In a world where unsupervised learning was easier to apply, it might be possible to start with data, compress it to a low-dimensional manifold, and use that understanding as inspiration for better directions/questions to pose. So I guess the real problem is that humans expect anything branded as "intelligence" to have the ability to learn in an unsupervised manner, while breakthroughs are closer to the regime where the goals are supervised while the solutions are discovered. Of course, the amount of supervision probably lies on a continuum, between everything being completely specified (classical programming) and everything unsupervised ("true" "human" intelligence).

taeric 8y ago
Lots of data does nothing to help when the person you are talking to is not given a) knobs to turn, and b) a goal to achieve. To that end, just having loads of data doesn't necessarily help. You also have to have loads of control over some part of the system. And even then, you can't expect ML to help, all on its own. So, the worst is folks that think just because you have full trace logs of your system, it should be trivial to apply anomaly detection. At face value, yes. Yes you can. But also at face value, your trace logs aren't free and it isn't like folks are keen on letting their detection system cut off anomalous transactions without some sort of kill switch.
- ssivark 8y ago
  I think my point may not have gotten across clearly, so let me clarify. I agree with most of what you said, which IMHO comes under the spirit of supervised machine learning paradigm, which is the most advanced tool we have pragmatically available to use today. Looking beyond, if we find a model+learner which can discover a low dimensional latent space (through certain indirectly specified biases) then that low dimensional formulation of the domain can guide us towards interesting questions worth asking. To have a factorized low-dimensional formulation is roughly what it means to "understand" a subject, so such a toolkit would be enormously useful. Many people (including some experts, rightly or wrongly) believe that neural networks might be that model class, and (clever tweaks of) gradient descent might be an acceptable learner. All I'm saying is that the current hype about AI fails to separate the potential of the latter class from the currently available successful tools of the former class.