5 ms·
Forecasting with uncertainty
- refrigerator 7y agoHey folks, I'm one of the authors of the post. Would love to hear people's thoughts, and happy to answer any questions :) If you liked this, then I'd definitely recommend reading "The Flaw of Averages" by Sam Savage. It talks about this stuff in a lot more detail, and the style is very entertaining — not at all what you'd expect from a statistics book.
- 03199618 7y agoIs there anything causal as in Pearl’s graphical models or the potential outcome framework?
- refrigerator 7y agoAfraid not, sorry — right now it's more of a Monte Carlo simulation tool. Can see how the name might be a bit misleading!
- 03199618 7y agoIf causal.app does not make causal statements of that kind, which kind of causal statements can it make then?
- _delirium 7y agoNot involved with causal.app, but I see it as going the other way as far as causality goes. Pearl-style models are trying to infer causation from data. Here, you give it a causal model that encapsulates how you think a scenario works (which variables affect the outcome, how, and a range of likely values for them), and it simulates your model to give you ranges of likely outcomes, plus some other things like sensitivity analysis (which variables impact likely outcomes the most). I like the comparison they make to spreadsheet models. It's that style of modeling but with Monte Carlo simulations, so you can put ranges instead of single numbers in cells, and the ranges propagate through the model to output ranges.
- refrigerator 7y agoCouldn't have put it better myself :)
- sails 7y agoThis is great. I think you're well positioned to help people understand this very important concept in a familiar way. Have you thought to incorporate the "Minimise Maximum Regret" concept? I suppose it is quite an opinionated strategy, but it is a very interesting approach to decisioning, and supposedly leads to robust decisions.
- sails 7y agoIn the visualisation it would be very useful if the "best/worst" cases included the inputs that were relevant to make that happen. It is somewhat obvious, but would drive the point home that should X & Y occur, you'd be left in scenario Z.
- deleted 7y ago[deleted]
- jerednel 7y agoI like the concept. Who is this targeted at though? Having been an analytics mgr at couple of ad agencies my common questions were... Given current impression share for my campaigns, what would incremental spend do for CPA and position? What would be the expected CPA to move from avg position of 1.8 to 1.4 Can I tie CRM data to get value of leads/opportunities and optimize for that?
- diab0lic 7y agoI would imagine it is targeted at a similar audience, it just lets you answer a better question. Given the current impression share for my campaigns, what would incremental spend do for CPA and position with 95% certainty? What would be the expected CPA in 50% of cases to move from avg position of 1.8 to 1.4? This question can be asked in reverse again with a degree of certainty attached. The answers to the questions you posted are likely to be averages and possibly medians if the posterior is Gaussian (or close enough). Mixing uncertainty into the equation allows you to provide a floor on probabilistic expectation, it also allows you to ask what the best case is, say the 95th percentile. I've stuck to 95 and medians here but you can use any percentile you'd feel comfortable with.
- refrigerator 7y agoGlad you like the concept! The post, and Causal, are targeted at anyone who needs to work with numbers — With those ad questions, there's no getting around the fact that you need to come up with a 'model' for how, e.g. incremental spend impacts CPA/position. Once you have a model you're happy with, there's always going to be uncertainty around aspects of it, and you'll ideally want to account for it. Simulation is a good way to do that. Causal has data connections that you can use to pull stuff from your CRM. Depending on what you mean by 'optimize', we might be able to do that too — happy to chat in more detail if you're interested: taimur @ causal . app
- jerednel 7y agoyeah i suppose i was just wondering how flexible it was and how it fits in the day-to-day of addressing client inquiries. and by optimize in this sense i guess i meant analyze - as in tie that data into forecast projections. i'll play with it and see!
- clairity 7y agotoo bad casual.app is so tightly coupled to the google ecosystem, even enough to say it's optimized for chrome (terrible for privacy/confidentiality). we really need deeper and wider understanding of decision-making under risk and uncertainty (see over-reaction to covid by many, under-reaction by some). tooling like this can help. it's hard to evaluate the modeling capabilities from a few pages and blog posts. how does it compare to crystal ball on excel, for example?
- refrigerator 7y agoCausal's a bit like a spreadsheet and a bit like programming — you can use it to construct arbitrary models by creating "variables" and linking them together with "formulas". Compared to Crystal Ball, it can do all the distribution stuff — variables can take on any kind of distribution you define. But we (try to) make it trivial to use these — you can write formulas like "5 to 10", and we map that onto a sensible distribution for you, so you don't need any probability/stats knowledge, whereas CB is a lot more technical. The CB suite also now includes some other tools that can do things like optimisation, which Causal doesn't do yet. Causal works in most modern browsers, and we're working on handling non-Chrome browsers better :) Not sure what you mean by "coupled to the google ecosystem"!
- owenshen24 7y agoThere's also Guesstimate, which I think is similar: https://www.getguesstimate.com/?ref=producthunt https://www.getguesstimate.com/?ref=producthunt