4 ms·

Having just read this book, I believe that this commentary is wrong on nearly all points. Specifically: - The ratings agencies...did not accidentally have bad

Having just read this book, I believe that this commentary is wrong on nearly all points. Specifically: - The ratings agencies...did not accidentally have bad underlying models. He first talks about how the models were defective, and then goes on at length to describe the perverse incentives for developing and keeping these models, and holds the responsible parties to the fire for willful ignorance and unabated greed. He could have made some of these points more strongly, but I don't think that he skirted over the real issues. - the only goal of a modeler is to produce an accurate model. Actually he speaks quite a bit about the reasons that various forecasters generate models in the first place, and their motivations and responsibilities. For example, the Weather Channel's skewed "wet" forecasts. They will, for example, predict a 20% chance of rain if their models show only a 5% chance. On the one hand it is a bit dishonest, but on the other hand there is a net utility in doing so. Part of the problem is that the general public doesn't understand what the percentages really signify, so it may be of some benefit to emphasize that it really could rain, it's just not very likely, so it behooves people to at least have some backup plan for rain. The other side of this is that people will not remember the good forecasts as much as the bad, and the Weather Channel has business considerations in mind. - He spends very little time on the question of how people act inside larger systems, where a given modeler might be more interested in keeping their job or getting a big bonus than in making their model as accurate as possible. How much of the book do you suggest he devote to this? He addressed the issue directly, as the next couple paragraphs state. He also talks to quite a few institutional players at large organizations, both public and private. - Having said all that, I have major problems with this book and what it claims to explain. In fact, I’m angry. Really, stop with the faux outrage, because you don't seem all that angry throughout the post, you more seem to be disappointed he did not devote the book to your own pet topics. I find this kind of criticism to be mostly without merit. For me, there were three overriding themes in "The Signal and the Noise": Bayesian thinking (this being the primary one), the goals and motivations of forecasters, and examples of what forecasters have done right and what they have done wrong. I think it covered the bases pretty well, and it was an entertaining and lively read on the whole.

cube13 14y ago
The author is a woman, just FYI.
- flatline 14y ago
  The author of the book is a man; the author of the commentary is a woman. Hmm...did I mix up a pronoun somewhere?
  - cube13 14y ago
    Whoops, I thought you did early on. Sorry about that!
    - proksoup 14y ago
      I also read "this" as "his" in that first sentence, and had to do a double take.
001sky 14y ago
You seem to miss the point of Silver's conflict of interest. He is a Modeler, and has a financial gain in promoting certain views. Among these are (1) the notions that modelers are in general "experts"; and (2) that the intentions of people using the "product" of (1) are benign, or at any rate are not Opportunist. His PR agent would never let him say otherwise, it would be bad for business. The financial crisis, at its heart was not an issue of "mistaken models". It was driven by (1) bad legislation from washington; and (2) unethical opportunism exploiting #1. As a general rule, an Analyst at a Credit rating agency is a lowly position in wall street. Most important people (people acting of their own free will) disregard Rating Agency "analysis" out of hand. They do their own work. And people don't get promoted into Goldman Sachs, for example, from S&P et al very often. There is almosts a social stigma attached to the job, that would need to be cleansed with an MBA or some other "success" to pave the way. The people that cannot disregard rating agency work (due to law), of course are a built in market. And in general they are not on an even playing field with Wall Street with respect to their talent pool and access to information. So, bad analysis merely fills a vaccuum. Using this as one example, think of the consequences. First, the model/analysis of the rating agency is a "product" looking to be sold (like a used car). There are no buyers for even the best analysis (wall st does its own research). So the "buyers" are not wall street but people who are forced to rely on credit ratings by law (eg, pension funds or some other public actors). But this is a captive market. The goal of the rating agency then becomes to game the system by extending the market. That is, creat new "protected classes" or create new "Asset types" that are salable to protected classes of investor. Structurally, now, this is the "game". The models, data-sets, and actual implementaions are really beside the point, provided there is a lack of transparency ("black box", proprietary, just complicated, etc). The models will be reverse engineered to "work", they only need to be "plasuble" (as in plausible deniability). Remember, the rating agencies are <<not legally experts>>. If they were, they could be sued, They are just expressing their 1st amendment rights <<to have an opinion>>. None of this you seem to address in your commentary. Nate Silver is just one in a long line of "experts" looking for ways to "sell their services". The first step in doing this is to never undermine the notion of "experts", or the idea that the system is driven by "expert knowledge". As the financial crisis has shown, those people who know what they are doing in the world are operating well beyond the scope and bounds of information that is available to people who publicly claim these hats. There is a reason hedge funds are famously secretive.

cube13 14y ago

The author is a woman, just FYI.

flatline 14y ago
The author of the book is a man; the author of the commentary is a woman. Hmm...did I mix up a pronoun somewhere?
- cube13 14y ago
  Whoops, I thought you did early on. Sorry about that!
  - proksoup 14y ago
    I also read "this" as "his" in that first sentence, and had to do a double take.

001sky 14y ago

You seem to miss the point of Silver's conflict of interest. He is a Modeler, and has a financial gain in promoting certain views. Among these are (1) the notions that modelers are in general "experts"; and (2) that the intentions of people using the "product" of (1) are benign, or at any rate are not Opportunist. His PR agent would never let him say otherwise, it would be bad for business. The financial crisis, at its heart was not an issue of "mistaken models". It was driven by (1) bad legislation from washington; and (2) unethical opportunism exploiting #1. As a general rule, an Analyst at a Credit rating agency is a lowly position in wall street. Most important people (people acting of their own free will) disregard Rating Agency "analysis" out of hand. They do their own work. And people don't get promoted into Goldman Sachs, for example, from S&P et al very often. There is almosts a social stigma attached to the job, that would need to be cleansed with an MBA or some other "success" to pave the way. The people that cannot disregard rating agency work (due to law), of course are a built in market. And in general they are not on an even playing field with Wall Street with respect to their talent pool and access to information. So, bad analysis merely fills a vaccuum. Using this as one example, think of the consequences. First, the model/analysis of the rating agency is a "product" looking to be sold (like a used car). There are no buyers for even the best analysis (wall st does its own research). So the "buyers" are not wall street but people who are forced to rely on credit ratings by law (eg, pension funds or some other public actors). But this is a captive market. The goal of the rating agency then becomes to game the system by extending the market. That is, creat new "protected classes" or create new "Asset types" that are salable to protected classes of investor. Structurally, now, this is the "game". The models, data-sets, and actual implementaions are really beside the point, provided there is a lack of transparency ("black box", proprietary, just complicated, etc). The models will be reverse engineered to "work", they only need to be "plasuble" (as in plausible deniability). Remember, the rating agencies are <<not legally experts>>. If they were, they could be sued, They are just expressing their 1st amendment rights <<to have an opinion>>. None of this you seem to address in your commentary. Nate Silver is just one in a long line of "experts" looking for ways to "sell their services". The first step in doing this is to never undermine the notion of "experts", or the idea that the system is driven by "expert knowledge". As the financial crisis has shown, those people who know what they are doing in the world are operating well beyond the scope and bounds of information that is available to people who publicly claim these hats. There is a reason hedge funds are famously secretive.