6 ms·

‘’’ Reach out to me (email “jacob” at this website) if: You can do economic analysis of pricing data, and especially the interaction/correlation of multiple st

‘’’ Reach out to me (email “jacob” at this website) if: You can do economic analysis of pricing data, and especially the interaction/correlation of multiple streams of prices across time ‘’’ The item in that quote is significantly, significantly harder than the author is giving it credit for. Canada has a competition policy agency. They are (almost surely) entitled to demand data from firms as part of an investigation. Their data will be better than the data here. You can rarely if ever prove these cases on the basis of data analysis alone. (Indeed if you could the world’s antitrust agencies would just monitor such data! And anyone engaging in collusion would attack the monitoring adversarially - not that hard to predict!) In grocery data you will be looking at thousands and thousands of prices of different goods from different suppliers with different costs who are exposed to different shocks due to variation in the costs of their inputs or god knows what else. Not to be negative bc the idea is nice, but this is a complete waste of time. ~ an actual antitrust guy.

jfil 2y ago
Hey man, I'm just creating the hammer - you can throw it in the lake, make a treehouse for your kid, smash a window, up to you :-D
- huitzitziltzin 2y ago
  Assuming you’re the author (and again this is a nice idea!) there is a paper you should check out. This paper: https://www.aeaweb.org/articles?id=10.1257/aer.20170116 https://www.aeaweb.org/articles?id=10.1257/aer.20170116 Is the best case scenario (=easiest) for the exercise you want to do and will illustrate what is hard about the grocery case. In the paper there is: - One product - one clearly isolated market. - four or five firms. - A clear mechanism for collusion (a government price sharing website). Even then there is no statistical or econometric test out of the box which “shows collusion”. It’s a great paper (both in my opinion and that’s why it got into AER) but there is nothing they do that says “hey here’s collusion based on this t-stat.” The grocery case (overlapping markets, thousands of products, many firms) is vastly, vastly harder.
- huitzitziltzin 2y ago
  Actually I’m going to add to your reading list: “ A Study of Cartel Stability: The Joint Executive Committee, 1880-1886” By Rob Porter (northwestern) That paper is a classic and I hope porter gets the Nobel prize for the work he has done. In that paper, we actually know for sure that there was collusion. Porter is trying to assess how hard it was to sustain collusion using the data. (One thing you are not crediting in your website is that collusion is hard to sustain bc all of the colluders have the incentive to cheat!) It takes a lot of assumptions and serious econometric chops to make the analysis credible. That paper is 40 years old but what you’re really trying to do with your project is called “structural industrial organization.” It’s been around in the economics literature for a long time. It’s my field. It’s hard. If you want a recent overview which still won’t answer the question the way you wanted to, get “the handbook of industrial organization” edited by hortacsu et al, or the book draft “empirical methods in industrial organization” by aguirregabiria (who is at U Toronto!) or “quantitative techniques for competition and antitrust” by Davis and Garcés. Good luck.
  - jfil 2y ago
    huitzitziltzin, I really appreciate that you've replied with so much detail, named the field of study, and gave references to research for me to follow up. I am going to be reading up on these topics.
- zxexz 2y ago
  In retail (or any market with elastic pricing, not sure if I'm using that term correctly), collusion is really hard to gain hard evidence (evidence that will hold up in court) using data alone. You need to use the data as a tool for on-the-ground and in-depth qualitative analysis. I would not be surprised if this holds "even more true" for consumer goods like this than it does for other, highly regulated fields like finance and healthcare. I personally like the idea of focusing on an individual good or category of item, and seeing if you can find similar patterns, in other goods.
surgicalcolor 2y ago
Conversely, I think you're overestimating the technical capailities of Canadian federal or provincial regulatory organizations. Government departments notoriously have serious technical debt, so while they might entitled to demand this kind of data, I sincerely doubt they are doing it with anything resembling modern data analysis. These orgs aren't going to have working data lakes, spark clusters or data warehouses on cloud infrastructure. They maybe have legacy SQL databases being pulled into spreadsheets. So, while their data access might be better theoretically, I doubt they truly are able to even remotely analyze the data effectively. Data science in govt is notoriously poor outside of Statscan or other tech heavy departments.
- the_cat_kittles 2y ago
  never underestimate a nerd with moral high ground
- huitzitziltzin 2y ago
  Canadian economics departments are full of really smart economists. They are happy to help. The issue here has nothing to do with the size of the data. A grocery store will have 100,000 products at most, probably sets prices about weekly and maybe you have ten years of data. The market is an oligopoly so you have 10-15 store chains. That fits in memory. The issue is not at all the mechanics of running the analysis but what analysis you run. Or don’t run bc as I said there is no easy way to just “identify collusion” in data like this or every agency would already be doing it
  - glitchc 2y ago
    > Canadian economics departments are full of really smart economists. They are happy to help. The appropriate incentive is lacking.
    - chii 2y ago
      it should be easy to make good incentives - the analysts that produce the evidence can be awarded a (large?) portion of the fines levied to those companies that are colluding. Just like how the SEC whistleblower laws will awarded part of the fines to those who they reported.
the_cat_kittles 2y ago
isnt the difference that when you have an open dataset, you get many many more eyeballs, and some very passionate ones. and they can make it their hobby, unconstrained by day to day needs of a job. eh?
- huitzitziltzin 2y ago
  No. They have to know what they are doing. Also imagine the false discovery rate of <whatever analysis you think you can do> with the thousands of products which exist in a grocery store! “Ah look we found collusion in salty snacks between weeks 38 and 45 of 2022 in stores 1, 5 and 13!” Even if you knew what analysis to do (and you don’t) you are going to end up crying wolf a million times and solving no problem at all. Again - a worthy effort but the methods just don’t exist.
  - the_cat_kittles 2y ago
    ok. i believe you, but hope that your maybe a bit overly pessimistic
  - MichaelZuo 2y ago
    You get the top 100 most plausible hunches and then investigate further from there… why focus on proving something in the data itself? Investigators work on hunches just fine if you pay them enough.
    - deleted 2y ago
      [deleted]
  - mistermann 2y ago
    > Even if you knew what analysis to do (and you don’t) you are going to end up crying wolf a million times and solving no problem at all. Mocking the epistemic skills of others while engaging in literal soothsaying (stating facts about future events) shows how culturally conditioned North Americans have become regarding government wrong doing. I don't expect these people to be successful in their primary goal, but an enticing second order effect is it could accidentally catalyze public interest in epistemology, logic, rhetoric, the nature of Human culture and cognition, and a variety of the other things that keep commoners locked in a sophisticated virtual reality without their awareness (in any sophisticated manner at least). Now that would make this whole game a lot more interesting, because it applies to everything.
tazu 2y ago
> ~ an actual antitrust guy So, unfirable? Essentially useless and beholden to little if any performance metrics?
__MatrixMan__ 2y ago
Is it necessary to prove it? I think it's sufficient to come up with an unbiased metric for which players are the most likely to be colluding at any given time. That would be a starting point for encouraging them to behave such that they're obviously not colluding: compete louder please, or you'll be at the top of this list.
- huitzitziltzin 2y ago
  Well… what do you want to happen ? You can publish some correlations between prices. (Remember there is no known method where you can look at a time series of prices across stores and say “that’s collusion!”). And hey if you have 10,000 - 100,000 products per store you probably have correlations as high as you want. None of that will actually show collusion or more importantly suggest a policy response. Even very high correlation _within a category_ across stores isn’t going to do that. Say you find price movements in tomato paste across all N chains are 0.95… ok now what? There are 10,000 or 2,000 product categories so you found a high correlation in one of them what do you want to do about it?
  - __MatrixMan__ 2y ago
    I think there are a lot of people in desperate situations who feel like the deck is stacked against them. They end up lashing out in completely irrational, often violent ways. You see them on the news all the time. What I want to happen is for those people to realize that they're not alone, and that the whole world is not working to put them at a disadvantage. Really, it's a rather small group of people who are setting the rest of us up to be rats in a maze with no exit. Then I want us to find those people, and go after their bottom line. Nothing violent, just like... welding the doors shut on opening day, that sort of thing. It might be less rational than something more restrained like getting a law degree and working in the system to bring them to actual justice. But it has... a glimmer of rationality. You're no longer an insane lone wolf lashing out against an impossible situation... now you're a shitty activist, and even if you're not acting with a cohort, you know that cohort is out there. You're not stuck, you're not alone, there's a path forward--you can aspire towards being a less shitty activist. But it requires a justifiable target, a representative of the untenable situation. A list of potential enemies that's far less than everybody. Once that list exists, I want companies to compete to not appear on that list. Markets work without being rational, as does evolution (some predation required). This would be the same.
transcriptase 2y ago
Yeah we have a competition bureau that’s known for never doing its actual job. The fact that they keep approving mergers and acquisitions in telecom, banking, and groceries despite those being some of the most comically oligopoly-esque industries within a country on earth shows just how tragically wrong you are. If there was 2 telecom companies left standing in Canada, our competition bureau would pretend to study their desire to merge for a year before approving it, under the condition that their collection of subsidiary brands pretend to be different companies while advertising.
kwar13 2y ago
Nice try, Loblaws.
ActionHank 2y ago
They're still trying to resolve the bread price fixing from years before the pandemic. The agency is likely underfunded, understaffed, and disincentivized from actually fixing the issues which are so pervasive across the country that it's just business as usual for most people..
patcon 2y ago
> You can do economic analysis of pricing data, and especially the interaction/correlation of multiple streams of prices across time Would this be the sort of analysis that measured transfer entropy or granger causality?

jfil 2y ago

Hey man, I'm just creating the hammer - you can throw it in the lake, make a treehouse for your kid, smash a window, up to you :-D

huitzitziltzin 2y ago
Assuming you’re the author (and again this is a nice idea!) there is a paper you should check out. This paper: https://www.aeaweb.org/articles?id=10.1257/aer.20170116 https://www.aeaweb.org/articles?id=10.1257/aer.20170116 Is the best case scenario (=easiest) for the exercise you want to do and will illustrate what is hard about the grocery case. In the paper there is: - One product - one clearly isolated market. - four or five firms. - A clear mechanism for collusion (a government price sharing website). Even then there is no statistical or econometric test out of the box which “shows collusion”. It’s a great paper (both in my opinion and that’s why it got into AER) but there is nothing they do that says “hey here’s collusion based on this t-stat.” The grocery case (overlapping markets, thousands of products, many firms) is vastly, vastly harder.
huitzitziltzin 2y ago
Actually I’m going to add to your reading list: “ A Study of Cartel Stability: The Joint Executive Committee, 1880-1886” By Rob Porter (northwestern) That paper is a classic and I hope porter gets the Nobel prize for the work he has done. In that paper, we actually know for sure that there was collusion. Porter is trying to assess how hard it was to sustain collusion using the data. (One thing you are not crediting in your website is that collusion is hard to sustain bc all of the colluders have the incentive to cheat!) It takes a lot of assumptions and serious econometric chops to make the analysis credible. That paper is 40 years old but what you’re really trying to do with your project is called “structural industrial organization.” It’s been around in the economics literature for a long time. It’s my field. It’s hard. If you want a recent overview which still won’t answer the question the way you wanted to, get “the handbook of industrial organization” edited by hortacsu et al, or the book draft “empirical methods in industrial organization” by aguirregabiria (who is at U Toronto!) or “quantitative techniques for competition and antitrust” by Davis and Garcés. Good luck.
- jfil 2y ago
  huitzitziltzin, I really appreciate that you've replied with so much detail, named the field of study, and gave references to research for me to follow up. I am going to be reading up on these topics.
zxexz 2y ago
In retail (or any market with elastic pricing, not sure if I'm using that term correctly), collusion is really hard to gain hard evidence (evidence that will hold up in court) using data alone. You need to use the data as a tool for on-the-ground and in-depth qualitative analysis. I would not be surprised if this holds "even more true" for consumer goods like this than it does for other, highly regulated fields like finance and healthcare. I personally like the idea of focusing on an individual good or category of item, and seeing if you can find similar patterns, in other goods.

surgicalcolor 2y ago

Conversely, I think you're overestimating the technical capailities of Canadian federal or provincial regulatory organizations. Government departments notoriously have serious technical debt, so while they might entitled to demand this kind of data, I sincerely doubt they are doing it with anything resembling modern data analysis. These orgs aren't going to have working data lakes, spark clusters or data warehouses on cloud infrastructure. They maybe have legacy SQL databases being pulled into spreadsheets. So, while their data access might be better theoretically, I doubt they truly are able to even remotely analyze the data effectively. Data science in govt is notoriously poor outside of Statscan or other tech heavy departments.

the_cat_kittles 2y ago
never underestimate a nerd with moral high ground
huitzitziltzin 2y ago
Canadian economics departments are full of really smart economists. They are happy to help. The issue here has nothing to do with the size of the data. A grocery store will have 100,000 products at most, probably sets prices about weekly and maybe you have ten years of data. The market is an oligopoly so you have 10-15 store chains. That fits in memory. The issue is not at all the mechanics of running the analysis but what analysis you run. Or don’t run bc as I said there is no easy way to just “identify collusion” in data like this or every agency would already be doing it
- glitchc 2y ago
  > Canadian economics departments are full of really smart economists. They are happy to help. The appropriate incentive is lacking.
  - chii 2y ago
    it should be easy to make good incentives - the analysts that produce the evidence can be awarded a (large?) portion of the fines levied to those companies that are colluding. Just like how the SEC whistleblower laws will awarded part of the fines to those who they reported.

the_cat_kittles 2y ago

isnt the difference that when you have an open dataset, you get many many more eyeballs, and some very passionate ones. and they can make it their hobby, unconstrained by day to day needs of a job. eh?

huitzitziltzin 2y ago
No. They have to know what they are doing. Also imagine the false discovery rate of <whatever analysis you think you can do> with the thousands of products which exist in a grocery store! “Ah look we found collusion in salty snacks between weeks 38 and 45 of 2022 in stores 1, 5 and 13!” Even if you knew what analysis to do (and you don’t) you are going to end up crying wolf a million times and solving no problem at all. Again - a worthy effort but the methods just don’t exist.
- the_cat_kittles 2y ago
  ok. i believe you, but hope that your maybe a bit overly pessimistic
- MichaelZuo 2y ago
  You get the top 100 most plausible hunches and then investigate further from there… why focus on proving something in the data itself? Investigators work on hunches just fine if you pay them enough.
  - deleted 2y ago
    [deleted]
- mistermann 2y ago
  > Even if you knew what analysis to do (and you don’t) you are going to end up crying wolf a million times and solving no problem at all. Mocking the epistemic skills of others while engaging in literal soothsaying (stating facts about future events) shows how culturally conditioned North Americans have become regarding government wrong doing. I don't expect these people to be successful in their primary goal, but an enticing second order effect is it could accidentally catalyze public interest in epistemology, logic, rhetoric, the nature of Human culture and cognition, and a variety of the other things that keep commoners locked in a sophisticated virtual reality without their awareness (in any sophisticated manner at least). Now that would make this whole game a lot more interesting, because it applies to everything.

tazu 2y ago

> ~ an actual antitrust guy So, unfirable? Essentially useless and beholden to little if any performance metrics?

__MatrixMan__ 2y ago

Is it necessary to prove it? I think it's sufficient to come up with an unbiased metric for which players are the most likely to be colluding at any given time. That would be a starting point for encouraging them to behave such that they're obviously not colluding: compete louder please, or you'll be at the top of this list.

huitzitziltzin 2y ago
Well… what do you want to happen ? You can publish some correlations between prices. (Remember there is no known method where you can look at a time series of prices across stores and say “that’s collusion!”). And hey if you have 10,000 - 100,000 products per store you probably have correlations as high as you want. None of that will actually show collusion or more importantly suggest a policy response. Even very high correlation _within a category_ across stores isn’t going to do that. Say you find price movements in tomato paste across all N chains are 0.95… ok now what? There are 10,000 or 2,000 product categories so you found a high correlation in one of them what do you want to do about it?
- __MatrixMan__ 2y ago
  I think there are a lot of people in desperate situations who feel like the deck is stacked against them. They end up lashing out in completely irrational, often violent ways. You see them on the news all the time. What I want to happen is for those people to realize that they're not alone, and that the whole world is not working to put them at a disadvantage. Really, it's a rather small group of people who are setting the rest of us up to be rats in a maze with no exit. Then I want us to find those people, and go after their bottom line. Nothing violent, just like... welding the doors shut on opening day, that sort of thing. It might be less rational than something more restrained like getting a law degree and working in the system to bring them to actual justice. But it has... a glimmer of rationality. You're no longer an insane lone wolf lashing out against an impossible situation... now you're a shitty activist, and even if you're not acting with a cohort, you know that cohort is out there. You're not stuck, you're not alone, there's a path forward--you can aspire towards being a less shitty activist. But it requires a justifiable target, a representative of the untenable situation. A list of potential enemies that's far less than everybody. Once that list exists, I want companies to compete to not appear on that list. Markets work without being rational, as does evolution (some predation required). This would be the same.

transcriptase 2y ago

Yeah we have a competition bureau that’s known for never doing its actual job. The fact that they keep approving mergers and acquisitions in telecom, banking, and groceries despite those being some of the most comically oligopoly-esque industries within a country on earth shows just how tragically wrong you are. If there was 2 telecom companies left standing in Canada, our competition bureau would pretend to study their desire to merge for a year before approving it, under the condition that their collection of subsidiary brands pretend to be different companies while advertising.

kwar13 2y ago

Nice try, Loblaws.

ActionHank 2y ago

They're still trying to resolve the bread price fixing from years before the pandemic. The agency is likely underfunded, understaffed, and disincentivized from actually fixing the issues which are so pervasive across the country that it's just business as usual for most people..

patcon 2y ago

> You can do economic analysis of pricing data, and especially the interaction/correlation of multiple streams of prices across time Would this be the sort of analysis that measured transfer entropy or granger causality?