4 ms·

Conversely, I think you're overestimating the technical capailities of Canadian federal or provincial regulatory organizations. Government departments notoriou

Conversely, I think you're overestimating the technical capailities of Canadian federal or provincial regulatory organizations. Government departments notoriously have serious technical debt, so while they might entitled to demand this kind of data, I sincerely doubt they are doing it with anything resembling modern data analysis. These orgs aren't going to have working data lakes, spark clusters or data warehouses on cloud infrastructure. They maybe have legacy SQL databases being pulled into spreadsheets. So, while their data access might be better theoretically, I doubt they truly are able to even remotely analyze the data effectively. Data science in govt is notoriously poor outside of Statscan or other tech heavy departments.

the_cat_kittles 2y ago
never underestimate a nerd with moral high ground
huitzitziltzin 2y ago
Canadian economics departments are full of really smart economists. They are happy to help. The issue here has nothing to do with the size of the data. A grocery store will have 100,000 products at most, probably sets prices about weekly and maybe you have ten years of data. The market is an oligopoly so you have 10-15 store chains. That fits in memory. The issue is not at all the mechanics of running the analysis but what analysis you run. Or don’t run bc as I said there is no easy way to just “identify collusion” in data like this or every agency would already be doing it
- glitchc 2y ago
  > Canadian economics departments are full of really smart economists. They are happy to help. The appropriate incentive is lacking.
  - chii 2y ago
    it should be easy to make good incentives - the analysts that produce the evidence can be awarded a (large?) portion of the fines levied to those companies that are colluding. Just like how the SEC whistleblower laws will awarded part of the fines to those who they reported.
- ajb 2y ago
  Famously , academics analysis found (virtually proved) share options manipulation by statistical analysis alone, which was a bombshell in the tech industry a decade ago, threatening many executives with imprisonment. However , it's not obvious how you could employ the same process here. with share options there was a liquid market as a source of external information.
- wjnc 2y ago
  1. If anything, this public dataset will increase 'collusion' by giving the parties in the market access to the integral set of market prices. 2. Collusion is a strong word. It hints at breaking the law, willfull action, etc. Looking at the price of a competitor and acting upon that, and internal information, is not collusion. It is completely legal and economically rational behaviour. There are hints and information everywhere on which to act. Groceries are an oligopoly, and they are ogipolistic buyers as well. Lots of information for grocers in the price bids of their suppliers, that all groceries get in tandem. 3. I've worked as an economist at one of those government agencies. They are filled with keen PhD / MSc economists looking to bust kartels and find collusion. Geographical price dispersion models are so '00s. There is a lot of positive energy and interaction between academia and government. Actually _proving_ collusion is terribly hard. Prices that move in parallel is not collusion. Proving collusion is finding documents, emails, phone calls, tracking cars with staff members of various firms to motels , paper trails of inside information in another firm. That's collusion. And I fear that's not what happens in the grocery sector. The motel one comes to mind as real life example of how DoJ in the US prosecuted a proper cartel. The talk was by more a broad-shouldered officer type, than an economist. Certain set of transport firms always had a interesting way of responding to auctions. Always the second best offer was quite far behind and there was a pattern to which firms did or did not reply to a bid request. End of the investigation, they could trace employees of the firms to meeting in a motel before bidding. And somebody left a paper trail.
  - mistermann 2y ago
    > I've worked as an economist at one of those government agencies. They are filled with keen PhD / MSc economists looking to bust kartels and find collusion. There is the skillset of the employees, and then there is what management allows employees to work on. A decade or so ago, a Statistics Canada senior manager was interviewed and claimed as fact that they were unable to reproduce an Excel model that analyzed the % of real estate sales to recent immigrants to Canada, one that an individual CRA team (one person I suspect, it's not complicated) in Richmond BC had done a few years before. At the same time, they were given additional budget of many millions of dollars to accomplish this task and other things. They did not accomplish the task. > Proving collusion... The government has tipped their hand already as far as I'm concerned. I don't expect anything would ever be proven, I just want to see as much evidence as possible to help me decide what I am going to believe - I'm certainly not going to take epistemic advice from anyone in the Canadian government.
- lupusreal 2y ago
  > The market is an oligopoly so you have 10-15 store chains. Not that prices aren't consistent across a chain, individual stores in one chain will have different prices, calibrated to local supply and demand. In many cases they'll also have local products that aren't in most of the stores in that chain. I think you have to track prices in each and every store, not just per chain. Still, I think you're right about this fitting in memory. This isn't "big data".

the_cat_kittles 2y ago

never underestimate a nerd with moral high ground

huitzitziltzin 2y ago

Canadian economics departments are full of really smart economists. They are happy to help. The issue here has nothing to do with the size of the data. A grocery store will have 100,000 products at most, probably sets prices about weekly and maybe you have ten years of data. The market is an oligopoly so you have 10-15 store chains. That fits in memory. The issue is not at all the mechanics of running the analysis but what analysis you run. Or don’t run bc as I said there is no easy way to just “identify collusion” in data like this or every agency would already be doing it

glitchc 2y ago
> Canadian economics departments are full of really smart economists. They are happy to help. The appropriate incentive is lacking.
- chii 2y ago
  it should be easy to make good incentives - the analysts that produce the evidence can be awarded a (large?) portion of the fines levied to those companies that are colluding. Just like how the SEC whistleblower laws will awarded part of the fines to those who they reported.
ajb 2y ago
Famously , academics analysis found (virtually proved) share options manipulation by statistical analysis alone, which was a bombshell in the tech industry a decade ago, threatening many executives with imprisonment. However , it's not obvious how you could employ the same process here. with share options there was a liquid market as a source of external information.
wjnc 2y ago
1. If anything, this public dataset will increase 'collusion' by giving the parties in the market access to the integral set of market prices. 2. Collusion is a strong word. It hints at breaking the law, willfull action, etc. Looking at the price of a competitor and acting upon that, and internal information, is not collusion. It is completely legal and economically rational behaviour. There are hints and information everywhere on which to act. Groceries are an oligopoly, and they are ogipolistic buyers as well. Lots of information for grocers in the price bids of their suppliers, that all groceries get in tandem. 3. I've worked as an economist at one of those government agencies. They are filled with keen PhD / MSc economists looking to bust kartels and find collusion. Geographical price dispersion models are so '00s. There is a lot of positive energy and interaction between academia and government. Actually _proving_ collusion is terribly hard. Prices that move in parallel is not collusion. Proving collusion is finding documents, emails, phone calls, tracking cars with staff members of various firms to motels , paper trails of inside information in another firm. That's collusion. And I fear that's not what happens in the grocery sector. The motel one comes to mind as real life example of how DoJ in the US prosecuted a proper cartel. The talk was by more a broad-shouldered officer type, than an economist. Certain set of transport firms always had a interesting way of responding to auctions. Always the second best offer was quite far behind and there was a pattern to which firms did or did not reply to a bid request. End of the investigation, they could trace employees of the firms to meeting in a motel before bidding. And somebody left a paper trail.
- mistermann 2y ago
  > I've worked as an economist at one of those government agencies. They are filled with keen PhD / MSc economists looking to bust kartels and find collusion. There is the skillset of the employees, and then there is what management allows employees to work on. A decade or so ago, a Statistics Canada senior manager was interviewed and claimed as fact that they were unable to reproduce an Excel model that analyzed the % of real estate sales to recent immigrants to Canada, one that an individual CRA team (one person I suspect, it's not complicated) in Richmond BC had done a few years before. At the same time, they were given additional budget of many millions of dollars to accomplish this task and other things. They did not accomplish the task. > Proving collusion... The government has tipped their hand already as far as I'm concerned. I don't expect anything would ever be proven, I just want to see as much evidence as possible to help me decide what I am going to believe - I'm certainly not going to take epistemic advice from anyone in the Canadian government.
lupusreal 2y ago
> The market is an oligopoly so you have 10-15 store chains. Not that prices aren't consistent across a chain, individual stores in one chain will have different prices, calibrated to local supply and demand. In many cases they'll also have local products that aren't in most of the stores in that chain. I think you have to track prices in each and every store, not just per chain. Still, I think you're right about this fitting in memory. This isn't "big data".