4 ms·

Legality of this is rocky in Australia. I dare say that NZ is the same? There are so many scrapers that come and go doing this in AU but are usually shut down

Legality of this is rocky in Australia. I dare say that NZ is the same? There are so many scrapers that come and go doing this in AU but are usually shut down by the big supermarkets. It's a cycle of usefulness and "why doesn't this exist", except it had existed many times before.

russelg 2y ago
I think with the current climate wrt the big supermarkets in AU, now would be the time to push your luck. The court of public opinion will definitely not be on the supermarkets side, and the government may even step in.
- timrkn 2y ago
  Agreed. Should we make something and find out?
timrkn 2y ago
Agreed. Hopefully the govs price gouging mitigation strategy includes free flow of information (allowing scraping for price comparison). I’ve been interested in price comparison for Australia for a while, am a Product designer/manager with a concept prototype design, looking for others interested to work on it. My email is on my profile if you are.
jaza 2y ago
Aussie here. I hadn't heard that price scraping is only quasi-legal here and that scrapers get shut down by the big supermarkets - but then again I'm not surprised. I'm thinking of starting a little price comparison site, mainly to compare select products at Colesworths vs Aldi (I've just started doing more regular grocery shopping at Aldi myself). But as far as I know, Aldi don't have any prices / catalogues online, so my plan is to just manually enter the data myself in the short-term, and to appeal to crowdsourcing the data in the long-term. And plan is to just make it a simple SSG site (e.g. Hugo powered), data all in simple markdown / json files, data all sourced via github pull requests. Feel free to get in touch if you'd like to help out, or if you know of anything similar that already exists: greenash dot net dot au slash contact
- jasomill 2y ago
  But as far as I know, Aldi don't have any prices / catalogues online There are a few here, but more along the lines of a flyer than a catalog: https://www.aldi.com.au/groceries/ https://www.aldi.com.au/groceries/ Aldi US has a more-or-less complete catalog online, so it might be worth crowdsourcing suggestions to the parent company to implement the same in Australia.
- sumedh 2y ago
  > get shut down by the big supermarkets How do they shut them down?
  - seoulmetro 2y ago
    Threaten legal action if scraping continues or something similar as the scraping ends abruptly for every site but the prices still sit online at the supermarkets site.
- seoulmetro 2y ago
  Ironic that I get a few responses with "I am wanting to do the project". I did it around 10 years ago and I think I've seen one a year since that. I didn't bother once I saw that people shipping were doing well and increasing their dataset only for it to be severely reduced later (I assume due to threats of legal action.)
sumedh 2y ago
> Legality of this is rocky in Australia. I dare say that NZ is the same? You might be breaking the sites terms and conditions but that does not mean its illegal. Dan Murphy uses a similar thing, they have their own price checking algorithm.
- seoulmetro 2y ago
  Breaking ToS is 100%, but that wouldn't stop people scraping, in all cases the people scraping are receiving something that stops them scraping while the data still is available, continuing the "why doesnt someone do this" project.
  - jaza 2y ago
    If it's just a matter of (not) breaking ToS, then my idea of manually collating prices shouldn't be able to be shut down (or credibly threatened with being shut down). It might be a painstaking (and unsustainable) effort, but price tags in a physical store have no ToS.
_nivlac_ 2y ago
For the other commenters here - looks like this site does the job? https://hotprices.org/ https://hotprices.org/ With the corresponding repo too: https://github.com/Javex/hotprices-au https://github.com/Javex/hotprices-au

russelg 2y ago

I think with the current climate wrt the big supermarkets in AU, now would be the time to push your luck. The court of public opinion will definitely not be on the supermarkets side, and the government may even step in.

timrkn 2y ago
Agreed. Should we make something and find out?

timrkn 2y ago

Agreed. Hopefully the govs price gouging mitigation strategy includes free flow of information (allowing scraping for price comparison). I’ve been interested in price comparison for Australia for a while, am a Product designer/manager with a concept prototype design, looking for others interested to work on it. My email is on my profile if you are.

jaza 2y ago

Aussie here. I hadn't heard that price scraping is only quasi-legal here and that scrapers get shut down by the big supermarkets - but then again I'm not surprised. I'm thinking of starting a little price comparison site, mainly to compare select products at Colesworths vs Aldi (I've just started doing more regular grocery shopping at Aldi myself). But as far as I know, Aldi don't have any prices / catalogues online, so my plan is to just manually enter the data myself in the short-term, and to appeal to crowdsourcing the data in the long-term. And plan is to just make it a simple SSG site (e.g. Hugo powered), data all in simple markdown / json files, data all sourced via github pull requests. Feel free to get in touch if you'd like to help out, or if you know of anything similar that already exists: greenash dot net dot au slash contact

jasomill 2y ago
But as far as I know, Aldi don't have any prices / catalogues online There are a few here, but more along the lines of a flyer than a catalog: https://www.aldi.com.au/groceries/ https://www.aldi.com.au/groceries/ Aldi US has a more-or-less complete catalog online, so it might be worth crowdsourcing suggestions to the parent company to implement the same in Australia.
sumedh 2y ago
> get shut down by the big supermarkets How do they shut them down?
- seoulmetro 2y ago
  Threaten legal action if scraping continues or something similar as the scraping ends abruptly for every site but the prices still sit online at the supermarkets site.
seoulmetro 2y ago
Ironic that I get a few responses with "I am wanting to do the project". I did it around 10 years ago and I think I've seen one a year since that. I didn't bother once I saw that people shipping were doing well and increasing their dataset only for it to be severely reduced later (I assume due to threats of legal action.)

sumedh 2y ago

> Legality of this is rocky in Australia. I dare say that NZ is the same? You might be breaking the sites terms and conditions but that does not mean its illegal. Dan Murphy uses a similar thing, they have their own price checking algorithm.

seoulmetro 2y ago
Breaking ToS is 100%, but that wouldn't stop people scraping, in all cases the people scraping are receiving something that stops them scraping while the data still is available, continuing the "why doesnt someone do this" project.
- jaza 2y ago
  If it's just a matter of (not) breaking ToS, then my idea of manually collating prices shouldn't be able to be shut down (or credibly threatened with being shut down). It might be a painstaking (and unsustainable) effort, but price tags in a physical store have no ToS.

_nivlac_ 2y ago

For the other commenters here - looks like this site does the job? https://hotprices.org/ https://hotprices.org/ With the corresponding repo too: https://github.com/Javex/hotprices-au https://github.com/Javex/hotprices-au