7 ms·
So a while ago a i remembered one post from HN, it was some guy crawling google play store and keeping apps that were paid and people used them but had a low ra
by visox 5y ago
So a while ago a i remembered one post from HN, it was some guy crawling google play store and keeping apps that were paid and people used them but had a low rating. He was selling this list, not sure if it was successful but i made something similar and updated
So far checking only 3 marketplaces, my crawlers found so far about 100k apps, the default filters show only the more interesting once (paid and low rating)
Its free.
I will add more and more marketplaces eventually. I will also try to add some social features, like "working as a team on some idea" and posting own ideas, but will see how it goes.
Feedback welcome
EDIT: i just see it on mobile, it does not look ideal. Will need to work on that hm.
EDIT2: just added pagination :), hope it works for you.
- lambdasquirrel 5y agoCool idea. I’m not able to find some well-used apps with the search, like Headspace. So as we say, it’s about the execution.
- visox 5y agohmm shame, maybe remove some filters, maybe there are no hated apps like that.
- yoz-y 5y agoHeadspace is there, but you need to set the filter to 0usd min (the app is free) and max rating to 100 (the app is rated 92%)
- lambdasquirrel 5y agohmm, interesting... my search term was "meditation"
- akudha 5y agoDo you know if scraping this data violates their ToS?
- Scoundreller 5y agoDunno about Atlassian and Shopify, but I can’t imagine Google launching a case against someone for scraping them. I guess robots.txt is all that really matters. Here’s Play Store’s: https://play.google.com/robots.txt https://play.google.com/robots.txt
- minnehaha 5y agoagreed, provided the crawler complies with robots.txt
- akudha 5y agoThis data is quite valuable and services like this aren't competing with these app stores. If anything, they are probably inviting more competition to the app stores resulting in better apps. There is no reason for these companies to go after some small guy scraping the app store data. But who knows - Craigslist went after one man shops. These companies might too.
- bahmboo 5y agoGoogle's entire business is scraping websites
- catillac 5y agoMaybe that’s a reason to not feel bad if it violates the TOS but there are real consequences to doing so so perhaps your response isn’t that helpful.
- intricatedetail 5y agoShouldn't that be caught by anti-trust laws? Google does not pay websites for scraping, so anyone should be entitled to scrap Google to their hearts content.
- namdnay 5y ago
- ehsankia 5y ago> He was selling this list Can someone explain the value of such a list, am I missing something? Don't get me wrong, your project is neat and I enjoyed looking through it, but I don't understand how anyone would try to sell this or pay money for this.
- nemomarx 5y agoI guess it could be "here are apps that people need enough to pay for, but have complaints about" in order to develop alternatives? Maybe these are apps with bad user interfaces or some other issue and it's a list of low hanging fruit ideas?
- m463 5y agoI've never bought an app that was one-star, but over time many of the really useful apps I used were sold and did one-star kinds of things. One app kept track of my car info (vin, insurance, mileage, fillups) and was useful. But one day it was sold - uploaded my data and put it behind a login.
- thr0w__4w4y 5y ago"[but] did one-star kinds of things." That is a great line (seriously). I will probably end up working that phrase into something at one point, which is why I have this post bookmarked, so I can "hat tip to m463 on HN" when I do. P.S. Do you remember the name of that auto app? Name and shame! ;-)
- m463 5y agoit was gas cubby. Another app I had that got worse was camscanner - a scanner app that would take a picture of a document and create a .pdf out of it. It was sold to tencent. The "privacy policy" - when you could access it (broken links) was written in broken english that basically said it did anything it wanted with your .pdf files. The kinds of documents I would image to .pdf were extremely sensitive personal documents - think w2, documents with SSN, etc... At least apple notes eventually added a version of camera to .pdf (though it wasn't obvious how to use it when it came out)
- shoto_io 5y agoThanks for sharing this! Great idea! Q: How do you evaluate if the apps are still being paid for?
- visox 5y agogood question, i cant tell how many people did actually pay and when was the last time someone did. I guess its up to you to do the research further. One can at least often see when the last comment was posted.
- amelius 5y agoPerhaps also keep a list of people who are currently implementing improved versions of these apps, to minimize the possibility of redundant work.
- visox 5y agoyeap this is a social feature i will work on, actually thinking more about collaboration rather than competition but will see.
- visox 5y agoI just tried to add G2 to my list but they immediately get you on captcha, so they really dont wish to be crawled :D
- dkdk8283 5y agoWill the scraper code ever be open source?