3 ms·

Show HN: I made ChatGPT search 15,000 Reddit posts for side project ideas

4 points by Jkroeger123 3y ago

Jkroeger123 3y ago
Hey HN, Im an indiehacker who loves to solve for niche pain points I realized I spend a lot of time just scrolling through subreddits looking for user complaints, insights, etc I decided to have ChatGPT do the heavy lifting for me, and consolidate a more curated list of posts with insights for me to read Stats: Posts Processed: ~15k Insights Found: ~1K API Costs: ~$150 I hope a few of you may find this useful, would appreciate any feedback ~Justin
- XCSme 3y ago
  ~1K API Costs: ~$150 $150 for 1000 API requests? Isn't that way too high?
  - Jkroeger123 3y ago
    ah sorry poor formatting. insights found was 1k. api costs were $150 for the 15k posts processed
    - XCSme 3y ago
      Still, isn't that high? With $150 you can rent a powerful server for a month and have a big LLM like Llama 2 running on it.
      - Jkroeger123 3y ago
        yea definitely not cheap, to break it down further: I first ran each post title through gpt-3 to classify as either likely or unlikely to be insightful to founders. Then, if it is likely to be insightful I pass the entire post content including up to 100 comments into gpt 4. This comes down to around 13-14 million tokens used for the 15k posts, which for the latest gpt-4 model is ~$150. Im sure this could be done cheaper with something like Llama 2 but tbh I am fine with this for now since I was aiming for speed with the initial list to gauge interest :)
codingdave 3y ago
I think the feedback you'll get is that ideas are fairly worthless. They are not validated with a market, and it is all about implementation anyway. So this is the kind of thing I'd absolutely browse with interest. But there is zero chance I'd pay for it.
- Jkroeger123 3y ago
  thanks for the insight! - yes absolutely this is not a list of validated ideas by any means, more so a feed of inspiration and hints towards market interest (if you find 5-6 reddit posts complaining about a similar thing it has higher likely hood of validation then random idea you think of in the shower). I'm working on a free plan which gives 10 a day to scroll through so currently looking at ways of increasing the value here beyond just the static list
MattGaiser 3y ago
I look at your screenshot list and those are huge problems with huge companies trying to solve them. This seems more like a big opportunities startup list than something for side projects. That may not be representative of what is in the database, but those are huge pain points, not niche problems.
- Jkroeger123 3y ago
  yea great point, I chose those in the screenshot since they caught my eye when scrolling through. But in general yes I think i need to work on the database more to only contain the super niche opportunities
  - MattGaiser 3y ago
    I didn't realize the first round that it was actually just a nice and neat list of scraped information, which I am quite happy to spend time going through. Just purchased a membership.
    - Jkroeger123 3y ago
      happy to hear! hoping to 10x the value of the product over the next few months so stay tuned :)
mehmet_mhy 3y ago
What sub-reddits were analyzed in this dataset?
- Jkroeger123 3y ago
  the current list is ["startups, SaaS, bigdata, analytics, content_marketing, web_design, UXDesign, growthhacking, telecommuting, remotework, BusinessIntelligence, indiebiz, advertising, agency, PRConsulting, sales, AskMarketing, eventplanners, EventProduction, EventProfessionals, CustomerSuccess, userexperience, Marketresearch, VirtualAssistants, OfficeManagers, devops, sysadmin, webdev, LawFirm, legaltech, LegalAdvice, sustainability, Green, enviroaction, business, travelhacks, humanresources, InstructionalDesign, socialmedia, consulting, AskManagement, DataScience, networking, B2BForHire, retail, warehouse, QualityAssurance, AskHR, projectmanagement, Construction, ITManagers, SmallBusiness, CustomerService, Automate, cybersecurity, AskNetsec, finance, FinancialPlanning, Emailmarketing, marketing, TechRecruitment, BusinessProcess, techsupport, ITdept, copywriting, TeamBuilding, RemoteWork, smallbusiness, dataisbeautiful, visualization, ecommerce, EntrepreneurRideAlong, LeadGeneration"] and adding a bunch more in the coming weeks :)
- XCSme 3y ago
  "Hey ChatGPT, give me a list of 30 subreddits where I can find startup ideas" (...) (Beep. Boop.) ["r/startups", "r/leadership", "r/GrowMyBusiness", "r/venturecapital", "r/AdvancedEntrepreneur", "r/marketing", "r/LadyBusiness", "r/productivity", "r/socialmedia", "r/startups_promotion", "r/Entrepreneur", "r/EntrepreneurRideAlong", "r/Entrepreneurship", "r/advancedentrepreneur", "r/startup", "r/crowdspark", "r/sideproject", "r/InternetIsBeautiful", "r/Plugyourproduct", "r/Imadethis", "r/growthmemes"]

4 points by Jkroeger123 3y ago

Hey HN, Im an indiehacker who loves to solve for niche pain points I realized I spend a lot of time just scrolling through subreddits looking for user complaints, insights, etc I decided to have ChatGPT do the heavy lifting for me, and consolidate a more curated list of posts with insights for me to read Stats: Posts Processed: ~15k Insights Found: ~1K API Costs: ~$150 I hope a few of you may find this useful, would appreciate any feedback ~Justin

XCSme 3y ago
~1K API Costs: ~$150 $150 for 1000 API requests? Isn't that way too high?
- Jkroeger123 3y ago
  ah sorry poor formatting. insights found was 1k. api costs were $150 for the 15k posts processed
  - XCSme 3y ago
    Still, isn't that high? With $150 you can rent a powerful server for a month and have a big LLM like Llama 2 running on it.
    - Jkroeger123 3y ago
      yea definitely not cheap, to break it down further: I first ran each post title through gpt-3 to classify as either likely or unlikely to be insightful to founders. Then, if it is likely to be insightful I pass the entire post content including up to 100 comments into gpt 4. This comes down to around 13-14 million tokens used for the 15k posts, which for the latest gpt-4 model is ~$150. Im sure this could be done cheaper with something like Llama 2 but tbh I am fine with this for now since I was aiming for speed with the initial list to gauge interest :)

codingdave 3y ago

I think the feedback you'll get is that ideas are fairly worthless. They are not validated with a market, and it is all about implementation anyway. So this is the kind of thing I'd absolutely browse with interest. But there is zero chance I'd pay for it.

Jkroeger123 3y ago
thanks for the insight! - yes absolutely this is not a list of validated ideas by any means, more so a feed of inspiration and hints towards market interest (if you find 5-6 reddit posts complaining about a similar thing it has higher likely hood of validation then random idea you think of in the shower). I'm working on a free plan which gives 10 a day to scroll through so currently looking at ways of increasing the value here beyond just the static list

MattGaiser 3y ago

I look at your screenshot list and those are huge problems with huge companies trying to solve them. This seems more like a big opportunities startup list than something for side projects. That may not be representative of what is in the database, but those are huge pain points, not niche problems.

Jkroeger123 3y ago
yea great point, I chose those in the screenshot since they caught my eye when scrolling through. But in general yes I think i need to work on the database more to only contain the super niche opportunities
- MattGaiser 3y ago
  I didn't realize the first round that it was actually just a nice and neat list of scraped information, which I am quite happy to spend time going through. Just purchased a membership.
  - Jkroeger123 3y ago
    happy to hear! hoping to 10x the value of the product over the next few months so stay tuned :)

mehmet_mhy 3y ago

What sub-reddits were analyzed in this dataset?

Jkroeger123 3y ago
the current list is ["startups, SaaS, bigdata, analytics, content_marketing, web_design, UXDesign, growthhacking, telecommuting, remotework, BusinessIntelligence, indiebiz, advertising, agency, PRConsulting, sales, AskMarketing, eventplanners, EventProduction, EventProfessionals, CustomerSuccess, userexperience, Marketresearch, VirtualAssistants, OfficeManagers, devops, sysadmin, webdev, LawFirm, legaltech, LegalAdvice, sustainability, Green, enviroaction, business, travelhacks, humanresources, InstructionalDesign, socialmedia, consulting, AskManagement, DataScience, networking, B2BForHire, retail, warehouse, QualityAssurance, AskHR, projectmanagement, Construction, ITManagers, SmallBusiness, CustomerService, Automate, cybersecurity, AskNetsec, finance, FinancialPlanning, Emailmarketing, marketing, TechRecruitment, BusinessProcess, techsupport, ITdept, copywriting, TeamBuilding, RemoteWork, smallbusiness, dataisbeautiful, visualization, ecommerce, EntrepreneurRideAlong, LeadGeneration"] and adding a bunch more in the coming weeks :)
XCSme 3y ago
"Hey ChatGPT, give me a list of 30 subreddits where I can find startup ideas" (...) (Beep. Boop.) ["r/startups", "r/leadership", "r/GrowMyBusiness", "r/venturecapital", "r/AdvancedEntrepreneur", "r/marketing", "r/LadyBusiness", "r/productivity", "r/socialmedia", "r/startups_promotion", "r/Entrepreneur", "r/EntrepreneurRideAlong", "r/Entrepreneurship", "r/advancedentrepreneur", "r/startup", "r/crowdspark", "r/sideproject", "r/InternetIsBeautiful", "r/Plugyourproduct", "r/Imadethis", "r/growthmemes"]