5 ms·
Show HN: Morning Brief – Track any topic on HN, Reddit and others
TL:DR - https://morningbrief.ai/setup
Hi, HN --
We're two indie cofounders who bootstrapped a new product to scratch our own itch: there's so much great content out there, but it's hard to find it. Sites like HN serve us like-minded people, but there are many more topics in other non-HN niches.
Our idea is this: an email you get once a day, filled with only top articles (posted to HN, Reddit, etc) that match your specific interests. You can specify which interests, and how many links you want. Ranked by our signal score.
<tech-info>
To start, we built an ingestion engine. In the first few weeks we only processed HN posts (1k/day) then we added Reddit posts (1m/day) then we added some other sites like Indie Hackers and relevant Twitter accounts.
Then we added semantic tagging. To start we used a generalized tagging service, but then quickly realized we needed to build our own layer because they don't understand many of the current trendy topics.
Next up we wanted to add thumbnails to worthy articles (about 10k/day). It became apparent this was going to cost over $1k/month using a commercial API. So we built our own service using headless chrome, Laravel & puppeteer.
Our original plan was to just show headlines, HN-style. But after initial feedback users asked for article summaries. To start we used a 3rd party commercial summarizer. Aside from the cost (appx. $400/month for our needs) we were not super happy with the 3rd party results so we built our a summarizer.
Here's an example of a brief with summaries - https://morningbrief.ai/sent/24960b76-4d494be1-ae822280-14274649 - this example is crypto focused with no thumbnails (keep in mind every brief is 100% custom). Here is another example of a more eclectic brief without summaries - https://morningbrief.ai/sent/377f08da-18f04407-a655d594-b1b3043b
Our stack right now is Python for link processing and summaries, Laravel to grab screenshots, send emails, and be the web app. We use SaltStack to orchestrate infrastructure. We run 13 servers on AWS with a cost of approx. $300/month.
</tech-info>
When we started out we thought this would be a weekend project! Turns out solving timely, personalized, summarized, quality controlled, content aggregation at scale (in a cost effective way) is quite a difficult problem. That's how we managed to soak up 6+ months of our side project time just getting it ready for this Show HN!
Our early users have told us that we've sent them important articles that they missed -- which is exactly why we built this.
You can check out how the brief builder works (without registering): https://morningbrief.ai/setup. We're still experimenting with pricing, but we're trying a 2-week free trial, then $5/mo (annual).
Any feedback would be appreciated!
Justin & Joe
- deleted 5y ago[deleted]
- jv22222 5y agoCo-founder here happy to answer any questions!
- tomjen3 5y agoIs there somewhere I can suggest sources? I really love your product, but I couldn't see where you got your data from. Also a small but annoying thing, when I delete topics the confirmation box doesn't respond to the enter key, only the mouse. Anyway, those are relatively minor issues and I definitely look forward to my daily brief tomorrow.
- jv22222 5y agoThanks so much for your feedback. Oh yeah I hate it when you can't do stuff by keyboard. I'll have a look at fixing. Re sources. Right now it is select Twitter accounts, all of Reddit, Hacker News, Indie Hackers. Some we are interested in is github and dev.to - just need to wrap our heads around the details. What ideas do you have for sources?
- frontman1988 5y agoYou guys have built quite a nice and useful product. I was thinking of a similar project which required crawling all of reddit. But presently there are some hiccups due to the large amount of data involved. I wanted to ask how you guys manage the reddit data? Do you use praw/pushshift to get all the comments/post titles or is it limited to selected subreddits? Including all comments and all subreddits is quite a lot of data I guess. Any suggestions to collect that data and then work with it effectively? Thanks.
- mojoe 5y agoWe use praw, it works great if you fill out Reddit's forms to register your user-agent as per their terms of service. We get data about all posts, but don't get the comments themselves, just the post metadata (titles, likes, number of comments, etc). Good luck!