mfrye0

8 ms·

61.
by mfrye0 3y ago
Any chance you could link to where you see that? The back story is that we started as started as a B2B AMB product. We relied on another provider for data, then got hit with a bill one month that nearly bankrupted us. We ended up building o
62.
by mfrye0 3y ago
To be frank, we and others interpreted that entire thread as resolved and didn't need addressing. Scraping is clearly a grey area of the law. This is all public data. - https://techcrunch.com/2022/04/18/w
63.
by mfrye0 3y ago
Hey HN. This is a follow up to a post from earlier this year that seemed to have polarizing reactions: https://news.ycombinator.com/item?id=35977057 . Basically, we're addressing the community's feedback and provid
64.
ChatGPT Landed Us in Hot Water on Hacker News(blog.bigpicture.io)
1 points by mfrye0 3y ago|9 comments
65.
BigPicture Drops Q4 Free Company Dataset Update – 17M Records(docs.bigpicture.io)
1 points by mfrye0 3y ago|0 comments
66.
by mfrye0 3y ago
I 2nd this, minus Salesforce being hard to customize. It's probably too customizable. At a certain stage of a company, sales reps expect Salesforce. So much so, that even when we finally caved and got it, I had reps specifically turn o
67.
by mfrye0 3y ago
I've been tempted to do something similar over the years. I've worked with Salesforce quite a bit, and I dislike almost everything about it. I settled on doing open source company data: https://news.ycombinator.com&#x2F
68.
by mfrye0 3y ago
Apologies. We've had a huge problem with bots, so we have a number of security measures in place. The Google Captcha component is probably flagging you as a bot. Try disabling your VPN if you're on one, or use a different IP.
69.
by mfrye0 3y ago
Hey dang. That's fair. To be frank, we went back and forth on this, but in the end, thought the original title was ok. The only other large, "open source" dataset we could find was 9M. So after researching, we came to the con
70.
by mfrye0 3y ago
Fair points. I agree the wording could be better. No, the source code is not available. This dataset is a subset of the raw data our system collects. Our final product made available via the API does a variety of processing steps on the raw
71.
by mfrye0 3y ago
Ok. I'm not sure how this happened, but I think the dataset was somehow mislabeled. It appears that this dataset is the Q1 version, not the latest Q2. Can you please try re-downloading it? We're probably going to have to make an p
72.
by mfrye0 3y ago
Yes.
73.
by mfrye0 3y ago
How did you compute this? I just did another check to verify (wc -l) and it's coming to 15,980,531.
74.
by mfrye0 3y ago
So this blew up today. Reviewing all the comments now. 1. Yes, this is scraped from public sources. 2. Yes, this is free to use / is open source in the broadest sense. Apologies for the confusion on the lack of a license and no mention
75.
by mfrye0 3y ago
Hey HN, we're thrilled to announce our latest project - the World's Largest Open Source Company Dataset. Our team has been working hard on this product for the past few months, and we're excited to finally share it with you a
76.
A raw dump of companies from all over the world by LinkedIn handle(blog.bigpicture.io)
197 points by mfrye0 3y ago|100 comments
77.
by mfrye0 4y ago
This looks great. I'll have to play around with it. Related, we built a developer oriented Zapier clone for event scale automations awhile back for our internal product. We've since pivoted and have been debating on potentially op
78.
by mfrye0 4y ago
Fair point.
79.
by mfrye0 4y ago
I told my mom about the breakthrough. Her immediate question was, "will this help to save money on energy costs for the house?". I replied, "this will eventually result in nearly limitless energy that will fundamentally chang
80.
by mfrye0 4y ago
fxtentacle, I remember chatting with you briefly earlier this year about all the projects you had worked on, and you didn't even mention this one. You are one interesting dude.
81.
by mfrye0 4y ago
https://bigpicture.io/ I was trying to build a B2B sales/marketing app and all the ML stuff we wanted to do required business data, which was too expensive for us. Long story, but after a cofounder blow up, a failed ac
82.
by mfrye0 4y ago
Hey Ed. I'm trying to email you via the pattern you shared, but it's saying not found.
83.
by mfrye0 4y ago
I think you have a few options. One is the side hustle that slowly becomes profitable. Another path is having a unique story that you can raise money on. A significantly sized problem that you are passionate about and have unique domain exp
84.
by mfrye0 4y ago
I was in Delhi in June for the first time. The air was initially ok as it had rained the days before, but by day 3 our throats burned and we felt exhausted after a few hours outside. At that time, it was 110+F and the AQI was 300+. The impa
85.
by mfrye0 4y ago
I made the mistake of building too much internally at first vs "outsourcing" it to SaaS. Apart from the initial time to build it, there's all the debt of maintenance after the fact. All of that has a cost and takes away from
86.
by mfrye0 4y ago
Very cool. We've just started exploring customer requests to sync our data to their warehouses, so great timing. What sort of scale can you guys handle? One of our DBs is in the billions of rows. I assume we'd need to potentially
87.
by mfrye0 4y ago
This looks cool at first glance. I'll dig into it more. One note that may be helpful, if all you care about is the HTML, it's better to take a "snapshot" of the page by streaming the response directly to blob storage lik
88.
by mfrye0 4y ago
Yes. One of my friends is a lawyer in India. He shared a story with me of a particular lawyer that ends up being paid millions of dollars per hour, because he's able to leverage his connections to help corporate clients jump to the fro
89.
by mfrye0 4y ago
Good question. Tbh I haven't looked into Fargate too much. For our use case, we're processing millions of requests every day. So we needed more control for performance and cost. We're also fairly comfortable with lower level
90.
by mfrye0 4y ago
ECS runs on EC2. You basically register available servers and ECS automatically puts containers onto the instances where space is available. We have it setup with Github actions to automatically deploy to ECS as well.

More ›