8 ms·

Show HN: Monitor websites for change and get scraped data in a webhook

49 points by omneity 7y ago

kseo3l 7y ago
I think monitoro is a good name, especially if you know Spanish. Any way to integrate with proxycrawl before hitting the webhook endpoint?
omneity 7y ago
Hello HN, Monitoro is a service I built to watch websites for changes, scrape data, and whenever the data changes send it to a webhook of your choice. It started as a scratch-your-own-itch kind of situation where I wanted to monitor the prices for some stuff to buy, until I decided to make a product out of it to help others solve similar issues. I am available to answer any questions you might have.
darrenwestall 7y ago
This is ideal for a new feature I’d like to add to our service. My concern is that you won’t stick around - does the on premise version continue work if you were to close?
- omneity 7y ago
  Glad to hear that! We can agree on a license to ensure exactly what you asked. Please write me to omar -at- monitoro.xyz to discuss this further.
dplgk 7y ago
Writing a scraper is easy. So I assume the special sauce in these tools is caching, not getting banned or rated limited. Mostly could be done with Tor but still doesn't work with some sites like Craigslist. Is that right?
- omneity 7y ago
  It depends on your usage. If you're parsimoniously checking for data, the website will not be hostile. If your usage requires higher frequency against a given website, you'll indeed need a proxy (which we'll be rolling relatively soon, when there is demand for it).
abarrettwilsdon 7y ago
Love the idea. Totally understand the business use case. Not in love with the pricing. Feels like paying $49/mo is too high for a marketer to try out, but too low for the fact you can run an effective DDoS of some sites with it I'd suggest changing the buckets - up to 10 monitored pages on free, a Silver Tier of maybe $29/mo of up to 100 pages + frequency of as low as hourly, a Gold Tier of maybe $99/mo of up to 1000 pages w/ minute frequency, Zapier integration, better support, etc, and then a call-for-quote tier like you already have
- jjeaff 7y ago
  It looks like at most, you can update every 30 minutes. So you aren't going to be doing any DDoS'ing with this.
  - omneity 7y ago
    This is incorrect. You can update up to every minute.
    - AznHisoka 7y ago
      A request every minute is a DDOS attack?
      - omneity 7y ago
        I am correcting the parent to my comment, which claimed the frequency is at most every 30 min. As the creator of the service (which is not proposed as a DDoS service), I can confirm the fastest frequency is every minute. For on premise deployments, Monitoro supports updating every second. Using the service to explicitly harm a website is forbidden and the culprits will be banned, free and paid accounts alike.
- omneity 7y ago
  Hey, thank you for the kind words! Before being able to slice plans in these dimensions, I wanted to understand first which use cases are the most valuable, and what are the constraints that affect the value derived by the user from this service. Nevertheless, your point hit home with me and I will definitely experiment with the impact of other plans on conversion rates. Out of curiosity, are you a marketer (and/or do you have a use case for Monitoro) yourself? Also, were you personally prevented from subscribing to the service do to the current plan? Would love to hear your thoughts, please write me at omar -at- monitoro.xyz
hbcondo714 7y ago
When I click on the "Start Monitoring for FREE" button on the homepage, I get redirected to the /login web page. Shouldn't it link to the register page so I can create a FREE account?
- omneity 7y ago
  That's a great point. It's an unfortunate artefact that will be fixed very soon. Thank you for pointing it out!
jjeaff 7y ago
I am trying it out. But I get a 403 error when trying to create my first new monitor. It's a silent error in the background. Had to open the network inspector to see it because it just sits there.
- jjeaff 7y ago
  Actually, it was that I had not yet confirmed my email address. Might want to capture those errors and give users an error message.
- omneity 7y ago
  That’s a great point of feedback, thanks for raising it! I’ll make sure to fix that one on next release.
artur_makly 7y ago
Interesting.. this my be a good addition to our new UX platform https://VisualSitemaps.com https://VisualSitemaps.com thanks for making this.
- omneity 7y ago
  Hey Artur, thank you as well for your interest in Monitoro! I actually see a great synergy potential in fact. What would be the best way to get in touch with you? My email is omar -at- monitoro.xyz
Ladyady 7y ago
I don't think your confirmation emails are sending...? Nothing for me yet (I gave it 8 hours just in case) and nothing in my junk box.
- _def 7y ago
  Was marked as spam for me, but delivered fast
  - omneity 7y ago
    That really sucks. I'm sorry about the inconvenience! Thank you for your patience.
- omneity 7y ago
  I'm unfortunately having trouble with the email provider I'm using. Unsure if it's the volume, but it's a big name and it should be fine. Really sorry about that. Please write me at omar -at- monitoro.xyz and I'll try to fix that wrong.
bobbonew 7y ago
What will the pricing be after beta? I ask because I’d love to use your service and need to know if it’ll be affordable after beta. Or will my price stay locked in? Thanks!

49 points by omneity 7y ago

I think monitoro is a good name, especially if you know Spanish. Any way to integrate with proxycrawl before hitting the webhook endpoint?

omneity 7y ago

Hello HN, Monitoro is a service I built to watch websites for changes, scrape data, and whenever the data changes send it to a webhook of your choice. It started as a scratch-your-own-itch kind of situation where I wanted to monitor the prices for some stuff to buy, until I decided to make a product out of it to help others solve similar issues. I am available to answer any questions you might have.

darrenwestall 7y ago

This is ideal for a new feature I’d like to add to our service. My concern is that you won’t stick around - does the on premise version continue work if you were to close?

omneity 7y ago
Glad to hear that! We can agree on a license to ensure exactly what you asked. Please write me to omar -at- monitoro.xyz to discuss this further.

dplgk 7y ago

Writing a scraper is easy. So I assume the special sauce in these tools is caching, not getting banned or rated limited. Mostly could be done with Tor but still doesn't work with some sites like Craigslist. Is that right?

omneity 7y ago
It depends on your usage. If you're parsimoniously checking for data, the website will not be hostile. If your usage requires higher frequency against a given website, you'll indeed need a proxy (which we'll be rolling relatively soon, when there is demand for it).

abarrettwilsdon 7y ago

Love the idea. Totally understand the business use case. Not in love with the pricing. Feels like paying $49/mo is too high for a marketer to try out, but too low for the fact you can run an effective DDoS of some sites with it I'd suggest changing the buckets - up to 10 monitored pages on free, a Silver Tier of maybe $29/mo of up to 100 pages + frequency of as low as hourly, a Gold Tier of maybe $99/mo of up to 1000 pages w/ minute frequency, Zapier integration, better support, etc, and then a call-for-quote tier like you already have

jjeaff 7y ago
It looks like at most, you can update every 30 minutes. So you aren't going to be doing any DDoS'ing with this.
- omneity 7y ago
  This is incorrect. You can update up to every minute.
  - AznHisoka 7y ago
    A request every minute is a DDOS attack?
    - omneity 7y ago
      I am correcting the parent to my comment, which claimed the frequency is at most every 30 min. As the creator of the service (which is not proposed as a DDoS service), I can confirm the fastest frequency is every minute. For on premise deployments, Monitoro supports updating every second. Using the service to explicitly harm a website is forbidden and the culprits will be banned, free and paid accounts alike.
omneity 7y ago
Hey, thank you for the kind words! Before being able to slice plans in these dimensions, I wanted to understand first which use cases are the most valuable, and what are the constraints that affect the value derived by the user from this service. Nevertheless, your point hit home with me and I will definitely experiment with the impact of other plans on conversion rates. Out of curiosity, are you a marketer (and/or do you have a use case for Monitoro) yourself? Also, were you personally prevented from subscribing to the service do to the current plan? Would love to hear your thoughts, please write me at omar -at- monitoro.xyz

hbcondo714 7y ago

When I click on the "Start Monitoring for FREE" button on the homepage, I get redirected to the /login web page. Shouldn't it link to the register page so I can create a FREE account?

omneity 7y ago
That's a great point. It's an unfortunate artefact that will be fixed very soon. Thank you for pointing it out!

jjeaff 7y ago

I am trying it out. But I get a 403 error when trying to create my first new monitor. It's a silent error in the background. Had to open the network inspector to see it because it just sits there.

jjeaff 7y ago
Actually, it was that I had not yet confirmed my email address. Might want to capture those errors and give users an error message.
omneity 7y ago
That’s a great point of feedback, thanks for raising it! I’ll make sure to fix that one on next release.

artur_makly 7y ago

Interesting.. this my be a good addition to our new UX platform https://VisualSitemaps.com https://VisualSitemaps.com thanks for making this.

omneity 7y ago
Hey Artur, thank you as well for your interest in Monitoro! I actually see a great synergy potential in fact. What would be the best way to get in touch with you? My email is omar -at- monitoro.xyz

Ladyady 7y ago

I don't think your confirmation emails are sending...? Nothing for me yet (I gave it 8 hours just in case) and nothing in my junk box.

_def 7y ago
Was marked as spam for me, but delivered fast
- omneity 7y ago
  That really sucks. I'm sorry about the inconvenience! Thank you for your patience.
omneity 7y ago
I'm unfortunately having trouble with the email provider I'm using. Unsure if it's the volume, but it's a big name and it should be fine. Really sorry about that. Please write me at omar -at- monitoro.xyz and I'll try to fix that wrong.

bobbonew 7y ago

What will the pricing be after beta? I ask because I’d love to use your service and need to know if it’ll be affordable after beta. Or will my price stay locked in? Thanks!