22 ms·

Show HN: We built a Rotten Tomatoes-style website for VPNs

74 points by hubraumhugo 5y ago

Thorentis 5y ago
I'll save you, and everybody else, the time, and just say that Mullvad is the best VPN currently available, and the day they get inevitably bought out will be a sad day. But until then, they beat all the competition hands down easily.
- srivan 5y ago
  The ownership and future of Mullvad VPN "Mullvad VPN is here to stay, and we are not interested in ever selling it." https://mullvad.net/en/blog/2021/9/16/ownership-and-future-mullvad-vpn/ https://mullvad.net/en/blog/2021/9/16/ownership-and-future-m...
tptacek 5y ago
Rotten Tomatoes works because (1) its audience generally trusts some (maybe even most) of the reviewers, and (2) some movies are good. I don't see how Rotten Tomatoes for VPNs can work.
- Lifelarper 5y ago
  What rotten tomatoes considers the "best films of all time" is laughable across the spectrum. Some gems from the top 10: 4. Black Panther 6. Parasite 7. Avengers: Endgame 9. Knives Out 10. Us Its surely just coincidence that the half of the best films ever made in human history were produced in the last 5 years. https://www.rottentomatoes.com/top/bestofrt/ https://www.rottentomatoes.com/top/bestofrt/
  - addingnumbers 5y ago
    They're doing something fishy with that "sorted by adjusted score." If it sorted by tomato-meter value instead, the esteemed classics would rise to the top, and the entries you took issue with wouldn't even crack the top 50. Whatever "adjustment" they are doing seems to strongly favor recency.
hubraumhugo 5y ago
VPNs are a mess. It's becoming a very commercial market, and companies are bought up by less reputable and bigger ones, making it hard to know what to trust. The biggest pain points when researching a good VPN are: - Sponsored content / Ads - Fake reviews - It takes time - Inconsistent metrics and info across sites To make finding a good VPN even easier, we've aggregated over 500 online reviews, 3k Reddit posts, 4k HackerNews posts, and 400 Youtube videos. ## How it works Using GPT-3 and BERT, we've built a service that collects VPN reviews from all over the web, filters and aggregates them, and presents the summarized results in a short and understandable form. ## What's next? We're constantly improving SeekVPN and will soon add new features like: - Building a community for VPN enthusiasts that helps each other - Running statistical analysis to spot discrepancies and contradictions among reviews and filter out fake reviews - Increase catalog and add more filters - Improve summaries (pros and cons style) We eventually plan to do our analysis for other categories of SaaS products. It should become the Rotten-Tomatoes for software products. Let us know what you think! What should we add or improve?
- VHRanger 5y ago
  Why would you use NLP to collect reviews that are astroturfed? Garbage in, garbage out Just go wirecutter style and actually research the products a minimal amount to give an opinion
  - hubraumhugo 5y ago
    Wirecutter is a great resource, and we'll soon include their lists for our analysis. But most people visit 5 different websites and spend many hours doing research before they buy a product. Our goal is to build a service that gives you a holistic view about what the internet thinks of products.
    - dymk 5y ago
      Again, garbage in, garbage out - how is rampant astroturfing and review fraud taken into account? Edit: the site highly ranks the ones I see pushed on YouTube, so I’m guessing this is just “how often is this vendor referenced” and not “how quality is the vendor”.
jamescun 5y ago
What you'll find difficult is establishing trust. I have no reason to believe your review site is a shill for another company, but I have no reason not to believe that either. It was recently revealed that Kape Technologies, a former malware distributor and now owner of PrivateInternetAccess and ExpressVPN, are doing this. The first step would be clearly identifying ownership and benefactors.
- hubraumhugo 5y ago
  I fully agree; we need to be more transparent about our process, intentions, and ownership. We'll work on that. Here is a good Reddit post about the fake reviews from Kape Technologies: https://www.reddit.com/r/VPNTorrents/comments/r160ll/spotting_fake_reviews_from_kape_technologies/ https://www.reddit.com/r/VPNTorrents/comments/r160ll/spottin...
  - LarryDarrell 5y ago
    And yet ExpressVPN is rated 5th on your site. A reputable review site would have ExpressVPN and NordVPN rated 0.0, or even not rated at all and shoved into a separate section, "VPNs to Avoid."
    - hubraumhugo 5y ago
      We're working on improving the site and the data quality. Keep in mind that it's still an MVP.
      - LarryDarrell 5y ago
        ...but you are a review site... The data quality is your MVP.
        tpoacher 5y ago
        This.
    - 5y ago
jrockway 5y ago
I get the impression that VPNs massively overcharge for what they are; every YouTube video that is even moderately mainstream is sponsored by some sort of VPN provider. Is the idea behind choosing this industry in particular to charge a referral and suck away some of their sweet, sweet profit for yourself? Seems like a good idea.
- iudqnolq 5y ago
  Take a look at mullvad. There's no real way of knowing if they're telling the truth, but I think the fact the have no affiliate program or even advertising and don't collect your email address is a good thing. (They give you a token, you send them cash or use a credit card to add money to the token, you input the token in the app.)
ortusdux 5y ago
Just one data point, but the "No, I'm not interested in joining a fun community or a chance to win an awesome VPN" text on the solicitation for my email after ~20 seconds on the site was enough to get me to leave. I am currently unhappy with my VPN and shopping for a new one when my annual subscription ends early next month.
- hubraumhugo 5y ago
  You're right, the popup feels quite annoying when first visiting the page. We removed it.
  - iudqnolq 5y ago
    It's still there on blur. It seems to be useful, because it correctly gets across the impression of the sort of VPN you recommending.
  - ortusdux 5y ago
    Awesome, thanks!
- cassianoleal 5y ago
  I cannot recommend Mullvad enough. From past posts on this website, I think many here will agree.
  - 55873445216111 5y ago
    Mullvad doesn't even ask for your email address!
smoldesu 5y ago
Wow... the immediate email nag is pretty catastrophically bad already, but then your "top" VPNs are... Nord? ProtonVPN? I'm not here to point fingers or assign blame, but I've gone through a fair handful of VPNs and I can tell you firsthand that those are faaaaaar from the best options, especially if you're trying to get the most out of your service. Mullvad and Azire are head-and-shoulders above anything I see on the top. Kinda disappointed. Edit: Also, your "about" page is surprisingly non-informative.
latexr 5y ago
Haven’t been on the website for one minute and you already shoved a popup in my face asking for my email. The options are “submit” or “No, I'm not interested in joining a fun community or a chance to win an awesome VPN”. If you’re going to be as annoying and manipulative as every other scammy website out there, why should we trust you? My reaction was to close the page. Considering you’re posting this to HN, I doubt I’m in the minority.
- hubraumhugo 5y ago
  We removed the popup and we'll add some more information to the about section to make us more transparent and trustworthy.
criddell 5y ago
Do you earn affiliate fees if I sign up for something after being referred by your site? If so, I won't trust it.
0xy 5y ago
The #1 result NordVPN had servers hacked to root and then attempted to covered up the huge security breach rather than disclose it. [1] Not to mention their concealed relationship with Tesonet, a shadowy data mining company. On top of that, key personnel of Tesonet (including the CEO) are stationed in Lithuania, which has a "cozy" relationship with the CIA, which extends as far as knowingly hosting rendition sites there. Isn't this an immediate disqualification of the site's methodology? [1] https://en.wikipedia.org/wiki/NordVPN#Criticism https://en.wikipedia.org/wiki/NordVPN#Criticism
- sam_goody 5y ago
  Well, the site has caused some really insightful comments from knowledgeable people, which will definitely helpme choose a VPN. So in that sense, the site has already succeeded. All they need to do now is prettify this HN thread and replace the current garbage with it. ;)
antihero 5y ago
Why should I trust this in any way?
danlugo92 5y ago
Reminder that if you only need 1 specific location you need to appear in, you can get a VPS for less than a monthly VPN subscription. These VPS, while still distinguishable from normal residential connnections, are more likely to pass filters etc, it won't also trip up IP-change-detection mechanisms as you'll get an stable IP address.
bitwize 5y ago
Are you going to disable or reweight user scores when they dunk on a VPN you're in a strategic partnership with, in the name of "preventing review bombing by white supremacist alt-right trolls"? Because that'd be necessary in order to achieve feature parity with RT...
devmor 5y ago
The fact that NordVPN is somehow #1 immediately discounts whatever algorithm you've designed for this. Under no circumstances would I ever consider a VPN that's been breached even once as a #1 choice.
- sebazzz 5y ago
  Especially not if they tried to hide the fact that they just got hacked.
soared 5y ago
Is there any info on parent companies? I don't know much about VPNs but I thought ownership was conentrated into just a few firms, and that some of them are owned by chinese companies - both of which could be useful information when making a decision.
- hubraumhugo 5y ago
  You can find information about the ownership in the "Transparency: Ownership" section when expanding the specs on a specific provider.
lifeisgood99 5y ago
How do I pick a VPN from this website? Let's say I go with the #1 Trending VPN, it scores very well, but the Reddit threads are negative. The "Reviews by Topic" are then suspiciously almost all positive. Do you audit the reviews?
- hubraumhugo 5y ago
  To make it easier for people to pick a VPN, we plan to add lists for Top X vpns for absolute security, top ones for privacy, top ones for reliability. About the reviews: We collect the reviews per topic by doing a web search. Unfortunately, many VPN reviews are bought and paid for, so we need to improve the collection, filtering, and ranking of the reviews. We did this for other products, but VPNs seem to be a very tricky category.
LarryDarrell 5y ago
I thought the general consensus was that the only VPN worth a damn was Mullvad? I'm not interested in a review aggregator. Individual reviews are the problem in the first place. What I want is someone to have a set of criteria and then rate the services individually based on that criteria. Also, your family of review sites are for VPNs, backpacks and knives. This sounds like a writing prompt.
hunterb123 5y ago
No you built an affiliate links site where you say they are all "great".
jph 5y ago
Avoid ExpressVPN & NordVPN because of problems with support and shady practices. I suggest you delete these two from your list and/or show prominent warnings. I currently use SurfShark which works well for me.
- madjam002 5y ago
  What shady practices do NordVPN engage in?
  - addingnumbers 5y ago
    March 5, 2018: Hacker gets root access and key material on a NordVPN server. 404 days after hack, NordVPN finally discovers server is compromised 594 days after hack, a twitter user boasts of experiments with NordVPN private key material that had "been floating around mostly unnoticed" 595 days after hack: NordVPN finally acknowledges they were breached. Apology hammers on two themes, "it's not our fault we had no way to restrict access to that server" and "you should still trust us because we confirmed this guy didn't do many of the awful things we allowed him the capability to do" That's a whole year of running a compromised server, and 27 whole weeks of cover-up between discovering the breach and getting called out on twitter.
butz 5y ago
Needs DIY VPN option, being on HN and all.
mdni007 5y ago
Thanks all. I have cancelled my NordVPN subscription. I was not aware of all the shady stuff they were doing behind the scenes.
metaloha 5y ago
I notice that page 3 of 4 doesn't have a Next button to go to the next page - that seems wrong. In fact, the second-last page (even in a search result) doesn't have a Back button, but the last page does have a Previous button. Classic off-by-one error it looks like :)
- hubraumhugo 5y ago
  Should be fixed now :)
noxer 5y ago
Obligatory vpn.md https://gist.github.com/joepie91/5a9909939e6ce7d09e29 https://gist.github.com/joepie91/5a9909939e6ce7d09e29
0xdeadb00f 5y ago
NordVPN as #1 means I know for sure not to trust this website.

74 points by hubraumhugo 5y ago

I'll save you, and everybody else, the time, and just say that Mullvad is the best VPN currently available, and the day they get inevitably bought out will be a sad day. But until then, they beat all the competition hands down easily.

srivan 5y ago
The ownership and future of Mullvad VPN "Mullvad VPN is here to stay, and we are not interested in ever selling it." https://mullvad.net/en/blog/2021/9/16/ownership-and-future-mullvad-vpn/ https://mullvad.net/en/blog/2021/9/16/ownership-and-future-m...

tptacek 5y ago

Rotten Tomatoes works because (1) its audience generally trusts some (maybe even most) of the reviewers, and (2) some movies are good. I don't see how Rotten Tomatoes for VPNs can work.

Lifelarper 5y ago
What rotten tomatoes considers the "best films of all time" is laughable across the spectrum. Some gems from the top 10: 4. Black Panther 6. Parasite 7. Avengers: Endgame 9. Knives Out 10. Us Its surely just coincidence that the half of the best films ever made in human history were produced in the last 5 years. https://www.rottentomatoes.com/top/bestofrt/ https://www.rottentomatoes.com/top/bestofrt/
- addingnumbers 5y ago
  They're doing something fishy with that "sorted by adjusted score." If it sorted by tomato-meter value instead, the esteemed classics would rise to the top, and the entries you took issue with wouldn't even crack the top 50. Whatever "adjustment" they are doing seems to strongly favor recency.

hubraumhugo 5y ago

VPNs are a mess. It's becoming a very commercial market, and companies are bought up by less reputable and bigger ones, making it hard to know what to trust. The biggest pain points when researching a good VPN are: - Sponsored content / Ads - Fake reviews - It takes time - Inconsistent metrics and info across sites To make finding a good VPN even easier, we've aggregated over 500 online reviews, 3k Reddit posts, 4k HackerNews posts, and 400 Youtube videos. ## How it works Using GPT-3 and BERT, we've built a service that collects VPN reviews from all over the web, filters and aggregates them, and presents the summarized results in a short and understandable form. ## What's next? We're constantly improving SeekVPN and will soon add new features like: - Building a community for VPN enthusiasts that helps each other - Running statistical analysis to spot discrepancies and contradictions among reviews and filter out fake reviews - Increase catalog and add more filters - Improve summaries (pros and cons style) We eventually plan to do our analysis for other categories of SaaS products. It should become the Rotten-Tomatoes for software products. Let us know what you think! What should we add or improve?

VHRanger 5y ago
Why would you use NLP to collect reviews that are astroturfed? Garbage in, garbage out Just go wirecutter style and actually research the products a minimal amount to give an opinion
- hubraumhugo 5y ago
  Wirecutter is a great resource, and we'll soon include their lists for our analysis. But most people visit 5 different websites and spend many hours doing research before they buy a product. Our goal is to build a service that gives you a holistic view about what the internet thinks of products.
  - dymk 5y ago
    Again, garbage in, garbage out - how is rampant astroturfing and review fraud taken into account? Edit: the site highly ranks the ones I see pushed on YouTube, so I’m guessing this is just “how often is this vendor referenced” and not “how quality is the vendor”.

jamescun 5y ago

What you'll find difficult is establishing trust. I have no reason to believe your review site is a shill for another company, but I have no reason not to believe that either. It was recently revealed that Kape Technologies, a former malware distributor and now owner of PrivateInternetAccess and ExpressVPN, are doing this. The first step would be clearly identifying ownership and benefactors.

hubraumhugo 5y ago
I fully agree; we need to be more transparent about our process, intentions, and ownership. We'll work on that. Here is a good Reddit post about the fake reviews from Kape Technologies: https://www.reddit.com/r/VPNTorrents/comments/r160ll/spotting_fake_reviews_from_kape_technologies/ https://www.reddit.com/r/VPNTorrents/comments/r160ll/spottin...
- LarryDarrell 5y ago
  And yet ExpressVPN is rated 5th on your site. A reputable review site would have ExpressVPN and NordVPN rated 0.0, or even not rated at all and shoved into a separate section, "VPNs to Avoid."
  - hubraumhugo 5y ago
    We're working on improving the site and the data quality. Keep in mind that it's still an MVP.
    - LarryDarrell 5y ago
      ...but you are a review site... The data quality is your MVP.
      - tpoacher 5y ago
        This.
  - 5y ago

jrockway 5y ago

I get the impression that VPNs massively overcharge for what they are; every YouTube video that is even moderately mainstream is sponsored by some sort of VPN provider. Is the idea behind choosing this industry in particular to charge a referral and suck away some of their sweet, sweet profit for yourself? Seems like a good idea.

iudqnolq 5y ago
Take a look at mullvad. There's no real way of knowing if they're telling the truth, but I think the fact the have no affiliate program or even advertising and don't collect your email address is a good thing. (They give you a token, you send them cash or use a credit card to add money to the token, you input the token in the app.)

ortusdux 5y ago

Just one data point, but the "No, I'm not interested in joining a fun community or a chance to win an awesome VPN" text on the solicitation for my email after ~20 seconds on the site was enough to get me to leave. I am currently unhappy with my VPN and shopping for a new one when my annual subscription ends early next month.

hubraumhugo 5y ago
You're right, the popup feels quite annoying when first visiting the page. We removed it.
- iudqnolq 5y ago
  It's still there on blur. It seems to be useful, because it correctly gets across the impression of the sort of VPN you recommending.
- ortusdux 5y ago
  Awesome, thanks!
cassianoleal 5y ago
I cannot recommend Mullvad enough. From past posts on this website, I think many here will agree.
- 55873445216111 5y ago
  Mullvad doesn't even ask for your email address!

smoldesu 5y ago

Wow... the immediate email nag is pretty catastrophically bad already, but then your "top" VPNs are... Nord? ProtonVPN? I'm not here to point fingers or assign blame, but I've gone through a fair handful of VPNs and I can tell you firsthand that those are faaaaaar from the best options, especially if you're trying to get the most out of your service. Mullvad and Azire are head-and-shoulders above anything I see on the top. Kinda disappointed. Edit: Also, your "about" page is surprisingly non-informative.

latexr 5y ago

Haven’t been on the website for one minute and you already shoved a popup in my face asking for my email. The options are “submit” or “No, I'm not interested in joining a fun community or a chance to win an awesome VPN”. If you’re going to be as annoying and manipulative as every other scammy website out there, why should we trust you? My reaction was to close the page. Considering you’re posting this to HN, I doubt I’m in the minority.

hubraumhugo 5y ago
We removed the popup and we'll add some more information to the about section to make us more transparent and trustworthy.

criddell 5y ago

Do you earn affiliate fees if I sign up for something after being referred by your site? If so, I won't trust it.

0xy 5y ago

The #1 result NordVPN had servers hacked to root and then attempted to covered up the huge security breach rather than disclose it. [1] Not to mention their concealed relationship with Tesonet, a shadowy data mining company. On top of that, key personnel of Tesonet (including the CEO) are stationed in Lithuania, which has a "cozy" relationship with the CIA, which extends as far as knowingly hosting rendition sites there. Isn't this an immediate disqualification of the site's methodology? [1] https://en.wikipedia.org/wiki/NordVPN#Criticism https://en.wikipedia.org/wiki/NordVPN#Criticism

sam_goody 5y ago
Well, the site has caused some really insightful comments from knowledgeable people, which will definitely helpme choose a VPN. So in that sense, the site has already succeeded. All they need to do now is prettify this HN thread and replace the current garbage with it. ;)

antihero 5y ago

Why should I trust this in any way?

danlugo92 5y ago

Reminder that if you only need 1 specific location you need to appear in, you can get a VPS for less than a monthly VPN subscription. These VPS, while still distinguishable from normal residential connnections, are more likely to pass filters etc, it won't also trip up IP-change-detection mechanisms as you'll get an stable IP address.

bitwize 5y ago

Are you going to disable or reweight user scores when they dunk on a VPN you're in a strategic partnership with, in the name of "preventing review bombing by white supremacist alt-right trolls"? Because that'd be necessary in order to achieve feature parity with RT...

devmor 5y ago

The fact that NordVPN is somehow #1 immediately discounts whatever algorithm you've designed for this. Under no circumstances would I ever consider a VPN that's been breached even once as a #1 choice.

sebazzz 5y ago
Especially not if they tried to hide the fact that they just got hacked.

soared 5y ago

Is there any info on parent companies? I don't know much about VPNs but I thought ownership was conentrated into just a few firms, and that some of them are owned by chinese companies - both of which could be useful information when making a decision.

hubraumhugo 5y ago
You can find information about the ownership in the "Transparency: Ownership" section when expanding the specs on a specific provider.

lifeisgood99 5y ago

How do I pick a VPN from this website? Let's say I go with the #1 Trending VPN, it scores very well, but the Reddit threads are negative. The "Reviews by Topic" are then suspiciously almost all positive. Do you audit the reviews?

hubraumhugo 5y ago
To make it easier for people to pick a VPN, we plan to add lists for Top X vpns for absolute security, top ones for privacy, top ones for reliability. About the reviews: We collect the reviews per topic by doing a web search. Unfortunately, many VPN reviews are bought and paid for, so we need to improve the collection, filtering, and ranking of the reviews. We did this for other products, but VPNs seem to be a very tricky category.

LarryDarrell 5y ago

I thought the general consensus was that the only VPN worth a damn was Mullvad? I'm not interested in a review aggregator. Individual reviews are the problem in the first place. What I want is someone to have a set of criteria and then rate the services individually based on that criteria. Also, your family of review sites are for VPNs, backpacks and knives. This sounds like a writing prompt.

hunterb123 5y ago

No you built an affiliate links site where you say they are all "great".

jph 5y ago

Avoid ExpressVPN & NordVPN because of problems with support and shady practices. I suggest you delete these two from your list and/or show prominent warnings. I currently use SurfShark which works well for me.

madjam002 5y ago
What shady practices do NordVPN engage in?
- addingnumbers 5y ago
  March 5, 2018: Hacker gets root access and key material on a NordVPN server. 404 days after hack, NordVPN finally discovers server is compromised 594 days after hack, a twitter user boasts of experiments with NordVPN private key material that had "been floating around mostly unnoticed" 595 days after hack: NordVPN finally acknowledges they were breached. Apology hammers on two themes, "it's not our fault we had no way to restrict access to that server" and "you should still trust us because we confirmed this guy didn't do many of the awful things we allowed him the capability to do" That's a whole year of running a compromised server, and 27 whole weeks of cover-up between discovering the breach and getting called out on twitter.

butz 5y ago

Needs DIY VPN option, being on HN and all.

mdni007 5y ago

Thanks all. I have cancelled my NordVPN subscription. I was not aware of all the shady stuff they were doing behind the scenes.

metaloha 5y ago

I notice that page 3 of 4 doesn't have a Next button to go to the next page - that seems wrong. In fact, the second-last page (even in a search result) doesn't have a Back button, but the last page does have a Previous button. Classic off-by-one error it looks like :)

hubraumhugo 5y ago
Should be fixed now :)

noxer 5y ago

Obligatory vpn.md https://gist.github.com/joepie91/5a9909939e6ce7d09e29 https://gist.github.com/joepie91/5a9909939e6ce7d09e29

0xdeadb00f 5y ago

NordVPN as #1 means I know for sure not to trust this website.