9 ms·

I used to work at Desktop Metal which was (at the time) a pretty small startup here in Boston. When I left, I wrote a pretty negative review on Glassdoor about

I used to work at Desktop Metal which was (at the time) a pretty small startup here in Boston. When I left, I wrote a pretty negative review on Glassdoor about what I felt was a toxic work environment. At the time I submitted my review, there were probably about 10 reviews for the company. The very next day after my review was posted, about 7 positive reviews popped up. Obviously pretty suspicious behavior, but when I contacted Glassdoor they said there was no issue. A week later my own review was removed for a generic content policy violation. Goes without saying I don't trust Glassdoor for much.

pubutil 3y ago
I definitely believe that. Years ago, I had an interview scheduled with a bank’s IT department. Checked their Glassdoor page and it was littered with negative reviews, with obviously fake positive reviews peppered in. I took the interview anyway because I really needed work at the time. Got some 1-on-1 time with the IT guys and the first thing they said was, “You read the reviews on Glassdoor, didn’t you?” “Yes..” “Well, don’t worry, IT is pretty isolated from all that” Cool, I guess? Next, I got some time with a VP. He saw on my resume that I’d done some work for a Christian church. He said, “Yeah, I love to debate religion at the water cooler!” Had he taken some time to get to know me, he’d know I’m not religious and probably would have held his tongue. I got offered the job. Naturally, I didn’t take it. Like others have said, Glassdoor is good to see problem spots in negative reviews. Positive reviews are meaningless in my eyes. For reference, the bank was called Bank of Internet at the time. They’ve since rebranded to Axos and I continue to recommend against them. I know nothing about their financial credibility, but I can’t in good faith recommend anyone support a company that allows people in positions of power to “debate religion at the water cooler.” That’s a hostile work environment.
- aleph_minus_one 3y ago
  > I can’t in good faith recommend anyone support a company that allows people in positions of power to “debate religion at the water cooler.” That’s a hostile work environment. This is likely some cultural issue (I'm not from the United States), but I don't get what some person of power who loves debating religion at the water cooler makes the company a hostile work environment. Quite the opposite: in my gut feeling the fact that religion (a topic that has a tendency to cause heated discussions) can be discussed at the water cooler is rather a point of evidence that the work environment is really healthy.
  - jasonfarnon 3y ago
    Yes, this is the issue I have with all the comments on this thread saying that glassdoor has too many good reviews for a company that the commenter knows is "toxic". I have no idea what "toxic" means to the commenters here. Then I finally see an issue concretely identified: toxic can mean manager expresses an interest in "debating religion at the water cooler"? So what? I also wouldn't generalize from this poster's anecdote to conclude "This is likely some cultural issue" in the US. I guess religion is a touchy subject in the US compared to more homoegenous nations like in the EU, and a rule of thumb I'd always heard is to avoid it in the office. (They used to say the same thing about politics in the office but that definitely went out the window these last 10 years.)
    - lotsofpulp 3y ago
      Religion is a legally protected class in the US, and the fact that a boss would willingly want to discuss, much less debate it with a subordinate opening the business up to much unnecessary liability is a red flag at least for stupid leadership. Worst case scenario, the boss is trying to discriminate based on religion, and trying to find out more about the subordinate’s personal beliefs. Best case scenario, boss is for whatever reason curious, but displays lack of knowledge of labor laws and best practices (in the context of asking to discuss with a newbie who they have no prior relationship with). After a working relationship had been established, I could see religion as a as a casual topic being reasonable depending on how what their relationship is like, but as an introduction? Forget about it.
      - LandR 3y ago
        This is absolutely crazy to a European.
        erenyeager 3y ago
        Really? How would European business culture approach this? I feel like talking about debating religion when it’s not relevant to a potential employee is extremely short sighted or some type of backhanded mind game; it just doesn’t seem normal to say that at a job interview. After a working relationship has been established if it comes up, that is different, but if the VP just says that based on seeing a potential employee’s work with a church, it’s quite suspect.
lurknot 3y ago
My favorite thing about glassdoor is that I've had employers use their pay scales for negotiations. Which of course are many years old at this point. Needless to say I rejected their offer.
- Fire-Dragon-DoL 3y ago
  They also seem to be below what companies offer in reality...
Mandatum 3y ago
Glassdoor removed all of my reviews after submitting a negative view about 1 company. Previous reviews had been mostly positive or just neutral with provable facts. Having seen their internal Blind, social Slack channels and culture vs. the Glassdoor reviews, and then submitting my review which really "went against the grain" of Glassdoor, it was clear there was something going on. I just didn't care enough to find out. As these systems age, I think they're all missing one feature - "what are ratings or reviews from the past 6 months".
emeril 3y ago
Sounds like target.com I ordered a play kitchen based on reviews and then I found it to be poor quality, I wrote a review which was never published without any reason given As such, I don't really trust positive reviews anymore, especially, there I only mostly trust yelp reviews
- propogandist 3y ago
  [dead]
jchw 3y ago
Yep, I noticed the exact same thing. Glassdoor is a complete scam, and it has been for as long as I've had a career. I do not even check anymore.
- AlanSE 3y ago
  So is it basically true that there is no objective information out there anymore, because anyone running a successful site gets corrupted with money?
  - praptak 3y ago
    The only good source is a network of people you know in person. If you don't have one, working on open source projects is a good way to bootstrap one.
  - dylan604 3y ago
    That's a big broad brush you just swiped with, but yeah, I kind of agree with it.
  - rightbyte 3y ago
    Blind seems somewhat OK?
  - iambateman 3y ago
    I’d say Consumer Reports and NYT wire cutter are pretty objective… Their bigger issue is that company quality doesn’t change much from year to year, so the pans that were good in 2019 are probably still good now, which is boring.
    - loloquwowndueo 3y ago
      Wire cutter is hardly objective - check out their bike lock reviews. Every single recommended product is from Kryptonite.
      - Fire-Dragon-DoL 3y ago
Solvency 3y ago
I know a non-engineer who left DesktopMetal with a similar sentiment.
princevegeta89 3y ago
No one trusts these review companies actually. They simply exist to extort entities that get reviewed either to remove negative reviews or promote or even auto-post fake positive reviews. Yelp is a classic example that did this for years.
- black_puppydog 3y ago
  > No one trusts these review companies It's not so much that I'd read the reviews, but ranking of results is fatally flawed thanks to these practices. For many queries (restaurant in town X, book in genre Y) there are simply so many candidate results that ranking is the one and only thing to determine a (much, MUCH) smaller candidate set that the customer might actually engage with. How often do you click through to page 2 of your search results?
- camhart 3y ago
  > No one trusts these review companies actually. Most people do trust them, but shouldn't.
  - princevegeta89 3y ago
    Actually you are right, sorry. I was exaggerating but it is true that it is mostly the naive/average users of internet that do place any significant credibility in online reviews
    - deleted 3y ago
      [deleted]
- Mtinie 3y ago
  Can you foresee a scenario where a review site could legitimately exist without turning to synthetic reviews, dark UI patterns, or outright brand extortion?
  - alecthomas 3y ago
    https://www.productreview.com.au/ https://www.productreview.com.au/ is the only site I've ever seen that does a good job at user reviews. Each review has to be accompanied by a verifiable receipt. Here's their operating model, which sounds like a mix of paid brand manager subscriptions and advertising: https://support.productreview.com.au/hc/en-us/articles/360001254676-Our-Operating-Model-at-ProductReview-com-au https://support.productreview.com.au/hc/en-us/articles/36000...
pyeri 3y ago
A blockchain based solution can be developed for customer reviews. If your cryptographically signed and verifiable review went and sat on the disk of every computer on the planet, there will be no options to manipulate it.
- JALTU 3y ago
  But those "seven positive" reviews will still show up! When I was calculating the NPS scores at a former job, IIRC you need seven or eight 100% scores to offset one low score. Interesting that's what happened to the Desktop Metal poster.
  - pyeri 3y ago
    But (presumably) those seven posters would be motivated by big tech money and thus be identified by some algorithm or the other in that blockchain? And/or you being a genuine reviewer would have a more organic activity across the blockchain with other products too compared to theirs and that will add to the weight?
    - sverhagen 3y ago
      There's just very little organic activity on a site like Glassdoor. You changing jobs, and thus naturally having any kind of inclination to post something may be few and far between, and you might not even think of uploading a review unless you're disgruntled (read: motivated).
    - JohnFen 3y ago
      > (presumably) those seven posters would be motivated by big tech money Probably not, I think. They're more likely to be employees or friends/family of the company doing it because they were required or asked to.
- starttoaster 3y ago
  Yeah except there would still be a mechanism for revoking or moderating bad reviews because people would use the platform to review bomb any company they took issue with. The review might still be on the chain's history, but the moderation tool would still be created with good intentions, with a new transaction on a later block that basically revokes the "validity" of the bad review. Similar mechanisms can be seen in NFTs, where you can update the NFTs metadata.
Simon_O_Rourke 3y ago
I've experienced this almost verbatim - put up a factual (at least from my view) account of working with a previous employer, a "warts and all" review. Next week I see about ten 5-star reviews each with a single sentence something like "This is a great place to work!". However, when searching out the low-down on a potential new employer, I only read the one and two star reviews - if they're pretty spurious then I feel more comfortable taking a role there. If however the poor reviews contain lots of pretty specific complaints, I find them more believable that the single sentence "everything is awesome here" style reviews.
- pc86 3y ago
  I've considered making a browser extension where you can click a review to ignore it and have the average ratings recalculate automatically.
druvisc 3y ago
Same is happening to Google Maps. I and other reviewers had left negative reviews for Cosuno Ventures GmbH as it's an incredibly toxic and horrible place to work in. I received a take-down notice from a German Google Maps review employee saying that Cosuno is claiming it's a false review and they don't know how who am I. Even though I worked at the company for 4 months with the CTO and had responded to the e-mail with documents signed by the CEO and CTO. They're rocking a 5 star review now. I didn't receive a response from Google and my appeal has been ignored. I still have the company's t-shirt in the countryside.
- Kretinsky 3y ago
  I went to a pediatrician near where I lived. Long story short, she was very toxic, making ultimatums and giving what was revealed as dubious practice. I consulted other similar pediatricians, who all gave me radically different answers than hers. As a result, I chose to write a detailed review explaining why I thought she was unprofessional, citing examples. Of course, my review got nuked. Same happened with a photo shop that screw up a photo development for some films. I left a factual review, with a sample in pictures. I can see my review with my account, but no one else can.
  - swayvil 3y ago
    >can see my review with my account, but no one else can. Your review got shadow-removed. That's a new one on me.
    - ShadyOperation 3y ago
      [dead]
- i_am_a_peasant 3y ago
  Heh, one of my friends wrote a negative review about his landlord's Immobilien company. Got sued and had to pay him the amount of his deposit. I bet I'd be at risk of getting sued in Germany even if I wrote a positive review about a company. The moment you post an opinion online you are opening yourself as a target.
randycupertino 3y ago
At a healthcare tech startup I worked at for every negative review we got management made us write three positive reviews. The owners daughter would get all of her friends from college to write more and more positive reviews. It was so obvious, any fake review written by HR will always have "keep doing what you're doing!" in the "advice to management" section. Like, even a moderately happy employee will actually have something to say in the "advice to management" section. Only a shill will say, "Keep doing what you're doing!"
- briHass 3y ago
  I swear the 'keep doing what you're doing' must either be the standard HR planted review (I wasn't invited to the HR conference where they learned that!), or it's GD's own spam for pay. I've seen that too frequently at toxic jobs for it to be coincidence. Luckily, just like Amazon reviews, it's pretty easy to determine worthwhile info from how specific and balanced it is. The aggregate scores are obviously worthless, because either shills or spurned employees taint the mix. I enjoy when there are responses from the business: that can often show egotistical management that thinks their poo doesn't stink.
- sizzle 3y ago
  Heard the same from a friend of a friend, they would game Trip Advisor reviews by giving discounts to customers and the daughter would make accounts and leave positive reviews all the time. There seriously needs to be a watchdog group or AI service (startup idea!) that vets reviews with heuristics like anomaly detection for VPN ip address reviews in succession or browser fingerprinting. Also ChatGPT is gonna make writing unique reviews a thing of the past. Expect fake reviews sounding indistinguishable from genuine human writing. We are screwed.
- incone123 3y ago
  " The owners daughter would get all of her friends from college to write more and more positive reviews." A cottage industry of liars. I was pretty naive in college and I still would have been skeptical about that.
smcleod 3y ago
Yep. I’ve experience almost exactly the same in the past. I’ve always just assumed glassdoor was a fake / scam site since.
xnx 3y ago
Another case where web annotations would be useful. Recently: iAnnotate – Whatever happened to the web as an annotation system? (2013) https://news.ycombinator.com/item?id=36559758 https://news.ycombinator.com/item?id=36559758
- jiveturkey 3y ago
  huh? how so? what stops the exact sort of gaming at glassdoor et al, from also occurring via some web annotation system?

pubutil 3y ago

I definitely believe that. Years ago, I had an interview scheduled with a bank’s IT department. Checked their Glassdoor page and it was littered with negative reviews, with obviously fake positive reviews peppered in. I took the interview anyway because I really needed work at the time. Got some 1-on-1 time with the IT guys and the first thing they said was, “You read the reviews on Glassdoor, didn’t you?” “Yes..” “Well, don’t worry, IT is pretty isolated from all that” Cool, I guess? Next, I got some time with a VP. He saw on my resume that I’d done some work for a Christian church. He said, “Yeah, I love to debate religion at the water cooler!” Had he taken some time to get to know me, he’d know I’m not religious and probably would have held his tongue. I got offered the job. Naturally, I didn’t take it. Like others have said, Glassdoor is good to see problem spots in negative reviews. Positive reviews are meaningless in my eyes. For reference, the bank was called Bank of Internet at the time. They’ve since rebranded to Axos and I continue to recommend against them. I know nothing about their financial credibility, but I can’t in good faith recommend anyone support a company that allows people in positions of power to “debate religion at the water cooler.” That’s a hostile work environment.

aleph_minus_one 3y ago
> I can’t in good faith recommend anyone support a company that allows people in positions of power to “debate religion at the water cooler.” That’s a hostile work environment. This is likely some cultural issue (I'm not from the United States), but I don't get what some person of power who loves debating religion at the water cooler makes the company a hostile work environment. Quite the opposite: in my gut feeling the fact that religion (a topic that has a tendency to cause heated discussions) can be discussed at the water cooler is rather a point of evidence that the work environment is really healthy.
- jasonfarnon 3y ago
  Yes, this is the issue I have with all the comments on this thread saying that glassdoor has too many good reviews for a company that the commenter knows is "toxic". I have no idea what "toxic" means to the commenters here. Then I finally see an issue concretely identified: toxic can mean manager expresses an interest in "debating religion at the water cooler"? So what? I also wouldn't generalize from this poster's anecdote to conclude "This is likely some cultural issue" in the US. I guess religion is a touchy subject in the US compared to more homoegenous nations like in the EU, and a rule of thumb I'd always heard is to avoid it in the office. (They used to say the same thing about politics in the office but that definitely went out the window these last 10 years.)
  - lotsofpulp 3y ago
    Religion is a legally protected class in the US, and the fact that a boss would willingly want to discuss, much less debate it with a subordinate opening the business up to much unnecessary liability is a red flag at least for stupid leadership. Worst case scenario, the boss is trying to discriminate based on religion, and trying to find out more about the subordinate’s personal beliefs. Best case scenario, boss is for whatever reason curious, but displays lack of knowledge of labor laws and best practices (in the context of asking to discuss with a newbie who they have no prior relationship with). After a working relationship had been established, I could see religion as a as a casual topic being reasonable depending on how what their relationship is like, but as an introduction? Forget about it.
    - LandR 3y ago
      This is absolutely crazy to a European.
      - erenyeager 3y ago
        Really? How would European business culture approach this? I feel like talking about debating religion when it’s not relevant to a potential employee is extremely short sighted or some type of backhanded mind game; it just doesn’t seem normal to say that at a job interview. After a working relationship has been established if it comes up, that is different, but if the VP just says that based on seeing a potential employee’s work with a church, it’s quite suspect.

lurknot 3y ago

My favorite thing about glassdoor is that I've had employers use their pay scales for negotiations. Which of course are many years old at this point. Needless to say I rejected their offer.

Fire-Dragon-DoL 3y ago
They also seem to be below what companies offer in reality...

Mandatum 3y ago

Glassdoor removed all of my reviews after submitting a negative view about 1 company. Previous reviews had been mostly positive or just neutral with provable facts. Having seen their internal Blind, social Slack channels and culture vs. the Glassdoor reviews, and then submitting my review which really "went against the grain" of Glassdoor, it was clear there was something going on. I just didn't care enough to find out. As these systems age, I think they're all missing one feature - "what are ratings or reviews from the past 6 months".

emeril 3y ago

Sounds like target.com I ordered a play kitchen based on reviews and then I found it to be poor quality, I wrote a review which was never published without any reason given As such, I don't really trust positive reviews anymore, especially, there I only mostly trust yelp reviews

propogandist 3y ago
[dead]

jchw 3y ago

Yep, I noticed the exact same thing. Glassdoor is a complete scam, and it has been for as long as I've had a career. I do not even check anymore.

AlanSE 3y ago
So is it basically true that there is no objective information out there anymore, because anyone running a successful site gets corrupted with money?
- praptak 3y ago
  The only good source is a network of people you know in person. If you don't have one, working on open source projects is a good way to bootstrap one.
- dylan604 3y ago
  That's a big broad brush you just swiped with, but yeah, I kind of agree with it.
- rightbyte 3y ago
  Blind seems somewhat OK?
- iambateman 3y ago
  I’d say Consumer Reports and NYT wire cutter are pretty objective… Their bigger issue is that company quality doesn’t change much from year to year, so the pans that were good in 2019 are probably still good now, which is boring.
  - loloquwowndueo 3y ago
    Wire cutter is hardly objective - check out their bike lock reviews. Every single recommended product is from Kryptonite.
    - Fire-Dragon-DoL 3y ago

Solvency 3y ago

I know a non-engineer who left DesktopMetal with a similar sentiment.

princevegeta89 3y ago

No one trusts these review companies actually. They simply exist to extort entities that get reviewed either to remove negative reviews or promote or even auto-post fake positive reviews. Yelp is a classic example that did this for years.

black_puppydog 3y ago
> No one trusts these review companies It's not so much that I'd read the reviews, but ranking of results is fatally flawed thanks to these practices. For many queries (restaurant in town X, book in genre Y) there are simply so many candidate results that ranking is the one and only thing to determine a (much, MUCH) smaller candidate set that the customer might actually engage with. How often do you click through to page 2 of your search results?
camhart 3y ago
> No one trusts these review companies actually. Most people do trust them, but shouldn't.
- princevegeta89 3y ago
  Actually you are right, sorry. I was exaggerating but it is true that it is mostly the naive/average users of internet that do place any significant credibility in online reviews
  - deleted 3y ago
    [deleted]
Mtinie 3y ago
Can you foresee a scenario where a review site could legitimately exist without turning to synthetic reviews, dark UI patterns, or outright brand extortion?
- alecthomas 3y ago
  https://www.productreview.com.au/ https://www.productreview.com.au/ is the only site I've ever seen that does a good job at user reviews. Each review has to be accompanied by a verifiable receipt. Here's their operating model, which sounds like a mix of paid brand manager subscriptions and advertising: https://support.productreview.com.au/hc/en-us/articles/360001254676-Our-Operating-Model-at-ProductReview-com-au https://support.productreview.com.au/hc/en-us/articles/36000...

pyeri 3y ago

A blockchain based solution can be developed for customer reviews. If your cryptographically signed and verifiable review went and sat on the disk of every computer on the planet, there will be no options to manipulate it.

JALTU 3y ago
But those "seven positive" reviews will still show up! When I was calculating the NPS scores at a former job, IIRC you need seven or eight 100% scores to offset one low score. Interesting that's what happened to the Desktop Metal poster.
- pyeri 3y ago
  But (presumably) those seven posters would be motivated by big tech money and thus be identified by some algorithm or the other in that blockchain? And/or you being a genuine reviewer would have a more organic activity across the blockchain with other products too compared to theirs and that will add to the weight?
  - sverhagen 3y ago
    There's just very little organic activity on a site like Glassdoor. You changing jobs, and thus naturally having any kind of inclination to post something may be few and far between, and you might not even think of uploading a review unless you're disgruntled (read: motivated).
  - JohnFen 3y ago
    > (presumably) those seven posters would be motivated by big tech money Probably not, I think. They're more likely to be employees or friends/family of the company doing it because they were required or asked to.
starttoaster 3y ago
Yeah except there would still be a mechanism for revoking or moderating bad reviews because people would use the platform to review bomb any company they took issue with. The review might still be on the chain's history, but the moderation tool would still be created with good intentions, with a new transaction on a later block that basically revokes the "validity" of the bad review. Similar mechanisms can be seen in NFTs, where you can update the NFTs metadata.

Simon_O_Rourke 3y ago

I've experienced this almost verbatim - put up a factual (at least from my view) account of working with a previous employer, a "warts and all" review. Next week I see about ten 5-star reviews each with a single sentence something like "This is a great place to work!". However, when searching out the low-down on a potential new employer, I only read the one and two star reviews - if they're pretty spurious then I feel more comfortable taking a role there. If however the poor reviews contain lots of pretty specific complaints, I find them more believable that the single sentence "everything is awesome here" style reviews.

pc86 3y ago
I've considered making a browser extension where you can click a review to ignore it and have the average ratings recalculate automatically.

druvisc 3y ago

Same is happening to Google Maps. I and other reviewers had left negative reviews for Cosuno Ventures GmbH as it's an incredibly toxic and horrible place to work in. I received a take-down notice from a German Google Maps review employee saying that Cosuno is claiming it's a false review and they don't know how who am I. Even though I worked at the company for 4 months with the CTO and had responded to the e-mail with documents signed by the CEO and CTO. They're rocking a 5 star review now. I didn't receive a response from Google and my appeal has been ignored. I still have the company's t-shirt in the countryside.

Kretinsky 3y ago
I went to a pediatrician near where I lived. Long story short, she was very toxic, making ultimatums and giving what was revealed as dubious practice. I consulted other similar pediatricians, who all gave me radically different answers than hers. As a result, I chose to write a detailed review explaining why I thought she was unprofessional, citing examples. Of course, my review got nuked. Same happened with a photo shop that screw up a photo development for some films. I left a factual review, with a sample in pictures. I can see my review with my account, but no one else can.
- swayvil 3y ago
  >can see my review with my account, but no one else can. Your review got shadow-removed. That's a new one on me.
  - ShadyOperation 3y ago
    [dead]
i_am_a_peasant 3y ago
Heh, one of my friends wrote a negative review about his landlord's Immobilien company. Got sued and had to pay him the amount of his deposit. I bet I'd be at risk of getting sued in Germany even if I wrote a positive review about a company. The moment you post an opinion online you are opening yourself as a target.

randycupertino 3y ago

At a healthcare tech startup I worked at for every negative review we got management made us write three positive reviews. The owners daughter would get all of her friends from college to write more and more positive reviews. It was so obvious, any fake review written by HR will always have "keep doing what you're doing!" in the "advice to management" section. Like, even a moderately happy employee will actually have something to say in the "advice to management" section. Only a shill will say, "Keep doing what you're doing!"

briHass 3y ago
I swear the 'keep doing what you're doing' must either be the standard HR planted review (I wasn't invited to the HR conference where they learned that!), or it's GD's own spam for pay. I've seen that too frequently at toxic jobs for it to be coincidence. Luckily, just like Amazon reviews, it's pretty easy to determine worthwhile info from how specific and balanced it is. The aggregate scores are obviously worthless, because either shills or spurned employees taint the mix. I enjoy when there are responses from the business: that can often show egotistical management that thinks their poo doesn't stink.
sizzle 3y ago
Heard the same from a friend of a friend, they would game Trip Advisor reviews by giving discounts to customers and the daughter would make accounts and leave positive reviews all the time. There seriously needs to be a watchdog group or AI service (startup idea!) that vets reviews with heuristics like anomaly detection for VPN ip address reviews in succession or browser fingerprinting. Also ChatGPT is gonna make writing unique reviews a thing of the past. Expect fake reviews sounding indistinguishable from genuine human writing. We are screwed.
incone123 3y ago
" The owners daughter would get all of her friends from college to write more and more positive reviews." A cottage industry of liars. I was pretty naive in college and I still would have been skeptical about that.

smcleod 3y ago

Yep. I’ve experience almost exactly the same in the past. I’ve always just assumed glassdoor was a fake / scam site since.

xnx 3y ago

Another case where web annotations would be useful. Recently: iAnnotate – Whatever happened to the web as an annotation system? (2013) https://news.ycombinator.com/item?id=36559758 https://news.ycombinator.com/item?id=36559758

jiveturkey 3y ago
huh? how so? what stops the exact sort of gaming at glassdoor et al, from also occurring via some web annotation system?