7 ms·

Years ago I worked as a frontend dev for Newsweek & The Daily Beast and saw the other side of this. It boiled down to money, brand, advertising and engagement.

Years ago I worked as a frontend dev for Newsweek & The Daily Beast and saw the other side of this. It boiled down to money, brand, advertising and engagement. Wrapping the content in an iframe allowed the host to monetize our content by decorating it with ads. This would dillute the CTR on our ads, resulting in less money. Big sites enter into contracts with major brands for full page advertisements, which could include contractual requirements that theirs is the only ad on the page. Serving and decorating a site would create headaches for us to explain why their ads weren't alone. The iframe also reduced engagement as reader would only read the embedded article and move on. An actual visitor of the site had a far higher likelihood to navigate to other pages. Higher engagement resulted in more money and some brand loyalty. This also dramatically hurt sharing as the browser URL wasn't ours, but the hosts (which I also hate about AMP). Your site messed with a lot of the ways their business generated revenue, in an industry that was already struggling to adapt to "the internet".

mathattack 6y ago
It would seem like the content creator has the right to make their site as easy or hard to aggregate as they want. If you’re in the aggregation business, you have to convince them of the benefits to make it easier. If you can’t, you’re not adding value (to them) for what you’re taking.
- Chris2048 6y ago
  There was a time when corps pushed the idea that you'd need permission to link to their content, where does this begin/end? seems to me there should be some control on behalf of the user wrt to how data is displayed: It maybe your content, but it's running on my hardware; you say "3rd party not allowed to run ads, only us" I say "my hardware, you aren't allowed to run ads either".
  - mpclark 6y ago
    ...then the publisher says "OK, you can't show my content then"
    - martimarkov 6y ago
      Then I scrape the page using JS from the browser side and render it dynamically. It’s all client side. Where would this sit on that line? It’s just an example but for the 1000 ppl trying to create laws there are 10000x trying to bypass them. I don’t think restrictions are the way forward but as of now I haven’t come up with a viable alternative.
      - derivagral 6y ago
        If a brick and mortar store refuses services and kicks you out for disruption... breaking in at night, taking what you want, and leaving money is still illegal. I love web scraping (it and tampermonkey basically got me into CS) but lets not act as if this behavior has no moral precedent.
        monocasa 6y ago
        More like a brick and mortar starts harassing people on the sidewalk for writing down what is on the posters on the side of their building.
        Chris2048 6y ago
        Ah, the old "unauthorized content handling is the same as theft" nonsense, because a more fitting metaphor wouldn't justify police-state copyright enforcement. Also, what do you mean by "disruption"? Using "their" content in a way they don't like? content manipulation on my machine is my house, not theirs.
        cercatrova 6y ago
        Web scraping is completely legal, see the LinkedIn case recently
        feanaro 6y ago
        You really shouldn't expect to be able to put up a public website and then get upset when people start connecting to it using a standard, open protocol to retrieve the information you're deliberately exposing to the world.
  - watwut 6y ago
    iframe is super reasonable line to make. "You cant show my content in iframe" is alright rule to make.
  - mathattack 6y ago
    My sense is if you put it on the public internet, by default you are giving people the ability to link to it. It’s just a matter of how hard or easy you choose to make it. Copying copyrights is more complex.
    - Chris2048 6y ago
      Then where do ad-blockers fall? Am I violating copyright/rights-management by altering how content is viewed? If a content provider can dictate the terms by which their content is consumed, why not dictate linking rights too?
      - mathattack 6y ago
        We are getting further from my expertise. My inclination is on the public internet it’s ok to make an ad blocker and ok to make something that thwarts the ad blocker. Different on a private network.
- georgeecollins 6y ago
  The problem is that Google can place your content down the search list for not making it easy for them to aggregate it. That is the anti-trust issue.
nr2x 6y ago
I think what the "legacy" news media failed at more than adapting was effectively communicating the time and effort that goes into serious news gathering and reporting. I regularly get the impression a certain slice of HN commenters think "the media" is a bunch of opinion bloggers fabricating sources to drive "an agenda". In reality, the effort for producing a deep investigative story can be equivalent to an a mid-size software engineering project in terms of person hours and expertise. Three senior reporters working on a Pulitzer-level story can take years, numerous support staff, and legal fees. Most software engineers innately know that taking an open-source project, stripping the authorship, putting it your own site and running ads next to it is unethical, regardless of the licensing etc. Yet, when it comes to capturing news stories of equivalent production effort, "information wants to be free". I think that may largely be a failure to understand the work that real reporting takes. Ironically, as prestige newsrooms have shrunk, the percentage of high-quality stories has gone down because revenue has declined and those stories are expensive. So in fact we do perversely end up with more click-bait as Google et al engineered systems to optimize for attention and click-through (the opposite of a multi-part investigation).
- kbutler 6y ago
  > "the media" is a bunch of opinion bloggers fabricating sources to drive "an agenda". In reality, the effort for producing a deep investigative story can be equivalent to an a mid-size software engineering project in terms of person hours and expertise. Three senior reporters working on a Pulitzer-level story can take years, numerous support staff, and legal fees. You are suggesting these are mutually exclusive.
  - nr2x 6y ago
    Per my final point, I'm suggesting adtech has given us more of the former to the detriment of the latter.
    - kbutler 6y ago
      I don't think there is any evidence to suggest deep, unbiased, evidence-based journalism ever dominated the field, or that the "crass commercial" interests didn't drive the news rooms. See the "yellow journalism" of Joseph Pulitzer's era, for example. Further, "expensive, pulitzer level" work may also be agenda-driven. Walter Duranty's bias and agenda, for example, aren't really in doubt anymore.
josteink 6y ago
> The iframe also reduced engagement as reader would only read the embedded article and move on. The entire point of RSS was to put the reader into exactly that position though.
- tomc1985 6y ago
  Which is why RSS (and by extension the kinds of open standards that facilitate things like aggregation) is dead I blame marketeers trying to control the brand, messaging and experience.
  - Wowfunhappy 6y ago
    Most of the major news websites still offer RSS feeds, although you do have to search quite a bit to find them. NYTimes, WSJ, Vox Media, The Atlantic, Time Magazine, etc etc all have them. Those last two are even full-text feeds, which is super lovely! I have a suspicion that journalists rely on RSS more than the general population, which gives the technology support within news organizations themselves.
    - 52-6F-62 6y ago
      I can speak first hand to that, in Canada. RSS is actively supported. Specific RSS feeds are even run to (web, Apple News) aggregators after agreements are made to syndicate the content. And we maintain feeds for the Canadian government archives. I can only imagine American outlets do something similar.
- 52-6F-62 6y ago
  The reader independently, yes. Not aggregators that operate as a business.
deleted 6y ago
[deleted]
musicale 6y ago
> Your site messed with a lot of the ways their business generated revenue, in an industry that was already struggling to adapt to "the internet". In 1972 Alan Kay predicted that one of the first programs the user of a global network would write would be a "filter to eliminate advertising." However, I don't know if he foresaw that opportunistic republishers would strip the original ads and replace them with new ones.
leetrout 6y ago
WPNI?
Apotheos 6y ago
As someone who works at TDB now, I can tell you we did the analysis and we were losing money from AMP, so we completely opted out of it.
- notjustanymike 6y ago
  I wonder I any of my ads and analytics code still exists in CQ5, or whatever it's called now.
tripzilch 6y ago
If this is a case of one party trying to throw more ads at the "user", and being miffed about earning less of the ad network tracking points than the other (meanwhile the ad networks win either way), then what are you defending? Because in that case there is no good guy, no "other side", just garbage. Stuffing the WWW with ad networks isn't exactly saving it.
- notjustanymike 6y ago
  I'm not really defending anything, just explaining. But I do sympathize with any content creator trying to get paid for their work even if it is a large organization.

mathattack 6y ago

It would seem like the content creator has the right to make their site as easy or hard to aggregate as they want. If you’re in the aggregation business, you have to convince them of the benefits to make it easier. If you can’t, you’re not adding value (to them) for what you’re taking.

Chris2048 6y ago
There was a time when corps pushed the idea that you'd need permission to link to their content, where does this begin/end? seems to me there should be some control on behalf of the user wrt to how data is displayed: It maybe your content, but it's running on my hardware; you say "3rd party not allowed to run ads, only us" I say "my hardware, you aren't allowed to run ads either".
- mpclark 6y ago
  ...then the publisher says "OK, you can't show my content then"
  - martimarkov 6y ago
    Then I scrape the page using JS from the browser side and render it dynamically. It’s all client side. Where would this sit on that line? It’s just an example but for the 1000 ppl trying to create laws there are 10000x trying to bypass them. I don’t think restrictions are the way forward but as of now I haven’t come up with a viable alternative.
    - derivagral 6y ago
      If a brick and mortar store refuses services and kicks you out for disruption... breaking in at night, taking what you want, and leaving money is still illegal. I love web scraping (it and tampermonkey basically got me into CS) but lets not act as if this behavior has no moral precedent.
      - monocasa 6y ago
        More like a brick and mortar starts harassing people on the sidewalk for writing down what is on the posters on the side of their building.
      - Chris2048 6y ago
        Ah, the old "unauthorized content handling is the same as theft" nonsense, because a more fitting metaphor wouldn't justify police-state copyright enforcement. Also, what do you mean by "disruption"? Using "their" content in a way they don't like? content manipulation on my machine is my house, not theirs.
      - cercatrova 6y ago
        Web scraping is completely legal, see the LinkedIn case recently
      - feanaro 6y ago
        You really shouldn't expect to be able to put up a public website and then get upset when people start connecting to it using a standard, open protocol to retrieve the information you're deliberately exposing to the world.
- watwut 6y ago
  iframe is super reasonable line to make. "You cant show my content in iframe" is alright rule to make.
- mathattack 6y ago
  My sense is if you put it on the public internet, by default you are giving people the ability to link to it. It’s just a matter of how hard or easy you choose to make it. Copying copyrights is more complex.
  - Chris2048 6y ago
    Then where do ad-blockers fall? Am I violating copyright/rights-management by altering how content is viewed? If a content provider can dictate the terms by which their content is consumed, why not dictate linking rights too?
    - mathattack 6y ago
      We are getting further from my expertise. My inclination is on the public internet it’s ok to make an ad blocker and ok to make something that thwarts the ad blocker. Different on a private network.
georgeecollins 6y ago
The problem is that Google can place your content down the search list for not making it easy for them to aggregate it. That is the anti-trust issue.

nr2x 6y ago

I think what the "legacy" news media failed at more than adapting was effectively communicating the time and effort that goes into serious news gathering and reporting. I regularly get the impression a certain slice of HN commenters think "the media" is a bunch of opinion bloggers fabricating sources to drive "an agenda". In reality, the effort for producing a deep investigative story can be equivalent to an a mid-size software engineering project in terms of person hours and expertise. Three senior reporters working on a Pulitzer-level story can take years, numerous support staff, and legal fees. Most software engineers innately know that taking an open-source project, stripping the authorship, putting it your own site and running ads next to it is unethical, regardless of the licensing etc. Yet, when it comes to capturing news stories of equivalent production effort, "information wants to be free". I think that may largely be a failure to understand the work that real reporting takes. Ironically, as prestige newsrooms have shrunk, the percentage of high-quality stories has gone down because revenue has declined and those stories are expensive. So in fact we do perversely end up with more click-bait as Google et al engineered systems to optimize for attention and click-through (the opposite of a multi-part investigation).

kbutler 6y ago
> "the media" is a bunch of opinion bloggers fabricating sources to drive "an agenda". In reality, the effort for producing a deep investigative story can be equivalent to an a mid-size software engineering project in terms of person hours and expertise. Three senior reporters working on a Pulitzer-level story can take years, numerous support staff, and legal fees. You are suggesting these are mutually exclusive.
- nr2x 6y ago
  Per my final point, I'm suggesting adtech has given us more of the former to the detriment of the latter.
  - kbutler 6y ago
    I don't think there is any evidence to suggest deep, unbiased, evidence-based journalism ever dominated the field, or that the "crass commercial" interests didn't drive the news rooms. See the "yellow journalism" of Joseph Pulitzer's era, for example. Further, "expensive, pulitzer level" work may also be agenda-driven. Walter Duranty's bias and agenda, for example, aren't really in doubt anymore.

josteink 6y ago

> The iframe also reduced engagement as reader would only read the embedded article and move on. The entire point of RSS was to put the reader into exactly that position though.

tomc1985 6y ago
Which is why RSS (and by extension the kinds of open standards that facilitate things like aggregation) is dead I blame marketeers trying to control the brand, messaging and experience.
- Wowfunhappy 6y ago
  Most of the major news websites still offer RSS feeds, although you do have to search quite a bit to find them. NYTimes, WSJ, Vox Media, The Atlantic, Time Magazine, etc etc all have them. Those last two are even full-text feeds, which is super lovely! I have a suspicion that journalists rely on RSS more than the general population, which gives the technology support within news organizations themselves.
  - 52-6F-62 6y ago
    I can speak first hand to that, in Canada. RSS is actively supported. Specific RSS feeds are even run to (web, Apple News) aggregators after agreements are made to syndicate the content. And we maintain feeds for the Canadian government archives. I can only imagine American outlets do something similar.
52-6F-62 6y ago
The reader independently, yes. Not aggregators that operate as a business.

deleted 6y ago

[deleted]

musicale 6y ago

> Your site messed with a lot of the ways their business generated revenue, in an industry that was already struggling to adapt to "the internet". In 1972 Alan Kay predicted that one of the first programs the user of a global network would write would be a "filter to eliminate advertising." However, I don't know if he foresaw that opportunistic republishers would strip the original ads and replace them with new ones.

leetrout 6y ago

WPNI?

Apotheos 6y ago

As someone who works at TDB now, I can tell you we did the analysis and we were losing money from AMP, so we completely opted out of it.

notjustanymike 6y ago
I wonder I any of my ads and analytics code still exists in CQ5, or whatever it's called now.

tripzilch 6y ago

If this is a case of one party trying to throw more ads at the "user", and being miffed about earning less of the ad network tracking points than the other (meanwhile the ad networks win either way), then what are you defending? Because in that case there is no good guy, no "other side", just garbage. Stuffing the WWW with ad networks isn't exactly saving it.

notjustanymike 6y ago
I'm not really defending anything, just explaining. But I do sympathize with any content creator trying to get paid for their work even if it is a large organization.