5 ms·

"""The systemic issue occurs because marketers and AdTech companies unreflectively increase surveillance and data collection in the hopes of increasing media at

by dpflan 7y ago

"""The systemic issue occurs because marketers and AdTech companies unreflectively increase surveillance and data collection in the hopes of increasing media attribution. Even the most trivial data point, the most granular site visit or location data, stored on a person-by-person basis, could conceivably be used to demonstrate that a customer may have come into contact with one of your campaigns.""" I'm genuinely intrigued by AdTech. It exists, but no one likes it -- i.e. before internet ads --> TiVo, internet ads --> ad blockers. I understand it can attract talent due to profits and the engineering + data problems, but is that the main attraction for developers? If you worked in adTech, what attracted you to it? Is there a hope you could make more ethical or privacy aware or (<insert positive idea here>) ad tech systems? Does adTech even work? Is there an active campaign somewhere / grassroots movement to "boycott" recruitment to ad tech companies? (Just trying to get a better understanding here, ad tech has become symbiotic with the current state of the internet, so I would like to better understand it.)

tomlockwood 7y ago
Author here. I was attracted by the money, but also the exciting technical challenges. AdTech had all the largest datasets! At the time I entered the industry I wasn't really thinking about the ethical concerns. I just didn't think those issues would bother me as much as they ended up bothering me.
- dpflan 7y ago
  Thank you for sharing your perspective and authoring the article. Is there growing sentiment like yours amongst colleagues and peers in ad tech?
  - tomlockwood 7y ago
    I'm not sure since I've been out for a couple years, but at the time I'd often have conversations with marketers where I'd come away wondering how they could be so oblivious to the ethical concerns in what they were asking for. I do think as time goes on, more and more people are becoming aware of the issues, but I see that uniformly across society rather than in tech.
    - criddell 7y ago
      > I'd come away wondering how they could be so oblivious to the ethical concerns in what they were asking for Like Upton Sinclair said, "It is difficult to get a man to understand something, when his salary depends on his not understanding it."
      - chibg10 7y ago
        I'm not sure that's what's at play here. Marketers could easily be aware of the ethical concerns and simply disregard them. Their salary doesn't depend at all on their ignorance of the ethical issues involved. More likely, at the risk of overgeneralizing, I suspect a lot of the people going into adtech aren't really the philosophical-type (I think those probably tend to avoid marketing in general), a lot of them are young, and probably excited just to get a job offer in such a well-paying, high-profile industry.
        tomlockwood 7y ago
        I think criddell is right. I think the amount of money makes people gloss over some of the ethics, as cynical as that probably sounds. I didn't say it in the article specifically because of it sounding so cynical (even though the article is very cynical). I think you're right, that a lot of the people going into AdTech, and just marketing in general, aren't philosophical. But, you saying that made me chuckle, because I have a BA in Philosophy, and no computing qualification.
        theshadowknows 7y ago
        A lot of the people I work with are looking for cheap wins. So they bend the rules and ignore ethics. Many retailers for example are data rich on the aggregate level. And could very likely achieve great results without trying to spearfish customers further. But the analytics insights required to do that take time and work. And marketers don’t like to do that. So they’d rather “enrich” crm data with Experian or some other service. When really they can’t handle they data they have and adding to it will just make it worse.
  - cattlefarmer 7y ago
    One more datapoint here if you care. The industry is filled with hugely interesting technical challenges even if you're not at FAANG scale. Trivial things like building a cost efficient delivery infrastructure to reducing the number of steps between receiving that first page load to ad delivery and then on to analytics. A lot of it is just math and it's fun when you structure your challenges that way. It's like inverse Pinky and the Brain everyday. We're not scheming to erode everyone's privacy, we're just lab rats looking for the shortest route from one end of the maze to the other. I also left after a few years because all that maze running got boring eventually and I decided I'd like something with a little bit more meaning. Or, from another point of view, those years eroding everyone's privacy has enabled me to climb high enough up on Maslow's pyramid such that I now have the comfort to worry about self actualisation. edit: There were a few people that totally drank the kool aid, and believed that what we were doing was enabling people to gain a livelihood by running a popular website. Usually the really young ones or those at management level. Those from a developed country were mostly interested in the technical problems and improving their engineering skills. Those from developing countries were most definitely in it for the money.
adwww 7y ago
I also worked in Ad Tech - outside of a major tech area, it was the biggest, fastest growing tech employer around. The pay was only average for the area, but the company clearly had money and was experiencing rapid growth which is always exciting. As you mention, massive data sets, tens of billions of HTTP requests per day, etc, are simply not challenges you get outside of FAANG, so there was a lot of interesting work. The really shady stuff only came to light once you'd worked there a while, and realised the entire industry is basically built on fraud and money laundering with almost nobody incentivised to stop it.
- rbinv 7y ago
  > tens of billions of HTTP requests per day Could you elaborate? Seems very high, even for ad impressions.
  - fsaintjacques 7y ago
    rtb bid request volume can easily go 1m/sec, this is not impressions, but offers to buy for an impression.
    - rbinv 7y ago
      Makes sense, thanks.
- cattlefarmer 7y ago
  It's not often one can claim to have saved the company tens of thousands of dollars in operating costs each month just by tweaking a few AWS settings. I definitely miss being able to work on such a large volume of data but at least I don't have to worry about PagerDuty anymore.
fsaintjacques 7y ago
Technical challenges and salary. In my local market, most jobs are web shops and/or CRUD business applications with low pay.
ilikehurdles 7y ago
I worked in adTech. The company I worked for did not collect personal or demographic data, and we did not care about users' cookies, so I did not feel too conflicted from the ethical perspective, but still found it difficult to explain to non-tech people exactly what my job was. The thing that attracted me to it was the technology and scale of data involved. From a purely engineering perspective, it was highly attractive. It was a small company, and as a small company in ad tech that still meant rapid and 0-downtime deploys, deeply-ingrained devops culture, rapidly evolving technology, and indexing/reporting/decision-making on thousands of events events every second. During peak seasons, every day we'd have another terabyte of data in the database. AdTech also affords a lot more transparency into company finances than you would in other companies. You're building the tools used for reporting on revenue and costs that the entire business uses for almost all decision-making. Engineers don't live in a silo from that perspective. From a product perspective, it was much less rewarding. You know how often times internal-facing applications have the worst UX? Everything we built felt like that because ultimately it was. There is also a lot of pressure that comes with the points above -- every second our system takes to throttle an ad campaign that has reached their budget, encounters an unavoidable AWS downtime, or exposes a reporting bug is company dollars down the drain.
taormina 7y ago
I left an AdTech company about a year ago. I went to learn more about data engineering and applied machine learning. I learned a phenomenal amount, but I really grew to dislike the advertising industry. The incentives are hilarious misaligned, as the article discusses at length. To be honest, a large part of my ethical discomfort was not only about implicitly contributing to the surveillance state, but about the waste of the use of computing power and time and work. Even if we succeed, and deliver the perfect recommendation to every single shopper, then <Large Client> might make X% more ROI from their website. Maybe a yay for <Large Client>'s stockholders? Unless you know, Twitter talks about them this week... How is the world a better place because I'm working at my job today? I don't make my senior software engineer salary working for myself, but I've "digitized" 4 separate businesses since I left. It feels good to build tangible products and write software that make someone's life easier.
kasey_junk 7y ago
Building systems that support millions of requests per second, that have actual real time constraints, that generate tons of data and operate in global networks. That produces a lot of interesting technical challenges. AdTech absolutely works in some sense. Advertisers are fairly savvy about being able track improvements in sales of the things they are advertising and they have seen improvements using digital advertising. Does all the tracking/segmentation/attribution voodoo work? Does it make up for all the fraud and middlemen? I think those are open questions. As for a privacy aware ad tech system, I think it’s fairly trivial to do that with something like RTB/open ecosystems. The problem there is that its a low margin business and the race to the bottom doesn’t incentivize it. People operating in that space are competing with Facebook and Google who obliterate the privacy veil & dominate the landscape.
bongobongo 7y ago
In my experience, AdTech companies will pay top dollar for engineering leadership (CTO, VP, etc.) but then hire a legion of junior developers at entry-level salaries to execute. I do wonder if AdTech hiring managers focus on young and inexperienced people fresh out of school who are attracted to beer and ping-pong and a decent salary for a first job.
theshadowknows 7y ago
I’ve been in the game for about 8 years now and I agree a lot with the author. I also think one issue is in the talent pool and hiring. You get a lot of well meaning but by and large poorly trained people in the space. And the result is poor use of the tools. Look at Salesforce Marketing Cloud for example. It’s immensely powerful. But 9 out of 10 orgs that use it are a shit show. And their results are often terrible. Because you have teams that simply aren’t technical enough to use it properly. This is common across the entire ad tech stack.

by dpflan 7y ago

tomlockwood 7y ago

Author here. I was attracted by the money, but also the exciting technical challenges. AdTech had all the largest datasets! At the time I entered the industry I wasn't really thinking about the ethical concerns. I just didn't think those issues would bother me as much as they ended up bothering me.

dpflan 7y ago
Thank you for sharing your perspective and authoring the article. Is there growing sentiment like yours amongst colleagues and peers in ad tech?
- tomlockwood 7y ago
  I'm not sure since I've been out for a couple years, but at the time I'd often have conversations with marketers where I'd come away wondering how they could be so oblivious to the ethical concerns in what they were asking for. I do think as time goes on, more and more people are becoming aware of the issues, but I see that uniformly across society rather than in tech.
  - criddell 7y ago
    > I'd come away wondering how they could be so oblivious to the ethical concerns in what they were asking for Like Upton Sinclair said, "It is difficult to get a man to understand something, when his salary depends on his not understanding it."
    - chibg10 7y ago
      I'm not sure that's what's at play here. Marketers could easily be aware of the ethical concerns and simply disregard them. Their salary doesn't depend at all on their ignorance of the ethical issues involved. More likely, at the risk of overgeneralizing, I suspect a lot of the people going into adtech aren't really the philosophical-type (I think those probably tend to avoid marketing in general), a lot of them are young, and probably excited just to get a job offer in such a well-paying, high-profile industry.
      - tomlockwood 7y ago
        I think criddell is right. I think the amount of money makes people gloss over some of the ethics, as cynical as that probably sounds. I didn't say it in the article specifically because of it sounding so cynical (even though the article is very cynical). I think you're right, that a lot of the people going into AdTech, and just marketing in general, aren't philosophical. But, you saying that made me chuckle, because I have a BA in Philosophy, and no computing qualification.
        theshadowknows 7y ago
        A lot of the people I work with are looking for cheap wins. So they bend the rules and ignore ethics. Many retailers for example are data rich on the aggregate level. And could very likely achieve great results without trying to spearfish customers further. But the analytics insights required to do that take time and work. And marketers don’t like to do that. So they’d rather “enrich” crm data with Experian or some other service. When really they can’t handle they data they have and adding to it will just make it worse.
- cattlefarmer 7y ago
  One more datapoint here if you care. The industry is filled with hugely interesting technical challenges even if you're not at FAANG scale. Trivial things like building a cost efficient delivery infrastructure to reducing the number of steps between receiving that first page load to ad delivery and then on to analytics. A lot of it is just math and it's fun when you structure your challenges that way. It's like inverse Pinky and the Brain everyday. We're not scheming to erode everyone's privacy, we're just lab rats looking for the shortest route from one end of the maze to the other. I also left after a few years because all that maze running got boring eventually and I decided I'd like something with a little bit more meaning. Or, from another point of view, those years eroding everyone's privacy has enabled me to climb high enough up on Maslow's pyramid such that I now have the comfort to worry about self actualisation. edit: There were a few people that totally drank the kool aid, and believed that what we were doing was enabling people to gain a livelihood by running a popular website. Usually the really young ones or those at management level. Those from a developed country were mostly interested in the technical problems and improving their engineering skills. Those from developing countries were most definitely in it for the money.

adwww 7y ago

I also worked in Ad Tech - outside of a major tech area, it was the biggest, fastest growing tech employer around. The pay was only average for the area, but the company clearly had money and was experiencing rapid growth which is always exciting. As you mention, massive data sets, tens of billions of HTTP requests per day, etc, are simply not challenges you get outside of FAANG, so there was a lot of interesting work. The really shady stuff only came to light once you'd worked there a while, and realised the entire industry is basically built on fraud and money laundering with almost nobody incentivised to stop it.

rbinv 7y ago
> tens of billions of HTTP requests per day Could you elaborate? Seems very high, even for ad impressions.
- fsaintjacques 7y ago
  rtb bid request volume can easily go 1m/sec, this is not impressions, but offers to buy for an impression.
  - rbinv 7y ago
    Makes sense, thanks.
cattlefarmer 7y ago
It's not often one can claim to have saved the company tens of thousands of dollars in operating costs each month just by tweaking a few AWS settings. I definitely miss being able to work on such a large volume of data but at least I don't have to worry about PagerDuty anymore.

fsaintjacques 7y ago

Technical challenges and salary. In my local market, most jobs are web shops and/or CRUD business applications with low pay.

ilikehurdles 7y ago

I worked in adTech. The company I worked for did not collect personal or demographic data, and we did not care about users' cookies, so I did not feel too conflicted from the ethical perspective, but still found it difficult to explain to non-tech people exactly what my job was. The thing that attracted me to it was the technology and scale of data involved. From a purely engineering perspective, it was highly attractive. It was a small company, and as a small company in ad tech that still meant rapid and 0-downtime deploys, deeply-ingrained devops culture, rapidly evolving technology, and indexing/reporting/decision-making on thousands of events events every second. During peak seasons, every day we'd have another terabyte of data in the database. AdTech also affords a lot more transparency into company finances than you would in other companies. You're building the tools used for reporting on revenue and costs that the entire business uses for almost all decision-making. Engineers don't live in a silo from that perspective. From a product perspective, it was much less rewarding. You know how often times internal-facing applications have the worst UX? Everything we built felt like that because ultimately it was. There is also a lot of pressure that comes with the points above -- every second our system takes to throttle an ad campaign that has reached their budget, encounters an unavoidable AWS downtime, or exposes a reporting bug is company dollars down the drain.

taormina 7y ago

I left an AdTech company about a year ago. I went to learn more about data engineering and applied machine learning. I learned a phenomenal amount, but I really grew to dislike the advertising industry. The incentives are hilarious misaligned, as the article discusses at length. To be honest, a large part of my ethical discomfort was not only about implicitly contributing to the surveillance state, but about the waste of the use of computing power and time and work. Even if we succeed, and deliver the perfect recommendation to every single shopper, then <Large Client> might make X% more ROI from their website. Maybe a yay for <Large Client>'s stockholders? Unless you know, Twitter talks about them this week... How is the world a better place because I'm working at my job today? I don't make my senior software engineer salary working for myself, but I've "digitized" 4 separate businesses since I left. It feels good to build tangible products and write software that make someone's life easier.

kasey_junk 7y ago

Building systems that support millions of requests per second, that have actual real time constraints, that generate tons of data and operate in global networks. That produces a lot of interesting technical challenges. AdTech absolutely works in some sense. Advertisers are fairly savvy about being able track improvements in sales of the things they are advertising and they have seen improvements using digital advertising. Does all the tracking/segmentation/attribution voodoo work? Does it make up for all the fraud and middlemen? I think those are open questions. As for a privacy aware ad tech system, I think it’s fairly trivial to do that with something like RTB/open ecosystems. The problem there is that its a low margin business and the race to the bottom doesn’t incentivize it. People operating in that space are competing with Facebook and Google who obliterate the privacy veil & dominate the landscape.

bongobongo 7y ago

In my experience, AdTech companies will pay top dollar for engineering leadership (CTO, VP, etc.) but then hire a legion of junior developers at entry-level salaries to execute. I do wonder if AdTech hiring managers focus on young and inexperienced people fresh out of school who are attracted to beer and ping-pong and a decent salary for a first job.

theshadowknows 7y ago

I’ve been in the game for about 8 years now and I agree a lot with the author. I also think one issue is in the talent pool and hiring. You get a lot of well meaning but by and large poorly trained people in the space. And the result is poor use of the tools. Look at Salesforce Marketing Cloud for example. It’s immensely powerful. But 9 out of 10 orgs that use it are a shit show. And their results are often terrible. Because you have teams that simply aren’t technical enough to use it properly. This is common across the entire ad tech stack.