15 ms·

How to Lose Money on Paid Marketing

53 points by jamiequint 11y ago

DLarsen 11y ago
I'll also add that the attribution window is totally relevant as well. In the multi-touch example given, the 5-day reporting window does not account for the fact that tomorrow's revenue will be sprinkled into today. That means that your ROI for today's spend isn't final until you're beyond the attribution window.
- shostack 11y ago
  Yep. And when looking at data you have to be very clear on whether the conversion data you are seeing reflects "date of click" vs. "date of conversion." "Date of Click" data will continue to change throughout the duration of your reporting window whereas "date of conversion" does not. This can get sticky in things like AdWords where the main UI reports on "date of click" and keeps changing, whereas Search Funnel reports report by "date of conversion." Fortunately, most bid management platforms like Marin let you make that distinction in reporting.
  - jamiequint 11y ago
    Date of click is fine as long as you are understanding your reporting over fixed timeframes. e.g. What is my Day 7 ROAS on spend from 2 weeks ago, that will never change once you get past 7 days from the window of spend that you are looking at. Date-of-conversion has the different (and in my opinion worse) issue of being 100% accurate in terms of current cost-per-conversion but under-values all marketing channels. This is because it attributes the cost of visits only to conversions that have already happened, while more conversions may later be attributed to those visits. This means that a conversion based model is actually worse for channel analysis.
    - shostack 11y ago
      Agreed on both points. Ultimately, everyone should try to gain a better understanding of the duration and number of touch points in their sales cycle. This directly impacts the lookback window you set. It is also important to know what these are when comparing any data across platforms. For example, right now I'm going from a 90 day window for certain goals in AdWords because I know that while the majority convert on that goal in ~30 days, there's a long tail. Unfortunately, Marin restricts their window to 30 days, so any deeper digging will need to occur within Google Analytics or data from our data warehouse.
      - jamiequint 11y ago
        Would be great to learn more about solutions you've seen in the wild since it seems like you have a lot of experience on this topic. If you have time for a chat my email is in my profile (tried to find you but shostack is ambiguous :)
- rob_lh 11y ago
  Isn't that an argument for assessment on a campaign basis? I'm still wrapping my head around his calculations. It looked like he included the cost of the ads prior to the reporting window in his multi-touch model. That's a fair way of measuring the true cost of user acquisition, but if the ads you are paying for today won't pay dividends until you're out of the reporting window, it would skew the ROI. I'm not sure if this is the same thing as what you cited, as my argument is about the cost association during the reporting window as opposed to the future revenue association. I can't figure out a way to back those out in real-time (i.e. in enough time to make adjustments to the campaign) though.
  - DLarsen 11y ago
    In long-running campaigns, I still typically want to be able to measure changes to performance over time. With a fluency in the ramifications of the attribution models and windows, I like to look at multiple attribution models (last-click and parabolic multi-touch) to draw conclusion about performance. My takeaway from the post was simply, "Be aware of how much attribution models influence metrics," and I think that's a critical point.
    - rob_lh 11y ago
      Indeed, that is the most salient point. I like your suggestion of looking at many attribution models to assess true performance. This seems especially relevant in larger organizations where there is an incentive to game the budgeting process - presenting just the best possible case is what marketing is all about but can lead to skewed perspectives of success.
- jamiequint 11y ago
  This is definitely true, you can think of that as Day 1 ROAS/ROI vs Day N ROAS/ROI. So when looking at your marketing spend you can see how it pays back over time since depending on how you attribute it could change forever (e.g. if you give some value of each conversion back to the original conversion source/sources).
cassieramen 11y ago
This is a great argument for having a data scientist or other statistician run your marketing department. Tech is really changing the way people think about marketing expertise. What counted 10 or even 5 years ago is quickly growing irrelevant.
dfragnito 11y ago
Not factoring in Cost of Sales is also a good way to lose money on Paid Marketing.
dfragnito 11y ago
Revenue per visit really? If a visit does not convert how does it have any value (yes I understand the math)? Why not convert Cost-Per-Visit to Cost-Per-Conversion (total spend / # conversions). I am going through this now with a client. The CPC gurus need to justify the CPC budget and surprise surprise the more cpc dollars spent the more they make. What perverse incentive system they have set up for themselves.
- jamiequint 11y ago
  If a visit doesn't convert it has $0 value. You can convert cost-per-visit to cost-per-conversion instead (the #s will come out the same) but its more complicated to do technically.
  - dfragnito 11y ago
    (total spend / # conversions) is pretty easy to do
    - jamiequint 11y ago
      You can't do it that easily. If you just take total_spend/conversions the question that leads to is what spend do I measure over what timeframe and what conversions do I measure over what timeframe? For example, if I want to know cost-per-conversion for my Facebook traffic then which conversions should I consider in your equation above?
      - dfragnito 11y ago
        a time frame is assumed in my equation. Calculating conversions and ad spend over a time frame is easy if you have your analytics tracking set up properly.
        jamiequint 11y ago
        You can't just add up marketing spend and conversions over the same period of time and divide them. That's not an accurate method of measurement. The conversions and the visits they lead to them are not guaranteed to happen over similar timeframes. If you calculate this the way you're suggesting you are making all sorts of implicit assumptions about variance in visit volume, marketing spend, conversion rates, etc.
swehner 11y ago
I couldn't quite follow what the problem is. I would define the revenue in the ROAS formula as revenue per time period, and the marketing spend also per time period Then I would qualify the revenue, as that part of the revenue that can be tied to the marketing activity (cookies, refer URL, etc) And then the ROAS ratio becomes an ordanary division, because both values are known; no need to infer them as in the rest of the article.
- DLarsen 11y ago
  The whole issue is how you tie revenue to marketing activity. In a case where you had multiple touch points before a conversion, it's a question of which channel gets credit and in what time frame.
shostack 11y ago
Respectfully, this is a submarine PR piece. From the last paragraph: >"It’s a lot of work to build out accurate multi-touch attribution, but there are third-party solutions out there that can simplify the process. My company Interstate makes a pretty good one in my opinion!" That said, the importance of the topic, cross-channel attribution, cannot be emphasized enough for any company doing any sort of serious digital marketing these days. A fantastic piece that goes much deeper would be Avinash Kaushik's article[1] on various modeling approaches. It is a great primer for both people new to the concept and approaches, as well as people with familiarity who are looking for new ideas. The broader challenges these days are shifting towards the cross-device dilemma as well as reliance on 3rd party cookies and browsers' and users' increasingly aggressive stance against those. -Cross-Device Dilemma- I visit initially on mobile and then convert on desktop. Even Google has challenges with cross-device accuracy here, and depending on your mobile usage, it could dramatically impact how you view your performance. There are a variety of technology solutions out there that attempt to solve this, but ultimately there is no great solution that doesn't make me truly question the privacy aspects of what it takes to get to accurate data here. If someone signs up for a trial and gives you a unique user ID, great, you can join that to the data from usage on other devices and understand that user's path to conversion. If they don't take an action that requires logging in, it is much murkier (not counting cookie onboarding services like Liveramp etc.). -3rd Party Cookies- Google and Facebook are excellently positioned because most browsers recognize their tags as 1st party. Many other networks and adtech platforms are boned here unless they can have their tags served as 1st party. I've seen variances of around 15-18% or higher in some cases between 1st and 3rd party tracking setups. While it is REALLY hard to control for all of the unknowns (no two web analytics platforms will ever give you the exact same data), my hunch is that 3rd party vs. 1st party cookies are playing a significant role here along with cookie deletion. Would love any other data folks can share on this. Beyond that, the author talks about static models, but does not address their real limitations. At the end of the day, these look at aggregate data, when the reality is attribution occurs at the user level. Every touch point has some value, and that value is different for each user. For example, seeing a display ad on one site does not necessarily have the same value as seeing it on a different placement within that same site. Static models are very helpful directionally, but dangerous when taken as religion. The future is in dynamic attribution models that constantly shift based on the data and whatever contributing factors you can feed in as supplementary data to the model. I'm super excited to see what Google starts doing with Adometry (check out the TV release they just had), and the other big players like VisualIQ and Convertro will likely have their technology trickle out through other platforms over time. FB's Atlas acquisition is also a big statement that they take this seriously. Net net, attribution is the hardest challenge in the marketing/advertising world today. Period. End of discussion. It is plagued with data that is messy as hell and incredibly nuanced based on the infinite variables of everyone's unique setups. Look through a variety of attribution models as different lenses to help inform directional decisions, but be cautious on saying "this is the model we will use across everything." It might help initially, but without constantly checking it against other models, you can miss shifts in your efforts. [1]http://www.kaushik.net/avinash/multi-channel-attribution-modeling-good-bad-ugly-models/ http://www.kaushik.net/avinash/multi-channel-attribution-mod...

53 points by jamiequint 11y ago

I'll also add that the attribution window is totally relevant as well. In the multi-touch example given, the 5-day reporting window does not account for the fact that tomorrow's revenue will be sprinkled into today. That means that your ROI for today's spend isn't final until you're beyond the attribution window.

shostack 11y ago
Yep. And when looking at data you have to be very clear on whether the conversion data you are seeing reflects "date of click" vs. "date of conversion." "Date of Click" data will continue to change throughout the duration of your reporting window whereas "date of conversion" does not. This can get sticky in things like AdWords where the main UI reports on "date of click" and keeps changing, whereas Search Funnel reports report by "date of conversion." Fortunately, most bid management platforms like Marin let you make that distinction in reporting.
- jamiequint 11y ago
  Date of click is fine as long as you are understanding your reporting over fixed timeframes. e.g. What is my Day 7 ROAS on spend from 2 weeks ago, that will never change once you get past 7 days from the window of spend that you are looking at. Date-of-conversion has the different (and in my opinion worse) issue of being 100% accurate in terms of current cost-per-conversion but under-values all marketing channels. This is because it attributes the cost of visits only to conversions that have already happened, while more conversions may later be attributed to those visits. This means that a conversion based model is actually worse for channel analysis.
  - shostack 11y ago
    Agreed on both points. Ultimately, everyone should try to gain a better understanding of the duration and number of touch points in their sales cycle. This directly impacts the lookback window you set. It is also important to know what these are when comparing any data across platforms. For example, right now I'm going from a 90 day window for certain goals in AdWords because I know that while the majority convert on that goal in ~30 days, there's a long tail. Unfortunately, Marin restricts their window to 30 days, so any deeper digging will need to occur within Google Analytics or data from our data warehouse.
    - jamiequint 11y ago
      Would be great to learn more about solutions you've seen in the wild since it seems like you have a lot of experience on this topic. If you have time for a chat my email is in my profile (tried to find you but shostack is ambiguous :)
rob_lh 11y ago
Isn't that an argument for assessment on a campaign basis? I'm still wrapping my head around his calculations. It looked like he included the cost of the ads prior to the reporting window in his multi-touch model. That's a fair way of measuring the true cost of user acquisition, but if the ads you are paying for today won't pay dividends until you're out of the reporting window, it would skew the ROI. I'm not sure if this is the same thing as what you cited, as my argument is about the cost association during the reporting window as opposed to the future revenue association. I can't figure out a way to back those out in real-time (i.e. in enough time to make adjustments to the campaign) though.
- DLarsen 11y ago
  In long-running campaigns, I still typically want to be able to measure changes to performance over time. With a fluency in the ramifications of the attribution models and windows, I like to look at multiple attribution models (last-click and parabolic multi-touch) to draw conclusion about performance. My takeaway from the post was simply, "Be aware of how much attribution models influence metrics," and I think that's a critical point.
  - rob_lh 11y ago
    Indeed, that is the most salient point. I like your suggestion of looking at many attribution models to assess true performance. This seems especially relevant in larger organizations where there is an incentive to game the budgeting process - presenting just the best possible case is what marketing is all about but can lead to skewed perspectives of success.
jamiequint 11y ago
This is definitely true, you can think of that as Day 1 ROAS/ROI vs Day N ROAS/ROI. So when looking at your marketing spend you can see how it pays back over time since depending on how you attribute it could change forever (e.g. if you give some value of each conversion back to the original conversion source/sources).

cassieramen 11y ago

This is a great argument for having a data scientist or other statistician run your marketing department. Tech is really changing the way people think about marketing expertise. What counted 10 or even 5 years ago is quickly growing irrelevant.

dfragnito 11y ago

Not factoring in Cost of Sales is also a good way to lose money on Paid Marketing.

dfragnito 11y ago

Revenue per visit really? If a visit does not convert how does it have any value (yes I understand the math)? Why not convert Cost-Per-Visit to Cost-Per-Conversion (total spend / # conversions). I am going through this now with a client. The CPC gurus need to justify the CPC budget and surprise surprise the more cpc dollars spent the more they make. What perverse incentive system they have set up for themselves.

jamiequint 11y ago
If a visit doesn't convert it has $0 value. You can convert cost-per-visit to cost-per-conversion instead (the #s will come out the same) but its more complicated to do technically.
- dfragnito 11y ago
  (total spend / # conversions) is pretty easy to do
  - jamiequint 11y ago
    You can't do it that easily. If you just take total_spend/conversions the question that leads to is what spend do I measure over what timeframe and what conversions do I measure over what timeframe? For example, if I want to know cost-per-conversion for my Facebook traffic then which conversions should I consider in your equation above?
    - dfragnito 11y ago
      a time frame is assumed in my equation. Calculating conversions and ad spend over a time frame is easy if you have your analytics tracking set up properly.
      - jamiequint 11y ago
        You can't just add up marketing spend and conversions over the same period of time and divide them. That's not an accurate method of measurement. The conversions and the visits they lead to them are not guaranteed to happen over similar timeframes. If you calculate this the way you're suggesting you are making all sorts of implicit assumptions about variance in visit volume, marketing spend, conversion rates, etc.

swehner 11y ago

I couldn't quite follow what the problem is. I would define the revenue in the ROAS formula as revenue per time period, and the marketing spend also per time period Then I would qualify the revenue, as that part of the revenue that can be tied to the marketing activity (cookies, refer URL, etc) And then the ROAS ratio becomes an ordanary division, because both values are known; no need to infer them as in the rest of the article.

DLarsen 11y ago
The whole issue is how you tie revenue to marketing activity. In a case where you had multiple touch points before a conversion, it's a question of which channel gets credit and in what time frame.

shostack 11y ago

Respectfully, this is a submarine PR piece. From the last paragraph: >"It’s a lot of work to build out accurate multi-touch attribution, but there are third-party solutions out there that can simplify the process. My company Interstate makes a pretty good one in my opinion!" That said, the importance of the topic, cross-channel attribution, cannot be emphasized enough for any company doing any sort of serious digital marketing these days. A fantastic piece that goes much deeper would be Avinash Kaushik's article[1] on various modeling approaches. It is a great primer for both people new to the concept and approaches, as well as people with familiarity who are looking for new ideas. The broader challenges these days are shifting towards the cross-device dilemma as well as reliance on 3rd party cookies and browsers' and users' increasingly aggressive stance against those. -Cross-Device Dilemma- I visit initially on mobile and then convert on desktop. Even Google has challenges with cross-device accuracy here, and depending on your mobile usage, it could dramatically impact how you view your performance. There are a variety of technology solutions out there that attempt to solve this, but ultimately there is no great solution that doesn't make me truly question the privacy aspects of what it takes to get to accurate data here. If someone signs up for a trial and gives you a unique user ID, great, you can join that to the data from usage on other devices and understand that user's path to conversion. If they don't take an action that requires logging in, it is much murkier (not counting cookie onboarding services like Liveramp etc.). -3rd Party Cookies- Google and Facebook are excellently positioned because most browsers recognize their tags as 1st party. Many other networks and adtech platforms are boned here unless they can have their tags served as 1st party. I've seen variances of around 15-18% or higher in some cases between 1st and 3rd party tracking setups. While it is REALLY hard to control for all of the unknowns (no two web analytics platforms will ever give you the exact same data), my hunch is that 3rd party vs. 1st party cookies are playing a significant role here along with cookie deletion. Would love any other data folks can share on this. Beyond that, the author talks about static models, but does not address their real limitations. At the end of the day, these look at aggregate data, when the reality is attribution occurs at the user level. Every touch point has some value, and that value is different for each user. For example, seeing a display ad on one site does not necessarily have the same value as seeing it on a different placement within that same site. Static models are very helpful directionally, but dangerous when taken as religion. The future is in dynamic attribution models that constantly shift based on the data and whatever contributing factors you can feed in as supplementary data to the model. I'm super excited to see what Google starts doing with Adometry (check out the TV release they just had), and the other big players like VisualIQ and Convertro will likely have their technology trickle out through other platforms over time. FB's Atlas acquisition is also a big statement that they take this seriously. Net net, attribution is the hardest challenge in the marketing/advertising world today. Period. End of discussion. It is plagued with data that is messy as hell and incredibly nuanced based on the infinite variables of everyone's unique setups. Look through a variety of attribution models as different lenses to help inform directional decisions, but be cautious on saying "this is the model we will use across everything." It might help initially, but without constantly checking it against other models, you can miss shifts in your efforts. [1]http://www.kaushik.net/avinash/multi-channel-attribution-modeling-good-bad-ugly-models/ http://www.kaushik.net/avinash/multi-channel-attribution-mod...