4 ms·

We are built on top of the feeds from the affiliate networks - and yes, there are issues with some of the data :-) We have about 9 million products in our syst

by rahoulb 14y ago

We are built on top of the feeds from the affiliate networks - and yes, there are issues with some of the data :-) We have about 9 million products in our system (although not all are active) and we regularly inspect it and blacklist products (particularly adult products). The system also tracks which products attract clicks and more importantly convert well, and if they aren't performing, they are disabled. So crappy ads don't get shown, once the system has noticed that they are crappy.

thwarted 14y ago
Unfortunately, we never reached the scale where we could exercise product blacklisting, and even 50k products were difficult to vet manually. And I had written significant code to help seek and destroy affiliate feed data problems which helped a lot but still ate up a lot of time. The major data hygiene issue was that the affiliate networks tried to pigeon hole every product into the same schema. I think this was largely due to their major users being non-technical and their feed format being CSV based (so people could manipulate it in Excel)--this was the only time in my life that I wished for an XML set up, one that had different schemas for different products (this is why Amazon was attractive, they do this, but it's still not 100%. While Amazon has a large product category tree, a lot stuff still ends up being pigeon holed, especially if it's a specialty product). The worst part was the "extra" fields that the vendor could use for anything, some used them for keywords, others for deeper hierarchical categorization. I think my favorite was the "category" fields which were chevron delimited phrases within a single CSV field.
barkingllama 14y ago
Issues? I find that Commission Junction affiliates can't even get their links right all the time which is basically their entire business. Talk about a "shotgun" approach...

by rahoulb 14y ago

thwarted 14y ago

Unfortunately, we never reached the scale where we could exercise product blacklisting, and even 50k products were difficult to vet manually. And I had written significant code to help seek and destroy affiliate feed data problems which helped a lot but still ate up a lot of time. The major data hygiene issue was that the affiliate networks tried to pigeon hole every product into the same schema. I think this was largely due to their major users being non-technical and their feed format being CSV based (so people could manipulate it in Excel)--this was the only time in my life that I wished for an XML set up, one that had different schemas for different products (this is why Amazon was attractive, they do this, but it's still not 100%. While Amazon has a large product category tree, a lot stuff still ends up being pigeon holed, especially if it's a specialty product). The worst part was the "extra" fields that the vendor could use for anything, some used them for keywords, others for deeper hierarchical categorization. I think my favorite was the "category" fields which were chevron delimited phrases within a single CSV field.

barkingllama 14y ago

Issues? I find that Commission Junction affiliates can't even get their links right all the time which is basically their entire business. Talk about a "shotgun" approach...