5 ms·

I have been thinking about that exact service for a while. One big issue is that you need the reverse engineer the pricing models of retailers, i.e. mapping out

by baxtr 9y ago

I have been thinking about that exact service for a while. One big issue is that you need the reverse engineer the pricing models of retailers, i.e. mapping out a high-dimensional space. For that you need a lot of data from the retailers. They could probably ban you quickly if you start pulling price quotes en masse. Then you need to invest a lot in cloaking techniques etc.

edem 9y ago
We can have TensorFlow do it...one model to rule them all...there will be neural networks fitting retailers, then there will be adversarial networks deployed by retailers, then NCMs (network counter measure)... the price will be high, since more and more will be consumed by the networks, then as an emergent behavior Skynet will rise and it will launch the missiles...
- baxtr 9y ago
  What about blockchain technology? That would sure help, too...
  - alkonaut 9y ago
    Once the deep learning blockchains are on quantum computers this will all be easy
hobofan 9y ago
You can go an uncloaked route and offer customers that buy the product with an usual profile small cashbacks if they provide you with their data.
crdb 9y ago
You don't. At least we didn't. Here's the real problem: 1. modern shopping sites are built haphazardly without structure; 2. a typical page load time is 10-20 seconds for many (most?) shops. 1. matters because you cannot build a simple crawler. This is because sometimes, the brand is in the title, the description is in the brand, and the gender is in the quantity, which itself is in the photo as a pixellated image... You must load the entire page and figure out where the attributes are in a smart way. You can do this today with headless chrome, in the old days with Selenium, etc. ... which brings us to 2. A mainstream, regionally-dominating shop, particularly any that adopt the marketplace model (Amazon, Taobao, etc.) may have as many as 20 million products. Given 2., good luck refreshing this price list in a sane time, even if you crack the attribute extraction. If by some miraculous feat you solve these issues you now have to product match an inventory of billions of listings, most of which are misspelt, missing attributes (brand, etc.), and use photos that even a human could not match by sight. Think how hard it is to find exactly what you want on eBay... The only way is to get the data directly from the retailers. Assuming they will give it to you, you end up with the same problem of feed poisoning that you have with marketing.
- baxtr 9y ago
  Oh wow, so it’s far worse than I thought. So the only way is to best-guess pricing models?
- baxtr 9y ago
  Oh one thing you could do: Create a huge community of users who share their purchase data and other data.and then build the model through that died so if you want to participate and save you need to be part of the panel
- zild3d 9y ago
  > 2. a typical page load time is 10-20 seconds for many (most?) shops. That sounds very exaggerated, are you quoting this range from actual data? For a sanity check the first two shops that came to mind loaded in <3 seconds (guitarcenter.com, nike.com)
- dclowd9901 9y ago
  Funny, I had a lot of overlap in this kind of smart data quantizing when I was trying to build a venue page crawler that would intelligently mete out live music event dates, times and locations. I realized about 20% into the project I'd need some form of machine learning and then bailed.
digitalengineer 9y ago
You’re over-thinking it, I think. All you need to do is provide a service that makes the E-commerce website think you’re not a big spender: geo: live in a poorer neighborhood, and age/gender/preferences must be the most optimal for you not them. Each experience (as they call it was pre-defined and pre-build).
- TheCoelacanth 9y ago
  Except it's not necessarily that simple. They are also probably taking into account your ability to seek out other options. Researchers found that black women were getting cars sold to them at a much higher markup than white men. I doubt that is because they were getting profiled as wealthier.
- gruez 9y ago
  how are you going to hide your shipping address? by using reshipping services?
  - maksimum 9y ago
    You only provide the shipping address once you've been given a price quote.

by baxtr 9y ago

edem 9y ago

We can have TensorFlow do it...one model to rule them all...there will be neural networks fitting retailers, then there will be adversarial networks deployed by retailers, then NCMs (network counter measure)... the price will be high, since more and more will be consumed by the networks, then as an emergent behavior Skynet will rise and it will launch the missiles...

baxtr 9y ago
What about blockchain technology? That would sure help, too...
- alkonaut 9y ago
  Once the deep learning blockchains are on quantum computers this will all be easy

hobofan 9y ago

You can go an uncloaked route and offer customers that buy the product with an usual profile small cashbacks if they provide you with their data.

crdb 9y ago

You don't. At least we didn't. Here's the real problem: 1. modern shopping sites are built haphazardly without structure; 2. a typical page load time is 10-20 seconds for many (most?) shops. 1. matters because you cannot build a simple crawler. This is because sometimes, the brand is in the title, the description is in the brand, and the gender is in the quantity, which itself is in the photo as a pixellated image... You must load the entire page and figure out where the attributes are in a smart way. You can do this today with headless chrome, in the old days with Selenium, etc. ... which brings us to 2. A mainstream, regionally-dominating shop, particularly any that adopt the marketplace model (Amazon, Taobao, etc.) may have as many as 20 million products. Given 2., good luck refreshing this price list in a sane time, even if you crack the attribute extraction. If by some miraculous feat you solve these issues you now have to product match an inventory of billions of listings, most of which are misspelt, missing attributes (brand, etc.), and use photos that even a human could not match by sight. Think how hard it is to find exactly what you want on eBay... The only way is to get the data directly from the retailers. Assuming they will give it to you, you end up with the same problem of feed poisoning that you have with marketing.

baxtr 9y ago
Oh wow, so it’s far worse than I thought. So the only way is to best-guess pricing models?
baxtr 9y ago
Oh one thing you could do: Create a huge community of users who share their purchase data and other data.and then build the model through that died so if you want to participate and save you need to be part of the panel
zild3d 9y ago
> 2. a typical page load time is 10-20 seconds for many (most?) shops. That sounds very exaggerated, are you quoting this range from actual data? For a sanity check the first two shops that came to mind loaded in <3 seconds (guitarcenter.com, nike.com)
dclowd9901 9y ago
Funny, I had a lot of overlap in this kind of smart data quantizing when I was trying to build a venue page crawler that would intelligently mete out live music event dates, times and locations. I realized about 20% into the project I'd need some form of machine learning and then bailed.

digitalengineer 9y ago

You’re over-thinking it, I think. All you need to do is provide a service that makes the E-commerce website think you’re not a big spender: geo: live in a poorer neighborhood, and age/gender/preferences must be the most optimal for you not them. Each experience (as they call it was pre-defined and pre-build).

TheCoelacanth 9y ago
Except it's not necessarily that simple. They are also probably taking into account your ability to seek out other options. Researchers found that black women were getting cars sold to them at a much higher markup than white men. I doubt that is because they were getting profiled as wealthier.
gruez 9y ago
how are you going to hide your shipping address? by using reshipping services?
- maksimum 9y ago
  You only provide the shipping address once you've been given a price quote.