10 ms·
Devirtualizing nike.com's bot protection
- tsujamin 4y agoCool write up, reminds me of a .net app I had to RE a few years ago which used a custom bytecode/VM for sensitive functions, such as some string crypt/obfuscate operations which I was trying to grok. Giant pain in the ass. …until I saw that they re-implemented the same operations in some schema migration code without adding the magic ”obfuscate-me” annotations
- BoorishBears 4y agoI ran into the same thing reverse engineering a piece of exercise equipment The routines for communicating with sensors were heavily obfuscated in the main application... but the factory sensor test application had been left installed and was completely clear.
- justinjlynn 4y agoone of their (more senior) engineers: "OoPsiE. WhAt A hUgE MisTakE. wHaT eVeR shAll I Do."
- tw98521358 4y agoObfuscation instructions were never issued to the test team by senior manager lol
- Animats 4y agoWhy does Nike bother? Their home page just has some pictures of stuff they sell. It doesn't do much. It leads to apparel ordering pages. Those have a column down the left for selecting some options, and pictures of products. Just like everybody else that sells clothing on line.
- trollied 4y agoPeople seem to lose their minds when new sneakers become available, which instantly have a huge resale value. So the anti-bot measures are probably to try and counteract sniping of said sneakers.
- Animats 4y agoOh, right. They have an artificial scarcity thing going.
- benj111 4y agoSo Nike arbitrarily limit the availability of a shoe. People for 'reasons' want to buy the shoe because of limited availability. People are willing to pay more than what Nike is charging. 3rd parties come in to capture that wasted value. Nike wants to stop those 3rd parties for 'reasons'. There seems to be a lot wrong with this situation. A group using free market, capitalist tools in a free market capitalist country seems the least wrong part.
- gnomewascool 4y ago> 3rd parties come in to capture that wasted value. The value isn't wasted — it's consumer surplus.
- wizzwizz4 4y ago> 3rd parties come in to capture that wasted value. That value would otherwise have gone to the people. If all value is captured in this way, the marginal benefit of making a purchase becomes very, very low, and the stuff we actually care about (standards of living, happiness, liberty) become decoupled with economic metrics. This is one of the situations where lotteries are fairer than auctions.
- benj111 4y agoOk. "3rd parties come in to capture the value that Nike otherwise left on the table" I disagree that all value would be gone though. People value things differently, the 'scalpers' are a mechanism to get the goods to those who value it most highly. I personally don't value these trainers for what Nike is asking, so I don't buy them. Should Nike be required to sell them to me for what I value them at? Or should people be able to set their own price, and others buy if that price is worth it to them? If you actually cared about happiness, liberty, etc, I don't see how a pair of trainers, expensive of otherwise is going to change that, but if it does become an issue, just don't by the sodding trainers then! I don't see why a lottery is 'fairer'. the shortage is entirely synthetic, created by Nike. I'm not big on adding morality into business transactions, but all this stems directly from Nike's decision to create scarcity. The 'fairest' thing would be for Nike to satisfy demand. 'scalpers' are a just symptom of that. I don't get why 'scalpers' are only deemed to be a problem when the scarcity in question is synthetic. If there's a corn shortage are farmers scalping? Are oil traders scalping when the price goes up? What about when the price of $stock goes up? Are the sellers scalpers? This is how free markets work. You could at least make a moral case for the price of corn being socially bad, you can't make that case for trainers or gig tickets or whatever. The other obvious solution is for another company to come in and solve the supply constraint that nike created. Although that too is also apparently wrong. If I bought some 'Niko' trainers from the local market, who is being harmed here? I'm not being misled, Nike aren't losing sales, so why?
- ErneX 4y agoThey have these type of sales frequently: https://www.nike.com/help/a/nike-snkrs-drawings https://www.nike.com/help/a/nike-snkrs-drawings
- netrus 4y agoAfter seeing the URL, I was a bit disappointed they do not let sneaker heads make drawings of shoes as a CAPTCHA. Don't require any artistic skill, just some creative effort - worst case, the bot makers will make some interesting contributions to AI art.
- wewxjfq 4y agoLike trollied said, there are people who use bots to buy the latest shoes. You can read more about it if you search for "sneaker bots" in the archive.
- paulmd 4y agothe "scalper problem" is strikingly similar to many cryptographic problems - the "sybil problem" in token distribution ("distribute 25 bitcoin every 10 minutes to a random participant") is obviously similar to distributing scarce items which are scalpable (sold below market clearance price). "We'll just have a free-for-all (random distribution)"? Oh ok I'll register 2 million network participants (or IPs, or email addresses, or browser sessions, or privacy CC numbers, etc). Without some kind of outside oracle proving user uniqueness (and many many oracles that people think of, like CC number or billing address, are not actually unique!) it is extremely extremely difficult to construct this system. NVIDIA distributing some 4090s via GeForce Experience is another interesting example of a hardware oracle, the telemetry is a fairly strong signal of hardware authenticity (as proven by Overwatch's fairly effective hardware bans).
- Gigachad 4y agoI once met a guy who had a stack of 50 sim cards used for bot buying sneakers and playstations. Guy didn't work in tech but had learned javascript and selenium for scalping. You can also ship a lot of things to the same address by just altering it slightly like setting the unit/number to 1B or something when there is no 1B, the delivery people will get it to the right place but the unique check will think it's a different address. He had apparently made enough money doing this to pull himself out of poverty and get a decent life going.
- Jhsto 4y agoThere's a big resale market, sometimes shoes that cost $250 go for $10K resale. I used to develop bots for this as well and documented some of the experience: https://juuso.dev/northern-nike-nabob.html https://juuso.dev/northern-nike-nabob.html
- metadat 4y agoI strongly dislike that you engaged in the shoe scalping arbitrage business, but at the same time really appreciate you sharing this information. Excellent writing covering a fascinating adventure! Question: With the Twitter RSVPs, do you think Nike implemented automation to automatically give you the code when you responded with the correct code in a fraction of a second? If so it implies that on some level Nike was fine what you were doing.
- Jhsto 4y agoEventually Nike made changes into how they were giving out the codes, so they tried at least. E.g., 0 follower accounts could not gain a code. We switched to using accounts of my contact and his friends to solve this issue. I think the idea of adding a timer for artificial latency was also discussed, but I can't recall if we ever implemented it. In general we thought that Nike was complicit to an extent. Especially with the on-site checkout and mining of product IDs, they could have cancelled orders done before they shared the link in public. I think on their end, it was a balance of guaranteeing a low supply while ensuring that normal customers did not feel like they were being screwed _all the time_. Yet, even nowadays with the SNKRS app some continue to call it a rigged ballot. In hindsight, Nike is still positioned somehow uniquely in the sneaker community: it's not guaranteed that money alone will get you a pair. Compare this to luxury brands like Balenciaga, which retail sneakers with a price tag of $500-1000. With the latter everyone knows you have handed out a chunk of cash, but it's not the same thing as the story that you could have waited in line or woke up in the morning to enter a raffle, or been smart enough and early on to pay reseller prices to get a pair before stock ran out which you knew would become a coveted pair over time.
- hrpnk 4y agoIn the past, people needed to queue to get their hands on the fresh stuff and there was a clerk that cloud verify your identity. Of course folks could send in their friends or use fake documents, yet the degree of the procedure was limited to humans in their flesh. e-commerce sales open the gates to the shop at the same time for everyone. This incentivises massive parallelization of the purchases across accounts, IPs, and identities. Already back in 2017 [1] it's become known to the industry that bots are sniping the purchases faster than humans. Now, we're at a stage where retailers are adjusting their T&Cs to address the resell market [2]. StockX was valued at 2.8B in Dec 2020 [3] and they have to get stock somewhere, right? The bot industry [4] also developed from simple scripts to SaaS offerings with multi-thousand $ subscription prices just to get access to a bot that's proven to work. The user has to provide a list of proxies that will be used to execute the scripting as these are not provided out of the box (and are hard to get). There are numerous guides on YT and advertising on Twitter that show how to use the bots and how they've been used for purchases. And of course, scarcity is not just for streetwear, but for any type of highly sought for products [5]. [1] https://www.esquire.com/style/mens-fashion/news/a55301/using-bots-to-buy-supreme/ https://www.esquire.com/style/mens-fashion/news/a55301/using... [2] https://www.wsj.com/articles/nike-moves-to-crimp-resellers-and-might-cancel-orders-made-using-bots-11665506775 https://www.wsj.com/articles/nike-moves-to-crimp-resellers-a... [3] https://techcrunch.com/2020/12/16/stockx-raises-275m-series-e-valuing-the-retailer-at-2-8b https://techcrunch.com/2020/12/16/stockx-raises-275m-series-... [4] https://www.complex.com/sneakers/how-to-use-sneaker-bots/ https://www.complex.com/sneakers/how-to-use-sneaker-bots/ [5] https://www.wsj.com/articles/desperate-parents-turn-to-shopping-bots-to-hunt-for-hottest-christmas-gifts-11637417633 https://www.wsj.com/articles/desperate-parents-turn-to-shopp...
- mattlondon 4y agoAhhh this might explain why their site is so slow even on modern hardware. Several times I have given up trying to buy new running shoes directly from their site just because it is so frustrating to use.
- a2tech 4y agoBlame scalpers. I work on a small time website that happens to sell limited release shoes occasionally. Its almost impossible to keep the site functional during these releases because the buying bots are both aggressive and smart. They HAMMER sites trying to get at those shoes. Its nuts.
- 69463864883 4y agoWhy not raise the price to whatever the scalpers charge or sell them on an auction instead of selling them so far below market value? If you for some reason don't want to make more money, you can donate the difference to charity. You could also do things like giveaways to previous customers or make them prizes in competitions, etc.
- jacob019 4y agoThat will just raise the market price, scalpers will keep scalping and overall sales will decline.
- brookst 4y agoWait, what? Isn’t the price set by what people are willing to pay? If scalpers can get $1000, that means people will pay $1000; if the scalpers could get $2000,’that’s what they’d charge. So if Nike raises their prices to $1000, I don’t see how that increases market price.
- pixl97 4y agoScalpers destroy markets by injecting excess capital in the marker and creating a false price floor. The issue with scalpers is they quite often get left holding the bag, and/or hold the product for a long period of time lessening the actual demand or excitement about the product in the first place.
- jonatron 4y agoIf you assume that every bot protection script will look at navigator.webdriver, you can just set a breakpoint before the script runs, and set a debugger statement on the property to land somewhere useful: Object.defineProperty(navigator, 'webdriver', { get: () => {debugger; return false}, });
- cactusplant7374 4y agoI have used this chrome driver project. The author probably can create something more lightweight with his knowledge. For my purposes I couldn’t find a Cloudflare bypass script that worked so I had to use this instead. https://github.com/ultrafunkamsterdam/undetected-chromedriver https://github.com/ultrafunkamsterdam/undetected-chromedrive...
- jonatron 4y agoUsing defineProperty can be detected with navigator.hasOwnProperty("webdriver"), which is why I suggested using it for reverse engineering rather than avoiding detection.
- gernb 4y ago> One technique to protect against these attacks is browser fingerprinting. This method works by collecting data about a user's browser, which is then used to create a unique fingerprint for differentiating between genuine users and bots. This seems like it can't work. 50% of users are browsing from an iPhone. Every iPhone of the same model has the same fingerprint except for time of day and language preference. So for every time zone there are literally hundreds of thousands of devices that will have the same fingerprint.
- arkitaip 4y agoYour browser's fingerprint is much more unique than you think: https://amiunique.org/ https://amiunique.org/
- konha 4y agoI just tested my iPhone with the EFF tool [0]. Seems unique-ish. I use an adblocker and safari for browsing with default settings. > Within our dataset of several hundred thousand visitors tested in the past 45 days, only one in 32014.4 browsers have the same fingerprint as yours. [0] https://coveryourtracks.eff.org/ https://coveryourtracks.eff.org/
- ntauthority 4y agoFor me, running the iOS 16.3 developer beta seems to have been what made my iPhone 'unique' among the ones tested on that site.
- saagarjha 4y agoEFF's site is wrong.
- 411111111111111 4y agoWhat makes you confident of that?
- saagarjha 4y ago
- arianvanp 4y agoI wonder if relying on cryptography instead of obfuscation would be a better solution to fight shoe-buying bots? E.g. couldn't proof of work be used in order to complete the purchase process? If it works for bitcoin on large scale it ought to work for a purchase form in small scale without all the distributed systems gotchas. E.g. Nike could simply increase the difficulty whenever they see fit to tweak the amount of bots succesfully purchasing things.
- grogenaut 4y agoNike already does this. Checkout kasada. https://www.kasada.io/sneaker-bots-evolution-ecosystem/ https://www.kasada.io/sneaker-bots-evolution-ecosystem/
- worewood 4y agoYoutube does use proof of work to protect against bots. Not really effective, youtube-dl for example just runs the scripts in order to generate the proofs required.
- saagarjha 4y agoThere's a hard tradeoff here between how long a user on a low-end device is willing to wait to purchase something and how quickly a bot with powerful hardware can go through your proof of work.
- arianvanp 4y ago
- StreamBright 4y agoI am not sure why can't you use other features of the traffic that such website gets. Isn't there any better way of finding features that are great signals that a request is a bot? We used to work on a project where the traffic was inserted into a graph like data structure and we connected the known bot traffic nodes and calculated the fraud score based on that. It worked very well.
- grogenaut 4y agoThat would happen server side and be invisible to you reading the js. The cam easily be doing this and we'd never know unless Nike told us, which they will not for obvious reasons. But a lot of people just buy the shoes.
- StreamBright 4y ago> That would happen server side Absolutely. This is the best part. You do not impact the client side, no complexity pushed down to all of the clients, etc.
- mawise 4y agoDoes anyone else notice that those long tails of the 'f' glyphs make an article really hard to read? My brain stalls out noticing every single 'f'.
- HPsquared 4y agoAgreed, it's too eye-catching.
- Waterluvian 4y agoI just see functions. f(ingerprint)
- deleted 4y ago[deleted]
- spookie 4y agoAgreed. It's interesting to say the least... Removing the top pixels from characters greatly reduces readability, which is not as significant if you remove the bottom ones. Then why does this happen? Any ideas?
- irrational 4y agoWe don’t really read by looking at individual letters. At least, once we become proficient readers. Instead we read words as shapes. If the shape is too far off from the pattern stored in our head, then we have to stop and actually look at the letters to read it.
- deleted 4y ago[deleted]
- crazygringo 4y agoI came here to say exactly the same. Usually I can enjoy idiosyncratic fonts, but it's driving me crazy trying to read this. It's the form of "f" used in italics, only un-italicized. I think part of my brain keeps thinking it's the italic "f" in math notation "f(x)" and then gets confused to find it as part of a regular non-italic word. Or it might be the fact that our brain relies heavily on word shape for fast reading, which is heavily dependent on the presence or absence of ascenders and descenders in lowercase, so adding a descender on a letter than shouldn't have one totally breaks our natural pattern recognition. In any case, I've never come across it in any typeface before in my whole life. Now I'm starting to see why. :)
- paulpauper 4y agoWhy not just do it server side?
- deleted 4y ago[deleted]
- dns_snek 4y agoThere's not much you can "do" purely server-side that isn't easily bypassed by bots. That's how the arms race kicked off. websites rate limit IPs -> bot developers start using proxies websites block datacenter proxies -> bots use residential proxies websites add captchas -> bots start solving them websites inspect nuanced details of TLS fingerprints and header ordering -> bots start faking them websites add browser fingerprinting -> bots reverse engineer it and start faking it And so on. One thing these websites could do is place a lower limit on the total checkout time. Anyone who completes their checkout in less than 2-15 seconds (depending on the website) is quite obviously a bot.
- tw98521358 4y agoWhich they never do and even if they did they would get botted in using more volume. Only viable method I knew was sms verification, but that didn’t seem to be implemented correctly by anyone.
- paulirish 4y agoThere are a couple DevTools features that make this kind of reverse-engineering much easier. Oftentimes, the obfuscation relies on turning strings into functions. You can watch all this happen with: `monitor(eval); monitor(Function)` That'll log out the arguments passed to eval and the Function constructor. https://imgur.com/hMbKQZA https://imgur.com/hMbKQZA You can also use `debug()` to breakpoint on invocation. (Yes, this works with native fns as well as user-land fns) Then, comparing memory snapshots can help illuminate the work that happened. Take a snapshot of about:blank, then another after evaluating the ips.js payload. Select comparison. Then look through the results, especially (string). This appears to be the same "VM's strings" called out in the post: https://imgur.com/2v58VM5 https://imgur.com/2v58VM5 Can also see heavy use of WebGL and wasm based on allocations.
- nullpt_rs 4y agoLove these suggestions. Definitely makes dynamic analysis heaps easier. A lot of our blog posts tend to focus on static analysis but it may be worth mentioning these techniques going forward.
- umasi 4y agoHey! My friend linked me this comment. This is super dope, I used to RE more dynamically and knowing about monitor() and debug() would have helped me a lot haha. Also the memory snapshots, I've used that to try to debug memory leaks in the past but never thought about using it to RE. Super cool that you can also get the string that way. You can definitely get the same results a lot faster via dynamic RE for most obfuscation methods, but what attracted me to Kasada was the challenge of writing a disassembler from a static RE perspective. I also don't think dynamic RE methods are super easy to apply here once you get into the VM logic though, as I've discovered while working on part 2. If you want to chat with me about this more my socials are linked in the post, or you can message me on discord (umasi#3301) :)
- paulirish 4y agoMost advanced javascript developers don't know about these features, so no worries at all. :) (We're chatting now)
- wizwit999 4y agoI wonder, why doesnt Nike just add ID verification, is the friction too much? I mean these people queue in lines so I would think not, and the new ID verification apps are pretty seamless (upload a pic of self and ID).
- EthicalSimilar 4y agoA lot of people buying these sneakers will be quite young and/or not necessarily have ID.
- tw98521358 4y agoPeople would generate ids or use stolen ones. A lot of foreign players in the sneaker game don’t care for us laws