10 ms·
Pokemon Sword and Shield Are Crashing Roku Devices
- eu 7y agoISP's routers too have issues with this: https://www.dslreports.com/forum/r32578490-Need-help-on-a-possible-software-bug-on-Altice-Fiber-gateway https://www.dslreports.com/forum/r32578490-Need-help-on-a-po...
- jzollinger0804 7y agoSounds like a pesky Rotom to me ;)
- zapzupnz 7y agoUnrelated: Sword and Shield still have the Rotomdex, except it's a smartphone rather than a 3DS. In French, Rotom is called Motisma; the Rotomdex is called a Motismart (as in, smartphone). Pretty neat, I thought.
- grenoire 7y agoPokémon localisation for European languages (and English too) has always been full of puns and jokes like that; they really put a lot of care into them.
- inputError 7y agoThis has to be one of the most random issues of all time.
- rolltiide 7y agoI've seen similar. Very curious about the fix and post-mortem though
- genera1 7y agoTBH legendary issues like mails failing over 500 miles [0] or recent reddit post about MRI disaing all iPhones in the hospital [1] take the cake [0] http://web.mit.edu/jemorris/humor/500-miles http://web.mit.edu/jemorris/humor/500-miles [1]https://www.reddit.com/r/sysadmin/comments/9mk2o7/mri_disabled_every_ios_device_in_facility/ https://www.reddit.com/r/sysadmin/comments/9mk2o7/mri_disabl...
- murph-almighty 7y agoYour first story, in turn, reminds me of this: https://www.reddit.com/r/talesfromtechsupport/comments/420oan/companywide_email_30000_employees_autoresponders/ https://www.reddit.com/r/talesfromtechsupport/comments/420oa...
- rgoulter 7y agoHere's a good list of debugging stories: https://github.com/danluu/debugging-stories https://github.com/danluu/debugging-stories
- cjbprime 7y agoAt One Laptop Per Child we found an issue where the mesh networking wifi packets crashed (actually caused a packet storm) Linksys routers, so just bringing an OLPC laptop somewhere would DoS wifi there. This kind of bug where one device's packets cause another device that it was never tested against to fail is not that uncommon. Of course, it's funny that a specific game is involved here. Perhaps there are other Switch games that do it too?
- knute 7y agoWhen the first-gen iPhone was released, they could break certain wireless networks built on Cisco hardware. https://hardware.slashdot.org/story/07/07/21/1212217/duke-wireless-problem-caused-by-cisco-not-iphone https://hardware.slashdot.org/story/07/07/21/1212217/duke-wi...
- joshmn 7y agoPrevious: https://news.ycombinator.com/item?id=21560166 https://news.ycombinator.com/item?id=21560166
- Giho 7y agoIsn't this an attack vector? Why would even a device respond to wireless signals in such way that they reboot? So now you can send annoying packets instead of jamming the network.
- oneepic 7y agoI imagine that was not the exact intent. Perhaps it was designed to respond to wireless signals, but something triggered a fault.
- Driky 7y agoThe title wrongfully is accusatory. The switch is very bad when it comes to network "cleanliness" with its propensity to broadcast for anything. But the fact that some roku devices are unable to handle proprietary network calls is on Roku not on Nintendo...
- mattigames 7y agoAllowing videogames to make network calls using arbitrary port numbers seem like poor engineering by Nintendo, videogames should have to use an API to do network calls regardless of what port the console uses to connect to such networks (internet or other network).
- freehunter 7y agoPeer to peer protocols use arbitrary ports in almost every implementation. Edit to add: and Pokémon games have used peer to peer protocols before on different systems, so this use isn’t new to Pokémon or Nintendo systems.
- kjs3 7y agoSays who?
- mattigames 7y agoNobody you consider important.
- kjs3 7y agoWell, yeah...anyone who doesn't bother or isn't capable of explaining why they make absolutist pronouncements as to how other people should do thing aren't particularly important to anyone.
- klodolph 7y agoWhat is wrong with the Switch broadcasting?
- lilyball 7y ago> Most people familiar with the situation are attributing the problem to a Roku, in that it's the Roku devices that are malfunctioning and not the Nintendo Switch. I would certainly hope that "entering an endless boot loop" is considered a malfunction no matter what network traffic is occurring.
- wizzwizz4 7y agoCounterexample: 1. Hello! 2. Three-way handshake. 3. Authentication. 4. Please reboot. GOTO 1
- StuffedParrot 7y agoIf you can auth without intending to that seems like a bug to me.
- neohaven 7y agoFrom GP: > no matter what network traffic is occurring. The chain described there is "possible network traffic", and would be a valid type of network traffic, "while-meaning-to" if we so please, but there is no way to distinguish that and it is thus meaningless to code. A device that would never reboot, no matter what network traffic is occurring, would never allow you to reboot it remotely... SSH is network traffic. Your SSH authentication also is. The point wasn't about the Roku, it was a pointed reply to that particular point, I think. :)
- benchaney 7y agoI really wish we could stamp out this excessive pedantry. It is completely clear what OP meant. There is no need to pretend we are robots.
- Agentlien 7y agoWe may not be robots, but our code is still run by machines which are incredibly pedantic. Any imprecissions which require human intuition to untangle won't help solve practical problems.
- umvi 7y agoI bet there was someone on the Pokemon team that wanted to exercise esoteric, obscure networking features just because they read about it in some spec and wanted to try it out. It's like the person on your team that insists on shoehorning language features into the application from the dustiest corners of the language instead of sticking to the tried-and-true idioms.
- thirtyseven 7y agoUsing a broadcast address for service discovery is a perfectly normal use case, the Roku should be able to handle it.
- sgspace 7y agoThese kind of features have been in pokemon games for over a decade. All the DS pokemon games search for other local Nintendo DSs’
- Qwertystop 7y agoAnd the later ones (and all the 3DS ones) have the option to do that perpetually in the background, which would probably be why this issue comes up here instead of on other local multisystem Switch games that only do it at specific times.
- Qwertystop 7y agoThey've always been pretty conservative as far as game design goes; somehow I doubt they'd be pushing any boundaries on the backend.
- AnIdiotOnTheNet 7y ago> They've always been pretty conservative as far as game design goes Uh, considering that Nintendo became popular by breaking the arcade game mold with Donkey Kong, and continued that trend with Super Mario Bros, The Legend of Zelda, Pokemon, etc. then went on to create things like Wii Music, not to mention the Wiimotes, building a tablet into the WiiU, Amiibos, Labo... Conservative isn't the word I'd associate Nintendo with.
- PhasmaFelis 7y agoI'm not clear if this happens only if the Switch is on the same network as the Roku, or if it can affect Rokus in range even on different networks. The first seems most plausible, but the article seems to imply the second in a couple of places.
- Qwertystop 7y agoThe Switch is capable of communicating both over an existing network through a wired or wireless router, and through wireless peer-to-peer communications with other nearby Switches. Previous Pokemon games (and other games on the Nintendo 3DS) have used local wireless as a constant background feature in this manner, but to my knowledge, most Switch games that use local wireless only do it at specific times (when playing multiplayer) instead of always-on in the background, so it doesn't seem unreasonable that only the Pokemon game causes this problem, or that it would be heard by nearby devices on other networks. Other games would break it only when someone is actually playing local multi-system multiplayer Smash Bros or something, which is going to be rarer and you're probably less likely to be trying to use the Roku at the same time. The only thing I would find implausible about breaking Rokus by proximity would be why the Rokus are picking up the communication from a Switch, when presumably they didn't break whenever someone used a 3DS within ten yards (or we'd have heard these complaints years ago). But that could easily be down to changes in protocol by Nintendo between the two systems, such that one is ignored and the other is mistaken for relevant data. I don't have a Roku or an easy way to inspect nearby wifi packets, so I can't easily test this theory.
- pcroh 7y agoMore like "Roku Devices Are Letting Themselves Crash When They Receive Packets They Don't Understand"
- rtkwe 7y agoAccording to a reddit post it's actually that the Roku understands perfectly but the packet happens to exactly match the request for a reboot that Roku uses for managing P2P updates: https://www.reddit.com/r/ProgrammerHumor/comments/dy0p86/how_do_you_even_begin_to_debug_something_like_this/f7xzay7/ https://www.reddit.com/r/ProgrammerHumor/comments/dy0p86/how...
- deleted 7y ago[deleted]
- jsgo 7y agoRoku has released an update to hopefully fix this. https://www.theverge.com/2019/11/18/20970743/roku-pokemon-sword-and-shield-crashing-boot-loop-fix https://www.theverge.com/2019/11/18/20970743/roku-pokemon-sw...
- tantalor 7y agoThis device complies with part 15 of the FCC Rules. Operation is subject to the following two conditions: (1) This device may not cause harmful interference, and (2) this device must accept any interference received, including interference that may cause undesired operation.
- Qwertystop 7y agoSo, is the Switch (in certain circumstances caused primarily/only by this game) breaking 1, or is the Roku breaking 2? Neither seem to have broken either until this particular interaction.
- rtkwe 7y agoNeither the FCC literally means radio interference not random protocol clashes [0] like seems to be happening here. That rule basically means 'don't pollute the radio spectrum' and 'you have to accept the spectrum you receive'. [0] https://www.reddit.com/r/ProgrammerHumor/comments/dy0p86/how_do_you_even_begin_to_debug_something_like_this/f7xzay7/ https://www.reddit.com/r/ProgrammerHumor/comments/dy0p86/how...
- rsynnott 7y agoThat's not what the FCC means by interference: https://en.wikipedia.org/wiki/Title_47_CFR_Part_15 https://en.wikipedia.org/wiki/Title_47_CFR_Part_15
- bcaa7f3a8bbc 7y agoYes, this is not what FCC means by interference, and no Part 15 regulation is violated here. But there's no need to downvote. If we review the terms of Part 15, > (1) This device may not cause harmful interference > (2) this device must accept any interference received, including interference that may cause undesired operation. You'll see it's actually a general, universal principle of good communication, and applies to many cases other than RF spectrum. Internet pioneer Jon Postel once said TCP implementations should follow a general principle of robustness: > (1) Be conservative in what you send > (2) Be liberal in what you accept. It's almost identical to the principles of Part 15. A few years ago, when the Mirai botnet launched its massive DDoS attack, a HN user used Part 15 as an analogy of the future directions of IoT's security model. And in this example of protocol conflict between Pokemon and Roku, it also applies.
- wil421 7y agoPokemon is using a local WiFi feature I didn’t know existed until I started playing. The WiFi symbol on the switch has L on it. Meaning it’s searching for users connected to WiFi but it’s not connected to the internet. I might be able to sniff the packets later today. To me it looks like a Roku issue. A device shouldn’t go into a reboot loop if it encounters something unexpected. I also heard the switch uses its own version of Bluetooth to add 8 players but since its nonstandard they won’t let you connect any BT devices to it.
- rtkwe 7y agoWhat about going into a reboot loop if you see a signed request to reboot? [0] It's unverified but this story makes sense in a cosmic coincidence sort of way. [0] https://www.reddit.com/r/ProgrammerHumor/comments/dy0p86/how_do_you_even_begin_to_debug_something_like_this/f7xzay7/ https://www.reddit.com/r/ProgrammerHumor/comments/dy0p86/how...
- scarejunba 7y agoWhy sign the reboot code if you're always sending the same code? Like, the signing is useless. The signed code is now the new reboot code and it's effectively unsigned.
- rtkwe 7y agoTrue but omitting any challenge response means the master node can just push the info out without having to communicate with any of the child nodes. It's true it's basically just a reboot code but a cryptographically random number to a randomly chosen port is honestly a fairly safe choice Roku just got cosmically unlucky if the theory is correct.
- DSMan195276 7y agoI agree with your point, but there's just no way that's true. A bare minimum MD5 signature is 16-bytes (Obviously not secure, but this isn't safe from replay attacks anyway), with a more acceptable SHA256 obviously being 32-bytes. Any type of signature should be sufficiently random that Nintendo is never going to accidentally match them, so that means the odds of matching a random 64-bit integer is already 16 quintillion to one, and just for MD5 we're talking 16 quintillino times 16 quintillion - that's beyond hopeless. It's the same thing as calling heads or tails correctly either 128 or 256 times in a row. If I had to guess it's probably something silly like not actually checking the signature for validity, or (more likely, IMO) incorrectly checking the length of the packet and getting a buffer overflow/underflow that eventually crashes the Roku.
- rahuldottech 7y agoExplanation, from Reddit: > Pokémon sends a network discovery packet to each device on port 26037. Roku also listen on that port for LAN based updates so that multiple devices on the same network can update each other. It was an obvious decision. Saved Roku around a quarter million dollars in CDN traffic costs. Roku is popular in the commercial space where it’s often used as a media source to control sometimes 100s of TVs on the same network. It just so happens that Pokémon’s network discovery packet shares the exact same bytes as Roku’s signed bytecode to reboot. > The odds are astronomically low. We could have wound up with an alien planet full of Justin Timberlake clones, but the universe decided this was our colossal fluke.
- emiliobumachar 7y ago> It just so happens that Pokémon’s network discovery packet shares the exact same bytes as Roku’s signed bytecode to reboot. > The odds are astronomically low. Interesting. Any chance anyone here would know exactly how many bytes we're talking about?
- fpgaminer 7y agoAssuming by "signed" they mean cryptographically signed, and they're not doing something dumb, then the signature itself would be at least 16 bytes. It'd be ~impossible for a collision to ever occur by accident. So either "signed" means something different in this context or GameFreak replicated Roku's reboot packet for some reason.
- dragontamer 7y ago> then the signature itself would be at least 16 bytes. Indeed, and 16-bytes is SHA1 or MD5, both considered insecure at this point. 32-bytes (SHA256) seems more likely to me. So I'm going to bet that they aren't "signing" the packet.
- jlgaddis 7y agoAES-128 is also 16 bytes (128 bits), which is still (and likely will be for many, many years) completely secure, for all intents and purposes. (SHA1 is 20 bytes, BTW.)
- badrabbit 7y agoAnd this boys and girls is why they don't let us run vuln scans on production during work hours (or unplanned). I've worked at at least two companies where a scan caused some important device to crash and have business impact because the scan packets were unusual. They will fire you if you mistakenly ran the scan against a prod asset.
- yjftsjthsd-h 7y agoI understand that it should be structured to minimize risk, but if your prod stack can be taken down by a scan... you really do want to find that out in a controlled manner and not because a script kiddie decided to do the same test.
- wil421 7y agoYou don’t know what you don’t know. There was a UNIX bug that caused the system to crash or execute code just by being fingered.[1] > Connect to your fingerd daemon and type more than 528 (= 512 + 16) characters (any will do). If your daemon crashes or terminates the connection with no data sent back, you probably have the vulnerability. [1] http://seclab.cs.ucdavis.edu/projects/vulnerabilities/doves/1.html http://seclab.cs.ucdavis.edu/projects/vulnerabilities/doves/...