10 ms·

The greatest bug I never fixed (2010)

389 points by triskweline 12y ago

markbnj 12y ago
That's a great story. How would you have fixed it without having to find another channel to transmit over? Could the plugin sense the character's condition and delay until it abated?
- lmm 12y ago
  The transform looks fairly simple; just approach it the same way you do error-correction (i.e. reed-solomon), find vectors of characters that are far apart in phase space and use them rather than sending messages directly.
  - masklinn 12y ago
    ECC's way overkill, just use a CRC and cancel the exchange if users are drunk.
- StefanKarpinski 12y ago
  Error correcting codes?
- yuliyp 12y ago
  Convert to hex, replace c with g. That avoids the ...hic and s->sh transforms
- Brandon0 12y ago
  I would have either just trimmed the ... hic from the received text (if there was only one string causing problems), or maybe even better would be to use control codes to signal the beginning and end of the data.
  - ryeguy 12y ago
    it looks like it actually transforms the message too, not just adds "..hic"
- Serow225 12y ago
  For detection, have the receiver echo back and check that sent == received.
  - netrus 12y ago
    But what if both are drunk?!
    - ajanuary 12y ago
      If the drunk transformation is 's' -> 'sh' Sender sends 's' Receiver receives 'sh' Receiver sends back 'sh' Sender receives 'shh', compares it to the 's' it thought it sent, and correctly deduces something went wrong in the communication. It is overzealous in that it would fail if the sender wasn't drunk but the receiver was.
  - wingerlang 12y ago
    But if the reciever echoes "sho" (drunken "so") that would just be "sho" again wouldn't it?
    - ajanuary 12y ago
      Server sends 'so' Receiver receives 'sho' Receiver sends back 'sho' Sender receives 'sho', compares it to the 'so' it thought it sent, and correctly deduces something went wrong in the communication.
- drzaiusapelord 12y ago
  Send a known-good line to test for it. "FriendnetStart: Hello, I am speaking normally and am not drunk," as the first message. If the application sees anything but that as the first message it should assume drunk and retry in an hour. Or implement a checksum at the end of every line. I imagine he also had issues with the profanity filter as well. Some usernames may set it off if they have a string of characters in them like username "Breshith," which would set off the filter for the text "shit."
  - tgarv 12y ago
    The character could possibly become drunk immediately after sending the "I am not drunk" message, so you'd need to check if the character was still not drunk after sending the FriendNet messages. But assuming that drunkenness lasts much longer than it takes to send the FriendNet messages, you could be sure that if the character was not drunk before and after, then the messages in between were also valid.
    - masklinn 12y ago
      > so you'd need to check if the character was still not drunk after sending the FriendNet messages Just send a checksum with the message.
      - leorocky 12y ago
        Yes this would work, even though the message could get garbled including the checksum, you make the first set of characters be a checksum for the rest. If the checksum or message became compromised it would fail validation. This is the best solution so far. Send it with every message.
        MasterScrat 12y ago
        Or use error-correcting codes so you can get your list accross even if you're drunk :D
- ktsmith 12y ago
  WoW Patch 1.12 released in August of 2006 added a hidden channel for add-on communication that didn't suffer from the drunken text transform. FriendNet was in use and updated well after that patch was released but to the best of my knowledge never moved communication to that channel. Source: http://www.wowwiki.com/Patch_1.12 http://www.wowwiki.com/Patch_1.12 New API Functions Add-ons can send hidden chat messages to players in the party, raid or guild, using SendAddonMessage("prefix", "message", "PARTY"|"RAID"|"GUILD"|"BATTLEGROUND").
- cpeterso 12y ago
  The addon should send a checksum so the receiving end can detect any unexpected changes. Also, the addon should just avoid sending when the character is drunk.
- ajanuary 12y ago
  Add a checksum that the client uses the validate the message.
- ricree 12y ago
  If I'm remembering correctly, only certain characters get transformed. So maybe you could use Huffman coding to map the text into a safe character set before you sent it. (might still have to strip out unacceptable characters to get rid of the ...hic)
- armada651 12y ago
  Please let me know when you guys figure this one out. I always wanted to know the best way to flawlessly communicate with friends while drunk.
- tcgv 12y ago
  What about encoding messages with numbers, and then filtering the letters that are added to the message when you are drunk at the receiver side, just before decoding it, would that work?
superduper33 12y ago
Careface.
esquivalience 12y ago
I wonder if this could be exploited by registering usernames that accord with the errors? The user would[0] then be trojanned in to people's friend list with a presumed level of trust they did not deserve. Not sure if this would have any value, but I'm sure someone enterprising could find a way to exploit others trust. That's not a new concept. [0] (This of course assumes that the only lisped-up content is the usernames, not the whole syntax, which I think is an acceptable assumption given that it's fully out of date anyway)
twic 12y ago
So getting drunk makes it harder to make new friends? Oh computers, you so crazy!
_asciiker_ 12y ago
This is one of the most clever ways to blame a bug on booze I have ever read!
kemayo 12y ago
Blizzard added a function to get around this, in response to this sort of chat-tunneling: SendAddonMessage [1] It stops the drunk text-transform, and also doesn't have to worry about hiding the text from the channel you're trying to talk into. Overall, Blizzard has been very good about adapting to what their addon community is trying to do. They add official support for hacks if they like what the addon does for the game, and deliberately break some if they don't like its effect. [1]: http://www.wowwiki.com/API_SendAddonMessage http://www.wowwiki.com/API_SendAddonMessage
- vjeux 12y ago
  During the beta, the way we used to release CosmosUI was to send modified versions of the game interface files. On every patch, we had to manually go over all the changes we made and re-apply them to the new files. This process took several hours and in the process, no one could really play because they didn't have all their mods. Since it was becoming a real pain, I figured out that we could modify the existing functions from a new file instead of modifying the actual files[1]. I spend several weeks transforming all our code into "mods", essentially the first addons :) Once this migration was done and patches were no longer a pain point, Blizzard officially introduced the notion of addons. [1]: http://blog.vjeux.com/2011/javascript/hook.html http://blog.vjeux.com/2011/javascript/hook.html
  - kemayo 12y ago
    Nice! I didn't get into WoW until around 1.9 (I think), and I had no idea that CosmosUI was the genesis for the addon system. Imagining a parallel universe where we stuck with a process in which installing a new "addon" involves applying a .patch to your base UI and hoping that it'll kinda-sorta work with the other patches you applied... terrifying, really...
    - sveiss 12y ago
      The third-party addon story for Asheron's Call, a MMO which pre-dated WoW by around five years, was like a dark, gritty crapsack world version of that parallel universe. There was no supported add-on mechanism; instead, a player-written framework called Decal hooked into the executable at runtime, and Decal exposed an API for plugin authors. I can't remember all the details, but I think did a mixture of intercepting function calls inside the client, injecting calls to client functions, parsing client memory to expose data to plugins, and either parsing the network stream directly to capture events or hooking into the client's packet parser. Oh, and this wasn't a nice safe Lua or Ruby API. It was an unmanaged COM API, so you had all the headaches which come with dealing with reference counting, pointers, and all the rest. Every month (and every interim patch), there would be a great whine and moan from the players until the memlocs file was updated, and all the plugins started working again. (Unless something more substantial in the client had changed, of course, requiring more work.) It was a seriously impressive technical achievement by the people behind it. And looking back on it, rather scary...
chris_wot 12y ago
I wonder if there would be a way of sending drunk text that forces it to sober text?
- sejje 12y ago
  It appears to add characters, so I doubt it. If it sometimes removes characters, then perhaps so.
patio11 12y ago
The author is the principal of the company which produces RailsLTS, which I was involved in as a customer. I wasn't aware that we shared the WoW connection, but that makes me like them even more. (I sort of hope we do not need an advisory about RCE via session cookie tampering because Rails is drunk.)
- triskweline 12y ago
  Thanks Patrick :) I had to quit my WoW guild when I discovered I could only run a single company at the same time. Speaking of RCE: Distribution of new FriendNet versions was such a pain that for a while we considered a P2P auto-update mechanism where FriendNet users with newer versions would whisper Lua code to users with older versions, who would then eval() it locally. It seemed like a much better idea then than it does today...
  - comex 12y ago
    If you signed the updates properly then it would probably be fine :)
  - bradleyjg 12y ago
    And a great handle to boot!
deleted 12y ago
[deleted]
legacy2013 12y ago
I wish I had thought of this when I was trying to develop a WoW Addon. I was building an advanced party gui and wanted to communicate the whereabouts of each user, but was stmyed about how to send information between each instance
akc 12y ago
So you're saying it's a lossy format.
heterogenic 12y ago
I can verify that, when inebriated, I am a terrible judge of character, and trust many people I shouldn't (even when my friends try to warn me off)... The number of times I've gone out questing while drunk and ended up in the company of some loser (while wearing sub-optimal gear) is uncomfortably high. The next morning my memory is almost always corrupted. Only by sheer luck have I not yet been fragged in a PvP area (and having rolled female this life, I wouldn't have much chance of defending myself.) Clearly, this is not a bug, but a feature.
- hluska 12y ago
  That was too funny not to upvote! Nicely argued!! :)
akx 12y ago
Ahhh, how I do miss the times of WoW pre-1.3, when addons could trigger spells and such at will. I wrote a (likely very unsuccessful, as it's now disappeared from the internets) addon called DancingGnome, that allowed one to bind arbitrary input series to spells -- it was meant to bind dance pad moves to spells. Up-Up-Up for Fireball, Down-Up for Fireblast, those were (some of) my binds. More successfully, though, I wrote Chatr, which popped up "IM" windows for private chats. And another addon that allowed covert chats that looked like (to people with the addon) they were being said in public channels. Ah, those were the times (for a given value of times).
deleted 12y ago
[deleted]
vitamen 12y ago
As a Game Master we could see all these hidden communications betweens addons, and it would dwarf the amount of true communication a player would participate in. A raid group could be filled with 1,000 lines of addon chat a second, often contributing to lag that they would then complain about. Addons were powerful, but were certainly a source of many issues, and nobody wanted to hear that they needed to disable their addons to fix their issue.
- quaunaut 12y ago
  If this had been explained exactly as you did here, we might have listened. All we ever heard was, "Disable your addons", which from a technically-inclined user's point of view is akin to your ISP saying "power cycle your modem" for a problem that seemed unrelated. This is one of those times where I have to wonder why so many companies feel it's such a bad idea to get technical with their users.
  - thisone 12y ago
    I think one of the main problems would be that the loud and angry forum types wouldn't care about know why disabling addons (by the entire raid) would fix the issue. They would still see it as a problem with "the devs" and not a problem with the addon. Communicating it clearly to addon developers, that may have helped I think. You know, use the com channels sparingly and for short, targeted messages only, because "lag". From what I remember there was quite a bit of pride surrounding having the most used addon. And the ability to say "my raid addon doesn't cause lag like the old favourite" would have spurred on some devs to get even more creative with limited resources.
joeblau 12y ago
MIM attack from the server! Great post; I've never played WOW but I can see how this would be very hard to track down.
lugg 12y ago
There is a newish add on called oqueue it does a similar p2p hack to form groups across servers for rbgs and now raids (no in game feature for rbg forming). Very fun to hack the code and give yourself impossible stats for jokes and invites. Haven't done much more than that but the hack was useful when you are playing an under geared alternate character that punches way above its weight and can't find groups due to gear level. I'm almost certain you could get yourself invites and hijack group leadership with it (the addon takes over those functions)
cryowaffle 12y ago
Not to be that guy, but how was this hard to find if WoW was adding the ...hic to the text? Couldn't you examine the exchange, see the ...hic and immediately recognize that would be added because drunk?
- iancarroll 12y ago
  I would assume it scrambles it more then that. Lol.

389 points by triskweline 12y ago

That's a great story. How would you have fixed it without having to find another channel to transmit over? Could the plugin sense the character's condition and delay until it abated?

lmm 12y ago
The transform looks fairly simple; just approach it the same way you do error-correction (i.e. reed-solomon), find vectors of characters that are far apart in phase space and use them rather than sending messages directly.
- masklinn 12y ago
  ECC's way overkill, just use a CRC and cancel the exchange if users are drunk.
StefanKarpinski 12y ago
Error correcting codes?
yuliyp 12y ago
Convert to hex, replace c with g. That avoids the ...hic and s->sh transforms
Brandon0 12y ago
I would have either just trimmed the ... hic from the received text (if there was only one string causing problems), or maybe even better would be to use control codes to signal the beginning and end of the data.
- ryeguy 12y ago
  it looks like it actually transforms the message too, not just adds "..hic"
Serow225 12y ago
For detection, have the receiver echo back and check that sent == received.
- netrus 12y ago
  But what if both are drunk?!
  - ajanuary 12y ago
    If the drunk transformation is 's' -> 'sh' Sender sends 's' Receiver receives 'sh' Receiver sends back 'sh' Sender receives 'shh', compares it to the 's' it thought it sent, and correctly deduces something went wrong in the communication. It is overzealous in that it would fail if the sender wasn't drunk but the receiver was.
- wingerlang 12y ago
  But if the reciever echoes "sho" (drunken "so") that would just be "sho" again wouldn't it?
  - ajanuary 12y ago
    Server sends 'so' Receiver receives 'sho' Receiver sends back 'sho' Sender receives 'sho', compares it to the 'so' it thought it sent, and correctly deduces something went wrong in the communication.
drzaiusapelord 12y ago
Send a known-good line to test for it. "FriendnetStart: Hello, I am speaking normally and am not drunk," as the first message. If the application sees anything but that as the first message it should assume drunk and retry in an hour. Or implement a checksum at the end of every line. I imagine he also had issues with the profanity filter as well. Some usernames may set it off if they have a string of characters in them like username "Breshith," which would set off the filter for the text "shit."
- tgarv 12y ago
  The character could possibly become drunk immediately after sending the "I am not drunk" message, so you'd need to check if the character was still not drunk after sending the FriendNet messages. But assuming that drunkenness lasts much longer than it takes to send the FriendNet messages, you could be sure that if the character was not drunk before and after, then the messages in between were also valid.
  - masklinn 12y ago
    > so you'd need to check if the character was still not drunk after sending the FriendNet messages Just send a checksum with the message.
    - leorocky 12y ago
      Yes this would work, even though the message could get garbled including the checksum, you make the first set of characters be a checksum for the rest. If the checksum or message became compromised it would fail validation. This is the best solution so far. Send it with every message.
      - MasterScrat 12y ago
        Or use error-correcting codes so you can get your list accross even if you're drunk :D
ktsmith 12y ago
WoW Patch 1.12 released in August of 2006 added a hidden channel for add-on communication that didn't suffer from the drunken text transform. FriendNet was in use and updated well after that patch was released but to the best of my knowledge never moved communication to that channel. Source: http://www.wowwiki.com/Patch_1.12 http://www.wowwiki.com/Patch_1.12 New API Functions Add-ons can send hidden chat messages to players in the party, raid or guild, using SendAddonMessage("prefix", "message", "PARTY"|"RAID"|"GUILD"|"BATTLEGROUND").
cpeterso 12y ago
The addon should send a checksum so the receiving end can detect any unexpected changes. Also, the addon should just avoid sending when the character is drunk.
ajanuary 12y ago
Add a checksum that the client uses the validate the message.
ricree 12y ago
If I'm remembering correctly, only certain characters get transformed. So maybe you could use Huffman coding to map the text into a safe character set before you sent it. (might still have to strip out unacceptable characters to get rid of the ...hic)
armada651 12y ago
Please let me know when you guys figure this one out. I always wanted to know the best way to flawlessly communicate with friends while drunk.
tcgv 12y ago
What about encoding messages with numbers, and then filtering the letters that are added to the message when you are drunk at the receiver side, just before decoding it, would that work?

superduper33 12y ago

Careface.

esquivalience 12y ago

I wonder if this could be exploited by registering usernames that accord with the errors? The user would[0] then be trojanned in to people's friend list with a presumed level of trust they did not deserve. Not sure if this would have any value, but I'm sure someone enterprising could find a way to exploit others trust. That's not a new concept. [0] (This of course assumes that the only lisped-up content is the usernames, not the whole syntax, which I think is an acceptable assumption given that it's fully out of date anyway)

twic 12y ago

So getting drunk makes it harder to make new friends? Oh computers, you so crazy!

_asciiker_ 12y ago

This is one of the most clever ways to blame a bug on booze I have ever read!

kemayo 12y ago

Blizzard added a function to get around this, in response to this sort of chat-tunneling: SendAddonMessage [1] It stops the drunk text-transform, and also doesn't have to worry about hiding the text from the channel you're trying to talk into. Overall, Blizzard has been very good about adapting to what their addon community is trying to do. They add official support for hacks if they like what the addon does for the game, and deliberately break some if they don't like its effect. [1]: http://www.wowwiki.com/API_SendAddonMessage http://www.wowwiki.com/API_SendAddonMessage

vjeux 12y ago
During the beta, the way we used to release CosmosUI was to send modified versions of the game interface files. On every patch, we had to manually go over all the changes we made and re-apply them to the new files. This process took several hours and in the process, no one could really play because they didn't have all their mods. Since it was becoming a real pain, I figured out that we could modify the existing functions from a new file instead of modifying the actual files[1]. I spend several weeks transforming all our code into "mods", essentially the first addons :) Once this migration was done and patches were no longer a pain point, Blizzard officially introduced the notion of addons. [1]: http://blog.vjeux.com/2011/javascript/hook.html http://blog.vjeux.com/2011/javascript/hook.html
- kemayo 12y ago
  Nice! I didn't get into WoW until around 1.9 (I think), and I had no idea that CosmosUI was the genesis for the addon system. Imagining a parallel universe where we stuck with a process in which installing a new "addon" involves applying a .patch to your base UI and hoping that it'll kinda-sorta work with the other patches you applied... terrifying, really...
  - sveiss 12y ago
    The third-party addon story for Asheron's Call, a MMO which pre-dated WoW by around five years, was like a dark, gritty crapsack world version of that parallel universe. There was no supported add-on mechanism; instead, a player-written framework called Decal hooked into the executable at runtime, and Decal exposed an API for plugin authors. I can't remember all the details, but I think did a mixture of intercepting function calls inside the client, injecting calls to client functions, parsing client memory to expose data to plugins, and either parsing the network stream directly to capture events or hooking into the client's packet parser. Oh, and this wasn't a nice safe Lua or Ruby API. It was an unmanaged COM API, so you had all the headaches which come with dealing with reference counting, pointers, and all the rest. Every month (and every interim patch), there would be a great whine and moan from the players until the memlocs file was updated, and all the plugins started working again. (Unless something more substantial in the client had changed, of course, requiring more work.) It was a seriously impressive technical achievement by the people behind it. And looking back on it, rather scary...

chris_wot 12y ago

I wonder if there would be a way of sending drunk text that forces it to sober text?

sejje 12y ago
It appears to add characters, so I doubt it. If it sometimes removes characters, then perhaps so.

patio11 12y ago

The author is the principal of the company which produces RailsLTS, which I was involved in as a customer. I wasn't aware that we shared the WoW connection, but that makes me like them even more. (I sort of hope we do not need an advisory about RCE via session cookie tampering because Rails is drunk.)

triskweline 12y ago
Thanks Patrick :) I had to quit my WoW guild when I discovered I could only run a single company at the same time. Speaking of RCE: Distribution of new FriendNet versions was such a pain that for a while we considered a P2P auto-update mechanism where FriendNet users with newer versions would whisper Lua code to users with older versions, who would then eval() it locally. It seemed like a much better idea then than it does today...
- comex 12y ago
  If you signed the updates properly then it would probably be fine :)
- bradleyjg 12y ago
  And a great handle to boot!

deleted 12y ago

[deleted]

legacy2013 12y ago

I wish I had thought of this when I was trying to develop a WoW Addon. I was building an advanced party gui and wanted to communicate the whereabouts of each user, but was stmyed about how to send information between each instance

akc 12y ago

So you're saying it's a lossy format.

heterogenic 12y ago

I can verify that, when inebriated, I am a terrible judge of character, and trust many people I shouldn't (even when my friends try to warn me off)... The number of times I've gone out questing while drunk and ended up in the company of some loser (while wearing sub-optimal gear) is uncomfortably high. The next morning my memory is almost always corrupted. Only by sheer luck have I not yet been fragged in a PvP area (and having rolled female this life, I wouldn't have much chance of defending myself.) Clearly, this is not a bug, but a feature.

hluska 12y ago
That was too funny not to upvote! Nicely argued!! :)

akx 12y ago

Ahhh, how I do miss the times of WoW pre-1.3, when addons could trigger spells and such at will. I wrote a (likely very unsuccessful, as it's now disappeared from the internets) addon called DancingGnome, that allowed one to bind arbitrary input series to spells -- it was meant to bind dance pad moves to spells. Up-Up-Up for Fireball, Down-Up for Fireblast, those were (some of) my binds. More successfully, though, I wrote Chatr, which popped up "IM" windows for private chats. And another addon that allowed covert chats that looked like (to people with the addon) they were being said in public channels. Ah, those were the times (for a given value of times).

deleted 12y ago

[deleted]

vitamen 12y ago

As a Game Master we could see all these hidden communications betweens addons, and it would dwarf the amount of true communication a player would participate in. A raid group could be filled with 1,000 lines of addon chat a second, often contributing to lag that they would then complain about. Addons were powerful, but were certainly a source of many issues, and nobody wanted to hear that they needed to disable their addons to fix their issue.

quaunaut 12y ago
If this had been explained exactly as you did here, we might have listened. All we ever heard was, "Disable your addons", which from a technically-inclined user's point of view is akin to your ISP saying "power cycle your modem" for a problem that seemed unrelated. This is one of those times where I have to wonder why so many companies feel it's such a bad idea to get technical with their users.
- thisone 12y ago
  I think one of the main problems would be that the loud and angry forum types wouldn't care about know why disabling addons (by the entire raid) would fix the issue. They would still see it as a problem with "the devs" and not a problem with the addon. Communicating it clearly to addon developers, that may have helped I think. You know, use the com channels sparingly and for short, targeted messages only, because "lag". From what I remember there was quite a bit of pride surrounding having the most used addon. And the ability to say "my raid addon doesn't cause lag like the old favourite" would have spurred on some devs to get even more creative with limited resources.

joeblau 12y ago

MIM attack from the server! Great post; I've never played WOW but I can see how this would be very hard to track down.

lugg 12y ago

There is a newish add on called oqueue it does a similar p2p hack to form groups across servers for rbgs and now raids (no in game feature for rbg forming). Very fun to hack the code and give yourself impossible stats for jokes and invites. Haven't done much more than that but the hack was useful when you are playing an under geared alternate character that punches way above its weight and can't find groups due to gear level. I'm almost certain you could get yourself invites and hijack group leadership with it (the addon takes over those functions)

cryowaffle 12y ago

Not to be that guy, but how was this hard to find if WoW was adding the ...hic to the text? Couldn't you examine the exchange, see the ...hic and immediately recognize that would be added because drunk?

iancarroll 12y ago
I would assume it scrambles it more then that. Lol.