6 ms·

Last time I used this approach I ended up regretting it. The problem is that the links become ugly, unwieldy for the users to handle, and occasionally cause pro

• sk5t 12y ago
  '.' is not a standard base64 character; did you roll your own? Base36 and base32 are popular where portability and paranoia of being MIME/URL-mangled are concerns, e.g., Google Authenticator's OATH TOTP implementation for the secret key.
  • jimktrains2 12y ago
    if you check http://en.wikipedia.org/wiki/Base64 http://en.wikipedia.org/wiki/Base64 there are actually RFC-standard alphabets that use . and - because they're URL-safe.
    • bazzargh 12y ago
      I was surprised when I read that, since I've implemented those RFCs before and didn't remember this coming up - so I checked. Unfortunately, that's just a problem with the wikipedia article. The only one listed there as an RFC using '.' is RFC 4686, (in the padding column right now it says "= (optional, not recommended. Recommended use .)"). That's wrong though. In RFC 4648 it actually says: The remaining unreserved URI character is ".", but some file system environments do not permit multiple "." in a filename, thus making the "." character unattractive as well. And... The pad character "=" is typically percent-encoded when used in an URI, but if the data length is known implicitly, this can be avoided by skipping the padding The data length is known for a cryptographic hash, so you would omit the '='. I'll fix wikipedia.
• tinco 12y ago
  This is why people use base48 or base36 for binary data in URLs. There is a really good reason to store these things in the database though, and that's on-demand authorization retraction.
  • tracker1 12y ago
    you can always use base64-url structure too... base64, remove trailing "="'s, replace "+" with "-", replace "/" with "_". Though the "_" and "-" can still be ugly at the beginning or end. You can also encode them %## style.
  • bjt 12y ago
    You can still do that on-demand retraction as long as the hash is fed something from the database that you can change. IIRC, Django uses both password and last login timestamp inside the hash. If you needed to invalidate someone's token you could just add 1 millisecond to their last login time.
• morgante 12y ago
  Sorry, but I don't really see how this introduces new problems. Even if you're using a stateful approach (ie. storing the token in the database) you still need to store and send a fairly long and convoluted URL. So in either case you're going to have to deal with the usability problem of making sure your long URL is still easily clickable. As for the link invalidation, I'm confused as to why "." was included in the first place. It's not usually included in a base 64 encoding. (Still, that's why I use base 62 for sending URLs.)
  • jsnell 12y ago
    The difference is that the URL with just a lookup token will be a lot shorter than the URL with all the data embedded in it, and will have a static and predictable length. And longer URLs are a source of problems. They'll be more likely to be misinterpreted by a MUA, more likely to be copy-pasted badly (e.g. due to word-wrap issues), and will look more suspicious to users wary of phishing. Optimizing the URLs for length will then cause other problems, like my base64 one (where I started with hex, and then switched to a supposedly URL-safe base64 encoding). It'll also prevent you from having a fallback mechanism when the URL does break. With state, the email can simply include the token as plaintext ("If the link didn't work type in this 10 letter code"). It'll make it harder to later implement validation over SMS. On the flip side, the gains from not storing state are basically non-existent.
    • andrewchambers 12y ago
      I would rather do both, and store the short url to long url in a key value store with automatic deletion. I would rather keep that crap away from my customer data if its not needed.
    • StavrosK 12y ago
      Just truncate it to a collision probability that is acceptable to you. Nothing's forcing you to keep all of the signature bytes.
• sly010 12y ago
  Same problem here. Plus I am using sendgrid which automagically makes the link about 4 times longer using the same "trick" which further amplifies the problem. In fact I decided that I should go the other direction instead. Now every outgoing email is stored in the database before sending. This makes it trivial to track every email open, every click on every link and the links look beautiful, like "/e/<random_email_id>/open". Been happy ever since. Edit: This obviously doesn't invalidate the general idea.
  • tracker1 12y ago
    Another benefit of generating and saving an email to the database first, is at that point putting a "view this email on the web" type link into the top of the email becomes trivial. Formatting HTML emails is a significant pain, and mobile only exacerbates that... The point of a transactional database is, you know... handling transactions... and in this case, resetting a password is a transaction... Sure, you have to kill long standing transactions manually, but that's a fair bit better than having to deal with a url that is longer than https://foo.com/account/password-reset/(uuid) https://foo.com/account/password-reset/(uuid) for users that may well copy/paste ... While I do like and appreciate the idea.. it's easy enough to scale when you are either using a replica-set for your reads, or a sharded database...
    • geofft 12y ago
      A UUID is 128 bits, written in hex, so it's 36 bytes. If you use an appropriately URL-safe base64, you can encode a 216-bit value in the same number of bytes. That's long enough to be cryptographically safe for this application.
  • hvidgaard 12y ago
    This of course is an email layer on top of your core logic. Your core logic still thinks it's sending a mail with a very long URL. It's just shortened and when it's incoming it will be converted back. So you get the best of both worlds.
• geofft 12y ago
  It'd be nice for someone to standardize this. I believe that it's safe to use a cryptographic PRF that's too short to be a proper cryptographic hash, like the 64-bit SipHash https://131002.net/siphash/ https://131002.net/siphash/ : you're not worried about collisions, and you (hopefully) have your own ways of shutting down 2^64 brute-force attempts already. But I haven't thought hard so don't quote me on that. If you can get away with a 64-bit or even 128-bit hash (maybe truncated SHA256?), and you encode in some base32 scheme, then it's 13 or 26 characters — at worst https://example.com/resetpw/d232rgs5690y6nphk00tdmmw60 https://example.com/resetpw/d232rgs5690y6nphk00tdmmw60 (128-bit), which isn't so bad. base64 only gets you down to 11 / 22 characters, so it's not a huge win. The benefit of the scheme is that this is stateless and you don't need to access the database at all. This isn't about schema, this is so that you don't have to think about expiring fields from the DB at all. It's done entirely in the web server. That said, the input to the hashes should be well-structured (so you don't generate a collision by having two API functions that can be coerced to hash the same string with different meanings). That's a vulnerability in this writeup, and I think that would also be something a proper specification of this scheme should address.
• viraptor 12y ago
  > So 1/64 of my validation links ended up being invalid. Why not just try appending "." on the server side if what you get is too short? All of the links become valid again.
  • hvidgaard 12y ago
    It's symptomatic treatment rather than curing the problem. In this case it's GMails tendency to remove a character that's considered URL safe. So really the best you can do is using a different encoding that is not subject to this.
    • wtbob 12y ago
      I wonder if Gmail would understand the standard textual form <URL:http://example.net/stuff.> http://example.net/stuff.>. Edit: interestingly, HN's text formatter doesn't!
      • a1369209993 12y ago
        Also, that's usually <http://example.net/stuff.> http://example.net/stuff.> or [http://example.net/stuff.]; http://example.net/stuff.]; no 'URL:'.
• lucb1e 12y ago
  Can I ask, did you send an HTML email or did you just throw the link in and did GMail do you the service of making 63/64 links easily clickable without copying it to the address bar? Because I'm pretty sure it will work fine 64/64 times if you use an a-href.
  • jsnell 12y ago
    Text email. A scheme that requires HTML email to paper over the problem of ugly URLs, has in my mind already lost any ease of implementation argument.
• hvidgaard 12y ago
  The benefit is that you do not have to store any additional data in your DB, of if you need it for BI purpose you can store it in an appropiate BI system instead.
• vbezhenar 12y ago
  You should use URL-safe base 64 encoding described in [1]. It uses `-` and `_` and should be safe for Gmail. 1: http://tools.ietf.org/html/rfc4648#section-5 http://tools.ietf.org/html/rfc4648#section-5
• cm2187 12y ago
  In the cloud, I would think that a better alternative is to store temp data in Azure table storage / Amazon SimpleDB. This is cheap, scalable storage that avoid having to use the database when not needed.
• tripzilch 12y ago
  These regrets sound to me more like "your problems" than problems with this confirmation scheme. > The problem is that the links become ugly, unwieldy for the users to handle Why? The hash only needs to be big enough that brute-forcing becomes infeasible. Depending on the type of server-level flood control you're using, you can exactly determine the amount of bits required. Or if you choose not to, a conservative estimate of 48 bits (8 characters in base64) characters should be more than enough. Remember that offline brute-forcing is not possible here, and at a rate of, say, 1M tries per second you'll be brute-forcing yourself silly for somewhere in the order of 2 years. If you can make tighter estimates about flood control, these are rough (and still conservative x4) estimates: at a limit of 15k tries per second you can do with 7 bytes (times 6 bits). At a limit of 200 tries per second, just 6 bytes of base64 code (without proper flood control a large botnet could make this last one somewhat feasible, maybe). ... unless you mean to imply that the string "reset?user=John&expiry=987654321" is the ugly, unwieldy part :) > For example the base64 encoding I was using for the authentication token included '.' as a one of the characters. Well, turns out that the linkification code in gmail will ignore a '.' at the end of the link (which is kind of sensible). So 1/64 of my validation links ended up being invalid. Using the wrong encoding for your URLs has nothing to do with this technique. In fact, I don't think the article even suggested a particular type of encoding. Did you not know what characters your base64 routine would output, or something? There's more than just the period that you don't want to end a URL with because of ambiguity with the ways people might want to use URLs in a sentence: Closing parens, comma, exclamation, asterisk, etc. Regardless of what one particular webmail provider will linkify or not, it's just common sense, if you think about it. Closing parens are always a big problem with many Wikipedia links, it's not like this is some obscure "gotcha". That is what the %XX URL-encoding was made for. Or why not just use what YouTube does, they seem to have it figured out pretty well: A-Z, a-z, 0-9, _ and -