7 ms·

Implementing a toy version of TLS 1.3

224 points by jfarmer 5y ago

tptacek 5y ago
This is wonderful. I was super excited to dive in an find the RSA code so I could preen about Bleichenbacher's vulnerability, but she neatly sidestepped that by doing ECDH. Then I thought, well, maybe it's P-curve ECDH and I can preen about invalid curve attacks on static-ephemeral ECDH. But nope, X25519! My point here, apart from making fun of myself for being the kind of person who would write this stuff on a message board, is TLS 1.3 is pretty solid. The "block thing" that's kind of weird is, I assume, the TLS Record Layer. TLS runs (ordinarily) over TCP, which provides a non-demarcated stream of bytes. TLS breaks that stream up into records, and runs its handshake messages over one type of record, (say) HTTPS over another, and "alerts" over a third. The Record Layer also interacts, I think, with TLS's misbegotten compression system? In the same vein as this project (but with different goals) is Trevor Perrin's tlslite, which is implemented in pure Python: https://github.com/trevp/tlslite https://github.com/trevp/tlslite
- westurner 5y ago
  > tlslite "PEP 543 – A Unified TLS API for Python" #interfaces (-2016) https://peps.python.org/pep-0543/#interfaces https://peps.python.org/pep-0543/#interfaces
- benmmurphy 5y ago
  i guess TLS needs records because it wants to mix signalling data with the data stream (stuff like alerts, keyupdate, renegotiation for tls1.2). but i guess also they want to use block ciphers and for stream ciphers they want some kind of authentication so i'm not sure how you would do this without some form of framing. there is also some more strangeness with the TLS record layer because handshake messages and potentially other messages are allowed to be fragmented over multiple TLS records or you can have multiple handshake messages in the same TLS record (i think there are some restrictions on TLS1.3 about mixing different message types). this also might be a way to mess with TLS censorship middle boxes because they might not be robustly coded. i have a plugin for mitmproxy that does TLS interception using knowledge of the shared key but it doesn't handle fragmentation of handshake messages and a bunch of other quirks correctly.
  - ekr____ 5y ago
    The primary reason you need records is that AEAD algorithms run on blocks so you need to know where the end of one block is and the start of the next. And because you are running over TCP, you don't get that framing information from TLS. As you say, it's also the case that you have different types of data and so the record type indicates what you are processing, but you'd need records even if you didn't do that.
- tialaramex 5y ago
  Seems like you ought to know TLS 1.3 famously doesn't have a way to do RSA key exchange. There's a fun video where some presumably American EDCO people are like, "surely if we all turn up in person we've got more votes so we can get RSA back into TLS 1.3" and er, no, the IETF isn't a democracy. Looking at GCM or any other authenticated encryption without Sum Types makes me sad. If decrypting the message fails, there is no plaintext and I feel like an API should be able to actually express that properly not return a plaintext [even if it's empty] and a flag saying "Don't use the plaintext". :( The "block thing" is a necessary choice if we're going to have authenticated encryption, because authenticating costs us overhead. If we wanted to reliably authenticate individual bytes (so that we can have what naively feels like a stream of bytes) we'd be paying that overhead for each byte. That's pretty outrageous. On the other hand if I'm downloading a 10MB file with my 24kbps modem and none of that was authenticated yet because the authentication only happens once at the end of the encrypted data, I'm going to get very anxious about my download counter correctly showing 0% successfully downloaded for so long. So TLS authenticates whole records.
  - tptacek 5y ago
    Yes, the point was how much better 1.3 is than what preceded it. You don't need the TLS Record Layer in order to do authenticated encryption.
    - ekr____ 5y ago
      Agreed. Basically all you need is: 1. Something to delimit the boundaries of the message. 2. Some way to determine the nonce. OTOH, the TLS 1.3 record layer is barely more than authenticated encryption,
- ekr____ 5y ago
  I know that tptacek knows this, but for those who don't, we took data compression out of TLS 1.3!
- syncsynchalt 5y ago
  Yep, when digging through the TLS 1.3 RFC (after learning 1.2) the thing that jumped out at me is how many _fewer_ options there are, and how deliberately chosen they seemed to be. - suites (as far as covering cipher + exchange + mac) are gone as a concept - only 5 supported cipher algorithms (all AEAD, and all but one built on AES) - only non-PSK key exchange is (EC)DHE - no compression supported - all the various key derivations are built around a single new primitive function, HKDF I'm sure that given enough time the above options will balloon like we had with latter-day TLS 1.2 but at this point it feels so clean.
  - toast0 5y ago
    I'm sure the options will balloon a bit. After all, if someone develops a new good cipher/etc, we'll want to use it in TLS. But it's nice that as of now, there's no bad choices available (other than potentially allowing downgrading).
    - tialaramex 5y ago
      It's entirely possible we will never need to replace AES (or indeed ChaCha20). AES 128 GCM is mandatory to implement, and implementers are recommended to offer ChaCha20 too. If you don't have AES hardware, ChaCha20 looks very attractive and you'd be a fool not to. So while you can get code points for a vanity cipher, there not only isn't an appetite from the working group to endorse such ciphers, there is unlikely to ever be a scenario where a client or server would actually pick it unless you control both and choose to prefer the vanity cipher. Whereupon why bother? The downgrade prevention in TLS 1.3 is pretty effective, so an attack which needs a downgrade also needs to defeat the downgrade protection, an additional hurdle. Obviously people should get rid of TLS 1.0 and TLS 1.1 but meanwhile I think the anti-downgrade story is the best it has ever been.
      - KMag 5y ago
        We may never need to replace AES 256, but Grover's quantum search algorithm can break AES 128 with a work factor of 2^64. AES 256 GCM and a post-quantum authenticated key agreement algorithm may be all we ever need.
profmonocle 5y ago
Learning a protocol by writing a toy client (or toy server) is a blast. It's so satisfying to see a real, production-quality server sending real responses to your little mess.
- tptacek 5y ago
  You'd probably enjoy work as a software pentester, where the docket --- at least for non-web-applications, which admittedly are the most common project if you don't specialize --- is almost entirely building tooling-grade implementations of random protocols so you can test for vulnerabilities.
  - westurner 5y ago
    Here's caddy's go/tls wrapper with e.g. ACME, OCSP stapling: https://github.com/caddyserver/caddy/blob/master/modules/caddytls/tls.go https://github.com/caddyserver/caddy/blob/master/modules/cad... Django-ca also does OCSP and certbot-compatible ACMEv2 w/ known limitations: https://django-ca.readthedocs.io/en/latest/acme.html#known-limitations https://django-ca.readthedocs.io/en/latest/acme.html#known-l... E.g. https://google.github.io/clusterfuzzlite/ https://google.github.io/clusterfuzzlite/ is likely not so great at protocols because that requires testing concurrent and distributed systems and TLAplus, which at least currently can't find side channels FWIU. https://github.com/secfigo/Awesome-Fuzzing#network-protocol-fuzzers https://github.com/secfigo/Awesome-Fuzzing#network-protocol-... OSS-Fuzz runs CloudFuzz[Lite?] for many open source repos and feeds OSV OpenSSF Vulnerability Format: https://github.com/google/osv#current-data-sources https://github.com/google/osv#current-data-sources
tialaramex 5y ago
Hopefully I have actually insightful things to say about this fun toy. However before that > elliptic curve “multiplication”, where n * P means “add P to itself n times” Not very smoothly described but this is all multiplication meant for the natural numbers you learned in primary school too! Why is 7 x 7 = 49? Because if you start with zero and add 7, seven times, you get 49. Try it. This is an important and re-usable insight, it's part of a larger beautiful framework of mathematics and I believe is much better instructed via modern teaching of arithmetic in schools than "rote learning" of times tables did for my parents.
- bitwize 5y ago
  There are entire worlds of math locked away from you if you don't know your times table by heart. Algebra becomes particularly difficult if you do not know how to not only multiply, but factor, quickly. So yeah, it's great that you know 9 * 6 is 9 added to itself 6 times, but you also have to learn the times table by rote if you want to get anywhere in math.
  - tialaramex 5y ago
    > it's great that you know 96 is 9 added to itself 6 times For a while I tried to figure out what the joke was here, and then I realised all that's going on is HN emphasis asterisks Anyway, no, it's well known that mathematicians are often terrible at this sort of "basic arithmetic". There are a bunch of mathematical disciplines where arithmetic is utterly irrelevant, but even in the disciplines where it seems like it ought to be important if you're an actual mathematician you don't care anyway. That's just mechanical stuff, use a calculator, we're busy doing mathematics. It's a nice party trick (but you'll need to go a lot higher than 9 x 6 to impress anybody), and there are mathematicians who are good at arithmetic, but I assure you that by the time I gave up mathematics (just below degree level) there was no use for times tables, which I had never memorised, what was an obstacle was all the bloody integration. Pure, Mechanics, even the Statistics had integration in it by the time I was eighteen.
tedunangst 5y ago
> When the HTTP response is done, we get these bytes: []byte{48, 13, 10, 13, 10, 23}. I don’t know what this is supposed to mean exactly but it seems to signal the end of the connection. HTTP headers include Content-Length, so you should know when you get a truncated response, but TLS is supposed to be more general purpose, so it includes its own crypto secure end of connection indicator.
- tialaramex 5y ago
  These bytes are an ASCII zero, then two carriage-return/linefeed pairs and then the byte with value 23, so that 0\r\n\r\n might just be HTTP saying we're done sending you this chunked HTTP response. I think that 23 byte is the TLS content type, meaning "This is the actual application data you were transporting over TLS". In TLS 1.3 the original content type byte (from previous TLS versions) just always says 23 regardless so that middle boxes leave us alone, "Huh I guess everything is encrypted application data, I'm just a dumb middlebox so this isn't suspicious" This new, inner content type, safely encrypted and thus invisible to the middleboxes, is at the end of the data because it allows a clever trick with zero padding. See, content type zero isn't a thing. So if we get some data and the last byte is zero, that's not the content type it must be padding, remove it, try again. Still zero? Still padding, remove that and so on. Because the zeroes were encrypted, an adversary can't tell they're padding, but because they're zero the real recipient can discard them safely, yet if we don't need padding we aren't wasting space for a "no padding" indicator, we just don't add any of those zeroes. However, if 23 here is a content type, this means our intrepid TLS implementer got things a little wrong and all their HTTP data has stray 0x17 bytes at the end of each record received. I don't see code to handle that, but I may have missed it. It seems plausible that in terminal output you wouldn't notice?
  - userbinator 5y ago
    For those who want to see the details of this explanation expressed in a more... struct-ured fashion, this section of the spec describes that: https://datatracker.ietf.org/doc/html/rfc8446#section-5.2 https://datatracker.ietf.org/doc/html/rfc8446#section-5.2
  - 5y ago
drewg123 5y ago
I'm super impressed that they got it to work so quickly. What I hate about crypto is that is either wrong or right, there is no "mostly working" that you can identify easily and use as a debugging aid. This drove me crazy when I was working on kTLS in FreeBSD. When I worked on other features (like getting checksum offload right in NIC firmware) there were easy tricks I could use for debugging, like sending a stream of all zeros. For crypto, it was basically back to first principals and code examination..
- toast0 5y ago
  TLS 1.3 has some published test vectors where if you understand the inputs (which is tricky) and do the same operations, you'll end up with the same outputs as the test vectors and that should maybe work with a real server too. Still, it helps a ton to get error logs (or step through debugging) from a real server to figure out what you messed up. For every value you send in a TLS handshake, it feels like you've got to send the length three different times, in different numbers of bytes and sometimes adding the bytes it takes to send the length. But, if you can smash through all that, you can get a reasonably working TLS 1.3 client in about a week of fiddling. If your runtime has a decent api to validate certificates, you can call out to that too (the less you personally do with x.509, the better) Edit: link to vectors https://datatracker.ietf.org/doc/rfc8448/ https://datatracker.ietf.org/doc/rfc8448/
  - drewg123 5y ago
    Nice, I didn't know that. I did most of the work in ~2017 before TLS 1.3.
  - syncsynchalt 5y ago
    Agreed, I second that recommendation for anyone interested in this stuff to write a toy TLS implementation. Here's mine: https://github.com/syncsynchalt/tincan-tls https://github.com/syncsynchalt/tincan-tls
- deleted 5y ago
  [deleted]
syncsynchalt 5y ago
> an amazing TLS resource: tls13.ulfheim.net Readers, i made a "squee" noise just now. Glad it was useful, this is exactly the sort of thing I wrote it for. <3
- toast0 5y ago
  It's super nice. The only issue I have is you released it after I had done my TLS 1.3 work, so I couldn't have referred to it when it would have been most useful to me.
- reneme 5y ago
  We can relate to that! It is extremely helpful for our current efforts to implement TLS 1.3 for the Botan crypto library. Especially to get a foothold with TLS and gain a feeling for the protocol structure. Amazing work! https://github.com/randombit/botan/pull/2922 https://github.com/randombit/botan/pull/2922
silisili 5y ago
I was one of the folks kinda down on the DNS server implementation...but this, this is awesome. TLS is a serious black box to 90%+ of even veteran programmers I meet. To break it down and write a small implementation showing the steps is seriously both impressive and extremely generous. Kudos!
travisgriggs 5y ago
Did anyone else find this line interesting? > This was pretty annoying to get working because I kept passing the wrong arguments to things. I think this sentiment shows up twice in fact in the article. I’m not sure what language the listings are in, but it looked static-type-ish. Are there things the current batch of safety focused languages do to mitigate these types of ordering errors? In Kotlin, Python, and Swift, we make it a practice to always use the keywords in calls if there are multiple arguments of the same type (e.g. two or more ints). I wish I could figure out how to configure any of their linters to enforce it better. Despite its late bound unsafe nature, this was just never an error I made (argument ordering) back when I used to do a lot of Smalltalk with its interwoven keyword syntax.
- tyingq 5y ago
  >I’m not sure what language the listings are in, but it looked static-type-ish. It's golang. No keyword args.
- toast0 5y ago
  I assumed passing the wrong argument to things to include wrong streams of bytes to HKDF and what not. There's a lot of keys to generate, and it's easy to get the bits of stuff not quite right.
- deleted 5y ago
  [deleted]
- tialaramex 5y ago
  > Are there things the current batch of safety focused languages do to mitigate these types of ordering errors? Rust's NewType idiom can often help here. The author makes a bunch of arrays of bytes, like "Keys" is just all arrays of bytes with different meanings, and so if you mistakenly try to use the ServerPublic key as a ClientHandshake IV the types match up but it can't work. NewType would encourage you to have types named AESKey and InitialisationVector and X25519Public and so on, whereupon the compiler will tell you that your AESKey isn't an InitialisationVector etc. If you use an IDE of some sort, your IDE will probably even prompt you while writing the code, you should put an AESKey here, not an InitialisationVector, something like Intellisense probably even scans your variables for the type and suggests you should either use my_sending_key or my_receiving_key and at that point it does seem like the programmer is only required to pay a modest amount of attention to produce programs which do what they intended. NewType would apply for simple integers too. Rust not only has a type for the abstract concept of file handles (pretty common in high level languages, even C can do this with FILE*) but it has a type for Unix file descriptors, even though of course Unix file descriptors are just integers with some promises about which values they can have. This is a fundamental idea in the language, Rust's str built-in is an array of bytes, but you could write [u8; N] and that's also an array of bytes, however str promises that the array of bytes is definitely UTF8 encoded Unicode text whereas u8 claims no such thing. At runtime they're identical, but in your program they're very different. Rust gets to do this because it promises types don't actually exist at run time. The program you are writing, and which the compiler is compiling, may distinguish a CodeNumber, an ArticleID, a RowNum, a UserIdentifier, and a mere counter, but the machine code output just uses the same say, 64-bit machine register and 8 bytes of RAM to represent all five things interchangeably, the distinction existed only to make your program easier to understand.
userbinator 5y ago
This is certainly something interesting and worth trying, but I'd recommend starting with 1.2 first, because the handshake isn't encrypted so you can follow along more easily. I only looked at the TLS 1.3 RFC for a few small things. As someone who has written implementations of various widely-considered-nontrivial things (mostly image and video codecs, some crypto -- but not TLS, although upon seeing this article, that may change...) for self-educational purposes, I strongly recommend that the official standard be the first thing you look at and refer to during the process. It will clarify a lot of doubts, and in the case of TLS, the RFCs are relatively readable as far as standards go. I’m not sure why the blocks have the size they do (maybe it’s so that each one will fit inside a TCP packet ???), but in theory I think they could be up to 65535 bytes, since their size field is 2 bytes. That's due to either MTU or server-side write fragmentation, and the limit is 16K(+overhead). It is a stream to the application layer, but the record-layer protocol breaks it up into chunks of smaller limited size to allow for the integrity checks. I assume a real TLS implementation would use a thread pool or coroutines or something to manage this. That's a very strange assumption. Depending on the API, the ones I've used will either just read in a loop until the length is fulfilled or an error occurs; or return an "incomplete message" error meaning that the caller is the one to continue reading. Getting a partial read from a TCP socket is a very common "trap for young players" --- if you don't keep it in mind, you'll write code that appears to work on localhost or inside a fast LAN, but fails intermittently and sometimes mysteriously over the Internet or when there's more latency. You can never assume message boundaries when using TCP.
- eek2121 5y ago
  100% agree on reading the RFC. I've implemented various protocols as well (SMTP, IMAP, POP, FTP, some crypto and other things, all a couple decades ago), and the cases where I glossed over the RFC have always bitten me.
- andrewf 5y ago
  Getting a partial read from a TCP socket I wish common test frameworks and/or stdlibs made it easier to "fuzz" this. I've written more than one io.Reader implementation (Golang) that breaks up results from an underlying io.Reader according to a random seed.
olliej 5y ago
One thing I like about cryptography, and wish we could get politicians to understand, the mathematical steps to implement many of the core concepts are not hard. Obviously implementing them robustly such that they’re safe at scale is another matter, but “criminals” won’t have any difficulty getting access to secure implementations, and adding any kind of weakness only means that the implementations used by the public are more likely to have bugs, in addition to the legally mandated ones.

224 points by jfarmer 5y ago

This is wonderful. I was super excited to dive in an find the RSA code so I could preen about Bleichenbacher's vulnerability, but she neatly sidestepped that by doing ECDH. Then I thought, well, maybe it's P-curve ECDH and I can preen about invalid curve attacks on static-ephemeral ECDH. But nope, X25519! My point here, apart from making fun of myself for being the kind of person who would write this stuff on a message board, is TLS 1.3 is pretty solid. The "block thing" that's kind of weird is, I assume, the TLS Record Layer. TLS runs (ordinarily) over TCP, which provides a non-demarcated stream of bytes. TLS breaks that stream up into records, and runs its handshake messages over one type of record, (say) HTTPS over another, and "alerts" over a third. The Record Layer also interacts, I think, with TLS's misbegotten compression system? In the same vein as this project (but with different goals) is Trevor Perrin's tlslite, which is implemented in pure Python: https://github.com/trevp/tlslite https://github.com/trevp/tlslite

westurner 5y ago
> tlslite "PEP 543 – A Unified TLS API for Python" #interfaces (-2016) https://peps.python.org/pep-0543/#interfaces https://peps.python.org/pep-0543/#interfaces
benmmurphy 5y ago
i guess TLS needs records because it wants to mix signalling data with the data stream (stuff like alerts, keyupdate, renegotiation for tls1.2). but i guess also they want to use block ciphers and for stream ciphers they want some kind of authentication so i'm not sure how you would do this without some form of framing. there is also some more strangeness with the TLS record layer because handshake messages and potentially other messages are allowed to be fragmented over multiple TLS records or you can have multiple handshake messages in the same TLS record (i think there are some restrictions on TLS1.3 about mixing different message types). this also might be a way to mess with TLS censorship middle boxes because they might not be robustly coded. i have a plugin for mitmproxy that does TLS interception using knowledge of the shared key but it doesn't handle fragmentation of handshake messages and a bunch of other quirks correctly.
- ekr____ 5y ago
  The primary reason you need records is that AEAD algorithms run on blocks so you need to know where the end of one block is and the start of the next. And because you are running over TCP, you don't get that framing information from TLS. As you say, it's also the case that you have different types of data and so the record type indicates what you are processing, but you'd need records even if you didn't do that.
tialaramex 5y ago
Seems like you ought to know TLS 1.3 famously doesn't have a way to do RSA key exchange. There's a fun video where some presumably American EDCO people are like, "surely if we all turn up in person we've got more votes so we can get RSA back into TLS 1.3" and er, no, the IETF isn't a democracy. Looking at GCM or any other authenticated encryption without Sum Types makes me sad. If decrypting the message fails, there is no plaintext and I feel like an API should be able to actually express that properly not return a plaintext [even if it's empty] and a flag saying "Don't use the plaintext". :( The "block thing" is a necessary choice if we're going to have authenticated encryption, because authenticating costs us overhead. If we wanted to reliably authenticate individual bytes (so that we can have what naively feels like a stream of bytes) we'd be paying that overhead for each byte. That's pretty outrageous. On the other hand if I'm downloading a 10MB file with my 24kbps modem and none of that was authenticated yet because the authentication only happens once at the end of the encrypted data, I'm going to get very anxious about my download counter correctly showing 0% successfully downloaded for so long. So TLS authenticates whole records.
- tptacek 5y ago
  Yes, the point was how much better 1.3 is than what preceded it. You don't need the TLS Record Layer in order to do authenticated encryption.
  - ekr____ 5y ago
    Agreed. Basically all you need is: 1. Something to delimit the boundaries of the message. 2. Some way to determine the nonce. OTOH, the TLS 1.3 record layer is barely more than authenticated encryption,
ekr____ 5y ago
I know that tptacek knows this, but for those who don't, we took data compression out of TLS 1.3!
syncsynchalt 5y ago
Yep, when digging through the TLS 1.3 RFC (after learning 1.2) the thing that jumped out at me is how many _fewer_ options there are, and how deliberately chosen they seemed to be. - suites (as far as covering cipher + exchange + mac) are gone as a concept - only 5 supported cipher algorithms (all AEAD, and all but one built on AES) - only non-PSK key exchange is (EC)DHE - no compression supported - all the various key derivations are built around a single new primitive function, HKDF I'm sure that given enough time the above options will balloon like we had with latter-day TLS 1.2 but at this point it feels so clean.
- toast0 5y ago
  I'm sure the options will balloon a bit. After all, if someone develops a new good cipher/etc, we'll want to use it in TLS. But it's nice that as of now, there's no bad choices available (other than potentially allowing downgrading).
  - tialaramex 5y ago
    It's entirely possible we will never need to replace AES (or indeed ChaCha20). AES 128 GCM is mandatory to implement, and implementers are recommended to offer ChaCha20 too. If you don't have AES hardware, ChaCha20 looks very attractive and you'd be a fool not to. So while you can get code points for a vanity cipher, there not only isn't an appetite from the working group to endorse such ciphers, there is unlikely to ever be a scenario where a client or server would actually pick it unless you control both and choose to prefer the vanity cipher. Whereupon why bother? The downgrade prevention in TLS 1.3 is pretty effective, so an attack which needs a downgrade also needs to defeat the downgrade protection, an additional hurdle. Obviously people should get rid of TLS 1.0 and TLS 1.1 but meanwhile I think the anti-downgrade story is the best it has ever been.
    - KMag 5y ago
      We may never need to replace AES 256, but Grover's quantum search algorithm can break AES 128 with a work factor of 2^64. AES 256 GCM and a post-quantum authenticated key agreement algorithm may be all we ever need.

profmonocle 5y ago

Learning a protocol by writing a toy client (or toy server) is a blast. It's so satisfying to see a real, production-quality server sending real responses to your little mess.

tptacek 5y ago
You'd probably enjoy work as a software pentester, where the docket --- at least for non-web-applications, which admittedly are the most common project if you don't specialize --- is almost entirely building tooling-grade implementations of random protocols so you can test for vulnerabilities.
- westurner 5y ago
  Here's caddy's go/tls wrapper with e.g. ACME, OCSP stapling: https://github.com/caddyserver/caddy/blob/master/modules/caddytls/tls.go https://github.com/caddyserver/caddy/blob/master/modules/cad... Django-ca also does OCSP and certbot-compatible ACMEv2 w/ known limitations: https://django-ca.readthedocs.io/en/latest/acme.html#known-limitations https://django-ca.readthedocs.io/en/latest/acme.html#known-l... E.g. https://google.github.io/clusterfuzzlite/ https://google.github.io/clusterfuzzlite/ is likely not so great at protocols because that requires testing concurrent and distributed systems and TLAplus, which at least currently can't find side channels FWIU. https://github.com/secfigo/Awesome-Fuzzing#network-protocol-fuzzers https://github.com/secfigo/Awesome-Fuzzing#network-protocol-... OSS-Fuzz runs CloudFuzz[Lite?] for many open source repos and feeds OSV OpenSSF Vulnerability Format: https://github.com/google/osv#current-data-sources https://github.com/google/osv#current-data-sources

tialaramex 5y ago

Hopefully I have actually insightful things to say about this fun toy. However before that > elliptic curve “multiplication”, where n * P means “add P to itself n times” Not very smoothly described but this is all multiplication meant for the natural numbers you learned in primary school too! Why is 7 x 7 = 49? Because if you start with zero and add 7, seven times, you get 49. Try it. This is an important and re-usable insight, it's part of a larger beautiful framework of mathematics and I believe is much better instructed via modern teaching of arithmetic in schools than "rote learning" of times tables did for my parents.

bitwize 5y ago
There are entire worlds of math locked away from you if you don't know your times table by heart. Algebra becomes particularly difficult if you do not know how to not only multiply, but factor, quickly. So yeah, it's great that you know 9 * 6 is 9 added to itself 6 times, but you also have to learn the times table by rote if you want to get anywhere in math.
- tialaramex 5y ago
  > it's great that you know 96 is 9 added to itself 6 times For a while I tried to figure out what the joke was here, and then I realised all that's going on is HN emphasis asterisks Anyway, no, it's well known that mathematicians are often terrible at this sort of "basic arithmetic". There are a bunch of mathematical disciplines where arithmetic is utterly irrelevant, but even in the disciplines where it seems like it ought to be important if you're an actual mathematician you don't care anyway. That's just mechanical stuff, use a calculator, we're busy doing mathematics. It's a nice party trick (but you'll need to go a lot higher than 9 x 6 to impress anybody), and there are mathematicians who are good at arithmetic, but I assure you that by the time I gave up mathematics (just below degree level) there was no use for times tables, which I had never memorised, what was an obstacle was all the bloody integration. Pure, Mechanics, even the Statistics had integration in it by the time I was eighteen.

tedunangst 5y ago

> When the HTTP response is done, we get these bytes: []byte{48, 13, 10, 13, 10, 23}. I don’t know what this is supposed to mean exactly but it seems to signal the end of the connection. HTTP headers include Content-Length, so you should know when you get a truncated response, but TLS is supposed to be more general purpose, so it includes its own crypto secure end of connection indicator.

tialaramex 5y ago
These bytes are an ASCII zero, then two carriage-return/linefeed pairs and then the byte with value 23, so that 0\r\n\r\n might just be HTTP saying we're done sending you this chunked HTTP response. I think that 23 byte is the TLS content type, meaning "This is the actual application data you were transporting over TLS". In TLS 1.3 the original content type byte (from previous TLS versions) just always says 23 regardless so that middle boxes leave us alone, "Huh I guess everything is encrypted application data, I'm just a dumb middlebox so this isn't suspicious" This new, inner content type, safely encrypted and thus invisible to the middleboxes, is at the end of the data because it allows a clever trick with zero padding. See, content type zero isn't a thing. So if we get some data and the last byte is zero, that's not the content type it must be padding, remove it, try again. Still zero? Still padding, remove that and so on. Because the zeroes were encrypted, an adversary can't tell they're padding, but because they're zero the real recipient can discard them safely, yet if we don't need padding we aren't wasting space for a "no padding" indicator, we just don't add any of those zeroes. However, if 23 here is a content type, this means our intrepid TLS implementer got things a little wrong and all their HTTP data has stray 0x17 bytes at the end of each record received. I don't see code to handle that, but I may have missed it. It seems plausible that in terminal output you wouldn't notice?
- userbinator 5y ago
  For those who want to see the details of this explanation expressed in a more... struct-ured fashion, this section of the spec describes that: https://datatracker.ietf.org/doc/html/rfc8446#section-5.2 https://datatracker.ietf.org/doc/html/rfc8446#section-5.2
- 5y ago

drewg123 5y ago

I'm super impressed that they got it to work so quickly. What I hate about crypto is that is either wrong or right, there is no "mostly working" that you can identify easily and use as a debugging aid. This drove me crazy when I was working on kTLS in FreeBSD. When I worked on other features (like getting checksum offload right in NIC firmware) there were easy tricks I could use for debugging, like sending a stream of all zeros. For crypto, it was basically back to first principals and code examination..

toast0 5y ago
TLS 1.3 has some published test vectors where if you understand the inputs (which is tricky) and do the same operations, you'll end up with the same outputs as the test vectors and that should maybe work with a real server too. Still, it helps a ton to get error logs (or step through debugging) from a real server to figure out what you messed up. For every value you send in a TLS handshake, it feels like you've got to send the length three different times, in different numbers of bytes and sometimes adding the bytes it takes to send the length. But, if you can smash through all that, you can get a reasonably working TLS 1.3 client in about a week of fiddling. If your runtime has a decent api to validate certificates, you can call out to that too (the less you personally do with x.509, the better) Edit: link to vectors https://datatracker.ietf.org/doc/rfc8448/ https://datatracker.ietf.org/doc/rfc8448/
- drewg123 5y ago
  Nice, I didn't know that. I did most of the work in ~2017 before TLS 1.3.
- syncsynchalt 5y ago
  Agreed, I second that recommendation for anyone interested in this stuff to write a toy TLS implementation. Here's mine: https://github.com/syncsynchalt/tincan-tls https://github.com/syncsynchalt/tincan-tls
deleted 5y ago
[deleted]

syncsynchalt 5y ago

> an amazing TLS resource: tls13.ulfheim.net Readers, i made a "squee" noise just now. Glad it was useful, this is exactly the sort of thing I wrote it for. <3

toast0 5y ago
It's super nice. The only issue I have is you released it after I had done my TLS 1.3 work, so I couldn't have referred to it when it would have been most useful to me.
reneme 5y ago
We can relate to that! It is extremely helpful for our current efforts to implement TLS 1.3 for the Botan crypto library. Especially to get a foothold with TLS and gain a feeling for the protocol structure. Amazing work! https://github.com/randombit/botan/pull/2922 https://github.com/randombit/botan/pull/2922

silisili 5y ago

I was one of the folks kinda down on the DNS server implementation...but this, this is awesome. TLS is a serious black box to 90%+ of even veteran programmers I meet. To break it down and write a small implementation showing the steps is seriously both impressive and extremely generous. Kudos!

travisgriggs 5y ago

Did anyone else find this line interesting? > This was pretty annoying to get working because I kept passing the wrong arguments to things. I think this sentiment shows up twice in fact in the article. I’m not sure what language the listings are in, but it looked static-type-ish. Are there things the current batch of safety focused languages do to mitigate these types of ordering errors? In Kotlin, Python, and Swift, we make it a practice to always use the keywords in calls if there are multiple arguments of the same type (e.g. two or more ints). I wish I could figure out how to configure any of their linters to enforce it better. Despite its late bound unsafe nature, this was just never an error I made (argument ordering) back when I used to do a lot of Smalltalk with its interwoven keyword syntax.

tyingq 5y ago
>I’m not sure what language the listings are in, but it looked static-type-ish. It's golang. No keyword args.
toast0 5y ago
I assumed passing the wrong argument to things to include wrong streams of bytes to HKDF and what not. There's a lot of keys to generate, and it's easy to get the bits of stuff not quite right.
deleted 5y ago
[deleted]
tialaramex 5y ago
> Are there things the current batch of safety focused languages do to mitigate these types of ordering errors? Rust's NewType idiom can often help here. The author makes a bunch of arrays of bytes, like "Keys" is just all arrays of bytes with different meanings, and so if you mistakenly try to use the ServerPublic key as a ClientHandshake IV the types match up but it can't work. NewType would encourage you to have types named AESKey and InitialisationVector and X25519Public and so on, whereupon the compiler will tell you that your AESKey isn't an InitialisationVector etc. If you use an IDE of some sort, your IDE will probably even prompt you while writing the code, you should put an AESKey here, not an InitialisationVector, something like Intellisense probably even scans your variables for the type and suggests you should either use my_sending_key or my_receiving_key and at that point it does seem like the programmer is only required to pay a modest amount of attention to produce programs which do what they intended. NewType would apply for simple integers too. Rust not only has a type for the abstract concept of file handles (pretty common in high level languages, even C can do this with FILE*) but it has a type for Unix file descriptors, even though of course Unix file descriptors are just integers with some promises about which values they can have. This is a fundamental idea in the language, Rust's str built-in is an array of bytes, but you could write [u8; N] and that's also an array of bytes, however str promises that the array of bytes is definitely UTF8 encoded Unicode text whereas u8 claims no such thing. At runtime they're identical, but in your program they're very different. Rust gets to do this because it promises types don't actually exist at run time. The program you are writing, and which the compiler is compiling, may distinguish a CodeNumber, an ArticleID, a RowNum, a UserIdentifier, and a mere counter, but the machine code output just uses the same say, 64-bit machine register and 8 bytes of RAM to represent all five things interchangeably, the distinction existed only to make your program easier to understand.

userbinator 5y ago

This is certainly something interesting and worth trying, but I'd recommend starting with 1.2 first, because the handshake isn't encrypted so you can follow along more easily. I only looked at the TLS 1.3 RFC for a few small things. As someone who has written implementations of various widely-considered-nontrivial things (mostly image and video codecs, some crypto -- but not TLS, although upon seeing this article, that may change...) for self-educational purposes, I strongly recommend that the official standard be the first thing you look at and refer to during the process. It will clarify a lot of doubts, and in the case of TLS, the RFCs are relatively readable as far as standards go. I’m not sure why the blocks have the size they do (maybe it’s so that each one will fit inside a TCP packet ???), but in theory I think they could be up to 65535 bytes, since their size field is 2 bytes. That's due to either MTU or server-side write fragmentation, and the limit is 16K(+overhead). It is a stream to the application layer, but the record-layer protocol breaks it up into chunks of smaller limited size to allow for the integrity checks. I assume a real TLS implementation would use a thread pool or coroutines or something to manage this. That's a very strange assumption. Depending on the API, the ones I've used will either just read in a loop until the length is fulfilled or an error occurs; or return an "incomplete message" error meaning that the caller is the one to continue reading. Getting a partial read from a TCP socket is a very common "trap for young players" --- if you don't keep it in mind, you'll write code that appears to work on localhost or inside a fast LAN, but fails intermittently and sometimes mysteriously over the Internet or when there's more latency. You can never assume message boundaries when using TCP.

eek2121 5y ago
100% agree on reading the RFC. I've implemented various protocols as well (SMTP, IMAP, POP, FTP, some crypto and other things, all a couple decades ago), and the cases where I glossed over the RFC have always bitten me.
andrewf 5y ago
Getting a partial read from a TCP socket I wish common test frameworks and/or stdlibs made it easier to "fuzz" this. I've written more than one io.Reader implementation (Golang) that breaks up results from an underlying io.Reader according to a random seed.

olliej 5y ago

One thing I like about cryptography, and wish we could get politicians to understand, the mathematical steps to implement many of the core concepts are not hard. Obviously implementing them robustly such that they’re safe at scale is another matter, but “criminals” won’t have any difficulty getting access to secure implementations, and adding any kind of weakness only means that the implementations used by the public are more likely to have bugs, in addition to the legally mandated ones.