3 ms·

can anyone tl;dr? what is this, why is it better than sockets etc

js2 7y ago
From the introduction: Network.framework is a modern alternative to sockets. There are three primary areas in which it's very difficult to use sockets well. The first one is Connection Establishment. There's a whole host of reasons that establishing connections can be really difficult with sockets. For starters, sockets connect to addresses, so you have to, most of the time you have a host name, so you're going to have to resolve that host name to an address. When you do that, you often end up with way more than one address. You'll have some IPv4 addresses, some IPv6 addresses, and now you've got this challenge, which address should you try and connect to, in what order? How long do you wait before you try the next one? You can spend years trying to perfect this. I know because we have. Once you get past the dual stack host problems, you run into a whole bunch of other issues. There are some networks that use something called Proxy Automatic Configuration or PAC. On these networks, there's a JavaScript that you get, and you have to pass a URL into the JavaScript, and the JavaScript runs and spits out an answer that says either you can go direct or you have to use this SOCKS proxy over here or that HTTP connect proxy over there. And now your app has to support SOCKS proxies and HTTP connect proxies, and this can be really difficult to do well. And the most difficult thing is that you may not have one of these networks to test on, so you may get a bug report from one of your customers, and they may complain that it's not working well on their environment. And you may want to add code to fix the problem, but that once you've got it in there, you really don't have a good way to test it. You have to end up building the whole environment to reproduce the same environment they have. It can be a real challenge. So connecting with sockets is really hard. The second thing that becomes challenges with sockets is data transfers. There's a lot of reasons that transferring data with sockets can be really difficult. The primary problem is the read and write model itself. If you're using blocking sockets, it pretty simple, but you're tying up a thread, and it's really not a great idea to be tying up a thread while you're waiting to read or write data. You can switch to nonblocking, but then you end up with a whole lot of other challenges that you run into. When you're using nonblocking, you may tell the kernel I'd like 100 bytes, and the kernel will come back and say, I've got 10 bytes for you, why don't you come back later. And now you have to build a state machine to keep track of how many bytes you read versus how many bytes you want to read. This can be a lot of work, and getting it to perform well can be a real challenge. On top of all of that, you really shouldn't be reading and writing to sockets directly because you should be using something like transport layer security or TLS. Sockets don't support TLS, so you're probably using some other library that is handling TLS for you and reading and writing to the sockets on your behalf, or you're writing the glue code between that library and sockets, and you have to figure out how to get all this to work with all the crazy connection logic that you put in ahead of time. There's a lot here that can be really difficult. Finally, mobility can be really challenges with sockets for a variety of reasons. Today, we have these incredibly powerful devices in our pocket with multiple radios that may be on at the same time, and some of them are moving from network to network, and your application has to handle all these transitions well to provide a seamless experience to your customers. Sockets does nothing to help you with this. You can use routing sockets, but it's really, really difficult. We think a transport API should do better. Fortunately, on our platform as an application developer you have a great API in URLSession. URLSession handles all of these problems for you. It's really focused on HTTP, but it also has stream task that gives you raw access to TCP and TLS connections. You might think that URLSession is built on the same primitives that you would use yourself. But it turns out that's not the case. URLSession is built on top of something we call Network.framework. URLSession really focuses on all of the HTTP bits, and it offloads a lot of the transport functionality to Network.framework. Network.framework is something we've been working on for a number of years, and in supporting URLSession, we've learned a lot, and we've taken a lot of those lessons to the IETF. A number of our engineers regularly participate in the IETF and meet with engineers from other companies, and they've been discussing a lot of what we've learned in the transport services working group. And in those discussions, we've got some great feedback, and we've brought that back in and improved Network.framework based on that. We are really excited to announce this year that your applications can take advantage of this same library directly now. Now we know that one of the things people love about sockets is that it gives them very fine-grain control over just about everything, and they're really loathe to give that up. So as we developed Network.framework, we wanted to make sure that it did the right thing by default in the way that sockets doesn't, but it gave you all the knobs that sockets does. And it's kind of got this gradient, so the more knobs you turn, the more complex it becomes. It gives you all the power you need, but you don't have to pay for the complexity unless you actually need some of it. Network.framework has incredibly smart connection establishment. It handles the dual stack cases. It handles IPv6 only networks. It handles PAC. It handles proxies. It will help you connect on networks that are otherwise very difficult to deal with. It has an incredibly optimized data transfer path that lets you go way beyond the performance of anything you can do with sockets, and Tommy will cover that in a little bit. It has support for built-in security. It supports TLS and DTLS by default. It's really simple to use. It has great support for mobility. It provides notifications about network changes that are relevant to the connections that your application is establishing. It's available on iOS, macOS, and tvOS as a CAPI with automatic reference counting, so it's easy to use from Objective C, and it has an incredible Swift API.
- cocktailpeanuts 7y ago
  he asked for a TLDR, i'm sure he can read too.
  - deepakkarki 7y ago
    It is an hour long talk (which I wouldn't have watched). I found js2's explanation very clear and helpful. Please don't be outright dismissive like that.
- swiley 7y ago
  The dual stack thing is fixed with getaddrinfo. You’re probably speaking a protocol, I personally like to use regular languages with a reset byte (or octet or whatever.) at the beginning of every message. Since the data just goes into a state machine you can feed it partial messages and it will reconstruct them. This takes very little code, very little thought (just use Kleene’s algorithm) is extremely portable (unlike this which is exclusively apple) and very easy to mock and test. Yes that still doesn’t take care of SOCKS but that (IMO) is heading into the “give the user an option to connect using pipes and netcat” territory (and that’s probably the best way to fix it.)
  - TheTon 7y ago
    getaddrinfo isn’t a complete solution itself. This RFC describes what they’re doing: https://tools.ietf.org/rfc/rfc8305.txt https://tools.ietf.org/rfc/rfc8305.txt
  - aseipp 7y ago
    It's always hilarious to read the ever-puzzling reply that you see here, which is basically "Why don't you just manually re-build this design into every single application you've ever written, repeating yourself and scope for bugs 50x over, while I also ignore all other possible design constraints", and it never ceases to amaze me. The idea that a better abstraction for a component could exist or should be explored is strictly verboten.
    - deleted 7y ago

js2 7y ago

From the introduction: Network.framework is a modern alternative to sockets. There are three primary areas in which it's very difficult to use sockets well. The first one is Connection Establishment. There's a whole host of reasons that establishing connections can be really difficult with sockets. For starters, sockets connect to addresses, so you have to, most of the time you have a host name, so you're going to have to resolve that host name to an address. When you do that, you often end up with way more than one address. You'll have some IPv4 addresses, some IPv6 addresses, and now you've got this challenge, which address should you try and connect to, in what order? How long do you wait before you try the next one? You can spend years trying to perfect this. I know because we have. Once you get past the dual stack host problems, you run into a whole bunch of other issues. There are some networks that use something called Proxy Automatic Configuration or PAC. On these networks, there's a JavaScript that you get, and you have to pass a URL into the JavaScript, and the JavaScript runs and spits out an answer that says either you can go direct or you have to use this SOCKS proxy over here or that HTTP connect proxy over there. And now your app has to support SOCKS proxies and HTTP connect proxies, and this can be really difficult to do well. And the most difficult thing is that you may not have one of these networks to test on, so you may get a bug report from one of your customers, and they may complain that it's not working well on their environment. And you may want to add code to fix the problem, but that once you've got it in there, you really don't have a good way to test it. You have to end up building the whole environment to reproduce the same environment they have. It can be a real challenge. So connecting with sockets is really hard. The second thing that becomes challenges with sockets is data transfers. There's a lot of reasons that transferring data with sockets can be really difficult. The primary problem is the read and write model itself. If you're using blocking sockets, it pretty simple, but you're tying up a thread, and it's really not a great idea to be tying up a thread while you're waiting to read or write data. You can switch to nonblocking, but then you end up with a whole lot of other challenges that you run into. When you're using nonblocking, you may tell the kernel I'd like 100 bytes, and the kernel will come back and say, I've got 10 bytes for you, why don't you come back later. And now you have to build a state machine to keep track of how many bytes you read versus how many bytes you want to read. This can be a lot of work, and getting it to perform well can be a real challenge. On top of all of that, you really shouldn't be reading and writing to sockets directly because you should be using something like transport layer security or TLS. Sockets don't support TLS, so you're probably using some other library that is handling TLS for you and reading and writing to the sockets on your behalf, or you're writing the glue code between that library and sockets, and you have to figure out how to get all this to work with all the crazy connection logic that you put in ahead of time. There's a lot here that can be really difficult. Finally, mobility can be really challenges with sockets for a variety of reasons. Today, we have these incredibly powerful devices in our pocket with multiple radios that may be on at the same time, and some of them are moving from network to network, and your application has to handle all these transitions well to provide a seamless experience to your customers. Sockets does nothing to help you with this. You can use routing sockets, but it's really, really difficult. We think a transport API should do better. Fortunately, on our platform as an application developer you have a great API in URLSession. URLSession handles all of these problems for you. It's really focused on HTTP, but it also has stream task that gives you raw access to TCP and TLS connections. You might think that URLSession is built on the same primitives that you would use yourself. But it turns out that's not the case. URLSession is built on top of something we call Network.framework. URLSession really focuses on all of the HTTP bits, and it offloads a lot of the transport functionality to Network.framework. Network.framework is something we've been working on for a number of years, and in supporting URLSession, we've learned a lot, and we've taken a lot of those lessons to the IETF. A number of our engineers regularly participate in the IETF and meet with engineers from other companies, and they've been discussing a lot of what we've learned in the transport services working group. And in those discussions, we've got some great feedback, and we've brought that back in and improved Network.framework based on that. We are really excited to announce this year that your applications can take advantage of this same library directly now. Now we know that one of the things people love about sockets is that it gives them very fine-grain control over just about everything, and they're really loathe to give that up. So as we developed Network.framework, we wanted to make sure that it did the right thing by default in the way that sockets doesn't, but it gave you all the knobs that sockets does. And it's kind of got this gradient, so the more knobs you turn, the more complex it becomes. It gives you all the power you need, but you don't have to pay for the complexity unless you actually need some of it. Network.framework has incredibly smart connection establishment. It handles the dual stack cases. It handles IPv6 only networks. It handles PAC. It handles proxies. It will help you connect on networks that are otherwise very difficult to deal with. It has an incredibly optimized data transfer path that lets you go way beyond the performance of anything you can do with sockets, and Tommy will cover that in a little bit. It has support for built-in security. It supports TLS and DTLS by default. It's really simple to use. It has great support for mobility. It provides notifications about network changes that are relevant to the connections that your application is establishing. It's available on iOS, macOS, and tvOS as a CAPI with automatic reference counting, so it's easy to use from Objective C, and it has an incredible Swift API.

cocktailpeanuts 7y ago
he asked for a TLDR, i'm sure he can read too.
- deepakkarki 7y ago
  It is an hour long talk (which I wouldn't have watched). I found js2's explanation very clear and helpful. Please don't be outright dismissive like that.
swiley 7y ago
The dual stack thing is fixed with getaddrinfo. You’re probably speaking a protocol, I personally like to use regular languages with a reset byte (or octet or whatever.) at the beginning of every message. Since the data just goes into a state machine you can feed it partial messages and it will reconstruct them. This takes very little code, very little thought (just use Kleene’s algorithm) is extremely portable (unlike this which is exclusively apple) and very easy to mock and test. Yes that still doesn’t take care of SOCKS but that (IMO) is heading into the “give the user an option to connect using pipes and netcat” territory (and that’s probably the best way to fix it.)
- TheTon 7y ago
  getaddrinfo isn’t a complete solution itself. This RFC describes what they’re doing: https://tools.ietf.org/rfc/rfc8305.txt https://tools.ietf.org/rfc/rfc8305.txt
- aseipp 7y ago
  It's always hilarious to read the ever-puzzling reply that you see here, which is basically "Why don't you just manually re-build this design into every single application you've ever written, repeating yourself and scope for bugs 50x over, while I also ignore all other possible design constraints", and it never ceases to amaze me. The idea that a better abstraction for a component could exist or should be explored is strictly verboten.
  - deleted 7y ago