4 ms·

Oh wow. I did not expect this to hit hackernews considering the slides are totally not intended to be consumed without notes. Let me just quickly point out a

Oh wow. I did not expect this to hit hackernews considering the slides are totally not intended to be consumed without notes. Let me just quickly point out a few things before I write up something longer about this thing. First things first: > I'd like to add that maybe Fireteam should have not chosen HTTP as a transfer protocol for games, where state is everywhere and HTTP is stateless. Although I don't feel qualified to suggest alternatives. We have very good reasons for supporting HTTP but it's not the only protocol we're implementing. As pointed out on one slide: HTTP is better supported than raw socket connections on certain devices we care about. > Trying to over-abstract network communication is one of the classic mistakes in distributed system design. We're not doing that. If you look at the slides we're 100% HTTP compliant, we don't make some fancy schmancy protocol on that just uses HTTP as a transport layer but does not embrace it. I have the "pleasure" to use SOAP in the past and this is not what we're after. We're basically treating HTTP as an implementation detail for JSON/urlencoded etc. REST. and we do that by sprinkling a ton of meta information around in the code. Once the request was dispatched to the function it's returned to the valuable information. One layer higher all the HTTP logic happens. The general workflow however is fundamentally different from what I did in the past with Flask or how Django works which also explains the title. I will write something up because I think from the slides alone you get a completely wrong impression about why this is cool :-)

trentonstrong 15y ago
Interesting! This seems like something that might work well with 3.0's function annotation support with the ability to specify that the handler's input argument conform to either a the streaming or buffered interface. As a point of curiosity about best practices, how are you handling the meta information right now? It feels like decorators provide the cleanest solution currently, but I've always wondered if there are alternatives.
- the_mitsuhiko 15y ago
  > Interesting! This seems like something that might work well with 3.0's function annotation support with the ability to specify that the handler's input argument conform to either a the streaming or buffered interface. I thought so, but the annotations in Python 3 are largely useless. They would work if you could forward the signatures through a chain of decorators, but ther is no guarantee for that. I did play around with Python 3 but it's actually not in any way better for what we're doing here. > It feels like decorators provide the cleanest solution currently, but I've always wondered if there are alternatives. Decorators require a routing system that can resolve routing "dependencies". We're using the Werkzeug routing which was designed to do that. The basic idea is that it does not matter in which order you define the URL rules, the routing system figures out the ordering. If you don't have that, you would have to move the definitions to a central file (akin to urls.py in Django) to make this work because you need to define the ordering. We also experimented with having the meta information in JSON files but it was not really worth it. Having it next to the function makes it easier to maintain it.
chipsy 15y ago
Although this wasn't the entire message of the original - I'm intrigued by the general idea of committing to a line-in-the-sand approach on streaming vs. buffering. I've been writing a fair number of file format parsers lately for various audio data, and one of the basic architecture questions that comes up is whether to go for the streamed implementation, when the format allows it, but my use case clearly doesn't need it (and probably won't be subjected to production-grade tests at any time in the near future). Streaming is pretty much universally the more complex option - and when you use a streamed implementation in a buffered fashion, it's wasted engineering effort. Treating data as just data, with the associated ability to copy it around, will always be simpler than maintaining protocol state. So after reading this and reflecting on my own process, I've got a more solid feeling about how to make that call: Support streaming if a certain application demands it. Otherwise, do the simple thing. Do not try to entangle both implementations together, because they have different intentions. Write both independently and refactor the reusable parts instead.

trentonstrong 15y ago

Interesting! This seems like something that might work well with 3.0's function annotation support with the ability to specify that the handler's input argument conform to either a the streaming or buffered interface. As a point of curiosity about best practices, how are you handling the meta information right now? It feels like decorators provide the cleanest solution currently, but I've always wondered if there are alternatives.

the_mitsuhiko 15y ago
> Interesting! This seems like something that might work well with 3.0's function annotation support with the ability to specify that the handler's input argument conform to either a the streaming or buffered interface. I thought so, but the annotations in Python 3 are largely useless. They would work if you could forward the signatures through a chain of decorators, but ther is no guarantee for that. I did play around with Python 3 but it's actually not in any way better for what we're doing here. > It feels like decorators provide the cleanest solution currently, but I've always wondered if there are alternatives. Decorators require a routing system that can resolve routing "dependencies". We're using the Werkzeug routing which was designed to do that. The basic idea is that it does not matter in which order you define the URL rules, the routing system figures out the ordering. If you don't have that, you would have to move the definitions to a central file (akin to urls.py in Django) to make this work because you need to define the ordering. We also experimented with having the meta information in JSON files but it was not really worth it. Having it next to the function makes it easier to maintain it.

chipsy 15y ago

Although this wasn't the entire message of the original - I'm intrigued by the general idea of committing to a line-in-the-sand approach on streaming vs. buffering. I've been writing a fair number of file format parsers lately for various audio data, and one of the basic architecture questions that comes up is whether to go for the streamed implementation, when the format allows it, but my use case clearly doesn't need it (and probably won't be subjected to production-grade tests at any time in the near future). Streaming is pretty much universally the more complex option - and when you use a streamed implementation in a buffered fashion, it's wasted engineering effort. Treating data as just data, with the associated ability to copy it around, will always be simpler than maintaining protocol state. So after reading this and reflecting on my own process, I've got a more solid feeling about how to make that call: Support streaming if a certain application demands it. Otherwise, do the simple thing. Do not try to entangle both implementations together, because they have different intentions. Write both independently and refactor the reusable parts instead.