kentonv

46 ms·

721.
by kentonv 3y ago
Are you saying that each service would need to declare, separately, the subset of fields they operate on, and make sure that those fields are always a strict subset of the overall set of fields the protocol contains? This essentially means
722.
by kentonv 3y ago
The particular piece of infrastructure I worked on sat in the middle of the search pipeline, between the front-end that served HTML web pages, and the back-end index. This middle piece would request search results from the back-end, tweak t
723.
by kentonv 3y ago
Can confirm, ocdtrekkie is one of Sandstorm's most active maintainers over the last few years.
724.
by kentonv 3y ago
They are all projects I started. But other than who worked on them, and sharing some technology choices under the hood, there's mostly no relationship between Workers and Sandstorm.
725.
by kentonv 3y ago
> I think you're defining consumer as the literal line of code where the field is read I am. > After all it's usually better to abort early than half way through an operation. I realize this goes against common wisdom, but I
726.
by kentonv 3y ago
In the actual implementation, when making a pipelined call on a result X, you actually say something like "Call X.foo.bar.baz()" -- that is, you can specify a nested property of the results which names the object that you actually
727.
by kentonv 3y ago
While it's true that load-balancing long-lived resources is harder than short-lived, a lot of the difficulty of load balancing with stateful servers is actually in the protocol, because you somehow have to make sure subsequent requests
728.
by kentonv 3y ago
If you're thinking strictly about stateless backends that just convert every request into a SQL query, then yeah, promise pipelining might not be very helpful. I think where it shines is when interacting with stateful services. I think
729.
by kentonv 3y ago
If your goal is to serialize an arbitrary Python object, Pickle is the way to go. Cap'n Proto requires you to define a schema, in Cap'n Proto schema language, for whatever you wan to serialize. It can't just take an arbitrary
730.
by kentonv 3y ago
> The argument is actually more like: don't use badly written middleman software that tries to parse messages it doesn't need to parse. The middleman software in question often needed to process some part of the message but not
731.
by kentonv 3y ago
The main thing that Cap'n Proto PRC really requires about the serialization is that object references are a first-class type. That is, when you make an RPC, the parameters or results can contain references to new, remote RPC objects. U
732.
by kentonv 3y ago
This really hasn't been fleshed out at all, it's more like: "Well, we're built on Cap'n Proto, it'd be really easy to expose it for applications to use. But is it useful?" Arguably Cap'n Proto RPC mig
733.
by kentonv 3y ago
> Exchanging schemas at the beginning of a connection handshake is hardly burdensome. I dunno, that sounds extremely burdensome to me, especially if the actual payload is small. And how exactly does exchanging schemas solve the problem?
734.
by kentonv 3y ago
Capability, not capacity. https://en.wikipedia.org/wiki/Capability-based_security https://capnproto.org/rpc.html#distributed-objects The idea really goes way beyond security and RPC. It's hard to
735.
by kentonv 3y ago
Yes, if I were starting from scratch today I'd use Rust. Unfortunately it was a little too early when work on Workers started.
736.
by kentonv 3y ago
Yes, he was the most active developer over the last few years, although it wasn't a huge amount of activity. And that activity had dropped off this year as Ian shifted his focus to Tempest, a mostly-from-scratch rewrite. https:/&
737.
by kentonv 3y ago
Yes, it's true, the C++ implementation has become extremely opinionated. I didn't initially intend for KJ to become as all-encompassing as it has. I guess I kept running into things that didn't work well about the standard li
738.
by kentonv 3y ago
The argument is really more like: Always defer validation until the point where the data is actually consumed, because only the consumer actually knows what is valid. Which is definitely a counterpoint to the oft-stated argument that you sh
739.
by kentonv 3y ago
Of course, nothing is ever "solved". :) I assume you are talking about the cancellation change. This is interesting, actually. When originally designing Cap'n Proto, I was convinced by a capabilities expert I talked to that c
740.
by kentonv 3y ago
What part of the industry are you in where flatbuffers is seen as the de facto standard? Personally I've never randomly encountered a project using flatbuffers. I see protobuf all the time. (I've randomly run into Cap'n Proto
741.
by kentonv 3y ago
I do agree that the API required for zero-copy turns out a bit awkward, particularly on the writing side. The reading side doesn't look much different. Meanwhile zero-copy is really only a paradigm shift in certain scenarios, like when
742.
by kentonv 3y ago
Huh, that reference actually never occurred to me. The name Cap'n Proto actually originally meant "Capabilities and Protobufs" -- it was a capability-based RPC protocol based on Protocol Buffers. However, early on I decided I
743.
by kentonv 3y ago
Here's a blog post about Cloudflare's use of Cap'n Proto in 2014, three years before I joined: https://blog.cloudflare.com/introducing-lua-capnproto-better... To this day, Cloudflare's data pipeline (whi
744.
by kentonv 3y ago
Sadly, the person leading that participation, Ian "zenhack" Denhardt, recently and unexpectedly passed away. For my part, I'm a fan of OCapN, but I am not sure how much time I can personally commit to it, with everything on m
745.
by kentonv 3y ago
This is indeed the main weakness of Cap'n Proto. I only really maintain the C++ implementation. Other implementations come from various contributors which can lead to varying levels of completeness and quality. Unfortunately I can&#x27
746.
Cap'n Proto 1.0(capnproto.org)
720 points by kentonv 3y ago|218 comments
747.
by kentonv 3y ago
Workers KV is actually largely implemented as a Worker, in TypeScript. :) Think of `workerd` as the "kernel" of our "cloud operating system". We try to avoid putting things in the kernel if we can. Features like Workers
748.
by kentonv 3y ago
One of the benefits of our new storage engine[0] is that it'll be much easier for us to host it in any datacenter, rather than just the biggest, best-connected ones. We still have a lot of work to do to make this available to all durab
749.
by kentonv 3y ago
Yes, but outside of Java developers a lot of people didn't understand the power of this yet.
750.
by kentonv 3y ago
I remember this post from the time and I'm glad we've come so far since then. It turns out the reason static typing seemed like a pain at the time is because we didn't have good tools. You'd write code for a while, then

More ›