3 ms·
Hello Friends, My name is John and I am the author of FrogNet, and I have a favor to ask. I want you to step back and reimagine networking from a fundamental p
by jfawcett 3mo ago
Hello Friends,
My name is John and I am the author of FrogNet, and I have a favor to ask. I want you to step back and reimagine networking from a fundamental point of view. I want you to forget about stateless networks. I want to open your mind to the beauty and power that is the stateful network.
Begin by imagining a network that can stand alone. Not hard, right? The critter projects or supports a WiFi SSID that clients attach to. It may provide DNS, but it certainly provides DHCP. Pretty standard stuff.
Now connect a bunch of them together and make a local internet. Automate discovery and route establishment so nobody has to do anything, and suddenly you have a mobile network that can merge with others at will. Kinda cool, and sorta useful in some scenarios, but it's only the beginning.
Now give these networks the ability to connect to each other across the Internet. The relay point in the middle carries the traffic, so assume somebody owns it — it still holds no endpoint keys, runs none of the semantic stack, forwards in the kernel, and isn't itself a member of any FrogNet. Owning it gets you the transit point and nothing else.
OK, so now we have a self-forming internet that works across the Internet, wherein everybody can find everybody else, DNS works, routing is optimized, and birds sing. Oh, and along the way, optimize it all so discovery and routing don't monopolize the pathways.
Then, really think outside the box. What have you created? In many ways, it looks like a computer.
The network has I/O, storage, and all the other attributes of a computer, but it lacks the essential element — RAM. We can fix that.
Give the network a known name, say databasehost.frognet, and make the way of setting that deterministic so that every node calculates the same address given the same information, every time, and so that the assignment floats as the network splits and merges. The underlying address may change, but the name remains.
Now implement RAM. That means an addressable database that can store data. We chose the Linda tuple space as our model, using a three-element index, and it is there. One data store for everything on a network, and one value per tuple. No synchronization, and data values may be out of date — that's the application's problem, but the query will succeed. One row, last write wins, just like RAM. There is one memory at a time and it holds current state, never history, so a split gives each side its own and a merge produces one memory rather than two to reconcile. What you get is bounded-stale current state, not consensus and not a total ordering. For the few things that genuinely need an order — whose turn it is, who holds a lease, which of two commands won — elect an authority and route those through it. Design for merge, elect for order, and never let the shared picture pretend to be the order.
Now we can move our programming model from the assembly-language REST days to a more C-like time where we can just access memory. The messages, the retries, the serialization, the fallbacks and all the other crap every distributed programming developer has dealt with since forever haven't stopped existing — the conversation still happens underneath. You stopped writing it.
There's a way to tell whether that's real or just renaming things, and you can run it on your own code this afternoon without installing anything of mine. If your code depends on knowing where the data came from, you can't properly mock or simulate it. Removing that dependency is also what let me build a real simulator rather than a test harness — the kind you fly before you fly. It runs the actual discovery, routing and election code and fakes only the environment underneath: the kernel, the wire, the database driver. Hostile topologies, nodes killed mid-run, partitioned meshes, real code throughout.
FrogNet Memory, the network shared memory model I have implemented, made things possible that shouldn't have been. A 1280×720 bidirectional call at 22 fps over a 900 MHz link with a 1 Mbit ceiling, Seattle to New York. My collaborator drove a node trunk-mounted through Queens on that same radio and the mesh re-formed on every walk as the topology changed at traffic speed. On the site I put numbers next to every claim, each with the method attached — including where a figure is an estimate rather than a capture, because a number without a method is decoration. Most of it is on video too.
The horizons this architecture reveals are breathtaking, and I don't wish to do this alone. I haven't open sourced the code yet, and I'll be straight about why: this is infrastructure, it implements some capabilities at a fundamental level, and I'm one person with no organization behind me. I've chosen to be conservative about export control rather than find out the hard way, so I ask that you sign a simple attestation that you're not in a restricted country. It's a form, not an interview. If you think I've read the risk wrong, tell me — I'd rather be corrected than cautious for no reason.
And if you think a piece of this is wrong, argue it here — but even better, replace the algorithm with yours, pass the same oracles, and beat the measured result. Bring the oracle; you don't have to bring the fix. The 56-node pond ceiling is the most legible open problem in the system and it's all arithmetic — it's in the book, with the numbers.
This is my magnum opus, gang, and I invite you to play in the sandbox. Welcome!