7 ms·
The 7F52 has 16MB of cache per core. I'd love to see what is possible with a tiny runtime/OS in the kilobyte size and running a microservice written in a nativ
by com2kid 7y ago
The 7F52 has 16MB of cache per core.
I'd love to see what is possible with a tiny runtime/OS in the kilobyte size and running a microservice written in a native language off of each core, everything out of the L3 cache.
I imagine the throughput would be amazing. Single thread per core, e.g. cooperative multitasking. Do this for stream orientated workflows, or even for processing data that is in reasonable sized chunks, it might be screaming fast!
- bgirard 7y agoI'm not sure with cache aliasing you'd be guaranteed that it would stay all in L3.
- com2kid 7y agoIt's have to be a special setup for sure, a board with no memory even hanging off of it. In theory all you'd need is some tight integration with the NIC (or a few of em!) and tiny driver that knows how to DMA to the NIC. Larger problem of course is that L3 isn't addressable. I wonder how much money a cloud provider would have to pony up to AMD to get an exception to that? :) A huge # of workloads would fit in 16MB with no storage hanging off. Heck I think every microservice I've ever written could manage that. There have been some efforts of late to make a cloud first OS, honestly once you remove everything except for "talk to network controller" and "parse JSON", well so long as you don't mind writing in a language that isn't JS, a few MB is plenty for lots of workloads.
- gpderetta 7y agoApparently there are ways to treat the cache as a static scratch pad. It is used by BIOSes during early startup, but I'm not sure whether the magic incantations are publicly known. It is second hand knowledge though, so it might just be folklore.
- Dylan16807 7y agoIf you allocate one big block it would fit even in a 1-way cache. Modern chips appear to have 16-way L3, so you'd have to be trying to spill.
- ChuckMcM 7y agoThere is an interesting argument to be made for DSP like workloads for these chips. With x8 PCIe 4.0 lanes to carry inter-socket traffic. If I had infinite money to spend I would build a hypercube out of these bad boys. each "atom" having 6 x8 PCIe connectors (up,down,left,right,front,back) with corner nodes dedicating a x16 PCIe port for I/O in and out of the cube. Immerse that bad boy in flourinert and contemplates the deepest secrets of the universe :-)
- BeeOnRope 7y agoThat would just be a cube, right?
- dr_zoidberg 7y agoOP's referring to an old architecture which was called "the hypercube" which involved a couple (I can't remember if 16 32, 64 or what number) of microprocessors connected to one another in a certainly-not-2d topology. Edit: oh a bit of searching and it was in the order of thousands of microprocessors. It was called "Connection Machine", you can read about it in Wikipedia[0]. [0] https://en.wikipedia.org/wiki/Connection_Machine https://en.wikipedia.org/wiki/Connection_Machine
- josh2600 7y agoOne of my favorite essays ever is on the subject of the Connection Machine and Richard Feynman: http://longnow.org/essays/richard-feynman-connection-machine/ http://longnow.org/essays/richard-feynman-connection-machine...
- SkyPuncher 7y agoBesides cryptographic heavy services, I'm having trouble imagining a microservice that would benefit from this. Disk, Network, or in-memory artifact reference would seem to be much bigger bottlenecks.
- deleted 7y ago[deleted]
- com2kid 7y agoI'm thinking of simplicity. No VMs, no containers, just a single binary. The majority of microservices I've written would fit into a system like this just fine. NodeJS is built around the idea that a single thread of execution is enough for a stupidly large % of backend code. On top of that is ton of infrastructure to deploy slews of 100 line JS microservices that each do one thing. So, back to basics. No VM, no disk at runtime, a single statically linked binary running on top of nearly bare metal. Available memory is whatever space the program code doesn't take up. Ignoring l3 being unaddressable, I wonder if UEFI applications are powerful enough to pull something like image deployment off. This is probably all less efficient than the current Russian doll system of VMs and containers in use right now, but it is fun to imagine!
- wmf 7y agoBasically unikernels on the Jailhouse hypervisor.
- londons_explore 7y ago> Ignoring l3 being unaddressable If you set the right config registers, you can map the cache to a linear address space and use it like RAM. I think to do that, you loose access to actual RAM though, so it isn't awfully practical.
- ajflores1604 7y agoI can see this being used in hft houses
- 7y ago
- yjftsjthsd-h 7y agoWhen I was younger, I was very pleased when I stumbled across the trick of putting all of a DOS system on a RAM-disk, which made the system fly. Then I got to college and discovered in my CS classes that CPUs now had multi-megabyte on-CPU cache, which immediately led me to wonder how hard it would be to make a DOS system that ran completely out of cache. I'm not convinced that it would be practical for any number of reasons, but it still blows me away every time I look at what modern CPUs have in-package:)
- keanebean86 7y agoRun windows 95 in L3 cache and use this to store the install: https://github.com/prsyahmi/GpuRamDrive https://github.com/prsyahmi/GpuRamDrive
- jschwartzi 7y agoThis is effectively what you do with some of the smaller MCUs -- you're running out of a megabyte or two of on-die static RAM. This is more to eliminate latency in RTOSes than it is to improve throughput but it's the same idea.
- gpderetta 7y agoMore than eliminating latency I guess is done to eliminate latency variance, which is inherent in the relatively unpredictable behaviour of caches, and is required to characterize worst case scheduling of realtime systems.
- yetihehe 7y agoMegabyte of RAM? That's a very big MCU for most of embedded programmers. 20KB is plenty for most, and some even go to zero RAM (ATtiny11).
- vardump 7y agoWorking with embedded systems. Different MCUs' memory ranges from bytes to gigabytes. What's plenty for one project isn't for another.
- starpilot 7y agokdb
- smabie 7y agoI'm not sure why you are getting downvoted, because kdb+/q would be a perfect candidate. Too bad kOS is no longer still a thing (or maybe it never was).
- deleted 7y ago[deleted]
- lallysingh 7y agoUnikernel as a c++ header would be a good candidate: https://github.com/includeos/IncludeOS https://github.com/includeos/IncludeOS
- Paul-ish 7y agoI learned from an electrical engineer that works on systems for the power grid that running the whole OS in CPU cache is a requirement for some of those systems. I guess that makes sense when you are working with things that move at the speed of electricity.
- person_of_color 7y agoNo, the system is probably using tightly coupled memory; the CPU has single cycle access to all of system memory, effectively making a cache redundant.
- londons_explore 7y agoAnd the reason is likely "we are building something realtime, we can't afford variable latency". For the electricity grid in today's world, that's over-engineering things. The 60Hz electricity grid doesn't really need anything being changed at more than 600Hz, and considering your CPU core runs at 3,000,000,000Hz, you can totally waste a lot of cycles before you start missing deadlines.
- HelloNurse 7y agoEven if your CPU cycle budget is large, designing a realtime system requires proving thet it won't be exceeded, by adding up the worst-case running time of the worst-case sequence of events. Avoiding high latency memory gives a few orders of magnitude of margin, which could be needed to afford good functionality on a reasonably cheap CPU; it's not a mere useless improvement from "fast enough" to "as fast as possible".
- dx034 7y agoAnd yet, so much software on modern CPUs takes seconds rather than milliseconds to perform and output simple calculations. Especially with the mantra "developer time is more expensive than computing power", few developers nowadays have recent experience of optimizing programs for sub-ms reaction times. You could probably make it work with standard hardware. But since developers have to be retrained for that field anyway, why not use hardware that's battle-tested and gives you more leeway? In addition, the advantage of optimizing at every point is that you can use hardware not designed at the limit of what's possible, likely giving you lower error rates.
- jfkebwjsbx 7y agoI don't understand why you would want to run tiny systems for that. Properly configured Linux processes will do the same.
- tomlu 7y agoThis is kind of what the SPUs of the PS3 were like, except with way way less RAM.
- rckoepke 7y agoWould KolibriOS fit the constraints to do something like this? Per the description it "requires only a few megabyte disk space and 8MB of RAM to run." but has a recognizably usable GUI and office suite. https://kolibrios.org/en/ https://kolibrios.org/en/
- rurban 7y agoBetter the original MenuetOS which supports now 64bit, though this is not open source anymore.
- ashtonkem 7y agoFor most general purpose work, you’d probably lose throughout. Chances are you’d rather have the heap be in the cache rather than the code segments.
- lucio 7y ago>640k ought to be enough for anybody
- dx034 7y agokdb+ [1] used to advertise already about a decade ago that their database code fits into L3 cache, making it extremely fast in execution. With 16MB, you can probably fit the core of a pretty much fully featured database in there, accelerating data driven jobs that GPUs are suited for. [1] https://kx.com/ https://kx.com/