7 ms·
For all the talk of needing all the cloud infra to run even a simple website, Marginalia hits the frontpage of HN and we can't even bring a single PC sitting in
by dizhn 4y ago
For all the talk of needing all the cloud infra to run even a simple website, Marginalia hits the frontpage of HN and we can't even bring a single PC sitting in some guy's living room to its knees.
- marginalia_nu 4y agohttps://www.marginalia.nu/junk/just-a-fleshwound.webp https://www.marginalia.nu/junk/just-a-fleshwound.webp If anything it's running faster now. All you've done is warm up the caches and given the JVM a chance to optimize the hottest code. (real talk the SSDs are running pretty near 100% utilization though)
- mvdwoord 4y agoVery well done! On mobile now but will check out the site once home. Do you happen to have a writeup somewhere of your tech stack?
- btown 4y agoNot OP but https://github.com/MarginaliaSearch/MarginaliaSearch/blob/master/code/readme.md https://github.com/MarginaliaSearch/MarginaliaSearch/blob/ma... has a diagram and component overview!
- marginalia_nu 4y agoI'm working on the documentation. It's getting there, but it's still kinda thin in many places.
- btown 4y agoIMO it's actually incredibly well-documented and thoughtfully organized for a one-person project! You should be proud of what you've put together here!
- marginalia_nu 4y agoYeah I did a huge refactoring effort very recently. I put a lot of effort in making the code easy to poke around in and I feel that works very well. But besides that, there's still a lot left to be desired when it comes to how it actually works. Not everything is easy to glean from the code alond.
- marginalia_nu 4y agoIt's a Debian server running nginx into a bunch of custom java services that use the spark microframework[1]. I use a MariaDB server for link data, and I've built a bespoke index in Java. [1] https://sparkjava.com/ https://sparkjava.com/ I don't use springboot or anything like that, besides Spark I'm not using frameworks.
- dizhn 4y agoI love this!! :) How much does 24 real cores + 126GB ram cost on the cloud? A million dollars?
- marginalia_nu 4y agoYeah I wouldn't try to run this in the cloud. Would be broke as a joke in a week. This is $5000 worth of consumer hardware, give or take.
- TremendousJudge 4y agonice
- alex_sf 4y agoMore than the cores and RAM, you have bigger issues with I/O (both throughput and latency) to disk and the network from cloud providers. Physical hardware, even when comparing cores/RAM 1:1, is outrageously faster than cloud services.
- nine_k 4y agoDon't use EBS, use the local SSD which are an option for most cloud VMs.
- deleted 4y ago[deleted]
- Nextgrid 4y agoProblem is that those volumes are ephemeral and may not provide the reliability guarantees that the EBS volumes do, so they're only really good as cache and not for any persistent data you care about.
- amluto 4y agoI would argue that a rack-mounted chassis with lots of disks is also ephemeral, just less so: most failures in a server can be fixed by swapping some parts, and your data is still there. But AWS and its competitors don’t have an offering even close to comparable to what you can get in a commodity server. A 1U server with one or two CPU sockets and 8-12 hot-swap NVMe bays is easy to buy and not terribly expensive, and you can easily fill it with 100+ TiB of storage with several hundred Gbps of storage bandwidth and more IOPS than you are likely able to use. EC2 has no comparable offering at any price. (A Micron 9400 drive supposedly has 7GBps / 56Gbps of usable bandwidth. 10 of them gives 560Gbps, and a modern machine with lots of PCIe 5.0 lanes may actually be able to use a lot of this. As far as I can tell, you literally cannot pay AWS for anywhere near this much bandwidth, but you can buy it for $20k or so.)
- deleted 4y ago[deleted]
- adrian_mrd 4y agoKudos. [0] is the best URL I have come across in the past few years [0] https://www.marginalia.nu/junk/just-a-fleshwound.webp https://www.marginalia.nu/junk/just-a-fleshwound.webp
- phendrenad2 4y ago"/junk/just-a-fleshwound.webp" for those on mobile
- gary_0 4y agoI'm curious about your network bandwidth/load. You only serve text, right? [Edit: No, I see thumbnail images too!] Is the box in a datacenter? If not, what kind of Internet connection does it have?
- marginalia_nu 4y agoAverage load today has at worst been about 300 Kb/s TX, 200 Kb/s RX. I've got a 1000/100 mbit/s down/up connection. Seems to be holding without much trouble. Most pages with images do lazy loading so I'm not hit with 30 images all at once. They're also webp and cached via cloudflare, softens the blow quite a lot.
- winrid 4y agoIf the SSDs were really maxed out you'd see high CPU usage/load as the CPU as blocked by IOWAIT.
- marginalia_nu 4y agoNot in this case, it's all mmap.
- winrid 4y agoReally? Even with mmapp'ed memory won't the CPU still register user code waiting on reading pages from disk as iowait? I'm so surprised by that that if it doesn't it sounds like a bug.
- marginalia_nu 4y agoYeah it's at least what I've been seeing. Although it could alternatively be that a lot of the I/O activity is predictive reads, and the threads don't actually stall on page faults all I/O that often.
- rglover 4y agoSemi-related: do you just run a static IP out of your house?
- marginalia_nu 4y agoYup.
- yonrg 4y ago239 days up. That's brave too ;)
- marginalia_nu 4y agoRebooting is like a hour of downtime :-/ FWIW I'm going commando with no ECC ram too.
- rks404 4y agoLove this new definition of ‘going commando’ lol
- nine_k 4y agoI remember there is ksplice or something like that to upgrade even the kernel without a complete downtime. Everything else can be upgraded piecemeal, provided that worker processes can be restarted without downtime.
- Nextgrid 4y agoIf the hardware itself is the reason for the long startup time, kexec allows you to boot a new kernel from within an existing one and avoids the firmware/HW init.
- deleted 4y ago[deleted]
- deleted 4y ago[deleted]
- alex_sf 4y agoStackOverflow still just runs on a pair of (beefy) SQL servers. Modern web engineering is a joke.
- dizhn 4y agoI belive hackernews itself is a couple of servers too.
- winrid 4y agoOne big server with a flat text file DB on NVME drives AFAIK.
- amiga-workbench 4y agoI believe the application code is single threaded due to its interpreter too.
- TremendousJudge 4y agoit does run pretty slow under load though, and they have acknowledged it's due to this
- marginalia_nu 4y agoHN has mutable data though. That's a much harder problem than indexing a large amount of static data like a search engine.
- winrid 4y agoNot only that, but I don't think the custom datastore handles concurrent writes. One thread FTW. :)
- hedora 4y agoWikipedia too.
- MichaelZuo 4y agoI doubt anyone would be foolish enough to claim that the site NEEDS 'cloud infra' to run.
- marginalia_nu 4y agoI think it sort of depends on what you want. Every time I deploy a service it goes down for anything between 30 seconds and 5 minutes. When I switch indices, the entire search engine is down for a day or more. Since the entire project is essentially non-commercial, I think this is fine. I don't need five nines. If reliability was extremely important, scales would tilt differently, maybe cloud would be a good option. A lot of it is for CYA's sake as well. If I mess up with my server, that's both my problem and my responsibility. If a cloud provider messes up, then that's a SLA violation and maybe damages are due.
- nine_k 4y agoIt depends. Both https://google.com https://google.com and, say, https://www.medusa.priv.at/ https://www.medusa.priv.at/ are technically web sites, but the complexity of the tech that makes them work is pretty different.
- HarHarVeryFunny 4y agoThat is pretty impressive - not only not on its knees, but very responsive atm.
- deleted 4y ago[deleted]
- nine_k 4y ago24 cores, 128 GB RAM. One could run 10-20 EC2 instances to utilize a box like this, and produce an impression of sprawling backend infrastructure.
- bayindirh 4y agoFrom my experience simple systems perform better on average due to less number of interconnected gears. Much more complex systems do not perform as consistently as simple ones, and they are exponentially harder to debug, introspect and optimize at the end of the day.