4 ms·
Walgit – a Git server that is one binary in front of an object store
- tccole 1mo agoVery cool! I might use this later
- jauntywundrkind 1mo agoConditional Put (often called compare and swap) shipped 21 months ago, and it has just been absolutely wild the massive flocking towards disaggregated storage, towards object stores that has been going on since then. https://news.ycombinator.com/item?id=42240678 https://news.ycombinator.com/item?id=42240678 https://aws.amazon.com/about-aws/whats-new/2024/11/amazon-s3-functionality-conditional-writes/ https://aws.amazon.com/about-aws/whats-new/2024/11/amazon-s3... PicoMQ also showing up today, a Durable Streams implementation on object store (S3). https://news.ycombinator.com/item?id=49421806 https://news.ycombinator.com/item?id=49421806 https://picomq.com/ https://picomq.com/ Celld, a Durable Objects implementation, is also heavily heavily using conditional put, for all manners of coordination. https://news.ycombinator.com/item?id=49185430 https://news.ycombinator.com/item?id=49185430 https://celld.dev https://celld.dev Longstanding SlateDB is built around it too! https://news.ycombinator.com/item?id=41714858 https://news.ycombinator.com/item?id=41714858 https://slatedb.io/ https://slatedb.io/ Yet another fine example here! Data comes in, we make sure we don't overwrite work, we write what we have.
- merb 1mo agoFun thing Google cloud storage supported that for a long time now. Just AWS took its time.
- cavisne 1mo agoThe challenge is not the functionality, it’s doing it cheaply at scale. Google Cloud storage has raised prices a few times, and added various quotas and limits when they realized they couldn’t sustain the service as is.
- unified101 1mo agosource?
- cavisne 1mo agohttps://cloud.google.com/storage/pricing-announce https://cloud.google.com/storage/pricing-announce
- chanux 1mo agoI immediately headed to a search engine to see if Terraform state (lock) on S3 is finally unshackled from a DynamoDB. And it has since Terraform v1.11
- the_alchemist 1mo agoI'm confused by the usage of the word consensus in the explanation, to me it seems like a last-write-wins strategy, no consensus involved?
- ShinyLeftPad 1mo agoJust show us the prompt(s) ;)
- lloydatkinson 1mo agoEven the README… ugh. > no state that matters
- datastru 1mo agoHow did this project get more than 1000 stars in just a few days? Are GitHub and Ycombinator (both company and chatbot-community) just fake and manipulative?
- baalimago 1mo agoI am working on a very similar mcp server which allows agents to communicate and share notes without race conditions, using git and s3 as the driver [0]. Funny how "inventions" like these pop up independently. [0]: slivingdoc.dev
- ksajadi 1mo agolooks very similar to markbase.cloud
- icase 1mo ago[flagged]
- glemmaPaul 1mo agodude is so "based" he wants more voting rights for wealthy people.
- icase 1mo agoleftists: “the wealthy don’t pay taxes!” tobi: “let’s give more votes to people who pay more taxes” leftists: “the wealthy want more votes for themselves!”
- 0123456789ABCDE 1mo agounderlying idea of storing git repositories in object stores is identical to https://github.com/tigrisdata/objgit https://github.com/tigrisdata/objgit see: https://news.ycombinator.com/item?id=48661938 https://news.ycombinator.com/item?id=48661938
- inigyou 1mo agoThat one is upfront that it's vibecoded as an experiment. I can respect that.
- xena 1mo agoI've been thinking about circling back to improve it. Right now one of the major limitations is that it uses a filesystem abstraction as the main bridge from git to object storage. I really wanted to see how far I could go by kitbashing parts together from software I'd already made. Turns out you can go pretty damn far! Turns out this is kind of a horrible idea in practice in ways that only really turn up when you try to square peg -> round hole as hard as I did (pushing gcc.git took multiple hours due to filesystem iops latency, turns out a thing designed for nanoseconds of latency copes poorly with a setup that has milliseconds of latency). I'm probably going to end up using the filesystem as a staging area and then have the git server asynchronously unpack git objects into individual S3 objects. I'm still not sure how to best implement merging or other git operations, but this is just kind of a hard problem in practice.
- inigyou 1mo agoCould you modify Git itself to access several objects at the same time? Possibly not in C, but how about the Go reimplementation since concurrency is quite easy in Go?
- xena 1mo agoI don't know enough about how the relevant libraries and primitives work to give your question the meaningful response it deserves. A lot of the problem here is that most of this implementation requires me to really deeply understand the plumbing of Git in ways I currently don't. This is part of why my prototype was done with heavy assistance from agentic coding tools. I was hoping that those hard parts could be abstracted away for later with load-bearing hacks. It seems that the chickens came home to roost much faster than I expected.
- depr 1mo agoIt's not actually safe to use https://x.com/kellabyte/status/2092105110381908458 https://x.com/kellabyte/status/2092105110381908458
- rvz 1mo ago[flagged]
- hilti 1mo agoAre you aware of the author? He has some experience with Shopify ;-)
- inigyou 1mo agohttps://xcancel.com/kellabyte/status/2092105110381908458 https://xcancel.com/kellabyte/status/2092105110381908458
- sreekanth850 1mo agoHe forgot to add Make no mistakes in his prompt.
- infogulch 1mo agoSounds fixable.
- ekeric13 1mo agosure. but if they missed obvious stuff, there are probably more subtle bugs too. you still have to know what to fix.
- rtcode_io 1mo ago[flagged]
- lionkor 1mo agoI struggle very much to understand the "why". There's a git server, it's installed when you install git, and it really does work very well. > every instance is a disposable cache that revalidates with one conditional GET. No database, no Redis, no gossip, no leader, no node identity. Git famously doesn't use that either...? What is the point? > "just put the repositories on NFS" failed at every large host that tried it Yeah, that's a terrible idea. You can just host Git on a server or two. Where's the limit? How much does it scale to put a single 64 core server with a 10 gbit link on a local network and put ONLY git on it? Is that really that slow and bad? You can give it 20+ TB of RAID storage for VERY cheap. What is the limit you're hitting? 200ms ping to it from across the world? Is that the issue we're solving? > walgit takes that as-is, and adds what a monorepo on small machines needs: serving refs and web pages for a repository whose packs will never fit on the instance (a remote reader over HTTP range requests), keeping commits and trees local while blobs stay in the bucket (the history pack), and moving clone bytes out of the server entirely (bundle-uri: fresh clones and catch-ups are static files the bucket or a CDN hands out). So it's about monorepos that, for some reason, have blobs large enough to be impossible to serve via range requests, shallow clones, etc? Are people committing binary blobs of 1TB+ to their git? There's Git LFS, that hooks up an object store (like S3) to git for large files. As an aside, I kind of adore that Tobi's latest post on his website, from 2019, talks about shopify's carbon emissions, meanwhile today his github is full of AI slop projects and enabling others to "vibe" even harder. He also missed that his clanker wrote an announcement post; https://github.com/tobi/walgit/blob/main/docs/announcing-walgit.md https://github.com/tobi/walgit/blob/main/docs/announcing-wal...
- ljm 1mo agoThe irony is that S3 is a database, so the first assertion that there is no database is fundamentally incorrect.
- brunes 1mo agoYou're debating with AI slop... Tobi didn't write any of this. He probably has never even read it.
- gexla 1mo agoMy understanding was that it was totally an experiment to create something that works like Cursor's Origin or whatever it's called. Just the idea of it is probably not something you would care about for your own usage.
- themgt 1mo agoWe've created Walinux, a Linux that runs as a durable object in S3, so it's diskless, serverless, transistorless, electricity-less, scales to zero and infinity simultaneously, and can be traded as an NFT ape. Mathematicians are dumbfounded. The thirty-five thousand other "S3-backed CLI tool for AI" projects are now obsolete.
- pelagicAustral 1mo agoWhat would be really cool would be to host apps in your own brain, that way we can harness the power of your own heart to run the stack.
- oooyay 1mo agoWhy use my brain when I can rent space in yours? Hell, I could see building the fee structure in such a way that three close proximity humans run the same app just in case one gets hit by a bus.
- pelagicAustral 1mo agoCrazy enough, someone is already working on it https://finalspark.com/ https://finalspark.com/
- ethersteeds 1mo agoThat's a new one for the cyberpunk dystopia bingo card: poor people renting capacity in their brains to multinationals. Makes you stupider, and the DRM implants give you mood disorders, but have you seen how much apartments go for in 2036?!
- nine_k 1mo agoPrototypes have existed for a long time, but, as usual, they guarantee neither correctness nor availability.
- scosman 1mo agoNote this is from tobi, Shopify CEO Amazing he still codes. But likely more experiment than prod level.
- brunes 1mo agoHe didn't write this, some AI did. And a shoddy one at that given the bugs and AI slop readme.
- dezsiszabi 1mo agoWhat is amazing about prompting an AI to write this slop?
- glemmaPaul 1mo ago[flagged]
- zoobab 1mo agoTarballs over FTP was better.
- gravypod 1mo agoIs there a market for an actual GitHub competitor now that everyone is looking around and is very angry about stability?
- inigyou 1mo agoThere are already several but it turns out that network effects dominate all else, so there is not and never will be a market. Just like Sourceforge, nothing will ever replace it until it actually dies. But if you weaken what you want there are several already.
- bluerooibos 1mo agoOh cool, vibe-coded slop from Tobi Lutke, the right-wing fash CEO who wants wealthy folks' votes [1] to carry more weight than the rest of us (the plebs) - no thanks [1] https://www.cbc.ca/news/canada/shopify-ceo-endorses-idea-to-give-more-votes-to-wealthy-canadians-9.7286688 https://www.cbc.ca/news/canada/shopify-ceo-endorses-idea-to-...
- delf 1mo agoYou can use GitSocial CLI to push git repositories straight to S3 without any dependencies: https://gitsocial.org/ https://gitsocial.org/. The buckets then are actual remotes and double as static sites. GitSocial website itself is hosted on Cloudflare R2.
- p-o 1mo agoThis is going to be a meta comment, my apologies. But I now understand the growth in Github's activity that led to its instability recently. Folks just see a project (Cursor's Origin), open up a claude session with a couple harness, and just throw a lot of money, and external energy at generating what will most likely end up being garbage. But this garbage looks and feels good at the onset, and so a few iterations happen, more PRs, more commits until it dies down. In the end, almost nothing of value was created, it was just machine creating stuff for other machines, while humans were mostly spectators of an illusion. Rinse and repeat.
- horsawlarway 1mo agoI don't really see a problem. Prior to this it would have been the millions of throw-away "hello world" intro projects, or "todo app" websites. Hell - think of how many people wrote "facebook clones" back in the day. The author is likely still gaining some skills & experience - although they're more likely project management & ops style skills than technical coding skills (ex - what does interaction with this look like, what's the desired cli surface, how do I configure/deploy it, etc). Intellectual property is pretty much always going to be a case where 99.999% of everything gets thrown away. Because the cost of duplication is zero, so the model trends very hard towards a "winner take all" popularity contest. A very small number of projects will see large success, everything else will likely be gone in a couple years. --- If anything, I quite like this new space. I no longer have to worry about trying to make software "big enough" for other people to use. Previously, it would take a large enough investment that simply scratching my own itch was prohibitively expensive - so I had to be picky about what I would pick up and work on. Now I can bang out all sorts of things designed just for me. I needed a way to limit screen time for kids, and existing apps are both expensive and junk. Cool - local llms wrote me a new home app for my android tvs in an hour. It does exactly what I want, and nothing else. I wanted to point my Home assistant audio satellites at unsloth studio instead of ollama. Cool - local ai in unsloth picked apart the ollama implementation and updated it to work with unsloth. Wanted a new ph monitor for my hydroponics setup, with an auto-doser (some super cheap peristalitic pumps), awesome, an old arduino plus an llm and it runs just fine. etc... I love it. It's simple to get relatively effective results on a scale of 1, for a scale of 1. I don't have to worry about trying to get community help or onboard people, or pitch it as a product, etc... It's mine, it'll last as long as I want. It is trash? Sure - to you (seriously, it'd be hard to run, is explicitly designed for my stuff, has no docs, etc... I'm under no illusions that it's not garbage in the broader context). But it's treasure to me.
- fasdarasa2000 1mo agoPlease camouflage Claude use a little bit better, the README is so cringe it makes it actually hard to understand what the value proposition really is
- kimos 1mo agoHe’s goal isn’t to camouflage. Shopify is full AI maximalism, and Tobi’s goal as the CEO is to prove that he can change the world with AI and that his employees should follow. Up to the reader to believe it or not.
- pajop 1mo agoProposed: https://github.com/tobi/walgit/pull/8 https://github.com/tobi/walgit/pull/8