9 ms·
Piku: Allows git push deployments to your own servers
- lakomen 2y agoThis, but on a per user basis would be great. But uwsgi performance overhead is a concern. Altough last time I've done anything with uwsgi that was probably over a decade ago. And last time I checked, Go required to import the uwsgi package, maybe not anymore? Or is uwsgi used only for Python here? Also I wonder how to define nginx routes aka locations?
- rcarmo 2y agoVery happy to see this here - check out our freshly revamped docs at https://piku.github.io/ https://piku.github.io/
- stavros 2y agoIs this the successor to Dokku? I didn't know you had a second project.
- rcarmo 2y agoNope, just took inspiration from it because I couldn’t run Docker on some of my targets.
- ewalk153 2y agoGreat to see the updated docs.
- pixelmonkey 2y agoThe new docs look great!
- stavros 2y agoThis looks good, and Dokku has been very solid for me, but removing the Docker dependency means that now I'm beholden to my OS's choices. For apps that might run for years without maintenance, that's not ideal, as you'll quickly find you need a specific version of the OS for them.
- rcarmo 2y agoYou can use docker with it - I have a couple of things with "docker run" statements in the procfile, but of course it’s not designed for that. Most of the deployments I got wind of are on extremely stable distros - typically LTS versions where you will not need to upgrade your runtime every six months (and my website has been running in it for at least two Ubuntu LTS releases…) But you can trivially use pyenv/nvenv/etc. by just setting environment variables. My home automation system now needs two different Node versions, and I have one set per app.
- stavros 2y agoOh yes, I definitely use LTS distros, but my longest-running apps are from 2008, so even LTS won't cover that.
- lathiat 2y agoAt -some- point you actually need to update things. If you're using a 2008 docker container you have all manner of bugs and security issues.
- szundi 2y agoBut at least the attack vectors are limited
- vault 2y agoyes, limited to those that work 100%!
- poincaredisk 2y agoI agree. But 2008 is old enough that exploits may be lost in time. I recently pentested a client, and had great problems connecting to an old service still using SSL (i think it was 2.0). Every modern tool straight up refused to connect, there was no method to override that, oldest static curl binaries i could find were still too new, I couldn't easily compile curl from source because dependencies also changed in the meantime. Finally I've found sn acient docker image that worked. The service was ironically so old that no modern vulnerability scanner or programming language would be able to connect! This made me seeiously ponder the fleeting nature of modern world - SSL support was everywhere 15 years ago, and now I, as an expert, had great problems using it. What chance do we have in 100 years?
- mati365 2y agoIsn't it better to create local docker repository and then use Podman Quadlet with autopull images to run apps?
- pacifika 2y agoBetter in what way?
- mati365 2y ago1. Less dependencies (only Podman and registry is needed) 2. Rock solid rootless systemd service management 3. Easy integration with systemd-proxyd 4. Easy manage dependencies between containers (with healthchecks) 5. Rollbacks
- GordonS 2y agoSounds interesting! Is there any support for multi-node systems? Let's say I want to have an ingress Caddy proxy on one node, which reverse proxies to several backed APIs on other nodes - can this be done simply with Podman Quadlet? Also, what is the localdev UX like? With Docker Swarm I can easily run very similar setups in dev, test and prod, and it works with multi-node setups, has support for secrets etc. But the lack of work going into Docker Swarm becomes more concerning as the years pass by. Also, had no idea systemd-proxy was a thing - is there anything systemd doesn't have its tentacles into? :)
- mati365 2y agoIf your VPS is wired with another one using VPC or any other internal network it'll just work. Just point Caddy to specified internal IPs of your other servers. It's not designed to work on local envs. When I wanted to debug infra I used to run it on Vagrant though
- altairprime 2y agoDoes this all fit in 256MB of server RAM?
- ericaska 2y agopikku means tiny or little in Finnish. Is it where the name came from?
- cuu508 2y agoI don't know but my first association was "pico-dokku"
- ewalk153 2y agoMy guess has been they both originate from heroku; docker heroku to dokku, pico heroku to piku
- _njyk 2y agoCute, as in the sibling language, Estonian it means “big” or “tall”
- rcarmo 2y agoNope. https://piku.github.io/FAQ.html https://piku.github.io/FAQ.html
- theanonymousone 2y agoNice work. But why isn't Docker supported as a runtime? Or is it?
- rcarmo 2y agoThe FAQ explains it: https://piku.github.io/FAQ.html https://piku.github.io/FAQ.html You can use docker run commands, but that’s not the main goal.
- sneak 2y agoI think a more common use case than doing deploys by pushing to a different remote is to send git repo webhooks on PR merges to main to an API that has a deploy key and can fetch the repo itself. This afaik is missing from most PaaS tools (CapRover excluded, but it has been illegally relicensed to non-open-source). Perhaps watchtower or something could replace the functionality?
- adeptima 2y agoDidnt know ... "CapRover has built in anonymous usage analytics starting v1.11" https://github.com/caprover/caprover/blob/master/TERMS_AND_CONDITIONS.md https://github.com/caprover/caprover/blob/master/TERMS_AND_C... https://github.com/caprover/caprover/issues/1852 https://github.com/caprover/caprover/issues/1852 Was looking at CapRover to see if it has REST API Looks shaddy
- vault 2y agoYou scared me for a moment, as I've just setup a new VPS with CapRover and migrated all my projects from heroku. Doesn't look too shady for me, there's a oneliner to disable analytics, it seems enough for me.
- sneak 2y agoYou still have to agree to the terms and conditions of use of the nonfree application which can of course change at any time without notice. It’s a time bomb. I’m thinking of forking it and adding all his dumb and easy table stakes features (2fa etc) that he is trying to gate as subscriptionware.
- rcarmo 2y agoIf you want to contribute any of that to piku, we’ll welcome it. Might take a bit to review and merge, but we’re always looking for non-breaking improvements
- sneak 2y ago
- block_dagger 2y agoCool project, but I’ll stick with Dokku, which is a wonder for managing single server deploys via Docker/Git.
- deleted 2y ago[deleted]
- demarq 2y agoIs there support for secrets?
- dakiol 2y agoDoes someone know how it handles (if any) zero downtime deployments? Like, if your Python service is running in one machine on port 8080 behind nginx, how does piku switch to a fresh instance running in the same port?
- rcarmo 2y agoCurrently it will only kill running processes after it finishes deploying the new git push. Socket and session handling will vary depending on your code and whether you use uwsgi or run your own HTTP daemon. One thing it already does (optionally) is to kill off unused instances and idle, lazily starting them when a new connection comes in.
- mrweasel 2y agoSlightly off-topic, but you can do zero downtime deployments using systemd and socket activation.
- deleted 2y ago[deleted]
- rcarmo 2y agoThat gives me a couple of ideas...But picking a shorter name than "piku" is going to be hard... Maybe I can whip up a proof of concept and call it "syd".
- pixelmonkey 2y agoI love piku. I wrote a webapp tutorial for piku which got turned into a repo as part of the official GitHub piku org. You can find that here: https://github.com/piku/webapp-tutorial?tab=readme-ov-file#build-a-web-app-fast-with-piku https://github.com/piku/webapp-tutorial?tab=readme-ov-file#b... It explains how piku works under the hood, as well as showing a minimalistic Python web app example from a user standpoint.
- rc_kas 2y ago"What is a Heroku-style deploy?" thanks for that. I have no idea what heroku is or does.
- pixelmonkey 2y agoSure thing! Bit of cloud computing history. Covered a bit here: https://leerob.io/blog/heroku https://leerob.io/blog/heroku
- jslakro 2y agoBasically it was the first PaaS to improve the developer experience when working with server infrastructure. It had git integration and allowed to scale easily your apps from a CLI
- brightball 2y agoShort version: Git push deployment where it detects your stack and automatically builds, then deploys with zero downtime.
- rcarmo 2y agoThis is now linked from the docs home page.
- sesm 2y agoThanks for the explanation, official repo doesn't make it clear enough for me. So, did I understand correctly, that Pico installs both an agent on the remote machine and a commit hook on the local machine? Why didn't they minimize the overhead by just making the remote machine a Git remote and do all the work there when you push a specific branch to that remote?
- isoprophlex 2y agoThat is brilliant. Something complex, but not complicated. A project distilled down to its UNIX essence: hackable, lean, and magic. That said I want to give this a go but don't immediately see how I can migrate my overengineered 8-10 container spaghetti of a docker-compose file to a state where I can use piku instead of manual 'git pull && docker compose up' on the remote
- rcarmo 2y agoThat kind of situation was what drove me to go simpler :)
- isoprophlex 2y agoYes it's me, not you ;) Currently hyping myself up to drastically simplify everything, which will be a joy onto itself
- nanidin 2y agoCan't you use git hooks to automate the manual steps?
- cfiggers 2y agoThe initial commit was eight years ago?? I wish I had known about this project ~18 months ago. I was specifically looking for a way to have a Heroku-like dev experience deploying to my Raspberry Pi, and this looks like it's trying to be exactly that.
- vault 2y agoExactly. There's a visibility problem. I've just setup a new VPS with CapRover and never found any mention of piku in the hour I've spent checking for comparisons between "Heroku-style self-hosted PaaS" dokku, CapRover, coolify, and dokploy.
- rcarmo 2y agoWe’ve been using it for a long time, yes, but doing Marketing for a 1500 LOC Python script felt a little overblown :) Still, Chris did a public presentation on it near the beginning (video’s in the docs) and other folk did similar things, so…
- aaomidi 2y agoIs go support planned?
- rcarmo 2y agoIt works with Godeps. Module support was always a bit in flux when we added that, but it should be an easy first contribution…
- brigadier132 2y agoThese self-hosted open source paas alternatives are really cool. Off the top of my head I know of coolify dokku kamal and now piku
- vault 2y agoDon't forget CapRover. I'm just trying it on a new VPS and it just works as expected. I would have tried piku first if I knew about it, because it's even more minimal.
- deleted 2y ago[deleted]
- 1oooqooq 2y ago[flagged]
- snitty 2y ago> set -e > echo "Downloading piku-bootstrap here." > curl -s https://raw.githubusercontent.com/piku/piku-bootstrap/master/piku->bootstrap https://raw.githubusercontent.com/piku/piku-bootstrap/master... > piku-bootstrap > chmod 755 piku-bootstrap > echo "Now you can install Piku on `hostname` like this:" > echo "./piku-bootstrap install" Did you look at the script?
- quectophoton 2y agoIn defense of their argument (not their choice of words), the final script is doing a `git clone`, so if they're going to ask the user to copy-paste anyway, their copy-paste code could be something like: PUBLIC_KEY_FINGERPRINT='SHA256:KnownPublicKeyFingerprintHere' git clone 'https://github.com/piku/piku-bootstrap' 'piku-bootstrap' && \ cd 'piku-bootstrap' && \ (git verify-commit $(git log -1 --pretty='format:%H' -- README.md) | grep 'Good "git" signature with ED25519 key '"${PUBLIC_KEY_FINGERPRINT}") && \ (git verify-commit 'HEAD' | grep 'Good "git" signature with ED25519 key '"${PUBLIC_KEY_FINGERPRINT}") && \ ./whatever-command-from-this-repo (EDIT: Feel free to clean it up a bit, adding other variables to make it a bit more readable, etc.) From the user's POV it's the same thing: copy from somewhere, paste into terminal, press enter. But now you're trusting two fewer places: - piku.github.io (the deployed version) - github.com/piku/piku.github.io (the source code that supposedly generates the deployed version) I'm not saying to do exactly these steps, and they are not perfect anyway (you're still trusting a `git clone` before even doing any kind of verification), but at least you're not YOLOing[1] without first doing some minimum verification of integrity before something is executed, so you know at least the first installation step has not been tampered with[2][3]. The git repos don't have commit signatures, and the verification doesn't have to use specifically those kind of signatures (could be a minisign public key hosted somewhere), but you get the point. The updates are also a `git pull`, meaning, if the commits were signed, and the installation copy-pasta (or script) included a step that added the author's public key to the allowed signers file (TOFU-style), e.g. with a `curl` to the author's GitHub's keys URL, then `git pull --verify-signatures` could check new commits against the already-known key, and automatically warn about any changes so the user could decide if trust the new key or not. Stuff like that. [1]: Nowadays, saying "install with `curl | sh`, but you can also do install these other ways", is equivalent to saying "verify integrity with `md5sum`, but you can also check this other file if you want to see other hashes"; technically correct, but if the installation instructions already contain questionable defaults, it makes me question what other questionable defaults are in the actual code. [2]: "Has not been tampered with" as in "the last modification of `README.md` and the most recent commit are signed with the same key". Not perfect, but it's not YOLO. [3]: Emphasis on "first installation step". Obviously anything that happens after that simple sanity check would need more scrutiny in case the author tries to run unsigned/unverified code in one of the next steps.
- p-o 2y agoFirst time I read about piku. I have no idea why, but the feeling of `git push` to initiate a deployment like piku does always felt magical to me. There's nothing simpler than that. This is timely for me as well as I just open sourced (yesterday!) a project that is in the same space, but for Kubernetes (https://github.com/pier-oliviert/sequencer https://github.com/pier-oliviert/sequencer). All of this to say, congrats! It looks great.
- CaptainOfCoit 2y agoMaybe I'm missing something obvious, but how does sequencer use git to do deploys, if it's similar to Heroku/dokku/piku? Seems like you're dealing with kubernetes templates and kubectl rather than `git push` to deploy, which would put the project is a completely difference space.
- myaccountonhn 2y agoIt works like magic, but it's also extremely simple to DIY if you wanna learn. If you set up a server, you can create a git repo by just doing `git init --bare`, add the setting `git config receive.denyCurrentBranch updateInstead`. After that you can use git hooks (more specifically push-to-checkout hook), to receive uploads, compile and launch. The hook will just be a simple shell script, the most basic version could be a variant of `compile && install && systemctl restart service`. From there you'll be able to copy the repo locally and pushing your changes will now trigger the hook you've setup. git clone root@yourserver.com:/path/to/git/folder
- rcarmo 2y agoYou just described Piku, except that it’s a Python script that also sets up nginx and a process supervisor for your code :)
- myaccountonhn 2y agoYeah I love the simplicity of Piku, being able to actually understand what is happening behind the scenes is a great quality. :)
- zeroq 2y ago15 years ago it was common to deploy web applications as live SVN repositories with a hidden path executing 'svn update' on manual http request. Not quite the 'push deploy', but that was the way apps were developed back in the days, and for some reason I still prefer that approach. Commit, test, and at one point manually nominate and deploy stable version.
- eastbound 2y agoYes, when we didn’t want a build machine, we’d just build in production. Isolating production with no unauthorized binary (like Alpine) was a long path away…
- deleted 2y ago[deleted]
- rmbyrro 2y agoYou can do that in the git push by having a separate "stable" branch and linking deployment to it.
- tiffanyh 2y agoWith PHP, 1-line (no new tools): sftp user@host remoteFile localFile Joking aside, I’m a bit surprised such a tool would be developed in Python given its dependency’s and runtime (which is not easy on the user).
- rcarmo 2y agoTo be honest, Python made it stupendously simpler than anything else because it has a great standard library. The only dependency (click) is rock solid and made it a lot simpler to handle commands independently, but we could probably do without it and just use the built-in argparse—-but at the expense of a few more lines of code I didn’t want to maintain. Also, Python is everywhere, on every OS and Linux system, so it was a natural choice. I also wanted it to be easily hackable and extensible, and few languages would make it simpler to understand or extend.
- mlhpdx 2y agoThat’s pretty funny. You may want to look a little further field to discover that the machines with Python are far from “all the machines” out there. Particularly production servers, which, if they run responsibly, are hardened with every extraneous bit of software removed.
- jteppinette 2y agoI developed security software in Python that ran on 100k+ production nodes covering dozens of operating systems. They all had Python.
- hamandcheese 2y agoCounter-anecdote: none of my Linux PCs have python.
- jteppinette 2y agoCounter-counter-anecdote: my toaster has python.
- BryanLegend 2y agoWhat is a PaaS?
- snitty 2y agoPlatform as a Service. Which leaves me the same number of questions. So, what is Platform as a Service?
- cfiggers 2y agoA "Service" is when someone or something does something for you, usually in return for some fee. A "Platform," in the context of IT and software (and especially the internet), is some IT infrastructure, generally a server and the software installed on it, that you can host something on, such as an app or a website. A "Platform as a Service" is when someone else sets up your platform for you so that you don't have to do it yourself—you get access to the platform and can use it for your own stuff, but don't have to configure or maintain most of it.
- rcarmo 2y agoMaintainer and co-author here. If you like simple, minimalist deployment tools, check out https://github.com/rcarmo/ground-init https://github.com/rcarmo/ground-init for a very much down to earth take on cloud-init…
- e12e 2y agoYour readme doesn't really answer the question of why not cloud-init?
- red_admiral 2y agoEventually, we'll need something more secure than effectively `sudo curl INSTALLER | sh` as a way to install stuff. I can see why package managers aren't always the answer, but still. piku itself is neat and I like it.
- deleted 2y ago[deleted]
- rcarmo 2y agoActually, we had manual install steps as the only way to go for a while. You'd be surprised at how many people asked for a one-liner... I'm going to add a note to the docs about that, since I've seen a couple of comments here of people who were put off by it and didn't even read the rest. I actually only install piku via cloud-init, but there are plenty more options: https://piku.github.io/install/index.html https://piku.github.io/install/index.html
- wayoverthecloud 2y agoHas anybody used this for Ruby on Rails?
- rcarmo 2y agoYes. Not any of the maintainers, though.
- deleted 2y ago[deleted]
- deleted 2y ago[deleted]
- deleted 2y ago[deleted]
- nico 2y agoCan it be a good replacement for Capistrano (for deploying rails applications)? Love the focus on being lightweight Recently I wanted to create a super basic website, and discovered it’s actually pretty hard to create something simple And then, even if you manage to create something actually simple, you usually end up having to manage some not so simple deployment process together with hopefully some sort of version control Ended up settling for putting plain html/css/js files in a git repo, then configuring auto deploy to GitHub Pages on merge to master (via Actions)
- juliushuijnk 2y agoAlso an option, if it's just for you and with not too many updates, you can upload the new files to ftp as a manual step.
- nico 2y agoDoes GitHub pages support ftp? Or are you talking about some other potential hosting options? Yes, ftp is pretty easy for static sites. However, given I want to have version control, it’s nice to have automated deploys happen after a git push
- mati365 2y agoUse Podman Quadlet, I use it as replacement
- verisimi 2y agoBut what if you like 'big pass'? Looking at you sir mixalot: https://youtu.be/wKfMZOR8eWI?t=31 https://youtu.be/wKfMZOR8eWI?t=31
- kachurovskiy 2y agoI just added a magic URL in my app that GitHub calls whenever a commit is pushed and the server does `git pull` which in turn causes pm2 to reload the app. So committing anything shows up in production in seconds. Great for smaller projects.
- fprotthetarball 2y agoI have a similar setup, using snare to handle the webhook endpoint: https://github.com/softdevteam/snare https://github.com/softdevteam/snare GitHub will call the webhook after a push to main and a successful test suite run. Snare runs a shell script on my server to git pull, build, deploy, and call a cronitor.io hook for monitoring deploy success. I've been pretty happy with how relatively simple it is and how well it works.
- gitinit 2y agoHow did you set this up? Seems simple yet effective.
- pheggs 2y agointeresting project! what are the advantages of this over pushing to a normal ssh server with a server side git hook?
- acidburnNSA 2y agoCame to ask this same question. I have a post-receive hook on my server that instamagically deploys whenever I push to it. It is simple and awesome and is basically just a builtin git feature.
- eloh 2y agoI just "git push" using nixos-rebuild
- smmseller 2y ago[dead]
- schneems 2y agoNot related to “git push” deployments, but absolutely related to the PaaS experience, the team I’m working on is previewing Cloud Native Buildpacks (CNB) which is an open spec in the CNCF for Buildpacks that target OCI. What this means is that you can now generate a docker image locally using similar build tooling to Heroku’s “git push” logic that detects language support and does the right thing TM. Here’s a tutorial for building a Rails app with the buildpack I maintain https://www.schneems.com/2024/05/01/build-a-ruby-on-rails-application-image-in-5-minutes-no-dockerfile-required/ https://www.schneems.com/2024/05/01/build-a-ruby-on-rails-ap... Would love some feedback, if you try it. Please consider posting about the experience in the linked discussion (good, bad, indifferent, or whatever I just want more feedback to improve the experience).
- Alifatisk 2y agoRepeat after me: git, is, not, a, deployment, tool
- jedberg 2y agoWhy? Not?
- taberiand 2y agoI think the main reason is it's sensible to pass the source code through a process that organises and optimises it for release to a specific environment. My first assumption seeing a git repo used in this way would be that someone was cutting corners and probably doing bad things like committing secrets to the repo, things like that. If the person setting it up is aware of the potential pitfalls and has a good explanation for the process - particularly if there is no build step involved and secrets are managed appropriately, then it can be fine.
- jedberg 2y agoYou can do all that with a pre-commit or post-commit hook.
- taberiand 2y agoIt goes further than that, those were just examples. The principle of least knowledge, and the principle of least privilege, guide deployment to a process that does not include the source code on a production server. But like I said, there are ways for it to be a reasonable approach if properly justified
- ec109685 2y agoThen you’re checking in artifacts generated from source code (for this method that relies on git push to work).
- llIIllIIllIIl 2y agoI use dokku for my side gigs and it works best. The performance issue I’ve experienced was when the container with my app was being built and it dramatically increased the load on a $20 vm. Then I migrated to use container registry to utilize GitHub actions for building and pushing containers to the registry and then deploying the container directly on the dokku host. Does piku support that flow? Another question is subdomain support: to have a catch all virtual host that will respond to anything.domain.tld and have wildcard letsenrypt enabled ssl with DNS challenge. Those two problems make me think that my side gig has grown up enough to switch to ArgoCD/K8s, although there are many other problems that come with it (from my experience on the day job). For now I just do the certificate rotation manually, which is not much ideal but works with a couple make targets.
- withinboredom 2y ago> Those two problems make me think that my side gig has grown up enough to switch to ArgoCD/K8s If you have to think about it, it isn't worth it. You'll know when it is time to refactor your infra.
- mitjam 2y agoI like Epinio which does the same but on top of kubernetes. It is backed by Suse and lightweight compared to KNative (which is the basis of GCP CloudRun for example), but being kubernetes based still requires more Resources than dokku or Piku. I still prefer k8s due to the vast ecosystem of mature solutions. And I can still run everything on a single box, it just needs to be a bit bigger. The new Hetzner CX42 with 8 vCPUs, 16 GB of RAM, and 160 GB of disk space for € 16.40 a month (€ 0.0273 per hour) is sufficient, and with the Kube Hetzner Project I can set up a kubernetes cluster with auto updating microos in 5 minutes. https://github.com/epinio/epinio/ https://github.com/epinio/epinio/ https://github.com/kube-hetzner/terraform-hcloud-kube-hetzner https://github.com/kube-hetzner/terraform-hcloud-kube-hetzne...
- CaptainOfCoit 2y ago> I like Epinio which does the same but on top of kubernetes So basically not at all the same? :D The point of piku seems to be: Heroku experience without requiring docker, and with a really simple architecture, and that it works on ARM. Kubernetes works on ARM, I give you that. But AFAIK, Kubernetes requires you to use some sort of containers (Docker or otherwise) and its architecture is anything but simple (for obvious reasons). Besides that, I don't see how epinion enables the "git push" workflow, the quick start tutorial seems to tell you to run "epinio push manifest.yaml" or similar to deploy the application, so it doesn't fit with the "Heroku-like experience" either. So really, the only things they have in common is that they handle deployments?
- mitjam 2y agoYou are right it needs docker but so does Dokku and I see this more as an implementation detail (even a plus in my book for the flexibility). Epinio does admittedly not support git push but for me „epinio push --name myapp“ feels similar enough. In the end, I can just push my Django or next.js or rails or node.js code to the server. I should have said „I like Epinio, as well“ because I also like Piku, especially for its minimalistic approach and readable code, but when it comes to actually using it for deployments, I prefer Kubernetes.
- flemhans 2y agowatch -n 1 git pull
- reneberlin 2y agoI love the minimal approach - will try it! There is also: https://kamal-deploy.org/ https://kamal-deploy.org/ https://github.com/basecamp/kamal https://github.com/basecamp/kamal https://dokku.com/ https://dokku.com/ https://github.com/dokku/dokku/ https://github.com/dokku/dokku/ https://clace.io/ https://clace.io/ https://github.com/claceio/clace https://github.com/claceio/clace