9 ms·
Takahē: An efficient ActivityPub Server for small installs with multiple domains
- totetsu 4y agoTakahē are interesting birds too. There is a related bird the pūkeko which was also blown to NZ but at a different time. It has relatives in Australia and South America also. It was thought to be extinct at one point, caused by predation by introduced pests, and introduced deer eating the grasses they rely on for food. Now there is a population of about 400?
- bjowen 4y agoLove that you’ve named it after an utterly charming little bird
- Tepix 4y agoLooks interesting. Why does it need a Postgresql server? For just a handful of users, isn't sqlite the leaner, yet sufficient choice? How does it compare to GoToSocial, which requires 50-100MB of RAM? They are also in alpha stage and i like their approach of keeping the web UI separate.
- nielsole 4y agoif they don't have PSQL specific queries, it might be a trivial change: https://github.com/jointakahe/takahe/blob/main/takahe/settings/base.py#L60 https://github.com/jointakahe/takahe/blob/main/takahe/settin...
- simonw 4y agoI tried swapping that for SQLite and successfully ran the test suite about a week ago, but I've not tried that again against the large number of more recent changes.
- rch 4y agoI wonder if Postlite would work. Actually that looks more like an interactive client.... https://news.ycombinator.com/item?id=30875837 https://news.ycombinator.com/item?id=30875837
- scrollaway 4y agoSQLite is magical and incredibly lean, but it is not leaner than Postgres if you need real database features. You end up reimplementing a lot of features in code that belong in the db.
- simonw 4y agoWhat kind of features are you talking about here? This doesn't match my experience from the last few years. SQLite in WAL mode is extremely capable. The only thing I really miss from PostgreSQL is that PostgreSQL has more built-in functions for things like date handling - but SQLite custom functions are very easy to register when you need them.
- scrollaway 4y agoConstraints and validation for example. Efficient json store. Etc
- simonw 4y agoSQLite has check constraints - and recent versions can have STRICT tables: https://www.sqlite.org/stricttables.html https://www.sqlite.org/stricttables.html It also has excellent JSON features - JSON maybe stored as text rather than a binary format like JSONB in PostgreSQL, but the SQLite JSON functions crunch through it at multiple GBs per second so it doesn't seem to matter.
- deleted 4y ago[deleted]
- scrollaway 4y agoDidn't know about strict tables, very cool!
- andrewgodwin 4y agoAuthor here - it's just to reduce support surface area. I know I'll need PostgreSQL's full text indexing and GIN indexes for hashtags/search eventually, and I probably also want to use some of the upsert and other specialised queries, and it's easier to just target one DB I know is very capable. For reference, when I say "small to medium", in my head that means "up to about 1,000 people right now".
- Tepix 4y agoSQLite has full text search and upsert. GIN indexes sound cool - perhaps you can get away not using them however and instead support 2 DB backends? If you want to accomodate "small" and not just medium, that would be great! ;-)
- UncleEntity 4y agoI was poking into this a bit yesterday. Is there any advantage to using a traditional db as opposed to a graph db since json-ld is just a text representation of graph nodes? I was thinking the easiest path would be have the server deal with all the activityPub stuff and expose something like a graphQL interface for a bring your own client implementation. Of all the stuff they shoehorned graphQL into this seems like a valid fit, like they were made for each other. Anyhoo, just my random thoughts…
- manfre 4y agoFor better or worse, many servers are targeting Mastodon API compatibility to be able to leverage the existing clients. Adding GraphQL increases surface area without solving the bigger issue of creating the clients.
- UncleEntity 4y agoI didn’t get as far as looking into the mastodon API for clients but that makes perfect sense, I just assumed it was an overlay on the more general API. Mostly I was thinking how one could implement something in the most efficient way and graph databases/graphQL were literally designed for this stuff.
- goodpoint 4y ago
- turnsout 4y agoCounterpoint: If the fediverse has any hope of achieving critical mass, it can’t confuse users by constantly breaking federation for every minor disagreement.
- goodpoint 4y agoOne more reason to break federation. The federation was always meant to build communities around shared interests and values. If I wanted the twitter "experience" I would just use twitter. If I wanted a "free for all" environment I would be on 4chan.
- kzrdude 4y agoHas that happened? I was trying to understand how it work in practice. It would be relevant to picking which instance to use.
- egypturnash 4y agoConstantly, it's great. There's big air-gaps between the US Lefty and US Righty fediverses. If you want to talk to both sides of that divide then you'll want to have accounts on multiple servers. If you decide to only be on one side then it's easy to migrate your accounts around between servers.
- Ruthalas 4y agoIf I host my own instance, can I federate with both 'halves'? Or will associating with one likely get my instance blocked from federating with the other?
- charcircuit 4y agoAt least in the past I know that would get you blocked as it acted as a bridge to allow tweets from the blocked instance to get through.
- turnsout 4y agoNice project! I like the goals, and hope we see more good ActivityPub options like this. Just a note that the web UI isn’t formatting correctly on iOS!
- robga 4y agoBrief discussion of Takahe in the TalkPython podcast here https://youtu.be/LhBfMoR3bvI?t=2369 https://youtu.be/LhBfMoR3bvI?t=2369
- solarkraft 4y ago> Features on the long-term roadmap: > “Since you were gone” optional algorithmic timeline That's exciting! The fediverse is severely lacking algorithmic curation presumably due to the belief that it's inherently evil (I'd strongly disagree; it's merely the algorithm not being user-controllable what's bad).
- dewey 4y agoFully agree, the algorithmic timeline (sprinkling in some likes and comments from other people that might be interesting to you) is one of Twitters best features even if many people (who mostly use third party clients for that reason) would not agree.
- cxr 4y agoThe word "algorithm" has suffered wild semantic drift at the hands of journalists. Let's see if we can start to fix that now by making sure that on HN and in adjacent communities of all places we use the appropriate words for the thing we are talking about. We are talking about heuristics here, not algorithms.
- dewey 4y agoI would say that ship has sailed, just like "crypto" doesn't mean cryptography in the previous sense anymore.
- robocat 4y agoCrypto has its own meanings and cryptocurrency also has some of those other meanings (not just the historical connection to encoding). I especially like some of the biological metaphors: https://www.cdc.gov/parasites/crypto/index.html https://www.cdc.gov/parasites/crypto/index.html https://en.m.wikipedia.org/wiki/Cryptozoology https://en.m.wikipedia.org/wiki/Cryptozoology https://en.m.wikipedia.org/wiki/Aggressive_mimicry#Mimesis https://en.m.wikipedia.org/wiki/Aggressive_mimicry#Mimesis “cryptic aggressive mimicry is where the predator mimics an organism that its prey is indifferent to” i.e. wolf in sheep’s clothing.
- college_physics 4y ago+1 for adopting django. with django's roots in journalism it somehow feels a natural building block of federated information exchange
- datalopers 4y agoDjango does many things but “efficient” is not one of them.
- Nasreddin_Hodja 4y ago> Django does many things but “efficient” is not one of them. It depends on how you code. I wrote a user instance in django and and I'm happy with it's performance.
- scrollaway 4y agoYou’re wrong. Just because it’s not the absolute peak of efficiency, written in C with asm routines to talk to the db, doesn’t mean it’s not efficient.
- datalopers 4y agohttps://www.techempower.com/benchmarks/#section=data-r21&test=composite https://www.techempower.com/benchmarks/#section=data-r21&tes... Django ranks #137 out of #142 across numerous web frameworks and languages. It’s literally one of the least performant options that exist.
- scrollaway 4y agoPerformance and efficiency aren’t the same thing. Django does a lot of things other frameworks ranked here don’t do. Such framework rankings are also utterly irrelevant when you want something widely used enough to easily find contributors and integrations. That restricts you quite a bit more than “any so called framework that just handles http”. Did you even look at the top performers on that page? This is number 2: https://github.com/Xudong-Huang/may_minihttp https://github.com/Xudong-Huang/may_minihttp
- college_physics 4y agomastodon is ruby on rails, just sligthly more "efficient" from django according to those benchmarks
- 4y ago
- mg 4y agoNice to see a Python/Django implementation of ActivityPub. Having a nice, lean implementation of ActivityPub that I can customize to my liking is the only thing that keeps me from using the Fediverse more regularly. So I am watching the space closely. What I find a bit unfortunate about Takahe is the coupling with Docker. An even leaner ActivityPub implementation seems to be MicroBlogPub. I have not yet managed to set it up though. Anybody interested in collaborating on a MicroBlogPub install script that turns a fresh Ubuntu installation (or container) into a running MicroBlogPub instance?
- simonw 4y agoIt's not coupled with Docker. Docker is purely one suggested way of running it - it's a classic Django app so running it directly on Ubuntu should work the same as any other Django application.
- mg 4y agoGreat! When I saw "Prerequisites: Something that can run Docker/OCI images" in the documentation, my interpretation was that containers are needed. It also says "You need to run at least two copies of the Docker image". Maybe you want to change the wording a bit. I would also collaborate on writing a setup script for Takahe then! I really like to write a setup script instead of following manual installation guides. So for every software I try, my first step is to write a script that turns a fresh Debian installation into a running instance. (MicroBlogPub needs Python 3.10 which is not in Debian stable, so I would use Ubuntu)
- mg 4y agoHmm.. does not look good for the non-Docker setup. The developer replied with "I am deliberately avoiding offering a non-Docker install path" and closed the issue: https://github.com/jointakahe/takahe/issues/44 https://github.com/jointakahe/takahe/issues/44 Creating a non-Docker fork would then probably be an uphill battle.
- 4y ago
- pickpuck 4y agoAhh this is so exciting to see so much happening in this space all of a sudden! My quest to get a personal instance running has been a long slog for me personally. I had been working on an ActivityPub server in Node.js/TypeScript for a while before the Twitter migration. It's got most of the features I'd want in a small server but it's basically bring-your-own-client at the moment. https://github.com/michaelcpuckett/activitypub-core https://github.com/michaelcpuckett/activitypub-core Finding all the resources to build a complete server that can interact with other instances isn't easy, so maybe this can help someone. The spec is well worded, but the checklist is confusing, the test server is down, Mastodon has its own rules, etc. Plus you have to have at least a cursory knowledge of JSON-LD/RDF.
- anthropodie 4y ago>Ahh this is so exciting to see so much happening in this space all of a sudden! It's like Elon unknowingly funded this space!
- hunterb123 4y agoNever underestimate HN's pettiness and spite! Now it's mainstream to work on a a cool technology that's been around for awhile! Oh and everyone can act like they weren't bad mouthing the tech and saying it wasn't going to work before. Visit any Mastodon thread here before Elon's Twitter and it's nothing but negativity.
- ruined 4y agowell, there's more than one person on this website. also, a lot of these projects have been on a slow simmer for a long time, and are only just now starting to become complete and interesting. edit: though it does seem to be true that takahe's initial commit was nov 5 :) and personally i don't consider it complete and interesting yet
- hunterb123 4y agoYes there are multiple people here, but the general sentiment was negative among most threads. Go ahead and look over any Mastodon thread a year ago or before. Generally it was dismissed with "oh it's too niche" or "moderation will be too difficult". People ignored the communities already on it and the tech overall. Only until people got pissy about Elon running Twitter instead of hedge funds did the general sentiment here change. It wasn't about the tech, and not even about Elon specifically, it was the Twitter safe space got taken away. But now hopefully the people who want that safe space will isolate themselves in mastodon instances that block all others and we can all live in peace from them.
- infotogivenm 4y agoBSD3, yay - even though I’m worthless at python, love to see an implementation that is not AGPL
- anecdotal1 4y agoSurprised it hasn't been attacked over this yet as there's so much needless hand wringing about anything non-AGPL being a threat to the anti-capitalism views of the Fediverse
- deleted 4y ago[deleted]
- losvedir 4y agoI'm very interested in this federated renaissance happening, but having trouble understanding how all the pieces fit together. Is there a good overview I can read? I think ActivityPub is the (a?) protocol, and Mastodon is one particular implementation of it, just like the software linked here? Are there other relevant or competing protocols? How does Matrix fit in exactly? What about identity? Is OpenID a part of all this somehow?
- dsr_ 4y agoThe Fediverse is the loose network of servers that exchange data with the ActivityPub protocol. Mastodon is a server that implements a chunk of AP. Mastodon also specifies a client API that is not AP, but is fairly often implemented by other packages because it's convenient. AP is not entirely Twitter-style microblogging. It can be used to exchange (data or links) photos, video, audio, documents, invitations and meeting appointments. The default privacy assumption for all AP content is that it is basically public. Matrix is not built on AP. Matrix is a real-time communications protocol suitable for private messages and public chats. Its mission appears to be to bridge every other protocol, so there's at least one Matrix-ActivityPub bridge module, MXToot. OpenID is not part of these. Some Matrix servers can use OpenID for authentication. As far as I know, no ActivityPub servers currently use OpenID.
- simonw 4y agoHere are three specifically interesting things about Takahē: 1. The "multiple domains" feature. I'm running my own Mastodon instance right now purely so I can have my simonwillison.net domain as my identifier there (and protect myself from losing my identifier if the server I am using shuts down). This feels pretty wasteful! I'd much rather be able to point my domain at a Takahē instance shared with some of my friends, each with their own domains for it. 2. It's a Django app that's taking full advantage of the async features that have been added in the most recent releases of that framework. Async is a perfect match for ActivityPub due to the need to send thousands of outbound HTTP requests when publishing a message. And Takahē creator Andrew Godwin is the perfect person to build this because he's been driving the integration of async into Django for the past four years: https://www.aeracode.org/2018/06/04/django-async-roadmap/ https://www.aeracode.org/2018/06/04/django-async-roadmap/ 3. The way it handles task queueing is super interesting. I've not fully got my head around it yet but it's the part of the codebase called Stator and it's modeled on things like the Kubernetes reconciliation loop - Andrew wrote a bit more about that here: https://www.aeracode.org/2022/11/14/takahe-new-server/ https://www.aeracode.org/2022/11/14/takahe-new-server/ - Stator code is here: https://github.com/jointakahe/takahe/blob/main/stator/runner.py https://github.com/jointakahe/takahe/blob/main/stator/runner...
- phphphphp 4y agoPerhaps naive but is it possible to create some sort of Mastodon proxy that exists independent of any specific instance? Rather than run your own instance or point to a shared instance, a proxy could be a fairly simple system that uses DNS records (?) to route requests to the appropriate instance -- much like email.
- simonw 4y agoUnfortunately that doesn't quite work with out-of-the-box Mastodon. I'm running a bit of a proxy at https://simonwillison.net/.well-known/webfinger?resource=acct:simon@simonwillison.net https://simonwillison.net/.well-known/webfinger?resource=acc... but it still needs to point to my own dedicated instance, just because Mastodon can't have multiple domains pointed at a single instance of the software yet. I'm using this pattern (also shared by Andrew, before he started to spin up Takahē) https://aeracode.org/2022/11/01/fediverse-custom-domains/ https://aeracode.org/2022/11/01/fediverse-custom-domains/
- Robotbeat 4y agoAre there any ActivityPub benchmarks to compare various implementations of Mastodon-compatible instances? Ie written in different languages, etc. For instance, Go seems to be around an order of magnitude faster than Ruby, and I think I've seen a Golang implementation of ActivityPub somewhere. https://programming-language-benchmarks.vercel.app/go-vs-ruby https://programming-language-benchmarks.vercel.app/go-vs-rub...
- nandalism 4y agoI see honk hasn't been mentioned on this thread. It's also an activitypub server which is very lightweight (golang) and easy to set up your own server. https://humungus.tedunangst.com/r/honk https://humungus.tedunangst.com/r/honk
- mdaniel 4y agohttps://humungus.tedunangst.com/r/honk/v/tip/f/web.go https://humungus.tedunangst.com/r/honk/v/tip/f/web.go reads like it is either been sent through an obfuscator or is in the church of templeOS
- nandalism 4y agoYes I wrote the same thing here http://blog.deckc.hair/2022-11-19-ted-unangsts-golang-obfuscator.html http://blog.deckc.hair/2022-11-19-ted-unangsts-golang-obfusc...
- Terretta 4y ago> butwhatabout(mdaniel) I see the sibling comment about obfuscation, but not sure I follow either of you. Is this code not clear? To me the code reads with humor and creativity, while every bit as self-evident as a Gary Larson FarSide cartoon on second glance. I mean, what else is nomoroboto going to do than what it does? I've never seen this tone in the wild before, but got a kick out of it, might even find it refreshing maintaining it.
- mdaniel 4y agosquints I can't tell if this is trolling or not Anyway, you're right, all code should be written in haiku form, to maximize creativity and succinctness, plus keeping methods short! True elite coders ensure variable names are always a prime number of characters
- Terretta 4y agoallinjest(originate(mdaniel),j) // but not entirely jest ... "emptiness" and "thelistingoftheontologies" ... it's delightful!
- railsagainst 4y agoI have come here to rail against (pun intended) the use of the name Takahē for a piece of software. The author is well-intentioned and there is some aptness to the name, but many people here in Aotearoa / New Zealand, are sensitive to the use of the names of our tāonga / treasures for businesses, technologies and other objects. Specifically, the takahē is an indigenous species tāonga as described in https://www.taiuru.maori.nz/branding/ https://www.taiuru.maori.nz/branding/. As such it would be best to consult with the Ngāi Tahu people before using the takahē’s name in this manner. In this link from NZ’s Department of Conservation (https://www.doc.govt.nz/news/media-releases/2021-media-releases/fulton-hogan-continues-support-for-takahe/ https://www.doc.govt.nz/news/media-releases/2021-media-relea...), you can see that NZers take this sort of iwi partnership seriously. More broadly, I find it sad when the the names of natural species and features are adopted in the business and technology world without any deep connection. A canonical case would be Amazon the company, which has prospered and become a household name while the Amazon itself, with its people and ecosystems has suffered and declined. An egregious case relevant to NZ is Kiwi Farms. The trend of using species names in technology perhaps started with the O’Reilly books. The argument can be raised that such use raises awareness of endangered species such as the takahē. But perhaps that is best left to other means, for fear that the mauri of a species should be captured and harmed.
- gearsandbeers 4y agoI'm tangata whenua and quite frankly this is ridiculous - Ngāi Tahu doesn't speak for all Māori. I'm perfectly fine with it.
- railsagainst 4y agoJust to be clear, its actually the bird itself I care about. I think its identity is something that deserves respect and shouldn't just be randomly adopted by someone who thinks it is cool. I am glad that some Māori people are asserting a degree of hegemony over the use of NZ names and identities and I appreciate their intent. If you think this is ridiculous, just try and call your technology product Walmart, Amazon, The Warehouse or even the All Blacks and see what happens.