26 ms·

I Wrote an Activitypub Server in OCaml: Lessons Learnt, Weekends Lost

154 points by gopiandcode 3y ago

mikece 3y ago
Im looking forward to a solid ActivityPub server written in Go or Rust that can run on modest hardware/small resource Docker hosts.
- mariusor 3y ago
  Well, there is one already as the reference implementation for a suite of libraries I wrote. You can find it at https://github.com/go-ap/fedbox https://github.com/go-ap/fedbox. (Contributions welcome)
  - zimpenfish 3y ago
    Does it only support C2S as the API? Are there any clients which actually support C2S rather than the Mastodon API?
    - mariusor 3y ago
      It does support server to server, but currently it does not play well with Mastodon due to its limited support of HTTP Signatures algorithms. I didn't get bothered enough by this yet to actually fix it on my side. And there are a number of clients that work with this specific brand of client to server ActivityPub but I wrote all of them. The one that can be seen on the internet is a link aggregator similar to HN and (old) reddit, you can find a demo instance at https://brutalinks.tech https://brutalinks.tech.
      - zimpenfish 3y ago
        > It does support server to server Ah, sorry, I should have said "client API" rather than just API there.
        mariusor 3y ago
        ActivityPub has a section which deals with how clients and servers should communicate with each other (called Client to Server - C2S - in the spec). So it's the same vocabulary and operations with slightly different side effects, but most servers don't implement it because it's not "specified enough". That's why developers generally just use the Mastodon API.
        zimpenfish 3y ago
        > [client apis / c2s / mastodon] Yep, I know. That's why I was asking if there was any support other than C2S because that does somewhat limit your server to "basically zero client support" which reduces its usefulness as an example somewhat?
- jeroenhd 3y ago
  I think there is (was?) an attempt to rewrite Mastodon into Rust but I haven't heard much about it. A single user Mastodon instance takes an unreasonable amount of resources. I don't know if it's just because of Ruby (Gitlab has the same problem, so it might just be) or because everyone is wasting money on expensive servers, but an RSS feed on steroid shouldn't take this much RAM.
  - deleted 3y ago
    [deleted]
  - WorldMaker 3y ago
    Mastodon itself is designed for "flagship scale" (given lead developers run mastodon.social and mastodon.online, two of the biggest instances and the most "dogfooding" two instances) so it bundles an entire cluster of services: background processors (sidekiq), caches (redis, I think?), database server (postgres), optional ElastiCache, and more. I don't know how much Ruby itself accounts for expensive overhead, but just running all of those other things on a single server vertically for a single user instance is a massive, expensive overhead. (It's clearly built for horizontal scale where your background services and caches and database servers may all be different clusters of VMs/servers over vertical stack efficiency when "scaled down" from the "natural" "mastodon.social scale" that Mastodon is most optimized for.) It's an interesting optimization problem reminder that scaling factors are different for different needs and not everything scales cleanly to every use case. A single user instance should be able to use a much smaller vertical stack, but scaling down from a wide horizontal stack is not necessarily the best or cheapest place to start when building something like that. (There are some interesting projects I've seen to build single user instances with much less overhead, shorter vertical stacks. I'm curious to see where those efforts go. In my own usage of Mastodon my "single user" instance gets the benefits of the horizontal scaling Mastodon was built for because my hosting provider does a bunch of work to make sure that they take advantage of that economy of scale to host many small instances for cheaper than trying to run small instances in one-off VMs.)
    - Xeoncross 3y ago
      This is the same problem that plagued mail servers. It takes so many different components which all have their own configurations to optimize (and memory/cpu footprints) that it ends up being too complex for most people to actually run an instance. I'm also completely positive that a Go or Rust version on a $5 VPS with an in-memory database model (RocksDB, LevelDB, BadgerDB, etc..) could easily handle hundreds of thousands of users and gigs of content each day. It seems like it would be easy to expand that to a multi-node key-value store like TiDB, CockroachDB, etc.. if you needed to grow beyond a single host.
- mxuribe 3y ago
  There are several websites out there which hope to list many ActivityPub servers (and clients) in many (programming) languages, and other implemtnation aspects...Like, here's an oldie but goodie website: https://fediverse.party/en/miscellaneous/ https://fediverse.party/en/miscellaneous/ ...There are other wbsites of course. Just select your desired lang. and review! Now, of course, it might be early days for some languages (e.g. for Rust, etc.)...But, one reason why some languages are used over others...is due to ease of deploying on VPCs and VPC-like hosts (...historically the land that php ruled ;-) Enjoy, and I hope you find what you're looking for!
- knjllppppp 3y ago
  I've had a go at doing it in Go and the ActivityPub spec is so loosely defined that it's just a real challenge if you intend to actually unmarshal the JSON you receive It's not completely impossible but you have to be okay with discarding a lot of unknown options or essentially reverse engineering the objects used by the servers you are federated with That's not to say it's impossible, I was able to crawl the network successfully, but it hints at the reason that Mastodon and Pleroma use dynamic languages I'd be very interested to see a flexible/complete AP implementation in any statically typed language Fwiw WriteFreely is implemented in Go with go-fed but -- correct me if I'm wrong -- that library seemed more limited to me than what Pleroma and Mastodon support
  - zimpenfish 3y ago
    > I'd be very interested to see a flexible/complete AP implementation in any statically typed language Try Honk[1] or GotoSocial[2]? [1] https://humungus.tedunangst.com/r/honk https://humungus.tedunangst.com/r/honk [2] https://github.com/superseriousbusiness/gotosocial https://github.com/superseriousbusiness/gotosocial
    - mariusor 3y ago
      Neither is flexible, nor strives for completion. They are both implementations that try to map the ActivityPub vocabulary on an existing web-application domain. They are not ActivityPub servers, but web-apps that use the ActivityPub vocabulary to federate, which is what I meant in the grandparent post when I mentioned the classic mistake of ActivityPub implementers. :D
  - yawaramin 3y ago
    Here's the implementation described in OP: https://github.com/Gopiandcode/ocamlot https://github.com/Gopiandcode/ocamlot OCaml is a statically-typed language. It falls somewhere between Go and Haskell on the spectrum of type 'strength'.
  - mariusor 3y ago
    I'm surprised you didn't find my library because I managed to create a statically typed vocabulary library for Go that maps the specification verbatim: https://pkg.go.dev/github.com/go-ap/activitypub#Object https://pkg.go.dev/github.com/go-ap/activitypub#Object It wasn't easy indeed, and it locked me out of some options to support execution time vocabulary extensions, but hey, it works and it's relatively easy to use.
- SideburnsOfDoom 3y ago
  > Im looking forward to a solid ActivityPub server written in Go or Rust that can run on modest hardware/small resource The "Lightweight" GoLang ActivityPub server is GoToSocial https://github.com/superseriousbusiness/gotosocial https://github.com/superseriousbusiness/gotosocial The better-known lightweight servers are Pleroma and fork Akkoma, written in Elixir https://akkoma.dev/AkkomaGang/akkoma/ https://akkoma.dev/AkkomaGang/akkoma/ Some of this info I got via: https://social.treehouse.systems/@ariadne/110226729543740723 https://social.treehouse.systems/@ariadne/110226729543740723
  - zimpenfish 3y ago
    There's also Honk[1] which is written in Go but has slightly wacky source and doesn't support the Mastodon API (but does provide an inbuilt web UI.) [1] https://humungus.tedunangst.com/r/honk https://humungus.tedunangst.com/r/honk
- bgorman 3y ago
  Ocaml code compiles to native binaries, just like Go/Rust.
- yawaramin 3y ago
  Why specifically those languages? Others can also target modest hardware/small resource Docker hosts.
  - Xeoncross 3y ago
    Sure, Zig, Nim, D, Erlang, etc.. could also do this, but Go and Rust are both big enough and just about as fast and low memory as anything that is available. Java can be faster than Go, but not by much and I've always seen it to use 5-10x the memory. Scripting languages like Typescript, Python, PHP and Ruby can't hold a candle to the speed of Rust and Go while also using significantly more memory. They also don't natively support multiple cores / threads. Rust and Go represent the most approachable middle ground on all accounts of familiarity, performance (allocs and calculation speed), and large communities with libraries covering whatever you could want.
    - saagarjha 3y ago
      C?
      - Xeoncross 3y ago
        I mean, nothing beats C (yet), but that isn't actually what most devs are writing web services in. Rust overs better guarantees, but even that is too complex for the masses so Go is probably the next largest community that still offers great performance. Java, PHP, Python & Node.js have massive market share, yet none of them are as ideal for the job.
  - sangnoir 3y ago
    Pleroma (written in Elixir) is one of the lighter, Mastodon-compatible AP servers available. I recently read a post (a toot actually, but I hate that term) by a Mastodon administrator observing that that Pleroma is often a common thread to problematic Fediverse instances because it can run on cheap VPS boxes on throwaway domains. Spammers/griefers can cause a lot of moderation problems for the same amount as a Twitter Blue subscription. It is dubious endorsement, but I think it shows how much more efficient Pleroma is than other popular, easy-to-use-OOTB AP servers: 9 out of 10 price-conscious griefers use and endorse Pleroma
mariusor 3y ago
The author makes the basic mistake of most of the people implementing ActivityPub services: they want to map the logic of an existing type of web application and contort existing domain objects into encoding/decoding to an "impractically large number" of options. That happens because they want two things in one: a server and a client. The ActivityPub specification needs to be read with a goal similar to an email server in mind. It should do one thing: receive JSON-LD objects in inbox, process them according to the specification, and(maybe) store them on disk. The idea of "users", "friends", "posts", "feeds" etc, are concepts that belong to the clients on top of this server, not in the server itself. This separation between clients and server will also allow better interop/graceful degradation of object types that the client/server don't specifically understand.
- cratermoon 3y ago
  OK, but for someone who wants to build a useful tool that does what the author wants, "interacting with the Fediverse", such as federating with Mastodon, how useful is doing that one thing?
  - jeroenhd 3y ago
    It depends on your goal. If your server is just a tool you use, you can ignore lot of concepts. There is no local timeline, there are no users, all follows belong to a single user, etc. I can't find the link but a while back there was a post on the front page about how to get a findable, read only ActivityPub profile by just uploading some static JSON files. Not exactly a Twitter competitor, but you don't need much to start exchanging messages.
    - mdasen 3y ago
      I believe you're looking for this: https://blog.joinmastodon.org/2018/06/how-to-implement-a-basic-activitypub-server/ https://blog.joinmastodon.org/2018/06/how-to-implement-a-bas...
      - cratermoon 3y ago
throwaway290 3y ago
There's also LitePub, though development seems stalled (?)
- riffic 3y ago
  link for reference: https://litepub.social/ https://litepub.social/
- yawaramin 3y ago
  Was it developed at all? I'm not seeing any business logic in the repo: https://hacktivis.me/git/litepub.social/files.html https://hacktivis.me/git/litepub.social/files.html
  - throwaway290 3y ago
    It was and has some pros compared to AP...
erwinh 3y ago
A bit off-topic but the post title will probably attract relevant people. What are the thoughts on OCaml on HN?
- cccbbbaaa 3y ago
  It replaced Python for everything longer than a couple hundred of lines long for me. Fast language, fast compile times, clean(-ish) syntax, strong typing system, good ecosystem, and now multicore support? Yes please! I must be more nuanced, though: existing libraries in opam are generally very, very good (I really like cmdliner), but many things may be missing. There is no alternative to Django, for instance. No serious IDE, except emacs. The standard library was so lacking that there is at least an alternative. The situation improved, but there's still missing stuff compared to Python.
  - amelius 3y ago
    Do you make GUIs in OCaml, and which libraries do or would you use? And how about scientific computing (SciPy), deep learning (PyTorch etc.), or computational geometry (Shapely etc.)?
    - yw3410 3y ago
      GUIs are a PITA like in most languages. I think most people use something which binds to gtk (such as lablgtk) or Qt. For scientific computing there is Owl, but I haven't used it personally.
      - amelius 3y ago
        Hmm, I'll stick with Python for now. The ecosystem of libraries is just too good. Perhaps if OCaml made it very easy to interoperate with Python, I could give it a chance.
        yw3410 3y ago
        There is pyml for interopt but I've never used it.
SideburnsOfDoom 3y ago
My question is this: if I was to try to hack up an ActivityPub server in my platform of choice, how would I know how compliant it is? Is there any compliance test suite to verify this? "Try and load it up in a client app" seems suboptimal. "load it up and see" attitude is part of what made parsing and renderings HTML so hairy, and compliance test suites helped.
- mariusor 3y ago
  There was a suite of tests, that sadly fell to bitrot. One of the developers in the community created a parallel application that could test implementations, but then this too ended up unmaintained[1]. [1] https://github.com/go-fed/testsuite https://github.com/go-fed/testsuite
nologic01 3y ago
I found the post well written and informative. Though I am clueless about OCaml it feels as this would be useful for anybody working on a new server implementation in any language ecosystem as it highlights what needs to be done and potential bottlenecks. As for the activitypub spec and the currently popular implementations it doesnt take long exposure to the fediverse to realise there are some rough edges and historical accidents (e.g mastodon being actually the defacto interpretation of the standard). Imho now that there is substantial more mindshare devoted to decentralized social it would be opportune to revisit these things and if needed revise before they get backed in.
dahwolf 3y ago
Saw some comments on the protocol being fluffy and typical implementations resource hungry. This is an interesting guy to follow: https://universeodon.com/@supernovae https://universeodon.com/@supernovae He's the admin of universeodon, a mastodon instance with 13K MAU. He recently shared that in a month's time, 3TB of text was transferred just in ActivityPub events. Images a multiple of it. I don't know what the bill is, but I was pretty shocked by the stats...for "just" 13K users. And the cruel thing is that it still doesn't work properly. Likes/boosts and replies do not properly synchronize.
- still_grokking 3y ago
  But people are saying that the Fediverse could replace Twitter and Facebook and Tiktok and Instagram and what not, don't they? How much hardware would you need for 100 million MAUs? (And that's just a fraction of the current social media users.) If it really "scales" like indicated in the parent post this tech will never provide any alternative to centralized social media sites just for technical reasons no matter what people want or do. Maybe someone experienced in effective distributed systems should start to design an alternative. Otherwise there won't be any viable alternative to the commercial silos no matter how bad people would like one.
  - dahwolf 3y ago
    Mastodon absolutely does not scale that far and this should be considered fact. Far smaller instance owners sometimes complain about bills of hundreds of USD per month. So the only way to scale up "indefinitely" is by having many small/medium-sized instances, but not really. In a 100M+ network, instances will suffer due to the wasteful nature of federation plus social media being append-only. Costs will forever go up and this doesn't even mention the burden and liabilities of moderation. Amidst all our hate for big social, we've forgotten about all the things they do very well. They are (financially) free. They are reliable and scale up without you noticing. You do not have to generally worry about your entire account and content being gone because some mod gave up. If you don't do anything funny, all your content is preserved, forever. Moderation works reasonably well, even if never perfect. We've taken all that for granted. But it costs billions, an army of engineers, mods, legal, marketing, UX, top notch infrastructure to make it run and work this smoothly. The idea that a bunch of enthusiasts can replicate this, is misplaced.

154 points by gopiandcode 3y ago

Im looking forward to a solid ActivityPub server written in Go or Rust that can run on modest hardware/small resource Docker hosts.

mariusor 3y ago
Well, there is one already as the reference implementation for a suite of libraries I wrote. You can find it at https://github.com/go-ap/fedbox https://github.com/go-ap/fedbox. (Contributions welcome)
- zimpenfish 3y ago
  Does it only support C2S as the API? Are there any clients which actually support C2S rather than the Mastodon API?
  - mariusor 3y ago
    It does support server to server, but currently it does not play well with Mastodon due to its limited support of HTTP Signatures algorithms. I didn't get bothered enough by this yet to actually fix it on my side. And there are a number of clients that work with this specific brand of client to server ActivityPub but I wrote all of them. The one that can be seen on the internet is a link aggregator similar to HN and (old) reddit, you can find a demo instance at https://brutalinks.tech https://brutalinks.tech.
    - zimpenfish 3y ago
      > It does support server to server Ah, sorry, I should have said "client API" rather than just API there.
      - mariusor 3y ago
        ActivityPub has a section which deals with how clients and servers should communicate with each other (called Client to Server - C2S - in the spec). So it's the same vocabulary and operations with slightly different side effects, but most servers don't implement it because it's not "specified enough". That's why developers generally just use the Mastodon API.
        zimpenfish 3y ago
        > [client apis / c2s / mastodon] Yep, I know. That's why I was asking if there was any support other than C2S because that does somewhat limit your server to "basically zero client support" which reduces its usefulness as an example somewhat?
jeroenhd 3y ago
I think there is (was?) an attempt to rewrite Mastodon into Rust but I haven't heard much about it. A single user Mastodon instance takes an unreasonable amount of resources. I don't know if it's just because of Ruby (Gitlab has the same problem, so it might just be) or because everyone is wasting money on expensive servers, but an RSS feed on steroid shouldn't take this much RAM.
- deleted 3y ago
  [deleted]
- WorldMaker 3y ago
  Mastodon itself is designed for "flagship scale" (given lead developers run mastodon.social and mastodon.online, two of the biggest instances and the most "dogfooding" two instances) so it bundles an entire cluster of services: background processors (sidekiq), caches (redis, I think?), database server (postgres), optional ElastiCache, and more. I don't know how much Ruby itself accounts for expensive overhead, but just running all of those other things on a single server vertically for a single user instance is a massive, expensive overhead. (It's clearly built for horizontal scale where your background services and caches and database servers may all be different clusters of VMs/servers over vertical stack efficiency when "scaled down" from the "natural" "mastodon.social scale" that Mastodon is most optimized for.) It's an interesting optimization problem reminder that scaling factors are different for different needs and not everything scales cleanly to every use case. A single user instance should be able to use a much smaller vertical stack, but scaling down from a wide horizontal stack is not necessarily the best or cheapest place to start when building something like that. (There are some interesting projects I've seen to build single user instances with much less overhead, shorter vertical stacks. I'm curious to see where those efforts go. In my own usage of Mastodon my "single user" instance gets the benefits of the horizontal scaling Mastodon was built for because my hosting provider does a bunch of work to make sure that they take advantage of that economy of scale to host many small instances for cheaper than trying to run small instances in one-off VMs.)
  - Xeoncross 3y ago
    This is the same problem that plagued mail servers. It takes so many different components which all have their own configurations to optimize (and memory/cpu footprints) that it ends up being too complex for most people to actually run an instance. I'm also completely positive that a Go or Rust version on a $5 VPS with an in-memory database model (RocksDB, LevelDB, BadgerDB, etc..) could easily handle hundreds of thousands of users and gigs of content each day. It seems like it would be easy to expand that to a multi-node key-value store like TiDB, CockroachDB, etc.. if you needed to grow beyond a single host.
mxuribe 3y ago
There are several websites out there which hope to list many ActivityPub servers (and clients) in many (programming) languages, and other implemtnation aspects...Like, here's an oldie but goodie website: https://fediverse.party/en/miscellaneous/ https://fediverse.party/en/miscellaneous/ ...There are other wbsites of course. Just select your desired lang. and review! Now, of course, it might be early days for some languages (e.g. for Rust, etc.)...But, one reason why some languages are used over others...is due to ease of deploying on VPCs and VPC-like hosts (...historically the land that php ruled ;-) Enjoy, and I hope you find what you're looking for!
knjllppppp 3y ago
I've had a go at doing it in Go and the ActivityPub spec is so loosely defined that it's just a real challenge if you intend to actually unmarshal the JSON you receive It's not completely impossible but you have to be okay with discarding a lot of unknown options or essentially reverse engineering the objects used by the servers you are federated with That's not to say it's impossible, I was able to crawl the network successfully, but it hints at the reason that Mastodon and Pleroma use dynamic languages I'd be very interested to see a flexible/complete AP implementation in any statically typed language Fwiw WriteFreely is implemented in Go with go-fed but -- correct me if I'm wrong -- that library seemed more limited to me than what Pleroma and Mastodon support
- zimpenfish 3y ago
  > I'd be very interested to see a flexible/complete AP implementation in any statically typed language Try Honk[1] or GotoSocial[2]? [1] https://humungus.tedunangst.com/r/honk https://humungus.tedunangst.com/r/honk [2] https://github.com/superseriousbusiness/gotosocial https://github.com/superseriousbusiness/gotosocial
  - mariusor 3y ago
    Neither is flexible, nor strives for completion. They are both implementations that try to map the ActivityPub vocabulary on an existing web-application domain. They are not ActivityPub servers, but web-apps that use the ActivityPub vocabulary to federate, which is what I meant in the grandparent post when I mentioned the classic mistake of ActivityPub implementers. :D
- yawaramin 3y ago
  Here's the implementation described in OP: https://github.com/Gopiandcode/ocamlot https://github.com/Gopiandcode/ocamlot OCaml is a statically-typed language. It falls somewhere between Go and Haskell on the spectrum of type 'strength'.
- mariusor 3y ago
  I'm surprised you didn't find my library because I managed to create a statically typed vocabulary library for Go that maps the specification verbatim: https://pkg.go.dev/github.com/go-ap/activitypub#Object https://pkg.go.dev/github.com/go-ap/activitypub#Object It wasn't easy indeed, and it locked me out of some options to support execution time vocabulary extensions, but hey, it works and it's relatively easy to use.
SideburnsOfDoom 3y ago
> Im looking forward to a solid ActivityPub server written in Go or Rust that can run on modest hardware/small resource The "Lightweight" GoLang ActivityPub server is GoToSocial https://github.com/superseriousbusiness/gotosocial https://github.com/superseriousbusiness/gotosocial The better-known lightweight servers are Pleroma and fork Akkoma, written in Elixir https://akkoma.dev/AkkomaGang/akkoma/ https://akkoma.dev/AkkomaGang/akkoma/ Some of this info I got via: https://social.treehouse.systems/@ariadne/110226729543740723 https://social.treehouse.systems/@ariadne/110226729543740723
- zimpenfish 3y ago
  There's also Honk[1] which is written in Go but has slightly wacky source and doesn't support the Mastodon API (but does provide an inbuilt web UI.) [1] https://humungus.tedunangst.com/r/honk https://humungus.tedunangst.com/r/honk
bgorman 3y ago
Ocaml code compiles to native binaries, just like Go/Rust.
yawaramin 3y ago
Why specifically those languages? Others can also target modest hardware/small resource Docker hosts.
- Xeoncross 3y ago
  Sure, Zig, Nim, D, Erlang, etc.. could also do this, but Go and Rust are both big enough and just about as fast and low memory as anything that is available. Java can be faster than Go, but not by much and I've always seen it to use 5-10x the memory. Scripting languages like Typescript, Python, PHP and Ruby can't hold a candle to the speed of Rust and Go while also using significantly more memory. They also don't natively support multiple cores / threads. Rust and Go represent the most approachable middle ground on all accounts of familiarity, performance (allocs and calculation speed), and large communities with libraries covering whatever you could want.
  - saagarjha 3y ago
    C?
    - Xeoncross 3y ago
      I mean, nothing beats C (yet), but that isn't actually what most devs are writing web services in. Rust overs better guarantees, but even that is too complex for the masses so Go is probably the next largest community that still offers great performance. Java, PHP, Python & Node.js have massive market share, yet none of them are as ideal for the job.
- sangnoir 3y ago
  Pleroma (written in Elixir) is one of the lighter, Mastodon-compatible AP servers available. I recently read a post (a toot actually, but I hate that term) by a Mastodon administrator observing that that Pleroma is often a common thread to problematic Fediverse instances because it can run on cheap VPS boxes on throwaway domains. Spammers/griefers can cause a lot of moderation problems for the same amount as a Twitter Blue subscription. It is dubious endorsement, but I think it shows how much more efficient Pleroma is than other popular, easy-to-use-OOTB AP servers: 9 out of 10 price-conscious griefers use and endorse Pleroma

mariusor 3y ago

The author makes the basic mistake of most of the people implementing ActivityPub services: they want to map the logic of an existing type of web application and contort existing domain objects into encoding/decoding to an "impractically large number" of options. That happens because they want two things in one: a server and a client. The ActivityPub specification needs to be read with a goal similar to an email server in mind. It should do one thing: receive JSON-LD objects in inbox, process them according to the specification, and(maybe) store them on disk. The idea of "users", "friends", "posts", "feeds" etc, are concepts that belong to the clients on top of this server, not in the server itself. This separation between clients and server will also allow better interop/graceful degradation of object types that the client/server don't specifically understand.

cratermoon 3y ago
OK, but for someone who wants to build a useful tool that does what the author wants, "interacting with the Fediverse", such as federating with Mastodon, how useful is doing that one thing?
- jeroenhd 3y ago
  It depends on your goal. If your server is just a tool you use, you can ignore lot of concepts. There is no local timeline, there are no users, all follows belong to a single user, etc. I can't find the link but a while back there was a post on the front page about how to get a findable, read only ActivityPub profile by just uploading some static JSON files. Not exactly a Twitter competitor, but you don't need much to start exchanging messages.
  - mdasen 3y ago
    I believe you're looking for this: https://blog.joinmastodon.org/2018/06/how-to-implement-a-basic-activitypub-server/ https://blog.joinmastodon.org/2018/06/how-to-implement-a-bas...
    - cratermoon 3y ago

throwaway290 3y ago

There's also LitePub, though development seems stalled (?)

riffic 3y ago
link for reference: https://litepub.social/ https://litepub.social/
yawaramin 3y ago
Was it developed at all? I'm not seeing any business logic in the repo: https://hacktivis.me/git/litepub.social/files.html https://hacktivis.me/git/litepub.social/files.html
- throwaway290 3y ago
  It was and has some pros compared to AP...

erwinh 3y ago

A bit off-topic but the post title will probably attract relevant people. What are the thoughts on OCaml on HN?

cccbbbaaa 3y ago
It replaced Python for everything longer than a couple hundred of lines long for me. Fast language, fast compile times, clean(-ish) syntax, strong typing system, good ecosystem, and now multicore support? Yes please! I must be more nuanced, though: existing libraries in opam are generally very, very good (I really like cmdliner), but many things may be missing. There is no alternative to Django, for instance. No serious IDE, except emacs. The standard library was so lacking that there is at least an alternative. The situation improved, but there's still missing stuff compared to Python.
- amelius 3y ago
  Do you make GUIs in OCaml, and which libraries do or would you use? And how about scientific computing (SciPy), deep learning (PyTorch etc.), or computational geometry (Shapely etc.)?
  - yw3410 3y ago
    GUIs are a PITA like in most languages. I think most people use something which binds to gtk (such as lablgtk) or Qt. For scientific computing there is Owl, but I haven't used it personally.
    - amelius 3y ago
      Hmm, I'll stick with Python for now. The ecosystem of libraries is just too good. Perhaps if OCaml made it very easy to interoperate with Python, I could give it a chance.
      - yw3410 3y ago
        There is pyml for interopt but I've never used it.

SideburnsOfDoom 3y ago

My question is this: if I was to try to hack up an ActivityPub server in my platform of choice, how would I know how compliant it is? Is there any compliance test suite to verify this? "Try and load it up in a client app" seems suboptimal. "load it up and see" attitude is part of what made parsing and renderings HTML so hairy, and compliance test suites helped.

mariusor 3y ago
There was a suite of tests, that sadly fell to bitrot. One of the developers in the community created a parallel application that could test implementations, but then this too ended up unmaintained[1]. [1] https://github.com/go-fed/testsuite https://github.com/go-fed/testsuite

nologic01 3y ago

I found the post well written and informative. Though I am clueless about OCaml it feels as this would be useful for anybody working on a new server implementation in any language ecosystem as it highlights what needs to be done and potential bottlenecks. As for the activitypub spec and the currently popular implementations it doesnt take long exposure to the fediverse to realise there are some rough edges and historical accidents (e.g mastodon being actually the defacto interpretation of the standard). Imho now that there is substantial more mindshare devoted to decentralized social it would be opportune to revisit these things and if needed revise before they get backed in.

dahwolf 3y ago

Saw some comments on the protocol being fluffy and typical implementations resource hungry. This is an interesting guy to follow: https://universeodon.com/@supernovae https://universeodon.com/@supernovae He's the admin of universeodon, a mastodon instance with 13K MAU. He recently shared that in a month's time, 3TB of text was transferred just in ActivityPub events. Images a multiple of it. I don't know what the bill is, but I was pretty shocked by the stats...for "just" 13K users. And the cruel thing is that it still doesn't work properly. Likes/boosts and replies do not properly synchronize.

still_grokking 3y ago
But people are saying that the Fediverse could replace Twitter and Facebook and Tiktok and Instagram and what not, don't they? How much hardware would you need for 100 million MAUs? (And that's just a fraction of the current social media users.) If it really "scales" like indicated in the parent post this tech will never provide any alternative to centralized social media sites just for technical reasons no matter what people want or do. Maybe someone experienced in effective distributed systems should start to design an alternative. Otherwise there won't be any viable alternative to the commercial silos no matter how bad people would like one.
- dahwolf 3y ago
  Mastodon absolutely does not scale that far and this should be considered fact. Far smaller instance owners sometimes complain about bills of hundreds of USD per month. So the only way to scale up "indefinitely" is by having many small/medium-sized instances, but not really. In a 100M+ network, instances will suffer due to the wasteful nature of federation plus social media being append-only. Costs will forever go up and this doesn't even mention the burden and liabilities of moderation. Amidst all our hate for big social, we've forgotten about all the things they do very well. They are (financially) free. They are reliable and scale up without you noticing. You do not have to generally worry about your entire account and content being gone because some mod gave up. If you don't do anything funny, all your content is preserved, forever. Moderation works reasonably well, even if never perfect. We've taken all that for granted. But it costs billions, an army of engineers, mods, legal, marketing, UX, top notch infrastructure to make it run and work this smoothly. The idea that a bunch of enthusiasts can replicate this, is misplaced.