6 ms·

Wallabag: Self-hostable application for saving web pages and articles

132 points by xz18r 5y ago

4oo4 5y ago
Been using it for years, love it. Sad to see that development on it has been slow for a while, but it's pretty mature at this point.
mrtesthah 5y ago
That’s cool, but can I just have a regular app I can run on my computer with a regular GUI instead of having to “host” another persistently running service with all kinds of dependencies? Something like DevonThink, for example.
- rektide 5y ago
  It's almost viable as a standalone static backend-less webapp. But even PWAs lack the ability to talk to other endpoints & sites. Reserved for *native" apps, for now. :(
  - aaaaaaaaata 5y ago
    What are you referring to, with this? PWA is one of my pet topics, and I am not aware of any such limiting factors, and I'd like to be.
    - rektide 5y ago
      In this case Wallabag appears to use a lot of PHP, so porting it to be a self-contained standalone static page is going to be a bit tricky. But assuming porting the codebase weren't so hard, the real issue is that the web platform- even for PWAs- is cruel & iron-hearted about who can talk to who. Applications are still the only free & open communicators on the planet, and that's just fucked, so blasted wrong. CORS is probably going to block most basic interactions. Wallabag is a basic simple application, something that conceptually could easily run as a standalone static web page. But if that page tries to go fetch("https://www.washingtonpost.com https://www.washingtonpost.com") it'll probably immediately hit some CORS restrictions & die. The web got deeply mishandled when we decided it was unsafe to let sites access each other. There's been a deep freak out, an anti-connectedness that's been ruling us for the past ~2 decades, and few forces have moved the needle the other way. We're all afraid of ads & tracking, the security atmosphere is tight & hot, but it's also isolating & destroying the idea of the web, a cross-connected medium. PWAs have gotten all manner of new great APIs & access, but they still have ridiculously brutal & harsh connectivity constraints that have only gotten higher harsher more brutal year after year. (WebRTC aside, There's not been a single year where the web has gotten better able to interconnect, better able to communicate. The cruelty & hostility barometer has gone up and up and up. People misuse the web for bad, so we'll make the web worse, ad infinitum.
  - southerntofu 5y ago
    Why would you want a webapp without a backend? The web was explicitly conceived for "user agents" to interact with "servers" and the web ecosystem has very little benefits (and many downsides) outside of this use-case. If you're looking for cross-platform UI toolkits, you could take a look at Qt (for example) which easily produces static backendless (non-web)apps.
- ydlr 5y ago
  There is archivebox. https://archivebox.io/ https://archivebox.io/
- nerdponx 5y ago
  I think the Joplin model is great. The application is just an application, but you can sync your notes using WebDAV.
  - tomthe 5y ago
    Agree. I used wallabag for years, but now I just use joplin and its webclipper for the same purpose and it works really well, including the sync over syncthing! The only downside is that Joplin does not save the position where I stopped reading.
- mbreese 5y ago
  Normally I’d agree with you, but here, the major use-case is saving webpages to read on different devices. I use pocket for saving site to read later, and I don’t think it would work nearly as well if I had to restrict myself to one device. So, for this application, a web app makes a lot of sense.
  - dmje 5y ago
    Second that. Common use case is saving stuff from iPad or Android phone and doing the reading later on desktop..
- ta988 5y ago
  Zotero works ok for single pages.
- southerntofu 5y ago
  Not saying it's the best solution, but it should be really easy to run as a desktop application since it supports SQLite databases. Just have your script start a local web server serving the app, and start a web browser in app mode (or whatever that's called) on the local URL.
ohCh6zos 5y ago
I love wallabag. I just wish it loaded faster.
- southerntofu 5y ago
  I don't see a reason it should be slow. Are you close to your host on the network? If you're using a hosted version (which i believe is hosted somewhere in Europe), it may just be a connectivity problem.
  - ohCh6zos 5y ago
    I’m running it on decently nice hardware, but most of the pages take around ten seconds to load. I did some investigation and it sure looks like it gets to 100% cpu usage and stays there frequently. I’m wondering if I have PHP misconfigured.
    - southerntofu 5y ago
      Nextcloud recommends Redis so maybe that's what's missing? Otherwise check if PHP's opcache is enabled, or if your PHP version is not really old (PHP7/8 really helps).
wwaru 5y ago
I love Wallabag been using it for a year already. Tho sometimes it fails to fetch certain websites because of captchas or whatever. I kinda wish if there would be a way that would send the contents directly from my browser instead of re-fetching them.
- Havoc 5y ago
  Selfhost it and it’ll come from your ip
- kolaente 5y ago
  There's an issue about that: https://github.com/wallabag/wallabag/issues/5559 https://github.com/wallabag/wallabag/issues/5559
- danuker 5y ago
  I use archivebox + SingleFile. SingleFile is a bit hard to set up, but if it works, you get exactly what you see in the browser. It supports more extractors though - wget, curl, archive.org..., but I prefer running just one, for speed. https://archivebox.io/ https://archivebox.io/
- southerntofu 5y ago
  > Tho sometimes it fails to fetch certain websites because of captchas or whatever. I'm sorry for you. The problem is with the content provider, and i'm just as annoyed/angered as you are by this. I routinely send "complaint" emails to websites to let them know they're actively blocking users, because usually they don't even know that and they assume cloudflare & others are just blocking malicious traffic. The more complaints they receive, the more inclined they'll be to let us through.
diamondap 5y ago
I've been using this for a couple of years, and it's really nice. It works with SQLite, which makes it very lightweight. I think the default DB is MySQL/MariaDB, for multiuser installations.
nomilk 5y ago
For those already using Wallabag, curious to know what hosting options you tried, and what works best?
- dmje 5y ago
  I'm using it through Yunohost [1] on the lowest end DigitalOcean box. Works beautifully. I'm pondering bringing it in house on a RPi, just for shits and giggles and so I can truly own all my stuff... [1] https://yunohost.org/ https://yunohost.org/
- Havoc 5y ago
  I had it running on a home server for a while. Works in principle but didnt do much testing
- littlevache 5y ago
  I use it on a Yunohost instance, on a raspberry pi hosted behing the home router. Works great
alphadog 5y ago
I save links as PDFs using DevonThink. I've yet to find a compelling reason to replace it.
- kstrauser 5y ago
  Same. Capturing the contents is a good start, but being able to index them, store them next to related articles, summarize them, or get automated “see also” links is the real magic.
- jacques-noris 5y ago
  Same here. But fortunately the web clipper in DevonThink is not very good, and with the annoying cookie pop-ups it’s hit or miss if websites will be archived properly. I wish DT would work on this, but they seem to treat the web clipper as an afterthought.
anotherevan 5y ago
Now if I could only get it to integrate with my Kobo ereader the same way Pocket does, I'd jump across.
- pasdechance 5y ago
  It can: https://gitlab.com/anarcat/wallabako/ https://gitlab.com/anarcat/wallabako/ I have tried many times to switch to wallabag. That darn kobo integration is what holds me back. I've installed this on my Kobo, but it is obviously not as polished as the official Pocket integration.
- xz18r 5y ago
  I have installed KoReader[1] on my Kobo device and loving it. Less clutter and more functions in every way. Plus it has a NATIVE integration with Wallabag - I read all my articles on my eReader. [1] https://github.com/koreader/koreader https://github.com/koreader/koreader
Tomte 5y ago
I looked at Wallabag and went for Pinboard again. Even subscribed for two years. Seems, it was a mistake. Archiving stopped around 85% of my bookmarks. I've had those problems before, especially when the owner played U.S. politics and didn't care much about Pinboard anymore. I still went back. Pinboard's support doesn't reply to mails, Pinboard's Twitter account doesn't reply, either. I've asked for a refund (within the one week refund window). No reply. Maybe he'll reply here. Last time I had such problems he also ignored my mails for a long time and finally replied on HN. People, look at Wallabag seriously. Maybe Pocket. And maybe Archivebox (https://github.com/ArchiveBox/ArchiveBox https://github.com/ArchiveBox/ArchiveBox).
- ayewo 5y ago
  Have you taken a look at https://raindrop.io/ https://raindrop.io/ ? It’s actively being worked on with native apps, is competitively priced at $28/yr, though it carries the same risk as Pinboard (of being run by a solo founder).
  - southerntofu 5y ago
    It looks like raindrop is a for-profit company which does not publish source code under a free license, or at least i could not find it on the homepage. In comparison, wallabag is fully free software: you can self-host it, or you can have a commercial contract with the maintainers to host it for you. In both cases, you can donate money for development. The main developer publishes regularly about the finances of the project and the economic sustainability of free-software. The last update was only available in french though: https://nicolas.loeuillet.org/billets/2022/01/03/wallabagit-5-ans-dactivite/ https://nicolas.loeuillet.org/billets/2022/01/03/wallabagit-... To be fair, wallabag is also mostly maintained by a single person. However, there's a wide community so it's unlikely the software would go unmaintained if a bus accident occurred. I must admit i have no idea about the "hosted" side of things and if the maintainer has a backup plan for someone to take over in case they're not available.
- dak89 5y ago
  Same thing happened to me, I kept contacting him every six months or so for a couple of years and did not get a response for years after an advertised feature of the paid subcription did not work for me.. I asked for a fix or a refund, never got either. Kind of annoying to see him spend a lot of time on activism while paying for a service I never received and could not get a refund for
ilian 5y ago
For a simple solution, saving the web pages and bookmarks as plain files you can try https://www.tagspaces.org/products/webclipper/ https://www.tagspaces.org/products/webclipper/ Having my saved web pages as plain files, gives me countless possibilities in terms of portability of my collection... This way I am independent of any online service or product...
- mackrevinack 5y ago
  even simpler browser extension to download a page as a markdown file: https://github.com/deathau/markdownload https://github.com/deathau/markdownload
dusted 5y ago
Self-hostable.. Back in the dark ages, we called that "a program that runs on your computer", it seemed like a good idea at a time, and the past 10 years of putting everything back on the mainframe has confirmed that it was.
- rjzzleep 5y ago
  One thing about wallabag is that it has eink support, both through its own native app but also through third party apps like koreader. So you don't depend on your device manufacturer to support it like pocket. In that case hosted is actually the right solution. If you want a local program that stores websites with metadata as articles, you can look at zotero.
  - southerntofu 5y ago
    I think what the parent meant is that you're not tied to one exclusive service provider. Wallabag can be (and routinely is) packaged as a service by hosting coops. So you can have the benefits of having someone else hosting it for you, while not having the downside of having a single service provider whose fate may make you unable to use the product at some point in time. That's what selfhosting free-software is about.
- tenebrisalietum 5y ago
  Not quite. Self-hostable means I can run it on my home router or DMZ'ed and appropriately firewalled/secured home server and get to it via the Internet using a browser. A bit different than just a "program that runs on your computer". Unless you were dialing into your computer from your Osborne 1 using your AMPS 1G car cell phone in those dark ages, I'm sure somebody here was. It takes some skill and is not for everyone, but is a great way to truly have control of your data.
- unfocussed_mike 5y ago
  "Self-hostable" tends to just mean "do it like WordPress on your own hosting somewhere" these days; as distinct from SaaS.
southerntofu 5y ago
Love wallabag! unfortunately the new blog is almost unreadable due to very low contrast... just in case someone from wallabag project is reading this :)
pyunramura 5y ago
I was using Wallabag for years prior to around a month ago. It was largely working fine aside from being a bit slow and resource heavy. Looking around at other self-hosted options in that space brought me to a few other contenders like Shaarli (https://github.com/shaarli/Shaarli https://github.com/shaarli/Shaarli) and Shiori (https://github.com/go-shiori/shiori https://github.com/go-shiori/shiori). Shaarli has been around forever and is insanely quick to load and add links, I just wish it had some sort of web-clipping support. Shiori seems ideal as a replacement, but it was essentially unmaintained for 1+ years, though a new maintainer stepped in within the last few weeks and has been pretty active during that time, so hopefully optimistic. I'll be watching Shiori for the next few months and see where it stands with development, then decide to make the move or not.
- geppetto 5y ago
  One more vote for Shaarli. I just need tagged bookmarks and it does the job while being fast, reliable and simple. I'm also using the Firefox extension and Android app. Super easy.
d4rkp4ttern 5y ago
Wallabag does not seem to have a good way to quick-capture from mobile, unless I am missing something.
- corney91 5y ago
  I use Android's share function to add links via the Wallabag app. Would that solve what you're trying to do?
  - d4rkp4ttern 5y ago
    Thanks, but I’m on iOS. There are wallabag apps on iOS but don’t seem rated well.

132 points by xz18r 5y ago

4oo4 5y ago

Been using it for years, love it. Sad to see that development on it has been slow for a while, but it's pretty mature at this point.

mrtesthah 5y ago

That’s cool, but can I just have a regular app I can run on my computer with a regular GUI instead of having to “host” another persistently running service with all kinds of dependencies? Something like DevonThink, for example.

rektide 5y ago
It's almost viable as a standalone static backend-less webapp. But even PWAs lack the ability to talk to other endpoints & sites. Reserved for *native" apps, for now. :(
- aaaaaaaaata 5y ago
  What are you referring to, with this? PWA is one of my pet topics, and I am not aware of any such limiting factors, and I'd like to be.
  - rektide 5y ago
    In this case Wallabag appears to use a lot of PHP, so porting it to be a self-contained standalone static page is going to be a bit tricky. But assuming porting the codebase weren't so hard, the real issue is that the web platform- even for PWAs- is cruel & iron-hearted about who can talk to who. Applications are still the only free & open communicators on the planet, and that's just fucked, so blasted wrong. CORS is probably going to block most basic interactions. Wallabag is a basic simple application, something that conceptually could easily run as a standalone static web page. But if that page tries to go fetch("https://www.washingtonpost.com https://www.washingtonpost.com") it'll probably immediately hit some CORS restrictions & die. The web got deeply mishandled when we decided it was unsafe to let sites access each other. There's been a deep freak out, an anti-connectedness that's been ruling us for the past ~2 decades, and few forces have moved the needle the other way. We're all afraid of ads & tracking, the security atmosphere is tight & hot, but it's also isolating & destroying the idea of the web, a cross-connected medium. PWAs have gotten all manner of new great APIs & access, but they still have ridiculously brutal & harsh connectivity constraints that have only gotten higher harsher more brutal year after year. (WebRTC aside, There's not been a single year where the web has gotten better able to interconnect, better able to communicate. The cruelty & hostility barometer has gone up and up and up. People misuse the web for bad, so we'll make the web worse, ad infinitum.
- southerntofu 5y ago
  Why would you want a webapp without a backend? The web was explicitly conceived for "user agents" to interact with "servers" and the web ecosystem has very little benefits (and many downsides) outside of this use-case. If you're looking for cross-platform UI toolkits, you could take a look at Qt (for example) which easily produces static backendless (non-web)apps.
ydlr 5y ago
There is archivebox. https://archivebox.io/ https://archivebox.io/
nerdponx 5y ago
I think the Joplin model is great. The application is just an application, but you can sync your notes using WebDAV.
- tomthe 5y ago
  Agree. I used wallabag for years, but now I just use joplin and its webclipper for the same purpose and it works really well, including the sync over syncthing! The only downside is that Joplin does not save the position where I stopped reading.
mbreese 5y ago
Normally I’d agree with you, but here, the major use-case is saving webpages to read on different devices. I use pocket for saving site to read later, and I don’t think it would work nearly as well if I had to restrict myself to one device. So, for this application, a web app makes a lot of sense.
- dmje 5y ago
  Second that. Common use case is saving stuff from iPad or Android phone and doing the reading later on desktop..
ta988 5y ago
Zotero works ok for single pages.
southerntofu 5y ago
Not saying it's the best solution, but it should be really easy to run as a desktop application since it supports SQLite databases. Just have your script start a local web server serving the app, and start a web browser in app mode (or whatever that's called) on the local URL.

ohCh6zos 5y ago

I love wallabag. I just wish it loaded faster.

southerntofu 5y ago
I don't see a reason it should be slow. Are you close to your host on the network? If you're using a hosted version (which i believe is hosted somewhere in Europe), it may just be a connectivity problem.
- ohCh6zos 5y ago
  I’m running it on decently nice hardware, but most of the pages take around ten seconds to load. I did some investigation and it sure looks like it gets to 100% cpu usage and stays there frequently. I’m wondering if I have PHP misconfigured.
  - southerntofu 5y ago
    Nextcloud recommends Redis so maybe that's what's missing? Otherwise check if PHP's opcache is enabled, or if your PHP version is not really old (PHP7/8 really helps).

wwaru 5y ago

I love Wallabag been using it for a year already. Tho sometimes it fails to fetch certain websites because of captchas or whatever. I kinda wish if there would be a way that would send the contents directly from my browser instead of re-fetching them.

Havoc 5y ago
Selfhost it and it’ll come from your ip
kolaente 5y ago
There's an issue about that: https://github.com/wallabag/wallabag/issues/5559 https://github.com/wallabag/wallabag/issues/5559
danuker 5y ago
I use archivebox + SingleFile. SingleFile is a bit hard to set up, but if it works, you get exactly what you see in the browser. It supports more extractors though - wget, curl, archive.org..., but I prefer running just one, for speed. https://archivebox.io/ https://archivebox.io/
southerntofu 5y ago
> Tho sometimes it fails to fetch certain websites because of captchas or whatever. I'm sorry for you. The problem is with the content provider, and i'm just as annoyed/angered as you are by this. I routinely send "complaint" emails to websites to let them know they're actively blocking users, because usually they don't even know that and they assume cloudflare & others are just blocking malicious traffic. The more complaints they receive, the more inclined they'll be to let us through.

diamondap 5y ago

I've been using this for a couple of years, and it's really nice. It works with SQLite, which makes it very lightweight. I think the default DB is MySQL/MariaDB, for multiuser installations.

nomilk 5y ago

For those already using Wallabag, curious to know what hosting options you tried, and what works best?

dmje 5y ago
I'm using it through Yunohost [1] on the lowest end DigitalOcean box. Works beautifully. I'm pondering bringing it in house on a RPi, just for shits and giggles and so I can truly own all my stuff... [1] https://yunohost.org/ https://yunohost.org/
Havoc 5y ago
I had it running on a home server for a while. Works in principle but didnt do much testing
littlevache 5y ago
I use it on a Yunohost instance, on a raspberry pi hosted behing the home router. Works great

alphadog 5y ago

I save links as PDFs using DevonThink. I've yet to find a compelling reason to replace it.

kstrauser 5y ago
Same. Capturing the contents is a good start, but being able to index them, store them next to related articles, summarize them, or get automated “see also” links is the real magic.
jacques-noris 5y ago
Same here. But fortunately the web clipper in DevonThink is not very good, and with the annoying cookie pop-ups it’s hit or miss if websites will be archived properly. I wish DT would work on this, but they seem to treat the web clipper as an afterthought.

anotherevan 5y ago

Now if I could only get it to integrate with my Kobo ereader the same way Pocket does, I'd jump across.

pasdechance 5y ago
It can: https://gitlab.com/anarcat/wallabako/ https://gitlab.com/anarcat/wallabako/ I have tried many times to switch to wallabag. That darn kobo integration is what holds me back. I've installed this on my Kobo, but it is obviously not as polished as the official Pocket integration.
xz18r 5y ago
I have installed KoReader[1] on my Kobo device and loving it. Less clutter and more functions in every way. Plus it has a NATIVE integration with Wallabag - I read all my articles on my eReader. [1] https://github.com/koreader/koreader https://github.com/koreader/koreader

Tomte 5y ago

I looked at Wallabag and went for Pinboard again. Even subscribed for two years. Seems, it was a mistake. Archiving stopped around 85% of my bookmarks. I've had those problems before, especially when the owner played U.S. politics and didn't care much about Pinboard anymore. I still went back. Pinboard's support doesn't reply to mails, Pinboard's Twitter account doesn't reply, either. I've asked for a refund (within the one week refund window). No reply. Maybe he'll reply here. Last time I had such problems he also ignored my mails for a long time and finally replied on HN. People, look at Wallabag seriously. Maybe Pocket. And maybe Archivebox (https://github.com/ArchiveBox/ArchiveBox https://github.com/ArchiveBox/ArchiveBox).

ayewo 5y ago
Have you taken a look at https://raindrop.io/ https://raindrop.io/ ? It’s actively being worked on with native apps, is competitively priced at $28/yr, though it carries the same risk as Pinboard (of being run by a solo founder).
- southerntofu 5y ago
  It looks like raindrop is a for-profit company which does not publish source code under a free license, or at least i could not find it on the homepage. In comparison, wallabag is fully free software: you can self-host it, or you can have a commercial contract with the maintainers to host it for you. In both cases, you can donate money for development. The main developer publishes regularly about the finances of the project and the economic sustainability of free-software. The last update was only available in french though: https://nicolas.loeuillet.org/billets/2022/01/03/wallabagit-5-ans-dactivite/ https://nicolas.loeuillet.org/billets/2022/01/03/wallabagit-... To be fair, wallabag is also mostly maintained by a single person. However, there's a wide community so it's unlikely the software would go unmaintained if a bus accident occurred. I must admit i have no idea about the "hosted" side of things and if the maintainer has a backup plan for someone to take over in case they're not available.
dak89 5y ago
Same thing happened to me, I kept contacting him every six months or so for a couple of years and did not get a response for years after an advertised feature of the paid subcription did not work for me.. I asked for a fix or a refund, never got either. Kind of annoying to see him spend a lot of time on activism while paying for a service I never received and could not get a refund for

ilian 5y ago

For a simple solution, saving the web pages and bookmarks as plain files you can try https://www.tagspaces.org/products/webclipper/ https://www.tagspaces.org/products/webclipper/ Having my saved web pages as plain files, gives me countless possibilities in terms of portability of my collection... This way I am independent of any online service or product...

mackrevinack 5y ago
even simpler browser extension to download a page as a markdown file: https://github.com/deathau/markdownload https://github.com/deathau/markdownload

dusted 5y ago

Self-hostable.. Back in the dark ages, we called that "a program that runs on your computer", it seemed like a good idea at a time, and the past 10 years of putting everything back on the mainframe has confirmed that it was.

rjzzleep 5y ago
One thing about wallabag is that it has eink support, both through its own native app but also through third party apps like koreader. So you don't depend on your device manufacturer to support it like pocket. In that case hosted is actually the right solution. If you want a local program that stores websites with metadata as articles, you can look at zotero.
- southerntofu 5y ago
  I think what the parent meant is that you're not tied to one exclusive service provider. Wallabag can be (and routinely is) packaged as a service by hosting coops. So you can have the benefits of having someone else hosting it for you, while not having the downside of having a single service provider whose fate may make you unable to use the product at some point in time. That's what selfhosting free-software is about.
tenebrisalietum 5y ago
Not quite. Self-hostable means I can run it on my home router or DMZ'ed and appropriately firewalled/secured home server and get to it via the Internet using a browser. A bit different than just a "program that runs on your computer". Unless you were dialing into your computer from your Osborne 1 using your AMPS 1G car cell phone in those dark ages, I'm sure somebody here was. It takes some skill and is not for everyone, but is a great way to truly have control of your data.
unfocussed_mike 5y ago
"Self-hostable" tends to just mean "do it like WordPress on your own hosting somewhere" these days; as distinct from SaaS.

southerntofu 5y ago

Love wallabag! unfortunately the new blog is almost unreadable due to very low contrast... just in case someone from wallabag project is reading this :)

pyunramura 5y ago

I was using Wallabag for years prior to around a month ago. It was largely working fine aside from being a bit slow and resource heavy. Looking around at other self-hosted options in that space brought me to a few other contenders like Shaarli (https://github.com/shaarli/Shaarli https://github.com/shaarli/Shaarli) and Shiori (https://github.com/go-shiori/shiori https://github.com/go-shiori/shiori). Shaarli has been around forever and is insanely quick to load and add links, I just wish it had some sort of web-clipping support. Shiori seems ideal as a replacement, but it was essentially unmaintained for 1+ years, though a new maintainer stepped in within the last few weeks and has been pretty active during that time, so hopefully optimistic. I'll be watching Shiori for the next few months and see where it stands with development, then decide to make the move or not.

geppetto 5y ago
One more vote for Shaarli. I just need tagged bookmarks and it does the job while being fast, reliable and simple. I'm also using the Firefox extension and Android app. Super easy.

d4rkp4ttern 5y ago

Wallabag does not seem to have a good way to quick-capture from mobile, unless I am missing something.

corney91 5y ago
I use Android's share function to add links via the Wallabag app. Would that solve what you're trying to do?
- d4rkp4ttern 5y ago
  Thanks, but I’m on iOS. There are wallabag apps on iOS but don’t seem rated well.