12 ms·

Woob: Web Outside of Browsers

331 points by goranmoomin 5y ago

togaen 5y ago
but… why
btrettel 5y ago
A while back I heard about Z39.50 [0], a protocol that libraries use for their catalogs. In the 90s it seems there were native clients for the protocol so that one could interact with the library catalog without using a web interface. A lot of the current web interfaces are terribly slow JS monstrosities now so I'd like to try something faster. I never did figure out if any of the GUI clients [1] are still actively developed and I'd appreciate if anyone who knows about this could point me towards a good client. [0] https://en.wikipedia.org/wiki/Z39.50 https://en.wikipedia.org/wiki/Z39.50 [1] Some software listed here: http://www.loc.gov/z3950/agency/resources/software.html http://www.loc.gov/z3950/agency/resources/software.html
- librarianscott 5y ago
  The most well-known Z39.50 using software that still works is a paid citation manager called EndNote (the desktop client not the web client). You can ingest PDFs with that software too. I don't think the open-source Zotero has added that feature yet. Z39.50 is clunky, and our old LMS (library management software, used behind the scenes) can pull in MARC records for our staff when we order books, but I don't even want to think about the security.
pavlov 5y ago
It's like a peek at an alternative Internet where Gopher won instead of WWW.
- mxuribe 5y ago
  There has been a desire to go back to older days where text was more prevalent, and so Gemini has begin and is gaining popularity - though I'm sure very slowly. See: https://gemini.circumlunar.space/ https://gemini.circumlunar.space/ Also, separately (though i wouild not be surprised if frequented by same/similar folks who have interets in Gemini), there is also the tildeverse...again, more text-heavy environments. See: https://tildeverse.org/ https://tildeverse.org/ And, as i have stated in another comment there is the fediverse (e.g. Mastodon, pleroma, etc.), so the ability to leverage APIs to interact with other folks and their content without explicitly needing a typical web browser exists, and flourishes. I'll end by stating that there are a few exciting things - like the above items i mention as well as this neat Woob platform - which to me seem very fun, a little new, and yet at the same time in some ways nostalgic...maybe they won't make the morning news, and likely only attract geeks, but it is all still exciting - at least for me!
  - Qub3d 5y ago
    The tildeverse looks pretty cool, but why does it redirect me to a Rick Roll on Firefox? I can see the normal site just fine on Lynx browser (maybe that's the point?) Edit: ah, I see, they're doing a JWZ and redirecting based on referral, but going a step further and setting a cookie. Cute, but also terribly immature.
    - mxuribe 5y ago
      Oops, sorry about that. I never had an issue before...but funny after visiting HN, see exactly what you mean. (Clearing the cookies avoids the classic rick roll video). Anyway, yeah i guess the tilde folks are "characters". But, separate of that, the community i've interacted with is quite fun, respectful, and good-natured. I also failed to mention a sort of equivalent to HN, which i frequent (with similar topics to HN but often nicer crowd): https://tildes.net/ https://tildes.net/ ...Tilde.net used to be only open to invite not open to anyone creating an account...so if interested - and still not open to the public - i can trigger to send you an invite.
brutal_chaos_ 5y ago
This is clever and fantastic. I have been pondering a similar concept recently and I think I would like to contribute. I'm curious as to why LGPL-3 was chosen as the license, though, not that the license is a show stopper.
- Zababa 5y ago
  One explanation might be that the project is mostly French, and the GPL/LGPL seems more popular in France than in the USA.
res0nat0r 5y ago
Also: woob - 1994 is one of the best ambient albums ever made. https://www.youtube.com/watch?v=0S3owK3pN64 https://www.youtube.com/watch?v=0S3owK3pN64
notorandit 5y ago
I do exactly the other way around on my smartphone: if there is a web app i won't install the app.
dylan-m 5y ago
I like the idea of this. There's so much information on the web, but we still need a way to bring that information to other applications, without being tied to a particular source. That was really the dream of the semantic web, after all. This kind of idea would be really nicely paired with good Microformats[1] support, which continues to be a very good idea. That way we can find, say, a recipe or an address on a web page in a reusable way and without needing magical heuristics. (Of course, "reusable" in theory, with the caveat that everybody forgot about microformats around when Google decided they could machine learn their way out of everything). [1] http://microformats.org http://microformats.org
- MaxBarraclough 5y ago
  > I like the idea of this. There's so much information on the web, but we still need a way to bring that information to other applications, without being tied to a particular source. I'm not sure I'm interpreting you correctly here, but I think I'm on the other side of this. The problem is that many modern websites are godawful. I think the story pretty much ends there. If websites were not awful, we wouldn't find ourselves appalled by the idea of just embedding a browser. Modern web browsers feature a 'reader mode' as a countermeasure to that much modern web design is significantly worse than having no web design at all. If you're serious about a 'lightweight' alternative to the lumbering horror-show of the modern web, the way forward is either Gemini [0], or a formalised simple subset of HTML. [1] > That way we can find, say, a recipe or an address on a web page in a reusable way and without needing magical heuristics. I think the find and reusable aspects here are really two very different problems. The reusable part is easy. HTML is already reusable. A standardised simple subset would be even more so. [1] The find part is trickier. Discovering decent content is harder, as there's an arms race of ad-funded spammers trying to out-compete legitimate recipe sites in search-engine rankings. (There's also the possibility of search engines not being motivated to work on delivering good search results. [2]) The idea of having a choice between native GUI applications and web apps, has been with us for some time. Email is probably the best example, we've long had the choice between webmail and native email clients. Beyond webmail, these days even Microsoft Word has a web-based version. There are of course both advantages and disadvantages to web-based applications. [0] https://news.ycombinator.com/item?id=23730408 https://news.ycombinator.com/item?id=23730408 [1] https://news.ycombinator.com/item?id=29291392 https://news.ycombinator.com/item?id=29291392 [2] https://news.ycombinator.com/item?id=29772136 https://news.ycombinator.com/item?id=29772136
0x445442 5y ago
Believe it or not, around the turn of the century there were many thick client apps. But back then it was a challenge to ship and update these applications. This pain, along with the continued rollout of broadband led many to advocate for creating applications that would run in a web browser while being controlled on centralized servers. In practice, turning the platform that was designed to render markup text into an application host. This would allow applications to be shipped and updated with little interaction from the user. However, right about the same time web apps were taking over the world there were thick client apps that were solving the problems of installation and and updates. Two of the prominent thick client applications doing this were iTunes and the browsers themselves. Now fast forward a decade to the early teens and the ubiquitous use of smart phones. What is the single largest determining factor of platform success? Is it the ability for web apps to render on your platform's web browser or is it the breadth and depth of your platform's app store? My rant is over, I wish web apps would die. I've wished that for most of the 21st century.
- mikehollinger 5y ago
  > Now fast forward a decade to the early teens and the ubiquitous use of smart phones. An interesting thing happened here as well - because of the iOS app store and its auto-update policy, "thick" clients came back in vogue again. It just happened that they're running on phones, not full PCs. I'd argue these count as thick client apps. :-) Consider WhatsApp or WeChat - it has a complete and full ecosystem of messaging, apps, and utility function. It actually (I think) supercedes AOL 2 or 3 in terms of functionality. There's DMs, Chat, Apps...
- redsparrow 5y ago
  This is an interesting observation. I hadn't noticed this dynamic before; thanks for sharing. The current differences in attitudes between mobile and non-mobile apps is pretty interesting. If you visit Reddit or Twitter from a browser that's running on a mobile OS then they make it extremely clear that they would prefer that you use a dedicated app. When visiting from a non-mobile OS not only do you not see such messages, but a dedicated app doesn't even exist. In one place the browser is good enough to be the only option, in the other it's so inferior that they will bully and harass you until you stop using it.
kadomony 5y ago
Woobs out
kodablah 5y ago
I think a version of this is what the internet needs but using headless browsers from the client and with a somewhat-centrally curated set of scraper "recipes" if you will. Basically a community curated/updated set of scraper logic per site (yes some trust is required) that essentially provides JSON data and/or APIs based on the site. Even just a neutered HTML equivalent of sites (e.g. amp w/out the Google and ads stuff) would be good. Since it is all client side, it can be dubbed a "browser" not a "scraper" and one might hope popularity is high enough that active blocking of it is blatantly user hostile. Granted one hopes that, like EasyList and uBO and others have shown, the community can outpace site owners. Not appearing headless (tunneling captchas, literal mousemove events in pseudo-random human-like ways, etc) should be doable. It's something I have thought about and once dubbed "recapitate" (https://github.com/cretz/software-ideas/issues/82 https://github.com/cretz/software-ideas/issues/82) and plan to revisit. I have seen many versions of this attempted. We need to encourage shared data extraction tools.
louissan 5y ago
woob woob woob! Battletech pulse lasers anyone?
vmception 5y ago
Reading the linked site and some of the discussion, I highly recommend finding your nearest Chinese friend or person and getting authorized on Wechat. Its a whole parallel other internet! Kinda similar to how a private set of Facebook pages are otherwise inaccessible with an account, except in a parellel reality where people use them for all business and have no other internet presence. Yes, as a user another government gets to read your posts, but I mean yet another. To get on, I literally just knocked on a few doors in San Francisco and got authorized, so many people here can too. You could probably do it at a park. Note: Hong Kong citizens cannot do it for US citizens even I was trying. Has to be a mainland Chinese person.
- vorpalhex 5y ago
  Weibo is a heavily censored and manipulated platform. The CCP uses it to track dissidents abroad. Do not install or operate on any trusted device. Do not connect to your home network. Do not store personal details on your weibo device. Do not ever send sensitive information or talk to real contacts on Weibo.
  - vmception 5y ago
    As you know, it is not possible to separate corporate China from government China due to the structure of that system so no analogy exactly works, but our day to day experience is exceedingly similar as we rely on heavily censored and manipulated private platforms. Although we retain options to express ideas, there is not enough saturation of other people to view those ideas except to participate on heavily censored platforms and risk complete deplatforming. Its an almost daily topic here, for example. So the user experience on a Chinese service is simply not different enough for me to treat it differently.
    - vorpalhex 5y ago
      I'm someone who is heavily critical of Facebook and Twitter. However there is a world of difference between sloppily shutting down vaccine or election misinformation, and actively censoring a Tennis star reporting a sexual assault by a politiburo member. There is a world of difference between taking down shit talking politicians twitter profiles, and actively censoring the genocide of an ethnic minority. Ads that slurp up personal data are bad. Threatening political dissidents abroad directly is incomparable. To say "well they are all the same bad" is to be willfully blind to the basic facts. [1] - https://www.theatlantic.com/technology/archive/2019/03/what-went-wrong-chinas-weibo-social-network/584728/ https://www.theatlantic.com/technology/archive/2019/03/what-... [2] - https://www.thecut.com/2021/12/the-disappearance-of-peng-shuai-what-we-know.html https://www.thecut.com/2021/12/the-disappearance-of-peng-shu...
mahastore 5y ago
Good example of another solution without a problem.
- mkdirp 5y ago
  Really? You ever tried automating your own data from e.g. your bank? Cos I have, and it's a lot more annoying that it should be. Woob does a lot more than just banks. It allows you to get any of your data. Adding additional providers is piss easy too.
anderspitman 5y ago
Something I've been thinking about lately is how browsers have essentially become a dependency for any sort of auth on the internet. Pretty much everything uses OAuth2, which requires you to be able to render HTML and CSS, and in many implementations JavaScript. That's ~20M (Firefox) to ~30M (Chromium) lines of code as a dependency for your application, just for auth. This applies even if you have a slick CLI app like rclone. If you want to connect it to Google drive you still need a browser to do the OAuth2 flow. All of this just so we have a safe, known location to stash auth cookies. It would be sweet if there was a lightweight protocol where you could lay out a basic consent UI (maybe with a simple JSON format) that can be rendered outside the browser. Then you need a way to connect to a central trusted cookie store. You could still redirect to a separate app, but it wouldn't need to be nearly as complicated as a browser.
- jl2718 5y ago
  SSL cert auth is recognized by every major HTTP server and used by every major browser. Frankly I don’t understand how this still is or ever was an issue.
- dmitriid 5y ago
  > All of this just so we have a safe, known location to stash auth cookies. > It would be sweet if there was a lightweight protocol where you could lay out a basic consent UI This also feels like it should be a part of the OS, since it's "select user, input password, and maybe 2fa, store auth". It doesn't need a full-blown web-browser. It does, however, needs a properly defined protocol.
  - anderspitman 5y ago
    Maybe someone needs to pull a "WireGuard for auth" and make a slick stripped-down implementation and spec and get it included in the kernel.
- Wowfunhappy 5y ago
  While I absolutely share the philosophical concern, I do wonder how large of an issue this is in practice. On an IOT device, or a retrocomputing/hobbyist platform, you'd likely want to display a QR code and have the user authenticate with their phone, similar to how you log into Netflix on a smart TV. As an aside, to my knowledge OAuth2 still works in PaleMoon. I just downloaded the source and did a count with CLOC, and it looks like there's "only" ~13.5M lines of code. :)
synergy20 5y ago
Browser is the new UI for 90% of GUI use cases, I'm all in, be it Electron or just the browser.
cglong 5y ago
Not a single comparison to Sherlock/Watson? I always thought it was a great idea, because it allowed a consistent interface for theoretically finding anything on the internet.
1vuio0pswjnm7 5y ago
Been using web outside of browser for last 10+ years. I am addicted to it, having written many scripts and programs to fetch and process HTML and other filetypes outside the browser. I can retrieve and extract data/information much faster and more efficiently using simpler programs that are small and work together. For viewing HTML, I prefer my text-only browser. HTML looks much better, less variable, more uniform, than in a graphical browser. The only reason these methods are appealing to me is because the alternatives are so unappealing. I avoid having to deal with the downsides of using "modern" web browsers and the annoyances of trying to view the web through those browsers.
beermonster 5y ago
Eons ago there used to be a type of service on the web that allowed you to send a URL to a mail server. It would fetch the content at that URL and email the article to you. I can't for the life of me remember what that type of service was. It was back in the era of anonymous remailers.. any ideas?
junux 5y ago
Written in python? So it's trash. I won't use it. I care about my time And nerves.
autobash 5y ago
I'm mainly interested Youtube functionality, and wanted to check if it was well maintained. The developer was listed as Laurent Bachelier (https://github.com/laurentb https://github.com/laurentb). Searching him up, he unfortunately seems to have committed suicide a year ago. Bizarrely with some links to right winged political groups? (First result on google from "Laurent Bachelier") My deepest condolences to the programmers of this project who have lost someone who I assume was a close friend and co-worker.
pjerem 5y ago
Interesting to see Woob here. Most of the modules are for french environment (banks, dating websites, job boards ...). I always liked the irreverence of the module's names and logos (which are authentic MS Paint piece of work).
- zeeZ 5y ago
  My first guess was this had something to do with legal nonsense, and I guess I was right: > If provided, icons are preferred to be parodic or humorous in nature for legal reasons, however there are no restrictions on the quality or style of humor. https://gitlab.com/woob/woob/-/blob/master/CONTRIBUTING.md https://gitlab.com/woob/woob/-/blob/master/CONTRIBUTING.md
  - pjerem 5y ago
    Yes, but I think they are, also, just for making fun of some brands. Because, also a fun fact : this project changed its name recently, it was called Weboob before: https://weboob.org https://weboob.org
    - throwaway744678 5y ago
      The 6th contributor's (nick)name, can be translated to something like "Fuckthewhores", with a play on words with Belzebuth
    - Hackbraten 5y ago
      Good on them. I really disliked the old name.
    - userbinator 5y ago
      I remember weboob, and all the amusing names of the various pieces. Unfortunate that the (French) humour was killed by political correctness.
      - awrmc 5y ago
        Are you suggesting "political correctness" is the only reason someone might be discouraged from installing a program named after toilet humor and juvenile references to specific parts of the anatomy? I've let my kids watch plenty of children's movies with fart jokes in them, but I still don't want to hear more of that when I'm trying to use a tool to access a banking service. It seems there's even still a banking module with a crude poop icon: https://woob.tech/applications/bank.html https://woob.tech/applications/bank.html Doesn't really inspire confidence in their professionalism or trustworthiness with handling financial transactions, if you ask me.
diogenesjunior 5y ago
Small discussion earlier: https://news.ycombinator.com/item?id=29935634 https://news.ycombinator.com/item?id=29935634
- zepto 5y ago
  Smaller as in essentially zero, and with a link back to this one. An amusing detour.
0xbadcafebee 5y ago
This is.... bizarre. And I like it? At first I thought this was like an API to integrate web content into your own apps. But now it looks more like Groupware, in the sense that Woob is actually your user interface and there are just modules to consume content from random websites. It goes back to the old idea where you would have one dedicated desktop application for each thing you wanted to do on the internet, like read news, send mail, listen to music, view a calendar... turning your computer into a utilitarian appliance. Rather than a portal for businesses to spend a lot of time and money building their own dedicated user interfaces to lock you in. The latter has made life more difficult, where we have to constantly learn every business's new interface, there's always competition between missing features, and the dedicated UI (or platform) becomes a way for the business to squeeze more out of the user. And there are no ads. I just realized there's an entire generation who have never seen technology without advertisements. I wonder what they'd make of this.
- NavinF 5y ago
  Re: that last part. We’re probably thinking of different generations, but I agree. Demographics from a random unreliable source: Age vs ad blocker usage (female, male) 16-24 43.2%, 49.2% 25-34 43.0%, 47.6% 35-44 38.4%, 44.8% 45-54 33.5%, 39.1% 55-65 32.1%, 37.3% Those poor boomers. They grew up watching ads on every cable television channel and now they watch ads on every YouTube video.
  - jessaustin 5y ago
    [flagged]
    - CodeGlitch 5y ago
      Ageism at it's finest.
smm11 5y ago
France just can't shake the Minitel, that's for sure.
- OrangeMusic 5y ago
  Not sure I understand the Minitel reference. This tool adds APIs to sites that don't have one. It's made to easily use your data. Sure you can use the CLI part of it, but that's only a small part of the story.
- beders 5y ago
  was thinking the same thing ;)
matthewaveryusa 5y ago
This has Bloomberg terminal / minitel vibes. I think there's definitely a space for an alternative browser that can render guis with visually consistent widgets.
thih9 5y ago
I've been playing with it, but I keep running into errors. E.g.: in woob-weather, with weather.com backend, I've been getting "Error(weather): 401 Client Error: Unauthorized"; in woob-gallery, with imgur backend, when I attempt to download an image the module crashes with "FileNotFoundError: [Errno 2] No such file or directory: ''" I like the idea though and I'll keep trying further. --- Update: I resolved the image-gallery problem by specifying the foldername (so: using "download ID 1 foldername" instead of "download ID"). BUT: it looks like I'm unable to download text descriptions that sometimes accompany the images.
Ajedi32 5y ago
Wow! I actually love the idea of being able to interact with websites via a standard API rather being forced to use web-based UI they provide. It opens up a whole lot of possibles for things like alternate clients, standard UIs for interacting across multiple sites, etc. Also eliminates the possibility of sites engaging in annoying or abusive behavior by putting users in full control of the client rather than the site operator. Obviously it can't work for every site, but it's quite the interesting concept.
- sys_64738 5y ago
  Isn't this just what Electron provides?
- onion2k 5y ago
  I actually love the idea of being able to interact with websites via a standard API rather being forced to use web-based UI they provide. That's what HTTP is. You're free to write a client that isn't a browser that sends and receives the same API messages as any HTTP client app does. Most people use browsers, but there's also things like iOS and Android apps that consume the same APIs as browsers, or Postman that directly communicates with the APIs, etc. The APIs that sit in top of HTTP are even sort of standardized in the sense that HTTP verbs mean the same everywhere (in theory, but some devs get it wrong.) Thr only hard bit that no one has really solved in a nice way is how you discover the APIs in the first place. There's things like WSDL but it's horrible.
  - mst 5y ago
    GraphQL schemas almost seem like the least worst current option and I'm not entirely happy to be saying that.
- clone1018 5y ago
  I've had two ideas related to this in the past that I've always wanted to prototype: - A social media website without a frontend. We just provide a fully exposed API and Oauth, and devs can create their own client to interact with the social network. This would give devs the freedom to create their own experiences without locking users into one specific way of using the social network. - "Cloud" content hosting as a service. You'd be able to build your own frontend for interacting with a website / blog, and then include our JS code and your site's content will automatically be populated in. This would keep the frontend clean, simple, and cheap, while offloading posts, comments, and other advanced functionality to the service. Of course both are purely experimental ideas, with no potential real world meaning :D
matheusmoreira 5y ago
This is so cool. A custom client for websites. Essentially a web scraper with a GUI on top. You can define your own user experience instead of accepting what they designed for you.
jokethrowaway 5y ago
This is great. In my ideal world all the web should be like this.

331 points by goranmoomin 5y ago

but… why

A while back I heard about Z39.50 [0], a protocol that libraries use for their catalogs. In the 90s it seems there were native clients for the protocol so that one could interact with the library catalog without using a web interface. A lot of the current web interfaces are terribly slow JS monstrosities now so I'd like to try something faster. I never did figure out if any of the GUI clients [1] are still actively developed and I'd appreciate if anyone who knows about this could point me towards a good client. [0] https://en.wikipedia.org/wiki/Z39.50 https://en.wikipedia.org/wiki/Z39.50 [1] Some software listed here: http://www.loc.gov/z3950/agency/resources/software.html http://www.loc.gov/z3950/agency/resources/software.html

librarianscott 5y ago
The most well-known Z39.50 using software that still works is a paid citation manager called EndNote (the desktop client not the web client). You can ingest PDFs with that software too. I don't think the open-source Zotero has added that feature yet. Z39.50 is clunky, and our old LMS (library management software, used behind the scenes) can pull in MARC records for our staff when we order books, but I don't even want to think about the security.

pavlov 5y ago

It's like a peek at an alternative Internet where Gopher won instead of WWW.

mxuribe 5y ago
There has been a desire to go back to older days where text was more prevalent, and so Gemini has begin and is gaining popularity - though I'm sure very slowly. See: https://gemini.circumlunar.space/ https://gemini.circumlunar.space/ Also, separately (though i wouild not be surprised if frequented by same/similar folks who have interets in Gemini), there is also the tildeverse...again, more text-heavy environments. See: https://tildeverse.org/ https://tildeverse.org/ And, as i have stated in another comment there is the fediverse (e.g. Mastodon, pleroma, etc.), so the ability to leverage APIs to interact with other folks and their content without explicitly needing a typical web browser exists, and flourishes. I'll end by stating that there are a few exciting things - like the above items i mention as well as this neat Woob platform - which to me seem very fun, a little new, and yet at the same time in some ways nostalgic...maybe they won't make the morning news, and likely only attract geeks, but it is all still exciting - at least for me!
- Qub3d 5y ago
  The tildeverse looks pretty cool, but why does it redirect me to a Rick Roll on Firefox? I can see the normal site just fine on Lynx browser (maybe that's the point?) Edit: ah, I see, they're doing a JWZ and redirecting based on referral, but going a step further and setting a cookie. Cute, but also terribly immature.
  - mxuribe 5y ago
    Oops, sorry about that. I never had an issue before...but funny after visiting HN, see exactly what you mean. (Clearing the cookies avoids the classic rick roll video). Anyway, yeah i guess the tilde folks are "characters". But, separate of that, the community i've interacted with is quite fun, respectful, and good-natured. I also failed to mention a sort of equivalent to HN, which i frequent (with similar topics to HN but often nicer crowd): https://tildes.net/ https://tildes.net/ ...Tilde.net used to be only open to invite not open to anyone creating an account...so if interested - and still not open to the public - i can trigger to send you an invite.

brutal_chaos_ 5y ago

This is clever and fantastic. I have been pondering a similar concept recently and I think I would like to contribute. I'm curious as to why LGPL-3 was chosen as the license, though, not that the license is a show stopper.

Zababa 5y ago
One explanation might be that the project is mostly French, and the GPL/LGPL seems more popular in France than in the USA.

res0nat0r 5y ago

Also: woob - 1994 is one of the best ambient albums ever made. https://www.youtube.com/watch?v=0S3owK3pN64 https://www.youtube.com/watch?v=0S3owK3pN64

notorandit 5y ago

I do exactly the other way around on my smartphone: if there is a web app i won't install the app.

dylan-m 5y ago

I like the idea of this. There's so much information on the web, but we still need a way to bring that information to other applications, without being tied to a particular source. That was really the dream of the semantic web, after all. This kind of idea would be really nicely paired with good Microformats[1] support, which continues to be a very good idea. That way we can find, say, a recipe or an address on a web page in a reusable way and without needing magical heuristics. (Of course, "reusable" in theory, with the caveat that everybody forgot about microformats around when Google decided they could machine learn their way out of everything). [1] http://microformats.org http://microformats.org

MaxBarraclough 5y ago
> I like the idea of this. There's so much information on the web, but we still need a way to bring that information to other applications, without being tied to a particular source. I'm not sure I'm interpreting you correctly here, but I think I'm on the other side of this. The problem is that many modern websites are godawful. I think the story pretty much ends there. If websites were not awful, we wouldn't find ourselves appalled by the idea of just embedding a browser. Modern web browsers feature a 'reader mode' as a countermeasure to that much modern web design is significantly worse than having no web design at all. If you're serious about a 'lightweight' alternative to the lumbering horror-show of the modern web, the way forward is either Gemini [0], or a formalised simple subset of HTML. [1] > That way we can find, say, a recipe or an address on a web page in a reusable way and without needing magical heuristics. I think the find and reusable aspects here are really two very different problems. The reusable part is easy. HTML is already reusable. A standardised simple subset would be even more so. [1] The find part is trickier. Discovering decent content is harder, as there's an arms race of ad-funded spammers trying to out-compete legitimate recipe sites in search-engine rankings. (There's also the possibility of search engines not being motivated to work on delivering good search results. [2]) The idea of having a choice between native GUI applications and web apps, has been with us for some time. Email is probably the best example, we've long had the choice between webmail and native email clients. Beyond webmail, these days even Microsoft Word has a web-based version. There are of course both advantages and disadvantages to web-based applications. [0] https://news.ycombinator.com/item?id=23730408 https://news.ycombinator.com/item?id=23730408 [1] https://news.ycombinator.com/item?id=29291392 https://news.ycombinator.com/item?id=29291392 [2] https://news.ycombinator.com/item?id=29772136 https://news.ycombinator.com/item?id=29772136

0x445442 5y ago

Believe it or not, around the turn of the century there were many thick client apps. But back then it was a challenge to ship and update these applications. This pain, along with the continued rollout of broadband led many to advocate for creating applications that would run in a web browser while being controlled on centralized servers. In practice, turning the platform that was designed to render markup text into an application host. This would allow applications to be shipped and updated with little interaction from the user. However, right about the same time web apps were taking over the world there were thick client apps that were solving the problems of installation and and updates. Two of the prominent thick client applications doing this were iTunes and the browsers themselves. Now fast forward a decade to the early teens and the ubiquitous use of smart phones. What is the single largest determining factor of platform success? Is it the ability for web apps to render on your platform's web browser or is it the breadth and depth of your platform's app store? My rant is over, I wish web apps would die. I've wished that for most of the 21st century.

mikehollinger 5y ago
> Now fast forward a decade to the early teens and the ubiquitous use of smart phones. An interesting thing happened here as well - because of the iOS app store and its auto-update policy, "thick" clients came back in vogue again. It just happened that they're running on phones, not full PCs. I'd argue these count as thick client apps. :-) Consider WhatsApp or WeChat - it has a complete and full ecosystem of messaging, apps, and utility function. It actually (I think) supercedes AOL 2 or 3 in terms of functionality. There's DMs, Chat, Apps...
redsparrow 5y ago
This is an interesting observation. I hadn't noticed this dynamic before; thanks for sharing. The current differences in attitudes between mobile and non-mobile apps is pretty interesting. If you visit Reddit or Twitter from a browser that's running on a mobile OS then they make it extremely clear that they would prefer that you use a dedicated app. When visiting from a non-mobile OS not only do you not see such messages, but a dedicated app doesn't even exist. In one place the browser is good enough to be the only option, in the other it's so inferior that they will bully and harass you until you stop using it.

kadomony 5y ago

Woobs out

kodablah 5y ago

I think a version of this is what the internet needs but using headless browsers from the client and with a somewhat-centrally curated set of scraper "recipes" if you will. Basically a community curated/updated set of scraper logic per site (yes some trust is required) that essentially provides JSON data and/or APIs based on the site. Even just a neutered HTML equivalent of sites (e.g. amp w/out the Google and ads stuff) would be good. Since it is all client side, it can be dubbed a "browser" not a "scraper" and one might hope popularity is high enough that active blocking of it is blatantly user hostile. Granted one hopes that, like EasyList and uBO and others have shown, the community can outpace site owners. Not appearing headless (tunneling captchas, literal mousemove events in pseudo-random human-like ways, etc) should be doable. It's something I have thought about and once dubbed "recapitate" (https://github.com/cretz/software-ideas/issues/82 https://github.com/cretz/software-ideas/issues/82) and plan to revisit. I have seen many versions of this attempted. We need to encourage shared data extraction tools.

louissan 5y ago

woob woob woob! Battletech pulse lasers anyone?

vmception 5y ago

Reading the linked site and some of the discussion, I highly recommend finding your nearest Chinese friend or person and getting authorized on Wechat. Its a whole parallel other internet! Kinda similar to how a private set of Facebook pages are otherwise inaccessible with an account, except in a parellel reality where people use them for all business and have no other internet presence. Yes, as a user another government gets to read your posts, but I mean yet another. To get on, I literally just knocked on a few doors in San Francisco and got authorized, so many people here can too. You could probably do it at a park. Note: Hong Kong citizens cannot do it for US citizens even I was trying. Has to be a mainland Chinese person.

vorpalhex 5y ago
Weibo is a heavily censored and manipulated platform. The CCP uses it to track dissidents abroad. Do not install or operate on any trusted device. Do not connect to your home network. Do not store personal details on your weibo device. Do not ever send sensitive information or talk to real contacts on Weibo.
- vmception 5y ago
  As you know, it is not possible to separate corporate China from government China due to the structure of that system so no analogy exactly works, but our day to day experience is exceedingly similar as we rely on heavily censored and manipulated private platforms. Although we retain options to express ideas, there is not enough saturation of other people to view those ideas except to participate on heavily censored platforms and risk complete deplatforming. Its an almost daily topic here, for example. So the user experience on a Chinese service is simply not different enough for me to treat it differently.
  - vorpalhex 5y ago
    I'm someone who is heavily critical of Facebook and Twitter. However there is a world of difference between sloppily shutting down vaccine or election misinformation, and actively censoring a Tennis star reporting a sexual assault by a politiburo member. There is a world of difference between taking down shit talking politicians twitter profiles, and actively censoring the genocide of an ethnic minority. Ads that slurp up personal data are bad. Threatening political dissidents abroad directly is incomparable. To say "well they are all the same bad" is to be willfully blind to the basic facts. [1] - https://www.theatlantic.com/technology/archive/2019/03/what-went-wrong-chinas-weibo-social-network/584728/ https://www.theatlantic.com/technology/archive/2019/03/what-... [2] - https://www.thecut.com/2021/12/the-disappearance-of-peng-shuai-what-we-know.html https://www.thecut.com/2021/12/the-disappearance-of-peng-shu...

mahastore 5y ago

Good example of another solution without a problem.

mkdirp 5y ago
Really? You ever tried automating your own data from e.g. your bank? Cos I have, and it's a lot more annoying that it should be. Woob does a lot more than just banks. It allows you to get any of your data. Adding additional providers is piss easy too.

anderspitman 5y ago

Something I've been thinking about lately is how browsers have essentially become a dependency for any sort of auth on the internet. Pretty much everything uses OAuth2, which requires you to be able to render HTML and CSS, and in many implementations JavaScript. That's ~20M (Firefox) to ~30M (Chromium) lines of code as a dependency for your application, just for auth. This applies even if you have a slick CLI app like rclone. If you want to connect it to Google drive you still need a browser to do the OAuth2 flow. All of this just so we have a safe, known location to stash auth cookies. It would be sweet if there was a lightweight protocol where you could lay out a basic consent UI (maybe with a simple JSON format) that can be rendered outside the browser. Then you need a way to connect to a central trusted cookie store. You could still redirect to a separate app, but it wouldn't need to be nearly as complicated as a browser.

jl2718 5y ago
SSL cert auth is recognized by every major HTTP server and used by every major browser. Frankly I don’t understand how this still is or ever was an issue.
dmitriid 5y ago
> All of this just so we have a safe, known location to stash auth cookies. > It would be sweet if there was a lightweight protocol where you could lay out a basic consent UI This also feels like it should be a part of the OS, since it's "select user, input password, and maybe 2fa, store auth". It doesn't need a full-blown web-browser. It does, however, needs a properly defined protocol.
- anderspitman 5y ago
  Maybe someone needs to pull a "WireGuard for auth" and make a slick stripped-down implementation and spec and get it included in the kernel.
Wowfunhappy 5y ago
While I absolutely share the philosophical concern, I do wonder how large of an issue this is in practice. On an IOT device, or a retrocomputing/hobbyist platform, you'd likely want to display a QR code and have the user authenticate with their phone, similar to how you log into Netflix on a smart TV. As an aside, to my knowledge OAuth2 still works in PaleMoon. I just downloaded the source and did a count with CLOC, and it looks like there's "only" ~13.5M lines of code. :)

synergy20 5y ago

Browser is the new UI for 90% of GUI use cases, I'm all in, be it Electron or just the browser.

cglong 5y ago

Not a single comparison to Sherlock/Watson? I always thought it was a great idea, because it allowed a consistent interface for theoretically finding anything on the internet.

1vuio0pswjnm7 5y ago

Been using web outside of browser for last 10+ years. I am addicted to it, having written many scripts and programs to fetch and process HTML and other filetypes outside the browser. I can retrieve and extract data/information much faster and more efficiently using simpler programs that are small and work together. For viewing HTML, I prefer my text-only browser. HTML looks much better, less variable, more uniform, than in a graphical browser. The only reason these methods are appealing to me is because the alternatives are so unappealing. I avoid having to deal with the downsides of using "modern" web browsers and the annoyances of trying to view the web through those browsers.

beermonster 5y ago

Eons ago there used to be a type of service on the web that allowed you to send a URL to a mail server. It would fetch the content at that URL and email the article to you. I can't for the life of me remember what that type of service was. It was back in the era of anonymous remailers.. any ideas?

junux 5y ago

Written in python? So it's trash. I won't use it. I care about my time And nerves.

autobash 5y ago

I'm mainly interested Youtube functionality, and wanted to check if it was well maintained. The developer was listed as Laurent Bachelier (https://github.com/laurentb https://github.com/laurentb). Searching him up, he unfortunately seems to have committed suicide a year ago. Bizarrely with some links to right winged political groups? (First result on google from "Laurent Bachelier") My deepest condolences to the programmers of this project who have lost someone who I assume was a close friend and co-worker.

pjerem 5y ago

Interesting to see Woob here. Most of the modules are for french environment (banks, dating websites, job boards ...). I always liked the irreverence of the module's names and logos (which are authentic MS Paint piece of work).

zeeZ 5y ago
My first guess was this had something to do with legal nonsense, and I guess I was right: > If provided, icons are preferred to be parodic or humorous in nature for legal reasons, however there are no restrictions on the quality or style of humor. https://gitlab.com/woob/woob/-/blob/master/CONTRIBUTING.md https://gitlab.com/woob/woob/-/blob/master/CONTRIBUTING.md
- pjerem 5y ago
  Yes, but I think they are, also, just for making fun of some brands. Because, also a fun fact : this project changed its name recently, it was called Weboob before: https://weboob.org https://weboob.org
  - throwaway744678 5y ago
    The 6th contributor's (nick)name, can be translated to something like "Fuckthewhores", with a play on words with Belzebuth
  - Hackbraten 5y ago
    Good on them. I really disliked the old name.
  - userbinator 5y ago
    I remember weboob, and all the amusing names of the various pieces. Unfortunate that the (French) humour was killed by political correctness.
    - awrmc 5y ago
      Are you suggesting "political correctness" is the only reason someone might be discouraged from installing a program named after toilet humor and juvenile references to specific parts of the anatomy? I've let my kids watch plenty of children's movies with fart jokes in them, but I still don't want to hear more of that when I'm trying to use a tool to access a banking service. It seems there's even still a banking module with a crude poop icon: https://woob.tech/applications/bank.html https://woob.tech/applications/bank.html Doesn't really inspire confidence in their professionalism or trustworthiness with handling financial transactions, if you ask me.

diogenesjunior 5y ago

Small discussion earlier: https://news.ycombinator.com/item?id=29935634 https://news.ycombinator.com/item?id=29935634

zepto 5y ago
Smaller as in essentially zero, and with a link back to this one. An amusing detour.

0xbadcafebee 5y ago

This is.... bizarre. And I like it? At first I thought this was like an API to integrate web content into your own apps. But now it looks more like Groupware, in the sense that Woob is actually your user interface and there are just modules to consume content from random websites. It goes back to the old idea where you would have one dedicated desktop application for each thing you wanted to do on the internet, like read news, send mail, listen to music, view a calendar... turning your computer into a utilitarian appliance. Rather than a portal for businesses to spend a lot of time and money building their own dedicated user interfaces to lock you in. The latter has made life more difficult, where we have to constantly learn every business's new interface, there's always competition between missing features, and the dedicated UI (or platform) becomes a way for the business to squeeze more out of the user. And there are no ads. I just realized there's an entire generation who have never seen technology without advertisements. I wonder what they'd make of this.

NavinF 5y ago
Re: that last part. We’re probably thinking of different generations, but I agree. Demographics from a random unreliable source: Age vs ad blocker usage (female, male) 16-24 43.2%, 49.2% 25-34 43.0%, 47.6% 35-44 38.4%, 44.8% 45-54 33.5%, 39.1% 55-65 32.1%, 37.3% Those poor boomers. They grew up watching ads on every cable television channel and now they watch ads on every YouTube video.
- jessaustin 5y ago
  [flagged]
  - CodeGlitch 5y ago
    Ageism at it's finest.

smm11 5y ago

France just can't shake the Minitel, that's for sure.

OrangeMusic 5y ago
Not sure I understand the Minitel reference. This tool adds APIs to sites that don't have one. It's made to easily use your data. Sure you can use the CLI part of it, but that's only a small part of the story.
beders 5y ago
was thinking the same thing ;)

matthewaveryusa 5y ago

This has Bloomberg terminal / minitel vibes. I think there's definitely a space for an alternative browser that can render guis with visually consistent widgets.

thih9 5y ago

I've been playing with it, but I keep running into errors. E.g.: in woob-weather, with weather.com backend, I've been getting "Error(weather): 401 Client Error: Unauthorized"; in woob-gallery, with imgur backend, when I attempt to download an image the module crashes with "FileNotFoundError: [Errno 2] No such file or directory: ''" I like the idea though and I'll keep trying further. --- Update: I resolved the image-gallery problem by specifying the foldername (so: using "download ID 1 foldername" instead of "download ID"). BUT: it looks like I'm unable to download text descriptions that sometimes accompany the images.

Ajedi32 5y ago

Wow! I actually love the idea of being able to interact with websites via a standard API rather being forced to use web-based UI they provide. It opens up a whole lot of possibles for things like alternate clients, standard UIs for interacting across multiple sites, etc. Also eliminates the possibility of sites engaging in annoying or abusive behavior by putting users in full control of the client rather than the site operator. Obviously it can't work for every site, but it's quite the interesting concept.

sys_64738 5y ago
Isn't this just what Electron provides?
onion2k 5y ago
I actually love the idea of being able to interact with websites via a standard API rather being forced to use web-based UI they provide. That's what HTTP is. You're free to write a client that isn't a browser that sends and receives the same API messages as any HTTP client app does. Most people use browsers, but there's also things like iOS and Android apps that consume the same APIs as browsers, or Postman that directly communicates with the APIs, etc. The APIs that sit in top of HTTP are even sort of standardized in the sense that HTTP verbs mean the same everywhere (in theory, but some devs get it wrong.) Thr only hard bit that no one has really solved in a nice way is how you discover the APIs in the first place. There's things like WSDL but it's horrible.
- mst 5y ago
  GraphQL schemas almost seem like the least worst current option and I'm not entirely happy to be saying that.
clone1018 5y ago
I've had two ideas related to this in the past that I've always wanted to prototype: - A social media website without a frontend. We just provide a fully exposed API and Oauth, and devs can create their own client to interact with the social network. This would give devs the freedom to create their own experiences without locking users into one specific way of using the social network. - "Cloud" content hosting as a service. You'd be able to build your own frontend for interacting with a website / blog, and then include our JS code and your site's content will automatically be populated in. This would keep the frontend clean, simple, and cheap, while offloading posts, comments, and other advanced functionality to the service. Of course both are purely experimental ideas, with no potential real world meaning :D

matheusmoreira 5y ago

This is so cool. A custom client for websites. Essentially a web scraper with a GUI on top. You can define your own user experience instead of accepting what they designed for you.

jokethrowaway 5y ago

This is great. In my ideal world all the web should be like this.