3 ms·

There should be a new category of apps, where you can still get the contents offered by a given service (Spotify, Youtube, your favourite news service etc.) whi

There should be a new category of apps, where you can still get the contents offered by a given service (Spotify, Youtube, your favourite news service etc.) while at the same time not using the built in UI. For example, imagine a web app that embeds Youtube videos, that will show you every new video published by channels you choose. No home page suggestions, no "trending", no "you might be interested in". You could do the same with Spotify or with one of those terrible news websites. Some examples of this already exist: Nitter (shows Twitter contents), Invido.us (shows Youtube contents) but they are not mainstream and not particularly polished. Also, I'm aware that this concept when applied to news websites results in something that is terribly close to an RSS reader.

mattbk1 5y ago
In a perfect world, all of those would work in an RSS reader.
- mastazi 5y ago
  Yes, true. I wish Spotify artists/playlists or Youtube channels etc. had RSS. So, the next idea would be a service that scrapes contents and builds an RSS feed.
  - HKH2 5y ago
    YouTube does have RSS.
    - mastazi 5y ago
      Thank you for pointing that out! I didn't know that until today. I also discovered that you can get RSS feeds from Reddit and a few other services (see my other comment in sibling sub thread)[1]. So, it sounds like I might be using RSS more. [1] https://news.ycombinator.com/item?id=28111810 https://news.ycombinator.com/item?id=28111810
- benrapscallion 5y ago
  NewsBlur can do this for YouTube videos.
  - mastazi 5y ago
    Right! I was curious about how they do it and found out that Youtube actually has RSS feeds! At least according to this and a few other pages I've found https://danielmiessler.com/blog/rss-feed-youtube-channel/ https://danielmiessler.com/blog/rss-feed-youtube-channel/ Also, I can see that for other services like Twitter, there are indeed third party providers that can create feeds https://rss.app/blog/how-to-create-twitter-rss-feeds https://rss.app/blog/how-to-create-twitter-rss-feeds Also Reddit subreddits and users: https://www.reddit.com/wiki/rss https://www.reddit.com/wiki/rss For podcasts, just don't use Spotify and instead get the RSS feed of the podcast you like So, yeah I'm now thinking of going to my RSS reader (which I currently only use for news) and adding all sorts of stuff. Thank you for posting your comment :-)
    - benrapscallion 5y ago
      For reddit, you can also look at inline-reddit.com NewsBlur and a couple other RSS feed readers have built-in Twitter clients. You can also use rss bridge to get Instagram in your RSS feeds.
      - mastazi 5y ago
        Thank you! This is all very useful info!
- hopesthoughts 5y ago
  They do. Well in a good RSS reader, anyway.
ekimekim 5y ago
The problem is that these apps come with a built-in doomsday. If they ever get popular enough to be meaningfully on the radar of the company whose app they're replacing, they'll be attacked both legally ("you're violating our TOS by using this app") and technically (removing API access or obfuscating internal APIs / pages to prevent scraping). Efforts like youtube-dl are a constant arms race between google making changes that prevent it working, and the youtube-dl devs pushing fixes. It's a piece of software where "too old to be useful" is measured in months as tons of things don't work anymore. These companies will never willingly give up control over the user experience (especially if they NEED that control in order to serve ads), and they'll use every method at their disposal in order to do so. Discord already threatens to ban anyone who tries to use a third party client - how long until twitter does the same?
- mastazi 5y ago
  I think these hypothetical apps should be based on html embeds. The big tech companies are not going to remove the ability to do web embeds of their posts/videos/tweets, because embeds are good for bringing traffic in. With those services that don't have RSS (Youtube does), you will still need a little scraping just to discover the URL of new tweets/posts in a given channel. But it would be just basic scraping and probably more manageable compared to something like youtube-dl.
wildmanx 5y ago
"Not mainstream" is key here. Once those become mainstream, they get shut down by the provider of the original service. The original provider wants to make money off this, so they use the UI to provide you with ads and a bunch of tricks to keep engagement up. No way they let you off the hook to use what's costly without being exposed to what drives the income stream. Open APIs, that's what you need. Obviously Facebook, Twitter, Instagram, Google Chat, etc don't have any, for the above reasons. I'm honestly surprised that email is still a thing. It's the last somewhat open API around. (Unless you count HTTP..)
- mastazi 5y ago
  > Open APIs, that's what you need. Well I agree in principle, but I also agree with you that the big guys aren't going to offer APIs. And even if they were, I think if you rely on some API you are going to be shut down in the long term (remember all those Twitter clients that were using the now defunct Twitter API?). So, I think the way to go is to base everything on embeds. They are never going to get rid of the ability to embed posts/videos/tweets because it brings more traffic to their sites. You still need a bit of scraping, in case the service in question doesn't have RSS (Youtube does for example). And yes, with scraping they will try and make your life impossible (dynamic class names etc. to make scraping hard), but you would only need it for discovering the URL of new posts/tweets/videos, and nothing else.
hopesthoughts 5y ago
When applied to everything, it's an RSS reader. For instance, Youtube offers RSS, so why not just put them in there? If you can find a podcast feed, you could put those in there, too. RSS readers aren't just for news sites.

mattbk1 5y ago

In a perfect world, all of those would work in an RSS reader.

mastazi 5y ago
Yes, true. I wish Spotify artists/playlists or Youtube channels etc. had RSS. So, the next idea would be a service that scrapes contents and builds an RSS feed.
- HKH2 5y ago
  YouTube does have RSS.
  - mastazi 5y ago
    Thank you for pointing that out! I didn't know that until today. I also discovered that you can get RSS feeds from Reddit and a few other services (see my other comment in sibling sub thread)[1]. So, it sounds like I might be using RSS more. [1] https://news.ycombinator.com/item?id=28111810 https://news.ycombinator.com/item?id=28111810
benrapscallion 5y ago
NewsBlur can do this for YouTube videos.
- mastazi 5y ago
  Right! I was curious about how they do it and found out that Youtube actually has RSS feeds! At least according to this and a few other pages I've found https://danielmiessler.com/blog/rss-feed-youtube-channel/ https://danielmiessler.com/blog/rss-feed-youtube-channel/ Also, I can see that for other services like Twitter, there are indeed third party providers that can create feeds https://rss.app/blog/how-to-create-twitter-rss-feeds https://rss.app/blog/how-to-create-twitter-rss-feeds Also Reddit subreddits and users: https://www.reddit.com/wiki/rss https://www.reddit.com/wiki/rss For podcasts, just don't use Spotify and instead get the RSS feed of the podcast you like So, yeah I'm now thinking of going to my RSS reader (which I currently only use for news) and adding all sorts of stuff. Thank you for posting your comment :-)
  - benrapscallion 5y ago
    For reddit, you can also look at inline-reddit.com NewsBlur and a couple other RSS feed readers have built-in Twitter clients. You can also use rss bridge to get Instagram in your RSS feeds.
    - mastazi 5y ago
      Thank you! This is all very useful info!
hopesthoughts 5y ago
They do. Well in a good RSS reader, anyway.

ekimekim 5y ago

The problem is that these apps come with a built-in doomsday. If they ever get popular enough to be meaningfully on the radar of the company whose app they're replacing, they'll be attacked both legally ("you're violating our TOS by using this app") and technically (removing API access or obfuscating internal APIs / pages to prevent scraping). Efforts like youtube-dl are a constant arms race between google making changes that prevent it working, and the youtube-dl devs pushing fixes. It's a piece of software where "too old to be useful" is measured in months as tons of things don't work anymore. These companies will never willingly give up control over the user experience (especially if they NEED that control in order to serve ads), and they'll use every method at their disposal in order to do so. Discord already threatens to ban anyone who tries to use a third party client - how long until twitter does the same?

mastazi 5y ago
I think these hypothetical apps should be based on html embeds. The big tech companies are not going to remove the ability to do web embeds of their posts/videos/tweets, because embeds are good for bringing traffic in. With those services that don't have RSS (Youtube does), you will still need a little scraping just to discover the URL of new tweets/posts in a given channel. But it would be just basic scraping and probably more manageable compared to something like youtube-dl.

wildmanx 5y ago

"Not mainstream" is key here. Once those become mainstream, they get shut down by the provider of the original service. The original provider wants to make money off this, so they use the UI to provide you with ads and a bunch of tricks to keep engagement up. No way they let you off the hook to use what's costly without being exposed to what drives the income stream. Open APIs, that's what you need. Obviously Facebook, Twitter, Instagram, Google Chat, etc don't have any, for the above reasons. I'm honestly surprised that email is still a thing. It's the last somewhat open API around. (Unless you count HTTP..)

mastazi 5y ago
> Open APIs, that's what you need. Well I agree in principle, but I also agree with you that the big guys aren't going to offer APIs. And even if they were, I think if you rely on some API you are going to be shut down in the long term (remember all those Twitter clients that were using the now defunct Twitter API?). So, I think the way to go is to base everything on embeds. They are never going to get rid of the ability to embed posts/videos/tweets because it brings more traffic to their sites. You still need a bit of scraping, in case the service in question doesn't have RSS (Youtube does for example). And yes, with scraping they will try and make your life impossible (dynamic class names etc. to make scraping hard), but you would only need it for discovering the URL of new posts/tweets/videos, and nothing else.

hopesthoughts 5y ago

When applied to everything, it's an RSS reader. For instance, Youtube offers RSS, so why not just put them in there? If you can find a podcast feed, you could put those in there, too. RSS readers aren't just for news sites.