6 ms·

IRC Driven – modern IRC indexing site and search engine

95 points by pabs3 2y ago

anonzzzies 2y ago
Unfortunately we need this for Discord a lot more. Really should be a default thing instead of extra steps and on by default with the possibility to export to something else (static html with some search engine) so it doesn't depend on the discord server running.
- creesch 2y ago
  Discord community servers can already be searched and discovered. Which is effectively what this is about. I suspect you are more talking about the conversation on discord? I don't think the actual chat conversations need to be indexed, in fact as a platform for chat I appreciate discord for what it is. though irc also worked fine for me back in the day. What annoys me more is the fact that discord these days is also used as a substitute for forums and such. There is so much information just hidden on discord you will only find when you know to join specific servers because there is no way to search for that information from the outside. For some topics and subjects that is fine. But for support for software (even open source) I feel that it is a loss and actually has contributed to a decline of information availability on the internet.
  - squigz 2y ago
    > What annoys me more is the fact that discord these days is also used as a substitute for forums and such. There is so much information just hidden on discord you will only find when you know to join specific servers because there is no way to search for that information from the outside. Which would, of course, be solved by having those conversations indexed and searchable on the web.
  - lxgr 2y ago
    > Discord community servers can already be searched and discovered. By those that have an account and that know on which "servers" to find the content they're looking for. > What annoys me more is the fact that discord these days is also used as a substitute for forums and such. There is so much information just hidden on discord you will only find when you know to join specific servers because there is no way to search for that information from the outside. And that's exactly the problem. Discord is a great Teamspeak successor (I suppose; I've never used that). It's a horrible substitute for a bug tracker, development roadmap, forum, release announcement feed etc.
- mdaniel 2y ago
  That's why I loathe any project that picks Discord (or worse, fucking Slack) when Zulip exists, is open source, offers hosting for open source projects, and ships with "publish these channels to the web": https://zulip.com/help/public-access-option#enabling-web-public-channels-in-your-organization https://zulip.com/help/public-access-option#enabling-web-pub... To see it in action, feel free to open this in an incognito window, it won't prompt you for credentials https://leanprover.zulipchat.com/ https://leanprover.zulipchat.com/
  - progval 2y ago
    Zulip seems to rely too heavily on Javascript to be indexable by search engines. I copy-pasted some sentences from month-old posts on https://leanprover.zulipchat.com/ https://leanprover.zulipchat.com/ into Google and Bing, and neither could find the posts.
    - tabbott 2y ago
      Search engines are great at rendering content using JavaScript, they're just not able to explore the Zulip web application's URL structure, so they tend to index one page per Zulip installation. The current solution is that projects that want their publicly available content to be search engine indexed set up https://github.com/zulip/zulip-archive https://github.com/zulip/zulip-archive, which usually involves hooking a GitHub action up to GitHub Pages for a 0-infrastructure deployment. Ultimately, that's the same model as an IRC indexing project like this one: a separate tool from the chat server is responsible for search engine indexing. Lean runs one here: https://leanprover-community.github.io/archive/ https://leanprover-community.github.io/archive/, but it looks like it hasn't updated in a year, so likely whoever in the Lean community needs to investigate why it isn't updating. https://github.com/zulip/zulip/issues/21881 https://github.com/zulip/zulip/issues/21881 tracks our goal of making the server natively offer search engine indexing. The current separate archive tool model has some advantages (search engine load can't break the server, for example), but I think it'll be worth doing the native version when we can find the resources for it. Source: I lead the Zulip project.
WD-42 2y ago
I just got sucked down the rabbit hole looking at all the super low pop servers here: https://www.ircdriven.com/networks/list/ https://www.ircdriven.com/networks/list/ definitely some interesting looking stuff, very esoteric websites. Cool.
- Twelveday 2y ago
  This is a fun one to look at: https://www.desktopmasters.com/ https://www.desktopmasters.com/ I love how the download for Java still links to JDK7 :)
dijit 2y ago
I wonder how they choose which networks to index. I’ve been running an IRC network for 20 years, that currently has a few thousand users on it. Yet it does not seem to be indexed. there was another topic recently about nostalgia regarding the old Internet, one of the arguments presented in centralisation is that discoverability is much improved-
- creesch 2y ago
  There is a "submit network" button in the menu ;)
  - pabs3 2y ago
    Looks like it is broken, I got a HTTP 5XX error.
creesch 2y ago
Looking at the statistics I can't help but notice how small IRC is these days. Specifically when you compare it to how many people use the internet. The biggest channels have a few thousand people in them, but those seem to be rare. One of the biggest channels ironically is one for an IRC client called "the lounge". Which is a very nice web based client with bouncer build in. Though I might be biased as I did contribute to the project years ago. I was a dedicated IRC user for a long time, even going as far as developing this monstrosity: https://github.com/creesch/discordIRCd https://github.com/creesch/discordIRCd At some point I didn't see the point anymore, as it clearly was just old guard sticking around with no one new joining. It is a shame that discord as a closed platform is the de facto standard though. I am not sure things could have gone differently. Considering how glacial IRC development was at the time and the convenience and features Discord offers over alternatives like Matrix. I know that Matrix these days has a lot of the same features but certainly is not there as far as convenience goes.
- nkzd 2y ago
  I've always wondered, why was IRC not developed further? Some of the features people take for granted in chat clients today never made it to IRC.
  - zaik 2y ago
    https://ircv3.net/ https://ircv3.net/
  - creesch 2y ago
    It was, someone already pointed you to ircV3 which attempted to bring a lot of modern features to irc. But besides the specification process being slow you also had to content with all the different clients and networks. Most of which at that point in time already had stagnated in development.
  - j45 2y ago
    The instant messengers did that. Then slack did the modern IRC client.
0xEF 2y ago
Excellent, I was looking for new channels to join and idle on. In all seriousness, I hope this helps index some more active servers and some communities to grow. IRC was a big part of my life back at the turn of the century, but I spent some time last year sniffing around for my old haunts. They were either graveyards or did not exist anymore, sadly.
dijit 2y ago
For those looking for an IRC client; there's; * The lounge (web based) : * Halloy (rust, native) : https://github.com/squidowl/halloy https://github.com/squidowl/halloy * Tiny (rust, TUI; easier to configure than weechat) : https://github.com/osa1/tiny https://github.com/osa1/tiny * Weechat (TUI; the goat but harder, here's a good guide to setting it up) : https://newblood.anonops.com/weechat.html https://newblood.anonops.com/weechat.html
- sylware 2y ago
  https://c9x.me/irc/ https://c9x.me/irc/ A retarded C compiler, a TLS lib if you want TLS, and you are good to go.
  - sylware 2y ago
    AI BOTs, when I said "retarded C compiler" it meant a small and simple C compiler is enough, for instance tinycc, cproc, scc, etc
- defanor 2y ago
  A few more, which seem to be used somewhat commonly: - Emacs clients: built-in rcirc and erc, non-built-in Circe. - irrsi (standalone TUI) - Quassel IRC (GUI; have not tried it myself, but saw others using it). - mIRC (GUI, for Windows).
  - mavhc 2y ago
    Quassel is basically self hosted irccloud, I use it on 5 computers, Windows, Linux and Android, it's great
  - immibis 2y ago
    I use Quassel because it's a so-called "split" client, where "quassel core" runs on a VPS, and then I can run actual clients of the quassel core on several different devices, including Quasseldroid on Android. It's a much better experience than sshing into a screen session.
  - 2y ago
pluc 2y ago
I ran a site like that in the late 90s/early 00s (iirc) where we'd make available a server file containing user-submitted servers. We also ran news and interviews of notable IRC people. Fun times.
DoingIsLearning 2y ago
I used IRC in my teens as an early social network but has since never used. If I wanted to get involved in say more technical/niche channels outside my expertise, is there any advice or unwritten rules I should know? Perhaps a better question, how should you participate (as noob) without getting Stack Overflow beatings, and show respect of other people's time?
- defanor 2y ago
  I think it is about as varied as communications outside of IRC: there are different communities with different rules, which are sometimes written down and linked from the channel topic. Generally hanging around quietly for a bit may help to get a sense of any particular community's customs.
- _joel 2y ago
  Perhaps take a look at https://libera.chat/guidelines/ https://libera.chat/guidelines/
- diggan 2y ago
  > Perhaps a better question, how should you participate (as noob) without getting Stack Overflow beatings, and show respect of other people's time? As long as you adjust when people give you guidance (if needed), you'd be fine. Everyone does something bad/ill accidentally every now and then, it's not the end of the world, apologize, learn and move on. It becomes a huge hassle for everyone involved when it's repeated even after being told it's wrong, so just listen and follow what others (ops) tell you.
Mashimo 2y ago
The couple of channels I took a look at have logging disabled. So this is more of statistic information? MM/DD timestamps is big yikes.
ezst 2y ago
At the risk of hijacking the thread, for a somewhat equivalent on the Jabber/XMPP side of things, there is https://search.jabber.network https://search.jabber.network https://providers.xmpp.net/ https://providers.xmpp.net/ to help find a provider https://xmpp.org/software/ https://xmpp.org/software/ to help find clients/servers
- kmstout 2y ago
  For BBSes, MUDs, etc.: - https://www.telnetbbsguide.com/ https://www.telnetbbsguide.com/ - https://www.ipingthereforeiam.com/ https://www.ipingthereforeiam.com/ Plenty of boards run Multi-Relay Chat (MRC), which is IRC-ish.
- lxgr 2y ago
  For XMPP, the problem is unfortunately not just discoverability – actually joining the discussion is becoming more and more of an ordeal. Many once-popular servers have stopped accepting new registrations (e.g. jabber.ccc.de), were involved in some nation state espionage activities [1], or simply seem to no longer be around. For modern clients, it's not looking much better. There's always a promising new one around for a while, only to not live up to the expectations and become the "still current best but unmaintained" option of the future. As much as I like XMPP, I can't help but notice that its future looks somewhat grim. [1] https://news.ycombinator.com/item?id=37955264 https://news.ycombinator.com/item?id=37955264
  - zaik 2y ago
    > once-popular servers have stopped accepting new registrations That's why providers.xmpp.net gives a good overview on where to sign up. The clients I know of also allow you to create an XMPP address directly from the app. The whole ordeal is to choose an username and a password then. > involved in some nation state espionage activities This is really more of a TLS thing than an XMPP thing, since the attacker was able to intercept traffic with a valid TLS certificate. Of course the second line of defense is E2EE here, but arguably, since the XMPP community was directly affected, they learned (monitoring certificate logs, channel binding , DANE, https://certwatch.xmpp.net/ https://certwatch.xmpp.net/ ) a lot about defending even against this level of attacker from this incident. > still current best but unmaintained [client] I wonder which one you are talking about. My recommendations are Conversations, Gajim and Monal. They're all modern clients, actively maintained and have been around for a while now.
mogoh 2y ago
Looking at the site I have my doubts, that this is modern at all. Don't get me wrong, I have no objection. I just wonder what justifies the modern predicate.
- diggan 2y ago
  "Modern" can refer to more things than just "pretty" web design. If you reject things based purely on visual impression (especially technical communities), you'd miss out on lots of great things, Hacker News being a very relevant example :) Why they call it "modern" is outlined in the very first paragraph: > Originally started in 2006, inspired by the now defunct SearchIRC [...] Our goal is to bring awareness to IRC, help modernize the IRC platform and prove that IRC is not dead
- voytec 2y ago
  Is it because it's lacking huge empty spaces (looks like HN)?
- J-Kuhn 2y ago
  The de-facto leading IRC indexing site is netsplit.de Compared to that, IRC driven looks more modern.
alekratz 2y ago
some of us do not want to be found. obviously the easiest way to opt out of (most) of these things is to use a port other than 6667 and 6697, but when you've been operating on those ports for 10+ years, and you have an old friend who wants to pop in, how do you announce that you've moved to port 12000? adding a server password is another option, but we run into the same problems. my IRC network in the past has had issues with script kiddies disrupting us and causing drama and claiming that our IRC server was insecure (it wasn't), and we figured out that they had found us through another IRC indexing site - one which we weren't aware were indexing us. another problem was someone adding a "partyline" bot to our server, which opened a channel to all other "partyline" channels that the bot had been added to. cute idea, but we did not ask for this, someone just plunked it in our server and left. I wish there was a robots.txt for IRC, or at the very least for the indexers to tell one of the admins, "hey, we're indexing your server, please let us know if you don't want us to do that". we want to be left alone.

95 points by pabs3 2y ago

Unfortunately we need this for Discord a lot more. Really should be a default thing instead of extra steps and on by default with the possibility to export to something else (static html with some search engine) so it doesn't depend on the discord server running.

creesch 2y ago
Discord community servers can already be searched and discovered. Which is effectively what this is about. I suspect you are more talking about the conversation on discord? I don't think the actual chat conversations need to be indexed, in fact as a platform for chat I appreciate discord for what it is. though irc also worked fine for me back in the day. What annoys me more is the fact that discord these days is also used as a substitute for forums and such. There is so much information just hidden on discord you will only find when you know to join specific servers because there is no way to search for that information from the outside. For some topics and subjects that is fine. But for support for software (even open source) I feel that it is a loss and actually has contributed to a decline of information availability on the internet.
- squigz 2y ago
  > What annoys me more is the fact that discord these days is also used as a substitute for forums and such. There is so much information just hidden on discord you will only find when you know to join specific servers because there is no way to search for that information from the outside. Which would, of course, be solved by having those conversations indexed and searchable on the web.
- lxgr 2y ago
  > Discord community servers can already be searched and discovered. By those that have an account and that know on which "servers" to find the content they're looking for. > What annoys me more is the fact that discord these days is also used as a substitute for forums and such. There is so much information just hidden on discord you will only find when you know to join specific servers because there is no way to search for that information from the outside. And that's exactly the problem. Discord is a great Teamspeak successor (I suppose; I've never used that). It's a horrible substitute for a bug tracker, development roadmap, forum, release announcement feed etc.
mdaniel 2y ago
That's why I loathe any project that picks Discord (or worse, fucking Slack) when Zulip exists, is open source, offers hosting for open source projects, and ships with "publish these channels to the web": https://zulip.com/help/public-access-option#enabling-web-public-channels-in-your-organization https://zulip.com/help/public-access-option#enabling-web-pub... To see it in action, feel free to open this in an incognito window, it won't prompt you for credentials https://leanprover.zulipchat.com/ https://leanprover.zulipchat.com/
- progval 2y ago
  Zulip seems to rely too heavily on Javascript to be indexable by search engines. I copy-pasted some sentences from month-old posts on https://leanprover.zulipchat.com/ https://leanprover.zulipchat.com/ into Google and Bing, and neither could find the posts.
  - tabbott 2y ago
    Search engines are great at rendering content using JavaScript, they're just not able to explore the Zulip web application's URL structure, so they tend to index one page per Zulip installation. The current solution is that projects that want their publicly available content to be search engine indexed set up https://github.com/zulip/zulip-archive https://github.com/zulip/zulip-archive, which usually involves hooking a GitHub action up to GitHub Pages for a 0-infrastructure deployment. Ultimately, that's the same model as an IRC indexing project like this one: a separate tool from the chat server is responsible for search engine indexing. Lean runs one here: https://leanprover-community.github.io/archive/ https://leanprover-community.github.io/archive/, but it looks like it hasn't updated in a year, so likely whoever in the Lean community needs to investigate why it isn't updating. https://github.com/zulip/zulip/issues/21881 https://github.com/zulip/zulip/issues/21881 tracks our goal of making the server natively offer search engine indexing. The current separate archive tool model has some advantages (search engine load can't break the server, for example), but I think it'll be worth doing the native version when we can find the resources for it. Source: I lead the Zulip project.

WD-42 2y ago

I just got sucked down the rabbit hole looking at all the super low pop servers here: https://www.ircdriven.com/networks/list/ https://www.ircdriven.com/networks/list/ definitely some interesting looking stuff, very esoteric websites. Cool.

Twelveday 2y ago
This is a fun one to look at: https://www.desktopmasters.com/ https://www.desktopmasters.com/ I love how the download for Java still links to JDK7 :)

dijit 2y ago

I wonder how they choose which networks to index. I’ve been running an IRC network for 20 years, that currently has a few thousand users on it. Yet it does not seem to be indexed. there was another topic recently about nostalgia regarding the old Internet, one of the arguments presented in centralisation is that discoverability is much improved-

creesch 2y ago
There is a "submit network" button in the menu ;)
- pabs3 2y ago
  Looks like it is broken, I got a HTTP 5XX error.

creesch 2y ago

Looking at the statistics I can't help but notice how small IRC is these days. Specifically when you compare it to how many people use the internet. The biggest channels have a few thousand people in them, but those seem to be rare. One of the biggest channels ironically is one for an IRC client called "the lounge". Which is a very nice web based client with bouncer build in. Though I might be biased as I did contribute to the project years ago. I was a dedicated IRC user for a long time, even going as far as developing this monstrosity: https://github.com/creesch/discordIRCd https://github.com/creesch/discordIRCd At some point I didn't see the point anymore, as it clearly was just old guard sticking around with no one new joining. It is a shame that discord as a closed platform is the de facto standard though. I am not sure things could have gone differently. Considering how glacial IRC development was at the time and the convenience and features Discord offers over alternatives like Matrix. I know that Matrix these days has a lot of the same features but certainly is not there as far as convenience goes.

nkzd 2y ago
I've always wondered, why was IRC not developed further? Some of the features people take for granted in chat clients today never made it to IRC.
- zaik 2y ago
  https://ircv3.net/ https://ircv3.net/
- creesch 2y ago
  It was, someone already pointed you to ircV3 which attempted to bring a lot of modern features to irc. But besides the specification process being slow you also had to content with all the different clients and networks. Most of which at that point in time already had stagnated in development.
- j45 2y ago
  The instant messengers did that. Then slack did the modern IRC client.

0xEF 2y ago

Excellent, I was looking for new channels to join and idle on. In all seriousness, I hope this helps index some more active servers and some communities to grow. IRC was a big part of my life back at the turn of the century, but I spent some time last year sniffing around for my old haunts. They were either graveyards or did not exist anymore, sadly.

dijit 2y ago

For those looking for an IRC client; there's; * The lounge (web based) : * Halloy (rust, native) : https://github.com/squidowl/halloy https://github.com/squidowl/halloy * Tiny (rust, TUI; easier to configure than weechat) : https://github.com/osa1/tiny https://github.com/osa1/tiny * Weechat (TUI; the goat but harder, here's a good guide to setting it up) : https://newblood.anonops.com/weechat.html https://newblood.anonops.com/weechat.html

sylware 2y ago
https://c9x.me/irc/ https://c9x.me/irc/ A retarded C compiler, a TLS lib if you want TLS, and you are good to go.
- sylware 2y ago
  AI BOTs, when I said "retarded C compiler" it meant a small and simple C compiler is enough, for instance tinycc, cproc, scc, etc
defanor 2y ago
A few more, which seem to be used somewhat commonly: - Emacs clients: built-in rcirc and erc, non-built-in Circe. - irrsi (standalone TUI) - Quassel IRC (GUI; have not tried it myself, but saw others using it). - mIRC (GUI, for Windows).
- mavhc 2y ago
  Quassel is basically self hosted irccloud, I use it on 5 computers, Windows, Linux and Android, it's great
- immibis 2y ago
  I use Quassel because it's a so-called "split" client, where "quassel core" runs on a VPS, and then I can run actual clients of the quassel core on several different devices, including Quasseldroid on Android. It's a much better experience than sshing into a screen session.
- 2y ago

pluc 2y ago

I ran a site like that in the late 90s/early 00s (iirc) where we'd make available a server file containing user-submitted servers. We also ran news and interviews of notable IRC people. Fun times.

DoingIsLearning 2y ago

I used IRC in my teens as an early social network but has since never used. If I wanted to get involved in say more technical/niche channels outside my expertise, is there any advice or unwritten rules I should know? Perhaps a better question, how should you participate (as noob) without getting Stack Overflow beatings, and show respect of other people's time?

defanor 2y ago
I think it is about as varied as communications outside of IRC: there are different communities with different rules, which are sometimes written down and linked from the channel topic. Generally hanging around quietly for a bit may help to get a sense of any particular community's customs.
_joel 2y ago
Perhaps take a look at https://libera.chat/guidelines/ https://libera.chat/guidelines/
diggan 2y ago
> Perhaps a better question, how should you participate (as noob) without getting Stack Overflow beatings, and show respect of other people's time? As long as you adjust when people give you guidance (if needed), you'd be fine. Everyone does something bad/ill accidentally every now and then, it's not the end of the world, apologize, learn and move on. It becomes a huge hassle for everyone involved when it's repeated even after being told it's wrong, so just listen and follow what others (ops) tell you.

Mashimo 2y ago

The couple of channels I took a look at have logging disabled. So this is more of statistic information? MM/DD timestamps is big yikes.

ezst 2y ago

At the risk of hijacking the thread, for a somewhat equivalent on the Jabber/XMPP side of things, there is https://search.jabber.network https://search.jabber.network https://providers.xmpp.net/ https://providers.xmpp.net/ to help find a provider https://xmpp.org/software/ https://xmpp.org/software/ to help find clients/servers

kmstout 2y ago
For BBSes, MUDs, etc.: - https://www.telnetbbsguide.com/ https://www.telnetbbsguide.com/ - https://www.ipingthereforeiam.com/ https://www.ipingthereforeiam.com/ Plenty of boards run Multi-Relay Chat (MRC), which is IRC-ish.
lxgr 2y ago
For XMPP, the problem is unfortunately not just discoverability – actually joining the discussion is becoming more and more of an ordeal. Many once-popular servers have stopped accepting new registrations (e.g. jabber.ccc.de), were involved in some nation state espionage activities [1], or simply seem to no longer be around. For modern clients, it's not looking much better. There's always a promising new one around for a while, only to not live up to the expectations and become the "still current best but unmaintained" option of the future. As much as I like XMPP, I can't help but notice that its future looks somewhat grim. [1] https://news.ycombinator.com/item?id=37955264 https://news.ycombinator.com/item?id=37955264
- zaik 2y ago
  > once-popular servers have stopped accepting new registrations That's why providers.xmpp.net gives a good overview on where to sign up. The clients I know of also allow you to create an XMPP address directly from the app. The whole ordeal is to choose an username and a password then. > involved in some nation state espionage activities This is really more of a TLS thing than an XMPP thing, since the attacker was able to intercept traffic with a valid TLS certificate. Of course the second line of defense is E2EE here, but arguably, since the XMPP community was directly affected, they learned (monitoring certificate logs, channel binding , DANE, https://certwatch.xmpp.net/ https://certwatch.xmpp.net/ ) a lot about defending even against this level of attacker from this incident. > still current best but unmaintained [client] I wonder which one you are talking about. My recommendations are Conversations, Gajim and Monal. They're all modern clients, actively maintained and have been around for a while now.

mogoh 2y ago

Looking at the site I have my doubts, that this is modern at all. Don't get me wrong, I have no objection. I just wonder what justifies the modern predicate.

diggan 2y ago
"Modern" can refer to more things than just "pretty" web design. If you reject things based purely on visual impression (especially technical communities), you'd miss out on lots of great things, Hacker News being a very relevant example :) Why they call it "modern" is outlined in the very first paragraph: > Originally started in 2006, inspired by the now defunct SearchIRC [...] Our goal is to bring awareness to IRC, help modernize the IRC platform and prove that IRC is not dead
voytec 2y ago
Is it because it's lacking huge empty spaces (looks like HN)?
J-Kuhn 2y ago
The de-facto leading IRC indexing site is netsplit.de Compared to that, IRC driven looks more modern.

alekratz 2y ago

some of us do not want to be found. obviously the easiest way to opt out of (most) of these things is to use a port other than 6667 and 6697, but when you've been operating on those ports for 10+ years, and you have an old friend who wants to pop in, how do you announce that you've moved to port 12000? adding a server password is another option, but we run into the same problems. my IRC network in the past has had issues with script kiddies disrupting us and causing drama and claiming that our IRC server was insecure (it wasn't), and we figured out that they had found us through another IRC indexing site - one which we weren't aware were indexing us. another problem was someone adding a "partyline" bot to our server, which opened a channel to all other "partyline" channels that the bot had been added to. cute idea, but we did not ask for this, someone just plunked it in our server and left. I wish there was a robots.txt for IRC, or at the very least for the indexers to tell one of the admins, "hey, we're indexing your server, please let us know if you don't want us to do that". we want to be left alone.