8 ms·

Folks, I am the guy behind this project. A friend of mine mentioned he saw the site mentioned on hacker news, so I came to check it out. If you have any questi

Folks, I am the guy behind this project. A friend of mine mentioned he saw the site mentioned on hacker news, so I came to check it out. If you have any questions for me, don't hesitate to ask, as time permits (and two little boys), I'll do my best to answer them.

CarelessExpert 6y ago
I just want to say thank you to you and everyone involved! I ran a little mini project a while ago to archive my old Usenet posts by using puppeteer to scrape the content from Google groups, knowing full well that they could shut it down by fiat any time. I've frankly never trusted them as a steward for this and I'm glad to see someone stepping in. Do you have any plans to team up with the Internet Archive on this?
- jozefjarosciak 6y ago
  Once I have everything in DB format, I'll certainly attempt to post it there.
  - codetrotter 6y ago
    Excellent! Btw, in the DB format the posts are still uncensored or?
  - toomuchtodo 6y ago
    Please get in touch with collections-service@archive.org (Internet Archive Patron Services) when you’re ready. They will assist you with uploading the corpus to the archive (item creation, collection creation and assignment, metadata hygiene, etc). Thank you for your efforts!
KyleSanderson 6y ago
Nice work my friend. As many demo-scene groups were active on newsgroups before going very dark almost 30 years ago I'm curious if you also have a collection that you can contribute to this basic collection https://archive.org/details/scenenotices https://archive.org/details/scenenotices . There's lots of this stuff going on throughout the decades but you may be in a position to help preserve history / lineage on the entire environment.
- jozefjarosciak 6y ago
  I won't have much when it comes to binaries, I am text mostly, but in terms of text I should have things going that far back.
- scruffyherder 6y ago
  Archive.org destroyed their utzoo mirror. It’s sadly not good enough.
throwaway89201 6y ago
Why are many words censored? To take a completely random example (I just took one with many asterisks) where it makes the post completely illegible: https://www.usenetarchives.com/view.php?id=soc.sexuality.general&g=79909&p=0 https://www.usenetarchives.com/view.php?id=soc.sexuality.gen... For an archive this is a big no-no. Respect the source material! Otherwise, thank you for the time spent doing this.
- jolmg 6y ago
  > For an archive this is a big no-no. Respect the source material! Though I'm also curious, that's perhaps not the tone I would have used when asking. After all, better a censored archive than no archive. I'm just speculating, but it may be the policy of usenetarchives.com, in order to accept their upload. Censoring seems to be done around email addresses, names, and offensive words. Perhaps, this is done to reduce the chances of people later asking for the posts to be taken down entirely. For example, I believe the takedown by Google of comp.lang.lisp and comp.lang.forth commented elsewhere was done because there was offensive content present. The Google support request that mentioned that reason was taken down, but it's what I remember.
  - jozefjarosciak 6y ago
    I've decided to remove bad word filtering and all other censoring. Let's see how it goes.
- usefulcat 6y ago
  For that example, it's at least fairly obvious why it was censored, but this one really puzzles me: "Getting good FP performance from a micro seems to require pipelining. Keeping the p<asterisk><asterisk>e(s) full seems to require a certain amount of parallelism and regularity." https://www.usenetarchives.com/view.php?id=comp.arch&g=149653&p=0 https://www.usenetarchives.com/view.php?id=comp.arch&g=14965... Edit: Ah, the irony. HN markdown causes two consecutive asterisks to disappear.
  - jolmg 6y ago
Multicomp 6y ago
THANK you for doing this (I said, having never so much as logged in to a usenet thingie)! I do have one question. Those tapes...how old were they? Were they contemporary to the postings? (bonus question: if so wow, but under what justification?. tapes were expensive right and nobody valued archiving at the time (except maybe this guy)?) this individual you got the tapes from meticulously copy onto new media and all the overhead that entails? Today I make personal backups of things I want to keep, whether local files or web snippets and burn 'em to a DVD , blu-ray, or optical disk of some kind with the justification of 'can't ransomware WORM media'. However, I don't do the internet archiving guru stuff of '3 copies, 2 mediums, 1 routine' (or something like that) so in a sense my backups are cruising for a bruising in the case that optical disks could go bad, hard drives could die (these I do migrate to new media when I get around to it, CDs I just read and burn to a fresh one), house could burn down, etc. Partially looking to see if I can get justification for my backup slovenliness ;)
- ericbarrett 6y ago
  To satisfy the "third place" you might look into storing these files in Amazon S3 Glacier, which is about $1/TB/month, so long as you don't read them back. (Retrieval is delayed, batched, and expensive.) It would take some engineering: I might store each optical disk image as a compressed image file, for example (zstd would be good for the large amount of data), to avoid metadata charges. Fun to think about.
gravitas 6y ago
Thank you! Is 2003 a hard limit to how far back was captured, or just a current point in the import? I was looking to comp.os.linux.* groups from a bit earlier than that, a lot of the 90s. Thanks again :)
- thimkerbell 6y ago
  People, once you get to a newsgroup, click on 'all years' to see which years are in the archive, and select which year you want. Don't expect posts from last millennium to be present, at least not yet.
  - gravitas 6y ago
    At the time of my comment, 2003 was the oldest year present for the group(s) i mentioned. I searched for a FAQ before posting, it's not clearly marked "imports are still in progress". Please don't be condescending, OP specifically stated "ask me questions." You are not OP. Edit: I am now on a desktop, and confirm it's not a mobile glitch; the archives for comp.os.linux are imported back to 2003 only at this time. https://www.usenetarchives.com/threads.php?id=comp.os.linux&p=0 https://www.usenetarchives.com/threads.php?id=comp.os.linux&...
cmrdporcupine 6y ago
I'm getting this: Fatal error: Uncaught Error: Call to a member function real_escape_string() on null in /var/www/html/usenetarchives.com/3.vars.php:80 Stack trace: #0 /var/www/html/usenetarchives.com/1.header.php(190): include() #1 /var/www/html/usenetarchives.com/view.php(22): include_once('/var/www/html/u...') #2 {main} thrown in /var/www/html/usenetarchives.com/3.vars.php on line 80 Pretty consistently.
- jozefjarosciak 6y ago
  on which pages, please share some examples... I was doing some changes there, let me know...
epc 6y ago
There's weird gaps where groups have some posts from a given date but not all of the posts (e.g. I know I posted to csm.hypercard on 24 April 1990 and can see it in Google's dejanews cache, but not in this cache which has other posts to the group on that date). Was it just luck of the draw from what was cut to tape? How do you handle cross posting?
- jozefjarosciak 6y ago
  I am still in the process of moving posts to the online DB.
CamperBob2 6y ago
Agree with throwaway89201. Don't do that. Reproduce the text as you found it.
zeveb 6y ago
Do you have any concerns about the so-called 'right' to be forgotten? Are you concerned that anyone who was active back then might lose a job now due to his postings a couple of decades ago?
davidwritesbugs 6y ago
Will you be making this available on nntp servers? Might be nice to see it on a text only nntp server. Sure one of the big usenet providers would want help seems on their interest.
cerebrum 6y ago
Do you know where I can find an archive with the de.* (German) groups?
microtherion 6y ago
It's great that your service comes into existence at all, and if you set up an opencollective or patreon account, I'd gladly contribute to the operating costs. As it exists right now, there are some glaring omissions. I would imagine that showing authors in the search results next to the titles would not be prohibitively difficult. Proper threading of posts would probably be considerably harder, but would provide immense value IMHO.
lixtra 6y ago
How do you deal with the (European) right to be forgotten? On one hand it’s nice that we can see all this old data. On the other hand, as far as I remember in the 90s you typically subscribed to a newsgroup and only then got recent posts. So the expectations was a limited life time of the post. Of course it would still be stored in thousands of readers. Obviously the copyright of each post is with the author. You just assume that you have a right to redistribute. I’m also happy if I get a pointer elsewhere if you have an url.
- msla 6y ago
  > How do you deal with the (European) right to be forgotten? Probably by not being European. Europe doesn't make laws for the whole world anymore. Colonialism is dead.
  - lixtra 6y ago
    What about copyright?
    - SturgeonsLaw 6y ago
      Archiving is considered fair use
scruffyherder 6y ago
Any chance at getting access to all the other raw data? The UTZOO archives have been invaluable, and the only reason they survive today is because they were mirrored. DMCA took this incredible resource off line. Please don’t let this happen again, as even archive.org found it easier to destroy it all than to fight Marty. Plus it allows all us little people build fun things with it, like my altavista based search of utzoo: altavista.superglobalmegacorp.com Thanks
joeyh 6y ago
I could have sworn I did this 9 years ago. :-) http://kitenet.net/~joey/blog/entry/announcing_olduse.net/ http://kitenet.net/~joey/blog/entry/announcing_olduse.net/ Did you find additional utzoo material beyond what was already on archive.org in tarballs?

CarelessExpert 6y ago

I just want to say thank you to you and everyone involved! I ran a little mini project a while ago to archive my old Usenet posts by using puppeteer to scrape the content from Google groups, knowing full well that they could shut it down by fiat any time. I've frankly never trusted them as a steward for this and I'm glad to see someone stepping in. Do you have any plans to team up with the Internet Archive on this?

jozefjarosciak 6y ago
Once I have everything in DB format, I'll certainly attempt to post it there.
- codetrotter 6y ago
  Excellent! Btw, in the DB format the posts are still uncensored or?
- toomuchtodo 6y ago
  Please get in touch with collections-service@archive.org (Internet Archive Patron Services) when you’re ready. They will assist you with uploading the corpus to the archive (item creation, collection creation and assignment, metadata hygiene, etc). Thank you for your efforts!

KyleSanderson 6y ago

Nice work my friend. As many demo-scene groups were active on newsgroups before going very dark almost 30 years ago I'm curious if you also have a collection that you can contribute to this basic collection https://archive.org/details/scenenotices https://archive.org/details/scenenotices . There's lots of this stuff going on throughout the decades but you may be in a position to help preserve history / lineage on the entire environment.

jozefjarosciak 6y ago
I won't have much when it comes to binaries, I am text mostly, but in terms of text I should have things going that far back.
scruffyherder 6y ago
Archive.org destroyed their utzoo mirror. It’s sadly not good enough.

throwaway89201 6y ago

Why are many words censored? To take a completely random example (I just took one with many asterisks) where it makes the post completely illegible: https://www.usenetarchives.com/view.php?id=soc.sexuality.general&g=79909&p=0 https://www.usenetarchives.com/view.php?id=soc.sexuality.gen... For an archive this is a big no-no. Respect the source material! Otherwise, thank you for the time spent doing this.

jolmg 6y ago
> For an archive this is a big no-no. Respect the source material! Though I'm also curious, that's perhaps not the tone I would have used when asking. After all, better a censored archive than no archive. I'm just speculating, but it may be the policy of usenetarchives.com, in order to accept their upload. Censoring seems to be done around email addresses, names, and offensive words. Perhaps, this is done to reduce the chances of people later asking for the posts to be taken down entirely. For example, I believe the takedown by Google of comp.lang.lisp and comp.lang.forth commented elsewhere was done because there was offensive content present. The Google support request that mentioned that reason was taken down, but it's what I remember.
- jozefjarosciak 6y ago
  I've decided to remove bad word filtering and all other censoring. Let's see how it goes.
usefulcat 6y ago
For that example, it's at least fairly obvious why it was censored, but this one really puzzles me: "Getting good FP performance from a micro seems to require pipelining. Keeping the p<asterisk><asterisk>e(s) full seems to require a certain amount of parallelism and regularity." https://www.usenetarchives.com/view.php?id=comp.arch&g=149653&p=0 https://www.usenetarchives.com/view.php?id=comp.arch&g=14965... Edit: Ah, the irony. HN markdown causes two consecutive asterisks to disappear.
- jolmg 6y ago

Multicomp 6y ago

THANK you for doing this (I said, having never so much as logged in to a usenet thingie)! I do have one question. Those tapes...how old were they? Were they contemporary to the postings? (bonus question: if so wow, but under what justification?. tapes were expensive right and nobody valued archiving at the time (except maybe this guy)?) this individual you got the tapes from meticulously copy onto new media and all the overhead that entails? Today I make personal backups of things I want to keep, whether local files or web snippets and burn 'em to a DVD , blu-ray, or optical disk of some kind with the justification of 'can't ransomware WORM media'. However, I don't do the internet archiving guru stuff of '3 copies, 2 mediums, 1 routine' (or something like that) so in a sense my backups are cruising for a bruising in the case that optical disks could go bad, hard drives could die (these I do migrate to new media when I get around to it, CDs I just read and burn to a fresh one), house could burn down, etc. Partially looking to see if I can get justification for my backup slovenliness ;)

ericbarrett 6y ago
To satisfy the "third place" you might look into storing these files in Amazon S3 Glacier, which is about $1/TB/month, so long as you don't read them back. (Retrieval is delayed, batched, and expensive.) It would take some engineering: I might store each optical disk image as a compressed image file, for example (zstd would be good for the large amount of data), to avoid metadata charges. Fun to think about.

gravitas 6y ago

Thank you! Is 2003 a hard limit to how far back was captured, or just a current point in the import? I was looking to comp.os.linux.* groups from a bit earlier than that, a lot of the 90s. Thanks again :)

thimkerbell 6y ago
People, once you get to a newsgroup, click on 'all years' to see which years are in the archive, and select which year you want. Don't expect posts from last millennium to be present, at least not yet.
- gravitas 6y ago
  At the time of my comment, 2003 was the oldest year present for the group(s) i mentioned. I searched for a FAQ before posting, it's not clearly marked "imports are still in progress". Please don't be condescending, OP specifically stated "ask me questions." You are not OP. Edit: I am now on a desktop, and confirm it's not a mobile glitch; the archives for comp.os.linux are imported back to 2003 only at this time. https://www.usenetarchives.com/threads.php?id=comp.os.linux&p=0 https://www.usenetarchives.com/threads.php?id=comp.os.linux&...

cmrdporcupine 6y ago

I'm getting this: Fatal error: Uncaught Error: Call to a member function real_escape_string() on null in /var/www/html/usenetarchives.com/3.vars.php:80 Stack trace: #0 /var/www/html/usenetarchives.com/1.header.php(190): include() #1 /var/www/html/usenetarchives.com/view.php(22): include_once('/var/www/html/u...') #2 {main} thrown in /var/www/html/usenetarchives.com/3.vars.php on line 80 Pretty consistently.

jozefjarosciak 6y ago
on which pages, please share some examples... I was doing some changes there, let me know...

epc 6y ago

There's weird gaps where groups have some posts from a given date but not all of the posts (e.g. I know I posted to csm.hypercard on 24 April 1990 and can see it in Google's dejanews cache, but not in this cache which has other posts to the group on that date). Was it just luck of the draw from what was cut to tape? How do you handle cross posting?

jozefjarosciak 6y ago
I am still in the process of moving posts to the online DB.

CamperBob2 6y ago

Agree with throwaway89201. Don't do that. Reproduce the text as you found it.

zeveb 6y ago

Do you have any concerns about the so-called 'right' to be forgotten? Are you concerned that anyone who was active back then might lose a job now due to his postings a couple of decades ago?

davidwritesbugs 6y ago

Will you be making this available on nntp servers? Might be nice to see it on a text only nntp server. Sure one of the big usenet providers would want help seems on their interest.

cerebrum 6y ago

Do you know where I can find an archive with the de.* (German) groups?

microtherion 6y ago

It's great that your service comes into existence at all, and if you set up an opencollective or patreon account, I'd gladly contribute to the operating costs. As it exists right now, there are some glaring omissions. I would imagine that showing authors in the search results next to the titles would not be prohibitively difficult. Proper threading of posts would probably be considerably harder, but would provide immense value IMHO.

lixtra 6y ago

How do you deal with the (European) right to be forgotten? On one hand it’s nice that we can see all this old data. On the other hand, as far as I remember in the 90s you typically subscribed to a newsgroup and only then got recent posts. So the expectations was a limited life time of the post. Of course it would still be stored in thousands of readers. Obviously the copyright of each post is with the author. You just assume that you have a right to redistribute. I’m also happy if I get a pointer elsewhere if you have an url.

msla 6y ago
> How do you deal with the (European) right to be forgotten? Probably by not being European. Europe doesn't make laws for the whole world anymore. Colonialism is dead.
- lixtra 6y ago
  What about copyright?
  - SturgeonsLaw 6y ago
    Archiving is considered fair use

scruffyherder 6y ago

Any chance at getting access to all the other raw data? The UTZOO archives have been invaluable, and the only reason they survive today is because they were mirrored. DMCA took this incredible resource off line. Please don’t let this happen again, as even archive.org found it easier to destroy it all than to fight Marty. Plus it allows all us little people build fun things with it, like my altavista based search of utzoo: altavista.superglobalmegacorp.com Thanks

joeyh 6y ago

I could have sworn I did this 9 years ago. :-) http://kitenet.net/~joey/blog/entry/announcing_olduse.net/ http://kitenet.net/~joey/blog/entry/announcing_olduse.net/ Did you find additional utzoo material beyond what was already on archive.org in tarballs?