6 ms·

I have a group I still care about. Here's how I'm archiving it: git clone git@github.com:andrewferguson/YahooGroups-Archiver.git python archive_group.

by jefftk 7y ago

I have a group I still care about. Here's how I'm archiving it: git clone git@github.com:andrewferguson/YahooGroups-Archiver.git python archive_group.py <group-name> See https://github.com/andrewferguson/YahooGroups-Archiver https://github.com/andrewferguson/YahooGroups-Archiver and https://www.archiveteam.org/index.php?title=Yahoo!_Groups https://www.archiveteam.org/index.php?title=Yahoo!_Groups

spullara 7y ago
If you don't use git authentication and use https, this works better: git clone https://github.com/andrewferguson/YahooGroups-Archiver.git
raincom 7y ago
Noob question: how to convert the downloaded json file into a readable message?
- icebraining 7y ago
  First run those commands, then run python make_Yearly_Text_Archive_html.py <group-name>
  - Alex3917 7y ago
    Happen to have a screenshot of the output of that? Just from skimming the source, I would imagine that messages in longer threads get pretty unreadable.
    - raincom 7y ago
      https://i.imgur.com/s7IiHO6.png https://i.imgur.com/s7IiHO6.png https://i.imgur.com/V6QzKdT.png https://i.imgur.com/V6QzKdT.png Run these scripts with python2, as it uses print statements. Also pip install natsort
      - milankragujevic 7y ago
        They meant a screenshot of the output file.
swyx 7y ago
i'm getting this:  YahooGroups-Archiver  master  python archive_group.py tns98 Archiving group 'tns98', mode: update , on Wed Oct 16 20:01:04 2019 Traceback (most recent call last): File "archive_group.py", line 143, in <module> archive_group(sys.argv[1]) File "archive_group.py", line 67, in archive_group max = group_messages_max(groupName) File "archive_group.py", line 89, in group_messages_max return pageJson["ygData"]["totalRecords"] UnboundLocalError: local variable 'pageJson' referenced before assignment but nobody else seems to be facing this issue?
- akovaski 7y ago
  I got this error when trying to archive a private group, see the README for how to archive private groups.
  - swyx 7y ago
    ah thanks, wish that was documented. thought it was public.
- raincom 7y ago
  tns98 is a private group. " To archive a private group using this tool, login to a Yahoo account that has access to the private groups, then extract the data from the cookies Y and T from the domain yahoo.com . Paste this data into the appropriate variables (cookie_Y and cookie_T) at the top of this script, and run the script again."
  - swyx 7y ago
    that was it, cheers. didnt realize it was private.
swyx 7y ago
trying this alternative: https://github.com/csaftoiu/yahoo-groups-backup https://github.com/csaftoiu/yahoo-groups-backup
- raincom 7y ago
  I tried this long time back. It requires a mongodb instance. I was not happy about it.
- logological 7y ago
  Doesn't work for me. The file scraper crashes with the default Firefox Selenium driver, and with the Chrome driver it immediately quits, falsely reporting that it successfully downloaded all the files.
hcs 7y ago
I'm using https://github.com/IgnoredAmbience/yahoo-group-archiver https://github.com/IgnoredAmbience/yahoo-group-archiver , it has support for databases and files which is important for the groups I want to archive. It's not my project, but I'm working on improving it, somewhat fixed the database support yesterday. Trying to figure out how to add support for links, but I'm stuck at link subdirectories/folders; I can't find a way to list the links that aren't in the root dir with the API (though once I have the link name I can get them with links?filename=DIR/LINK.url)
- logological 7y ago
  Does that script even work for you? When I run it, I'm prompted for my Yahoo! password, but after typing it in the program just spits out a "Login failed" message. The problem has been reported on the issue tracker but there's been no activity on it since February.
  - patja 7y ago
    You need to add some parameters for the Yahoo authentication cookie values. -tc and -ty should be set to the cookie values from your browser, after logging into Yahoo. It also needs some try/except handling to be added for all the cases where Yahoo fails to provide the requested asset. Once you sort those things out, it works pretty well.
    - dude17943 7y ago
      Could you please explain what this means and how to do it? This is not documented on the project page, and I'm having the same issue as the poster above. login.yahoo.com creates 2 cookies. The program's help identifies two flags called -ct and -cy which I assume you're referring to. Am I to understand that to properly run the program, the command should be entered as so: ./yahoo.py -ct %cookieOneContent -cy %cookieTwoContent -u username -p password groupname I don't know which cookie goes where so I tried it both ways, and both with and without the -u username and -p password flags; none of them worked, only returning HTTP Errors.

by jefftk 7y ago

spullara 7y ago

If you don't use git authentication and use https, this works better: git clone https://github.com/andrewferguson/YahooGroups-Archiver.git

raincom 7y ago

Noob question: how to convert the downloaded json file into a readable message?

icebraining 7y ago
First run those commands, then run python make_Yearly_Text_Archive_html.py <group-name>
- Alex3917 7y ago
  Happen to have a screenshot of the output of that? Just from skimming the source, I would imagine that messages in longer threads get pretty unreadable.
  - raincom 7y ago
    https://i.imgur.com/s7IiHO6.png https://i.imgur.com/s7IiHO6.png https://i.imgur.com/V6QzKdT.png https://i.imgur.com/V6QzKdT.png Run these scripts with python2, as it uses print statements. Also pip install natsort
    - milankragujevic 7y ago
      They meant a screenshot of the output file.

swyx 7y ago

i'm getting this:  YahooGroups-Archiver  master  python archive_group.py tns98 Archiving group 'tns98', mode: update , on Wed Oct 16 20:01:04 2019 Traceback (most recent call last): File "archive_group.py", line 143, in <module> archive_group(sys.argv[1]) File "archive_group.py", line 67, in archive_group max = group_messages_max(groupName) File "archive_group.py", line 89, in group_messages_max return pageJson["ygData"]["totalRecords"] UnboundLocalError: local variable 'pageJson' referenced before assignment but nobody else seems to be facing this issue?

akovaski 7y ago
I got this error when trying to archive a private group, see the README for how to archive private groups.
- swyx 7y ago
  ah thanks, wish that was documented. thought it was public.
raincom 7y ago
tns98 is a private group. " To archive a private group using this tool, login to a Yahoo account that has access to the private groups, then extract the data from the cookies Y and T from the domain yahoo.com . Paste this data into the appropriate variables (cookie_Y and cookie_T) at the top of this script, and run the script again."
- swyx 7y ago
  that was it, cheers. didnt realize it was private.

swyx 7y ago

trying this alternative: https://github.com/csaftoiu/yahoo-groups-backup https://github.com/csaftoiu/yahoo-groups-backup

raincom 7y ago
I tried this long time back. It requires a mongodb instance. I was not happy about it.
logological 7y ago
Doesn't work for me. The file scraper crashes with the default Firefox Selenium driver, and with the Chrome driver it immediately quits, falsely reporting that it successfully downloaded all the files.

hcs 7y ago

I'm using https://github.com/IgnoredAmbience/yahoo-group-archiver https://github.com/IgnoredAmbience/yahoo-group-archiver , it has support for databases and files which is important for the groups I want to archive. It's not my project, but I'm working on improving it, somewhat fixed the database support yesterday. Trying to figure out how to add support for links, but I'm stuck at link subdirectories/folders; I can't find a way to list the links that aren't in the root dir with the API (though once I have the link name I can get them with links?filename=DIR/LINK.url)

logological 7y ago
Does that script even work for you? When I run it, I'm prompted for my Yahoo! password, but after typing it in the program just spits out a "Login failed" message. The problem has been reported on the issue tracker but there's been no activity on it since February.
- patja 7y ago
  You need to add some parameters for the Yahoo authentication cookie values. -tc and -ty should be set to the cookie values from your browser, after logging into Yahoo. It also needs some try/except handling to be added for all the cases where Yahoo fails to provide the requested asset. Once you sort those things out, it works pretty well.
  - dude17943 7y ago
    Could you please explain what this means and how to do it? This is not documented on the project page, and I'm having the same issue as the poster above. login.yahoo.com creates 2 cookies. The program's help identifies two flags called -ct and -cy which I assume you're referring to. Am I to understand that to properly run the program, the command should be entered as so: ./yahoo.py -ct %cookieOneContent -cy %cookieTwoContent -u username -p password groupname I don't know which cookie goes where so I tried it both ways, and both with and without the -u username and -p password flags; none of them worked, only returning HTTP Errors.