5 ms·

How We Moved 34k Wired Pages to One Site

56 points by nols 11y ago

efnresident 11y ago
Pleasantly surprised by this post. Interesting project that could be applied to other archived publications.
- primus202 11y ago
  Thanks! I was the lead...well only dev on this project and wrote this article. I tried to build the tool to be as flexible and reusable as possible but some compromises had to be made. I hope to, eventually, make the changes necessary to have it be truly adaptable so we can open source it. Fingers crossed!
  - mthoms 11y ago
    Great stuff. I'd be keenly interested in this both as a contributor and user. My email is in my profile if/when you do open source it I'd love to hear about it.
aaronbrethorst 11y ago
While they serve as an interesting historic record of digital trends, they lack the read- ability and reusability of our current site. Yeah, especially those super-readable headlines on the new site.
- primus202 11y ago
  You can disable 3rd party fonts if you really dislike them that much :D
  - aaronbrethorst 11y ago
    I like good 3rd party font choices. Edit: Sorry, that was snarky and offers little value. I am generally a fan of creative web typography. I like the fact that we can use fonts beyond Times New Roman on the web. That said, I think that the headline font used on Wired is excruciatingly awful. Ten bucks that says if you did a survey of Wired readers, a sizable majority would agree with me. What sucks is that there is no financial incentive to change the headline font: you're not going to make more money by changing the font, which means that none of the business people at Wired will ever force ac change. So everyone who visits a Wired.com page has to suffer through your design director's bad font choice. :(
    - domador 11y ago
      Yes, the long, narrow font Wired now uses for headlines is hard to read and pretty ugly. At least it's not the even-harder to read, even narrower font used for the credits at the bottom of movie posters.
kyle6884 11y ago
Might be a good idea to put a canonical tag on the old to the new content. http://archive.wired.com/wired/archive/1.02/crypto.rebels_pr.html http://archive.wired.com/wired/archive/1.02/crypto.rebels_pr... > http://www.wired.com/1993/02/crypto-rebels/ http://www.wired.com/1993/02/crypto-rebels/
- primus202 11y ago
  Thanks for the catch! We're still fine tuning our redirects from archive to the new article locations. We no longer have print pages so we'll need to redirect those to the new posts. At least the actual post (i.e. http://archive.wired.com/wired/archive/1.02/crypto.rebels.html http://archive.wired.com/wired/archive/1.02/crypto.rebels.ht...) is redirecting correctly at the moment. Also you can see at the bottom of the actual article on WIRED.com we did add tags to archived magazine content, like the article you mentioned, to group content together a bit. Thanks!
  - cedricr 11y ago
    http://www.wired.com/1995/06/xanadu/ http://www.wired.com/1995/06/xanadu/ is missing the first few paragraphs from http://archive.wired.com/wired/archive//3.06/xanadu_pr.html http://archive.wired.com/wired/archive//3.06/xanadu_pr.html That being said, great work!
  - tvanantwerp 11y ago
    I'm about to transition an old Drupal site with 10+ years of content to WordPress with new URL patterns. I'm curious, how have you approached the doubtlessly massive number of redirects at Wired?
    - mschuster91 11y ago
      Once you have a list (which you already have if you've automigrated it) just generate a list of rewrite rules with header set to "Moved Permanently". But I'm curious, usually platform changes go from Wordpress to Drupal because people need features other than a blog and simple CMS system. What made you go the opposite route?
carbocation 11y ago
> How We Moved 34,000 WIRED Pages to One Site in 9 Hours > Starting in April, Cyphon quickly consumed most of my working hours. This juxtaposition either emphasizes the fantastically few hours worked (9 hours since April!) or the slight exaggeration about moving 34k pages in 9 hours.
- singlow 11y ago
  I read it as the transfer took 9 hours, not all of the work to prepare for the transfer.
  - carbocation 11y ago
    You are correct, of course. But in that more limited sense, the 9 hours is not particularly interesting. Actually, I would say that the long hours developing the tools are much more interesting!
anon5_ 11y ago
You guys were hardly successful - I still go through 27bstroke6 (yes, i remember the old name of threatlevel) and find the PDF affidavits are broken links. Disappointing guys. Do you care enough? Can you guys hire and intern, go back into pacer and bring back the PDF's? Make a log of your progress please. Do that, and you'll win my respect. Poulsen's writing was great.
teddyh 11y ago
When the Wayback Machine does a better job than you do in preserving your content, you have failed. I looked through some old bookmarks, and I couldn’t even find these articles on the new site: http://web.archive.org/web/20090220095241/http://www.wired.com/culture/lifestyle/news/2000/03/35143 http://web.archive.org/web/20090220095241/http://www.wired.c... http://archive.wired.com/politics/security/commentary/securitymatters/2006/05/70886 http://archive.wired.com/politics/security/commentary/securi...
pgrote 11y ago
Odd. There are no redirects for the old articles to the new ones. For instance, this example: http://www.wired.com/1999/07/pilgrims/ http://www.wired.com/1999/07/pilgrims/ Originally showed up here: https://archive.wired.com/wired/archive/7.07/pilgrims_pr.html https://archive.wired.com/wired/archive/7.07/pilgrims_pr.htm... It doesn't redirect.
gamesbrainiac 11y ago
Am I missing something here, and I'm not trying to be snarky, but how is this a big deal?

56 points by nols 11y ago

efnresident 11y ago

Pleasantly surprised by this post. Interesting project that could be applied to other archived publications.

primus202 11y ago
Thanks! I was the lead...well only dev on this project and wrote this article. I tried to build the tool to be as flexible and reusable as possible but some compromises had to be made. I hope to, eventually, make the changes necessary to have it be truly adaptable so we can open source it. Fingers crossed!
- mthoms 11y ago
  Great stuff. I'd be keenly interested in this both as a contributor and user. My email is in my profile if/when you do open source it I'd love to hear about it.

aaronbrethorst 11y ago

While they serve as an interesting historic record of digital trends, they lack the read- ability and reusability of our current site. Yeah, especially those super-readable headlines on the new site.

primus202 11y ago
You can disable 3rd party fonts if you really dislike them that much :D
- aaronbrethorst 11y ago
  I like good 3rd party font choices. Edit: Sorry, that was snarky and offers little value. I am generally a fan of creative web typography. I like the fact that we can use fonts beyond Times New Roman on the web. That said, I think that the headline font used on Wired is excruciatingly awful. Ten bucks that says if you did a survey of Wired readers, a sizable majority would agree with me. What sucks is that there is no financial incentive to change the headline font: you're not going to make more money by changing the font, which means that none of the business people at Wired will ever force ac change. So everyone who visits a Wired.com page has to suffer through your design director's bad font choice. :(
  - domador 11y ago
    Yes, the long, narrow font Wired now uses for headlines is hard to read and pretty ugly. At least it's not the even-harder to read, even narrower font used for the credits at the bottom of movie posters.

kyle6884 11y ago

Might be a good idea to put a canonical tag on the old to the new content. http://archive.wired.com/wired/archive/1.02/crypto.rebels_pr.html http://archive.wired.com/wired/archive/1.02/crypto.rebels_pr... > http://www.wired.com/1993/02/crypto-rebels/ http://www.wired.com/1993/02/crypto-rebels/

primus202 11y ago
Thanks for the catch! We're still fine tuning our redirects from archive to the new article locations. We no longer have print pages so we'll need to redirect those to the new posts. At least the actual post (i.e. http://archive.wired.com/wired/archive/1.02/crypto.rebels.html http://archive.wired.com/wired/archive/1.02/crypto.rebels.ht...) is redirecting correctly at the moment. Also you can see at the bottom of the actual article on WIRED.com we did add tags to archived magazine content, like the article you mentioned, to group content together a bit. Thanks!
- cedricr 11y ago
  http://www.wired.com/1995/06/xanadu/ http://www.wired.com/1995/06/xanadu/ is missing the first few paragraphs from http://archive.wired.com/wired/archive//3.06/xanadu_pr.html http://archive.wired.com/wired/archive//3.06/xanadu_pr.html That being said, great work!
- tvanantwerp 11y ago
  I'm about to transition an old Drupal site with 10+ years of content to WordPress with new URL patterns. I'm curious, how have you approached the doubtlessly massive number of redirects at Wired?
  - mschuster91 11y ago
    Once you have a list (which you already have if you've automigrated it) just generate a list of rewrite rules with header set to "Moved Permanently". But I'm curious, usually platform changes go from Wordpress to Drupal because people need features other than a blog and simple CMS system. What made you go the opposite route?

carbocation 11y ago

> How We Moved 34,000 WIRED Pages to One Site in 9 Hours > Starting in April, Cyphon quickly consumed most of my working hours. This juxtaposition either emphasizes the fantastically few hours worked (9 hours since April!) or the slight exaggeration about moving 34k pages in 9 hours.

singlow 11y ago
I read it as the transfer took 9 hours, not all of the work to prepare for the transfer.
- carbocation 11y ago
  You are correct, of course. But in that more limited sense, the 9 hours is not particularly interesting. Actually, I would say that the long hours developing the tools are much more interesting!

anon5_ 11y ago

You guys were hardly successful - I still go through 27bstroke6 (yes, i remember the old name of threatlevel) and find the PDF affidavits are broken links. Disappointing guys. Do you care enough? Can you guys hire and intern, go back into pacer and bring back the PDF's? Make a log of your progress please. Do that, and you'll win my respect. Poulsen's writing was great.

teddyh 11y ago

When the Wayback Machine does a better job than you do in preserving your content, you have failed. I looked through some old bookmarks, and I couldn’t even find these articles on the new site: http://web.archive.org/web/20090220095241/http://www.wired.com/culture/lifestyle/news/2000/03/35143 http://web.archive.org/web/20090220095241/http://www.wired.c... http://archive.wired.com/politics/security/commentary/securitymatters/2006/05/70886 http://archive.wired.com/politics/security/commentary/securi...

pgrote 11y ago

Odd. There are no redirects for the old articles to the new ones. For instance, this example: http://www.wired.com/1999/07/pilgrims/ http://www.wired.com/1999/07/pilgrims/ Originally showed up here: https://archive.wired.com/wired/archive/7.07/pilgrims_pr.html https://archive.wired.com/wired/archive/7.07/pilgrims_pr.htm... It doesn't redirect.

gamesbrainiac 11y ago

Am I missing something here, and I'm not trying to be snarky, but how is this a big deal?