You might talk to the guys at New Idea Engineering about their xpump technology http://www.ideaeng.com/ds/xpump.html http://www.ideaeng.com/ds/xpump.html I used

You might talk to the guys at New Idea Engineering about their xpump technology http://www.ideaeng.com/ds/xpump.html http://www.ideaeng.com/ds/xpump.html I used it a few years ago to process all of the hardware data sheets on the Cisco website and extract 13 parameters such as height, width, depth, weight, power consumption (AC and DC), etc...Because Cisco's products come from many different acquisitions the datasheets were in many different formats, which sounds similar to your problem. One point I would make is that a fast crawler is not always the best for this type of application: crawling at about the speed a user would click on pages is more friendly to a site and less likely to have them take steps to block your access.