You aren't going to find a magic module that does what you want. Break down the problem a bit. Parse xml (RSS/ATOM feeds) Parse tabular data (some times) -> p

You aren't going to find a magic module that does what you want. Break down the problem a bit. Parse xml (RSS/ATOM feeds) Parse tabular data (some times) -> parsing html *Parse html (DON'T USE REGEXES) You're on the right track where you're going to have to write a separate parser for each source. I would recommend structuring your app in the mindset of drivers for your computer's graphics card. The OS doesn't need to know about all the thousands of kinds of graphics card hardware. It just needs a set of drivers for the ones you're going to use with it. Each custom parser is a driver. The output of each driver should be some standard data format you've decided on (json is your best bet). Your app layer should live over this, so it can just ingest the various sources of data without having to worry about formats. We went through something like this on an internal project at my office last year, in Java. Its not fun. I'd honestly recommend going with Python. If you're a java guy or girl, it'll be easy to move over to Python. The language is much more expressive, so you will be able to work easier, faster, and with better results. Another benefit of using Python for something like this is all of the useful modules that are available. Beautifulsoup for html parsing is particularly useful.