> the db is getting large and difficult to handle. Did your store the raw content in the database? If so, you might consider writing files instead. These blobs

> the db is getting large and difficult to handle. Did your store the raw content in the database? If so, you might consider writing files instead. These blobs, like pictures, are better stored as files. In your database, you should keep relational data: url, time, adder, etc, properly indexed (probably by adder, time and maybe keywords). Then a 6M rows table is quite a small thing for any RDBMS (if your SELECTs are filtering on indexed columns).