As a DBA, I'm duty-bound to say this, though I'm sure it will be drowned out by calls to 'Denormalize All the Things!'. DON'T denormalize for performance. It is

As a DBA, I'm duty-bound to say this, though I'm sure it will be drowned out by calls to 'Denormalize All the Things!'. DON'T denormalize for performance. It is a siren's call. Don't take my word for it, see anything written by Fabian Pascal, Hugh Darwen, Chris Date. If I was faced with a dataset of 50GB in a text file, being loaded into a database every day, I would normalize as much as possible to ensure that I'm not adding 50GB to a database everyday. What's the plan, to have a 20TB database in one year? No, instead I would focus on micro-optimizations at the database level. I would inspect the datatypes you're storing your data in. If it's a value with a limited domain (an enum in programming parlance), make sure you determine the theoretical maximum amount of values it could be, and use the equivalent numeric value to serve as a surrogate key in a separate table. If I know a value is never going to exceed a few thousand distinct values, I could use a smallint, which only uses 2 bytes, verses recording a char/varchar which is going to probably burn at least 4-5 bytes every row, likely much more. I believe ordering of columns matters in Postgres, so consult the documentation/Stackoverflow where swapping around column ordering can sometimes save several bytes per row. You'll notice quickly that you may have repeating 'groups' of similar values across your surrogate keys. For instance, if you had a StateID and a CountryID in the same table, chances are good the combinations of StateID and CountryID are finite, and could themselves go into their own table, which has its own surrogate key, so instead of storing two surrogates on every row, you could use only one. (Incidentally, chances are even better that CountryID should just be a reference on the State table itself, but this is just to illustrate the point). Now imagine you have sets of attributes that describe whole subsets of the information in your tables that are mutually exclusive. You could make two different tables with identical data structures. Then placing a row in one 'means' those other implicit value combinations. Thus you record information without actually storing it. The price here is on retrieval, you have to union the tables if you want to consider the whole set. That said, you may shave a large amount of storage. Views could be created to make this transparent to end users. Don't use an auto-incrementing index on the table, instead store a value that is iterated separately, and 'take' ranges of values at a time that get cached by routines that mean to load the data. Through this style of process, one can often turn a 1TB table into something a fraction of that size, at the cost of much harder to write queries. That said, there are other benefits, suddenly you can load values into the supporting tables first, then cache those values and load the main dataset without doing lookups into the additional related tables. The 'growth' of a database will slow considerably. Then you just preprocess the files, and 'COPY' them in, which should allow you load them on the order of minutes, not hours.