Since you are using Postgres: https://www.postgresql.org/docs/current/static/populate.html https://www.postgresql.org/docs/current/static/populate.html Get fam

Since you are using Postgres: https://www.postgresql.org/docs/current/static/populate.html https://www.postgresql.org/docs/current/static/populate.html Get familiar with the COPY command (https://www.postgresql.org/docs/current/static/sql-copy.html https://www.postgresql.org/docs/current/static/sql-copy.html) for importing your data as it will save you a ton of time compared to doing single or even bulk inserts from a Python script. You can use COPY to either import directly from a CSV file, or you can use it to load Python tuples in memory w/ the Psycopg2 client and copy_expert(). For example: buffer = cStringIO.StringIO() csv_writer = csv.writer(buffer) csv_writer.writerow((col1_data_field, col2_data_field, col3_data_field)) # [write more rows to a buffer, probably in a loop] buffer.seek(0) curr.copy_expert('COPY table_name (col1, col2, col3) FROM STDIN CSV', buffer) For things too big for memory, transform your data to a secondary CSV file, then feed a fd to copy_expert() csv_file = open(csv_path, 'r') curr.copy_expert("COPY %s from STDIN DELIMITER ',' CSV" % table_name, csv_file) Import your data into tables with no indexes, and then create the indexes after the import. In your postgresql.conf consider the following options to help w/ write throughput: synchronous_commit = off Once your initial data is loaded into PG, consider doing some transformations on your data with materialized views (https://www.postgresql.org/docs/10/static/sql-creatematerializedview.html https://www.postgresql.org/docs/10/static/sql-creatematerial...) as oppose to doing the transforms with a Python script. PG will be able to create the views much faster than you can import data into a new table.