3 ms·
> Although pulling out a simple max is easy, the data shape is not ideal. It's in "wide" format - a Date column, and then an extra column for every currency. He
by nextaccountic 3y ago
> Although pulling out a simple max is easy, the data shape is not ideal. It's in "wide" format - a Date column, and then an extra column for every currency. Here's the csv header for that file:
> Date,USD,JPY,BGN,CYP,CZK,DKK,EEK,GBP,HUF,LTL,LVL,MTL,[and on, and on]
> When doing filters and aggregations, life is easier if the data is in "long" format, like this:
> Date,Currency,Rate
> Switching from wide to long is a simple operation, commonly called a "melt". Unfortunately, it's not available in SQL.
> No matter, you can melt with pandas:
> curl -s https://www.ecb.europa.eu/stats/eurofxref/eurofxref-hist.zip https://www.ecb.europa.eu/stats/eurofxref/eurofxref-hist.zip | \
gunzip | \
> python3 -c 'import sys, pandas as pd
> pd.read_csv(sys.stdin).melt("Date").to_csv(sys.stdout, index=False)'
You sure this can't be done in SQL? I myself don't know enough SQL, but I believe that you can decompose a single row into multiple rows in SQL by joining multiple queries using UNION ALL:
https://stackoverflow.com/questions/46217564/converting-single-row-into-multiple-rows-based-on-values-in-columns https://stackoverflow.com/questions/46217564/converting-sing...
https://stackoverflow.com/questions/52279384/how-do-i-melt-a-sql-table https://stackoverflow.com/questions/52279384/how-do-i-melt-a...
Now the question is, how to make this less verbose (so that the query doesn't need to mention each of the 42 columns)
- tbrownaw 3y agoThat sounds like PIVOT and UNPIVOT. But of course being SQL you have to list all the columns explicitly, which can get kind of annoying.