4 ms·
My first programming, instead of opening excel and making a plot I did (after setting up jupyter lab with anaconda): import pandas as pd data = pd
by teekert 5y ago
My first programming, instead of opening excel and making a plot I did (after setting up jupyter lab with anaconda):
import pandas as pd
data = pd.read_excel('my_excel_file.xlsx')
data.boxplot()
Amazing! And then someone showed me a loop [1] and I was plotting the columns I needed from hundreds of Excel files. Crazy efficient! Programming clicked right away for me. It was exactly what I needed at that time. It helps tremendously that in Jupyter notebooks you can just type a variable (in this case `data`) and it shows you what it looks like, and it looks like... An Excel file (indeed, a table).
You know what is hard? Opening 20, roughly the same, Excel files and copying some of the data into a new Excel file without making any mistakes (and that includes Excel not reformating your data to something else, like a dd-mm-yyyy to mm-dd-yyyy or making a date out of your Gene names [0]).
[0] https://www.theverge.com/2020/8/6/21355674/human-genes-rename-microsoft-excel-misreading-dates https://www.theverge.com/2020/8/6/21355674/human-genes-renam...
[1] The loop:
excel_files = os.listdir('.')
for excel_file in excel_files:
data = pd.read_excel('excel_file.xlsx')
data[['Column A', 'Column D', 'Column X']].boxplot()
- TeMPOraL 5y ago> (after setting up jupyter lab with anaconda) That right here is a very hard step for anyone who isn't an experienced software dev with some Python familiarity.
- teekert 5y agoAnaconda gives you a UI where you can click Jupyter-lab and it will just start. That said, I'm not claiming the learning curve is not there. It takes some investing. But learning to work with Excel's PMT function to understand your mortgage and making some plots also presents a learning curve.
- thablackbull 5y agoThe problem is that non-tech people have the exact opposite problem of what you experienced. > copying some of the data into a new Excel file without making any mistakes (and that includes Excel not reformating your data to something else, like a dd-mm-yyyy to mm-dd-yyyy or making a date out of your Gene names In your typical office environment, your regular analyst will run into issues where the date formatting is different between files, and there are one-off errors or issues here and there. In Excel, they are able to scan through and adjust as necessary. If you're asking them to figure out how to write the code to re-map and fix random issues in the data, that will be much more complicated to learn.
- teekert 5y agoYeah, this is probably true for many people. Still, I always offered people that click around in Excel all day a session with me to see what aspects we could automate. There is always a lot, but it never really sticks. Indeed, it is that dealing with edge case stuff. I admit I did a lot of data.to_excel() -> fix some stuff and then -> pd.read_excel() again back in the day...
- pjmlp 5y agoEverything is hard when one doesn't read the programming language manuals, or in this case, VBA, Lambda and PowerQuery.