3 ms·
Find your bottleneck and see what function calls are the slowest/fastest and find out the root cause of any perceived lack of performance. Just run python -m cP
by aburan28 11y ago
Find your bottleneck and see what function calls are the slowest/fastest and find out the root cause of any perceived lack of performance. Just run python -m cProfile <yourscript.py> and you'll get that information
- nekopa 11y agoThanks for this, I am very new to python, so I will try this out. But I think at the moment my bottleneck is the scraping part. The current program hits 365 pages (each day for a year) and managed to pull down the 20 million results. I am now trying to figure out how to pull more info from each day, and that's why I am thinking about trying to optimize. Does cProfile measure things like time on a webpage (via bs4)? Do you have any links on optimization "thinking", as in, I don't know if it is my approach that needs optimizing, vs my code.