At the abstract level, get a standardized process for your analysis that you can use from project to project. Doesn't have to be complicated, but it needs to be

At the abstract level, get a standardized process for your analysis that you can use from project to project. Doesn't have to be complicated, but it needs to be consistent and applied to each project. Here's what I'd recommend - Take 5-10 minutes after getting a project and think through it. Break it into chunks and note what you need to discover (i.e. go find a data set) and what you already can/know how to do. Create an ordered task list for yourself (1. get data set, 2. setup project db, 3. create schema for db etc. etc. etc). This gives you measurable milestones you can track and helps you keep track of what's next. You can calendar these out if you need to as well. Keep a running notebook / text file for each project that's dated with a brief explanation of what you did that day, what troubles you ran into, what needs fixing, what you need to do next, and any other random thoughts you had related to the project. It unloads your RAM and keeps it somewhere you can get at easily. Write in this at the end of each day or when you switch projects. It should take 3-5 minutes to update this. It's also the first thing you open when you start working on a project again. Get source control setup and give each project it's own folder. Store notes and source code in there and keep it up to date. Remember process matters right now more than specific implementations. Can be SVN or GIT. Can further sub-organize if you need to. I.E. SQL folder for scripts/stored procedures, BASH for shell scripts, PYTHON for python scripts etc. Write a README for your project, that gets stored in the project folder, that explains what pieces of software get run and in what order. That way you won't forget order of execution. Be systematic in your approach. The best process is the one you stick to and actually use, not the one that's perfect on paper.