6 ms·

Lab Notebooks (2020)

148 points by momonga 3y ago

OldGuyInTheClub 3y ago
Lab notebooks have saved me many times. The days run together in experimental work and having a record of what I tried or saw and on what day has paid off weeks or months later when a collection of observations snapped into focus. Never been able to keep a table of contents, though.
qetuo13579 3y ago
We call them “Engineering Notebooks” and after 5 years at a company they tell an interesting story of the projects worked on. I’m often reviewing notes from the past couple of months and occasionally have to go back a year or two to dig up some information that I know is there somewhere.
eternityforest 3y ago
I'll probably just use an app, rather than real paper, but I'll definitely keep this in mind next time I'm doing something experimental and not just "Stuff copilot could probably do". Preserving a record in detail seems like a natural fit, we're used to doing the same for our source code, I'm surprised it's not more common to do the same for the source of the source code.
analog31 3y ago
The article mentions, but doesn't elaborate, on the purpose of the notebook as a legal document. This was much more important when notebooks could be used to show priority in patent disputes, but only if well kept. My parents were both research scientists. Where my dad worked, the company patent lawyer read and critiqued your notebooks, and maintained the discipline. Many of those rules became unnecessary when the US patent system joined the rest of the world by adopting "first to file" priority. Rules such as: Always writing in pen, and crossing things with a single strike-through. Notebooks are still a great way to preserve the little details of "what the hell was I thinking?" A colleague told me a good notebook prevents you from having to repeat a study because you can't figure out what you did the first time. There are still industries that require legally defensible record management -- you'll know if you're in that boat. If not, you can DIY whatever method works for you. The success metric is if you actually use it. On paper, I write in pencil because I'm more concerned about readability than a precise narrative. On the computer, Jupyter, just because nothing else really lets me document "thinking in code."
- doctorpangloss 3y ago
  To your specific point, the legal issues are so idiosyncratic. You can’t write an article about them. You can’t tease out the difference between “this is a best practice everyone should follow” and “the patent lawyer is selling the holistic social experience of compliance,” which is the non-trial attorney’s primary product. It’s the same energy as DocuSign: when your biggest competitor is a pen, you know all the bullshit they peddle about timestamps and tracking and custody is just that, bullshit. It really comes down to, what is your role? Are you making discoveries or working for people who do? Both are very worthy. In my experience, the PIs - and their subordinates who become PIs - they aren’t thinking very hard about lab notebooks and their tactics vary widely. No best practices. However they are focused on maximizing the amount of serendipity they can have per unit time, which tends to devalue how you’re documenting stuff for patent lawyers. Does Benchling have a manifesto about why you should care about lab notebooks? No. It has a ton of little reactive opinions that are relevant to its sales pipeline. But nothing like “Agile, except for life sciences” which incorporates their software. It’s everything to all people, like most life science software. I know people receive huge grants on promising discoveries in life sciences, and some of their labs have Benchling licenses and don’t use them. The people first authoring the papers use lab notebooks of course, but see this as a small part of their process.
  - analog31 3y ago
    I just looked up Benchling. My first impression is that it's enterprise software. You're not expected to just buy a license and start using it, and being the first user at your site won't gain you much over what you're using right now. Where I think software like that gains its use is when you have multiple people working on similar research or processes, and trying to manage that kind of research and the data that it generates. At the extreme, I see job openings like this: Wanted, quality control chemist for third shift... On the other hand, I work in a "small lab" setting. I'm often the only person working on a project. No two of my projects resemble one another, though there are themes that progress through my work, and tools that I re-use and share. My choice of a notebook system, if you can grace it with such a description, is almost solely for my own benefit.
- 3y ago
fireynis 3y ago
I have started using Obsidian for daily notes (which you can categorize better than the default so it's more findable) and you can tag / create back links. I am thinking that these tips plus the tools Obsidian provides would make an excellent combo. Then the files can also be placed in git and backed up for protection!
nolamark 3y ago
Habit of lab notebook has stuck with me since college. Shoutout to reMarkable tablets and Neo Smartpens, for helping combine the analog process of handwriting with digital search. Likely a personal thing, just like to take hands off keyboard from time to time. Seen hints of research re benefits of handwriting, but never really dug into it.
- maCDzP 3y ago
  Haven’t heard of Neo before but it looks amazing. Could you explain your process and likes/dislikes with the Neo/reMarkable?
  - nolamark 3y ago
    Here is a first draft. I'll expand if I have more time and energy. Rule #1. Every thing gets a date (And more an aspiration than reality a time.) I am truely an oddball and have my own date format 2023_12Dec_26Tue_1312. Rational, when used a a corresponding file name, it sorts naturally assuming you used 0 prefix on one digit numbers. Day of week, just another memory clue to try to recall specific thought experiment. Rule #2. It is a process, not end product. Like the article mentions, no erasure. Point is re remember mistakes, not hide them. Rule #3. Take you time to write neatly. It is for your future you, and you will thank yourself. May feel like a debt due to slowness, but really an investment for your future self. Pros. reMarkable likes: feels like nice inkflow. Can get a fine line not quite comperable to a Pilot G-Tec-C 0.25 pen, but fine enought. I write small to increase information density per page. NeoSmart pen: doesn't just feel like ink flow, it is inflow Cons. NeoSmart pen: notebooks are expensive. Cheapskate in me, writes less than I should. Remarkable tablet: Miss the real estate of two pages open in a labnote book. Also, miss the leafthrough feel of physical notebook. Can't as easily binary seach to find something like one can in paper notebook Just saw Smart Plate at neosmartpen site, resisting urge for impulse purchase Edit - update to expand Additional cons or NeoSmartpen: Very limited number of pen widths. Note: When I worked for a living, it was much more likely to be greenfield project, not a repetition of a existing workflow. Even on repetitive projects, debugging is a good match for a notebook. Now my projects are for learning, never finished product. As always, your mileage may vary. Try to formalize your process and have repeatable actions done in repeatable ways. I have a collection of tokens. Before starting I write down a #GOAL, useful for climbing out of rabbit holes. Of course, the all important #ASSUMPTIONS. If the goal is not likely achievable in one session, write down a #PLAN. If need motivation to pursue a path, I write #BABYSTEPS for an action to start with. If I have a thought that would distract from present pursuit, I write a #STASH. If I do have a distraction I pursue, I note #DETOUR. End of day gets a few #LESSONSLEARNED and a #WRAPUP. I have a target audience of one, me. Thru trial and error I know what I'll want to know in the future, usually from stuff I regreted not having written down. From time to time, recall, revise, and document "process" currently in use in notebooks. Date_Time are links to other notes in notebook, Date_Time_Notebook are references to other notebooks. Date_Time and Date_Time_Notebook are also typed as comments in associated computer files, be they source code or what serves as documentation. To re-emphasize importance of Rule#1: Truth value of things changes with time. Time, usually, but not always, serves as a shortcut description of context, or at least provides enough hints to re-build the context
karencarits 3y ago
I wonder if we soon will get wearable lab cameras/microphones that can log what you do while keeping your hands free. More than once, I have been working with both hands and suddenly been distracted for a brief moment just to discover that I no longer remember if I really added stuff to that well. Also, having more exact timestamps at each stage of an experiment might be useful for later
- BasilPH 3y ago
  A former colleague of mine left to work at a company that builds a voice-controlled lab assistant[^0]. The idea is that you can record data and ideas while your hands are busy dealing with samples or a microscope. I don't know how well it works, but I've always found it to be an interesting niche. Pharmaceutical companies have deep pockets and security, compliance and specific requirements let you build a moat from more generic voice assistants. [^0]: https://www.labtwin.com/ https://www.labtwin.com/
atrettel 3y ago
One advantage of lab notebooks that the article does not mention is that by design, the pages are firmly bound to the book, so you cannot lose them. I started to use lab notebooks because I would otherwise just write thoughts on loose-leaf paper and have a difficult time keeping track of the pages and their order. The lab notebook (or any notebook with firmly bound pages) solves that problem. That said, I don't use lab notebooks for coding/software development. I tend to use Markdown files in a Git repository for that. That setup serves the same purpose, since I write down what I intend to do and whether I was able to achieve it or not, but it is much easier to use when coding in my experience.
- stanford_labrat 3y ago
  As a grad student I have an “official” lab notebook which sits on my lab bench and has all my technical details etc. I also have what I affectionately call my “real” lab notebook which is a stack of loose printer paper at my office desk, full of scribbles, designs, math, random planning and thoughts. I plan to get it bound and call it my “real” thesis.
robertlacok 3y ago
A huge downside of writing this down on paper is team collaboration. Imagine that instead this is kept in a tool like Notion which the entire team can access. At first this feels incredibly uncomfortable - your notes are there for everyone to see. However it’s a massive force multiplier because often the work is picked up by someone else later on, and instead of asking you they can self-serve. It’s pretty common that by the time they need it you would no longer be at the company. I don’t have experience writing in a lab journal format, but for documents like growth experiments and how they worked, or RFCs, this is a godsend. It takes a lot of work to keep it tidy, but it’s worth it.
- oslem 3y ago
  I agree that it’s useful for the rest of the team to have access to your notes, but I find that it’s far more useful to write my notes with a pen and paper rather than typing them out. It forces me to slow down and think through what I’m doing, in addition to aiding in memorizing the parts of my notes that I find important.
  - nolamark 3y ago
    Discussion prompted me to find NeoSmartpen accessory, seems like it could help with the sharing. https://shop.neosmartpen.com/collections/accessories/products/smart-plate https://shop.neosmartpen.com/collections/accessories/product... an impulse purchase I am fighting not make, not actual experience with the tablet. Do use Neo Smartpens
  - cratermoon 3y ago
    The important point is that the raw notes are for the note-taker. They not only document what you thought, they are themselves a part of and tool for the thinking process. Writing is thinking[1], and the initial results aren't going to be very useful for someone else, because they'll be rambling and messy, with lots of false starts, errors, and just wrong ideas. They're for the author to remember the path to the outcome[2], they are not the outcome itself. For the rest of the team, those raw notes are source material. That can be the input into lightweight documents like technical memos and decision records. 1. https://www.insidehighered.com/blogs/higher-ed-gamma/writing-thinking https://www.insidehighered.com/blogs/higher-ed-gamma/writing... 2. https://posts.oztamir.com/the-opposite-of-forgetting-is-writing/ https://posts.oztamir.com/the-opposite-of-forgetting-is-writ...
distortionfield 3y ago
I recommend the Leuchtturm 1917 Notebook for this kind of stuff. It comes with spine labels and a table of contents pages in the notebook already, and the paper is extremely high quality. They're the perfect daily-use notebook, IMO.
- wduquette 3y ago
  Agreed.
- hodgesrm 3y ago
  > I recommend the Leuchtturm 1917 Notebook for this kind of stuff. Just bought my first one last week. My previous notebook is full and I'm eager to try it out tomorrow!
bayindirh 3y ago
I do this for my personal software projects, research or otherwise. I have more stricter rules about paper and pen I use though. A single fountain pen, filled with the same permanent, pigment or cellulose reactive ink, on a hardcover, ink-safe notebook. However, the rest “at the speed of thought” rules are almost the same. Theorize, design and detail on paper, implement, document results, GOTO 10. It’s a greet speed regulator, clarifier and keeps everything documented. As a bonus, you get “That weird guy” merit badge, which I don’t care.
shcheklein 3y ago
It reminds me an approach to data science and ML. It's way more common and essential for ML teams to log their attempts (experiments). There is a whole set of tools for this - experiments trackers. Primarily to even being able to compare and pick the best direction, but also to ensure reproducibility (in some areas it might be required). ML/DS always seemed to me closer to science in its nature vs software engineering. Can be because of its nature as well - in a lot of cases it's a process of incremental improvements (vs - simplifying this a lot - let's say in SE we do a button that just works or not).
- rramadass 3y ago
  > ML/DS always seemed to me closer to science in its nature vs software engineering. Can be because of its nature as well - in a lot of cases it's a process of incremental improvements Right, i have felt this way too (even though i am just a noob at ML/DS). For me it is the use of Statistics/Probability/Mathematics in driving understanding/intuition about the problem.
cratermoon 3y ago
I know of two places to buy genuine engineering lab notebooks, for anyone interested. 1. Scientific Notebook Company <https://snco.com/ https://snco.com/> 2. BookFactory <https://www.bookfactory.com/ https://www.bookfactory.com/> Of the two, I slightly prefer BookFactory's offerings. Note that there are many, many variants. Unless you are employed somewhere that mandates a particular type – and if you do, they are likely available from your office supply closet – pick one you like. Do go with the bound hardcover style, though. They're much more durable than spiral or softbound.
jcpst 3y ago
I always have one of more pages of loose-leaf on my desk. It’s messy, but basically the lowest barrier to entry. And if I’m somewhere else and write something down, it goes in the pile with the other notes when I get back. A few times a year, I open an org-mode file and start summarizing my notes. Those sheets then get put away, to help keep things a little less messy.
charred_patina 3y ago
I started working at a lab where I am the only SWE and everyone else is a scientist. I have a lab notebook, but I don't use it properly (thankfully nobody gets on my case about it). I usually just use it as a short-term working memory, but it helps me keep track of the progress I've made. It's nice to look back on something I was working on two months ago and see all my thoughts written down about which library to use or architecture to use. Looking back at my past struggles gives me a sense of accomplishment and progress. That said, I usually try to preserve the results of my "experimentation" in documentation with the software, so I can share it with others. It's a good sign when people never talk about X to me, and I end up asking them if they ever used X and they tell me "I just used your documentation". Best feeling ever.
devsda 3y ago
Does anybody have a reference/example of a lab notebook for software engg ? If I'm forced to log my thoughts/actions it comes out as either too terse(everthing feels obvious) or it's too verbose and it's often difficult to find the right balance.
- rramadass 3y ago
  Don't over think it; just trust your intuition and do what seems natural. I also have the same problem but find that when it is too terse it acts like a "bookmark" into my "Memory Library" where i can recall the details. The process of "writing it down" is an aid to understanding/remembering/sorting/connecting ideas/concepts in one's head and it doesn't matter what system you follow.
iancmceachern 3y ago
I get these, modified to be regular notebooks, from https://snco.com/ https://snco.com/ You can get them custom printed with whatever paper you like, custom names, titles and numbering on the books, for no more cost than a regular notebook.
Fomite 3y ago
A good reminder to get back to keeping lab notebooks - they are extremely useful for being able to look back in time and go "What was I thinking and where did that number come from?"
photochemsyn 3y ago
One issue is whether to organize lab notebooks by project, or by time. In the latter case, this means the notebook is a record of your day-to-day activity, which has certain advantages (like having a single record to refer to), and disadvantages (reconstructing a project's timeline can be time-consuming if you're working concurrently on different projects). A decent compromise is to have a daily notebook, but never put work on two projects on the same page. Then if you really want to reconstruct a project's history from the daily journal, you can at least just copy pages. The primary reason for having a lab bench notebook is that, if properly used, you can always reconstruct the process you were documenting. A lot of bench research involves protocol development, for example. In comparison with software engineering, a protocol is like an algorithm, only you're applying it to physical samples. In a well-run lab, once someone has developed a protocol, they 'publish' it (stick it in the lab's manual of protocols, where other people can have access to it). So, you don't share your lab notebooks with others, you share the completed protocol, and then if there's some question about if the protocol can be improved, you can go back to the lab notebooks to see what was already tried (this seems similar to software library or device driver development). Some people don't like lab notebooks much, because properly recording what you did in enough detail that you can go back and repeat it eats up time and requires a certain discipline, and it often takes some unfortunate catastrophe (comparable to losing all your data backups) to learn the value of the practice. Incidentally, if a research lab, public or private, has no guidelines at all on lab notebook practices, it's an indication they may be engaging in shady behavior (e.g. Theranos) - it's comparable to a crypto exchange outfit like FTX not having an accounting division, and should be a red flag for investors. This is also a reason for the ink-only rule, with pencil you can go back and fudge data more easily.
carrozo 3y ago
Highly recommend Muji’s gridded notebooks (€1.95 for A5-size, 30 sheets): https://www.muji.eu/product/recycling-paper-notebook-dark-grey-a5-7160.muji https://www.muji.eu/product/recycling-paper-notebook-dark-gr...
crispyambulance 3y ago
I have seen meticulous, beautiful lab notebooks before. Paper ones. With numbered oversize greenish pages, faintly quadrille ruled, filled with exceptional INK penmanship, paragraph after paragraph of cogent technical observations, theoretical musings, responses to papers, thought-experiments and real experiments, taped graphs and images. Everything dated and signed. I've seen stacks of them spanning YEARS of work. I tried. Mine look like shit. If it's in ink, it's peppered ALL OVER with correction after correction after correction. Perhaps it's a matter of practice, but then I've not been at one place long enough where projects weren't constantly re-shuffled, killed, or lasted long enough to get a grip (sometimes the job and the company itself doesn't even last long enough). Perhaps these are becoming a relic of a lost era? Paper is too much of an "air-gap" when everything is done on computer. Moreover, the standard 2-week Agile "sprint" leaves no time for careful thinking and writing. One can't put epiphanies gained from writing up notes into "the deliverable" for a freaking Jira Ticket.
- PNewling 3y ago
  >Perhaps these are becoming a relic of a lost era? Paper is too much of an "air-gap" when everything is done on computer. Moreover, the standard 2-week Agile "sprint" leaves no time for careful thinking and writing. One can't put epiphanies gained from writing up notes into "the deliverable" for a freaking Jira Ticket So, I get what you're getting at here (I think), especially as someone who came from research labs and then moved into computer science, but teams should be building doco into the tickets and pushing back on anything that doesn't have some meaningful documentation built in. Easier said than done in some organizations, I know, but I still think we need to be the ones to push back.
  - crispyambulance 3y ago
    Yeah, it depends on the team. FWIW, some do structure a "data review" state prior to closure of a ticket where the deliverable is some type of report (often with a presentation). Obviously, this can't be done for every ticket but it's appropriate for periodic "capstone" tickets.
bobek 3y ago
I use lab notebook inspired process to share information across our engineering organization. It is like your daily standup, but written throughout the day. https://bobek.cz/blog/2021/written-standup/ https://bobek.cz/blog/2021/written-standup/
bobek 3y ago
I use lab notebook inspired process to share information across our engineering organization. It is like your daily standup, but written throughout the day. https://bobek.cz/blog/2021/written-standup/ https://bobek.cz/blog/2021/written-standup/
jstanley 3y ago
I've started writing a markdown file every day, with a separate heading for each project, and a tool that automatically creates an index page that says what topics (based on headings) were present each day. Apart from the format of the headings, it is deliberately informal and unstructured. I've found it very useful.
- plg 3y ago
  It would be great for you to share the code
  - jstanley 3y ago
    It's in https://github.com/jes/notes/tree/master/mkjesnotes https://github.com/jes/notes/tree/master/mkjesnotes but it's not particularly user-friendly for anyone but me. Nor is it supposed to be, it is very much a "home-cooked app".
ac50hz 3y ago
100% yes. And a new page per day (paper is cheap), with the date always on top outer corner of the page. Never share a page across dates. Thoughts, however bizarre, around the topics are very useful. Elaborate where possible. Names, keywords and classifications will be useful later, so do consider the future as you write.
physicles 3y ago
The main problem with paper is that it's not searchable, so I've been using [0] since 2018. Each invocation just appends a timestamped line of text. I have a crontab that pops up a box every hour and I type in what I'm working on. Sometimes I'll add extra notes if I'm working through a particularly thorny problem. It's always right there on the terminal. I'd say it's saved my ass at least a dozen times over the years, where I hit a problem that I've seen before but can't quite remember how to solve it. I also page through it quarterly to get a good summary of what I've worked on for that quarterly 1:1 with the boss. [0]: https://gist.github.com/mfischr/f1ee2967ec0181b934639c30f4e68f17 https://gist.github.com/mfischr/f1ee2967ec0181b934639c30f4e6...
- ac50hz 3y ago
  Whilst i agree with all of this, I like paper, and to set time aside (with coffee) to review batches of notes.
- hodgesrm 3y ago
  You need highlighting pens! I have a couple of them to circle anything significant--a bright circle means "this is probably interesting!" It makes searching vastly easier. The other thing is to date every page accurately. Notes are a time series.
rramadass 3y ago
Tangential: I find the website design to be particularly nice !

148 points by momonga 3y ago

Lab notebooks have saved me many times. The days run together in experimental work and having a record of what I tried or saw and on what day has paid off weeks or months later when a collection of observations snapped into focus. Never been able to keep a table of contents, though.

qetuo13579 3y ago

We call them “Engineering Notebooks” and after 5 years at a company they tell an interesting story of the projects worked on. I’m often reviewing notes from the past couple of months and occasionally have to go back a year or two to dig up some information that I know is there somewhere.

eternityforest 3y ago

I'll probably just use an app, rather than real paper, but I'll definitely keep this in mind next time I'm doing something experimental and not just "Stuff copilot could probably do". Preserving a record in detail seems like a natural fit, we're used to doing the same for our source code, I'm surprised it's not more common to do the same for the source of the source code.

analog31 3y ago

The article mentions, but doesn't elaborate, on the purpose of the notebook as a legal document. This was much more important when notebooks could be used to show priority in patent disputes, but only if well kept. My parents were both research scientists. Where my dad worked, the company patent lawyer read and critiqued your notebooks, and maintained the discipline. Many of those rules became unnecessary when the US patent system joined the rest of the world by adopting "first to file" priority. Rules such as: Always writing in pen, and crossing things with a single strike-through. Notebooks are still a great way to preserve the little details of "what the hell was I thinking?" A colleague told me a good notebook prevents you from having to repeat a study because you can't figure out what you did the first time. There are still industries that require legally defensible record management -- you'll know if you're in that boat. If not, you can DIY whatever method works for you. The success metric is if you actually use it. On paper, I write in pencil because I'm more concerned about readability than a precise narrative. On the computer, Jupyter, just because nothing else really lets me document "thinking in code."

doctorpangloss 3y ago
To your specific point, the legal issues are so idiosyncratic. You can’t write an article about them. You can’t tease out the difference between “this is a best practice everyone should follow” and “the patent lawyer is selling the holistic social experience of compliance,” which is the non-trial attorney’s primary product. It’s the same energy as DocuSign: when your biggest competitor is a pen, you know all the bullshit they peddle about timestamps and tracking and custody is just that, bullshit. It really comes down to, what is your role? Are you making discoveries or working for people who do? Both are very worthy. In my experience, the PIs - and their subordinates who become PIs - they aren’t thinking very hard about lab notebooks and their tactics vary widely. No best practices. However they are focused on maximizing the amount of serendipity they can have per unit time, which tends to devalue how you’re documenting stuff for patent lawyers. Does Benchling have a manifesto about why you should care about lab notebooks? No. It has a ton of little reactive opinions that are relevant to its sales pipeline. But nothing like “Agile, except for life sciences” which incorporates their software. It’s everything to all people, like most life science software. I know people receive huge grants on promising discoveries in life sciences, and some of their labs have Benchling licenses and don’t use them. The people first authoring the papers use lab notebooks of course, but see this as a small part of their process.
- analog31 3y ago
  I just looked up Benchling. My first impression is that it's enterprise software. You're not expected to just buy a license and start using it, and being the first user at your site won't gain you much over what you're using right now. Where I think software like that gains its use is when you have multiple people working on similar research or processes, and trying to manage that kind of research and the data that it generates. At the extreme, I see job openings like this: Wanted, quality control chemist for third shift... On the other hand, I work in a "small lab" setting. I'm often the only person working on a project. No two of my projects resemble one another, though there are themes that progress through my work, and tools that I re-use and share. My choice of a notebook system, if you can grace it with such a description, is almost solely for my own benefit.
3y ago

fireynis 3y ago

I have started using Obsidian for daily notes (which you can categorize better than the default so it's more findable) and you can tag / create back links. I am thinking that these tips plus the tools Obsidian provides would make an excellent combo. Then the files can also be placed in git and backed up for protection!

nolamark 3y ago

Habit of lab notebook has stuck with me since college. Shoutout to reMarkable tablets and Neo Smartpens, for helping combine the analog process of handwriting with digital search. Likely a personal thing, just like to take hands off keyboard from time to time. Seen hints of research re benefits of handwriting, but never really dug into it.

maCDzP 3y ago
Haven’t heard of Neo before but it looks amazing. Could you explain your process and likes/dislikes with the Neo/reMarkable?
- nolamark 3y ago
  Here is a first draft. I'll expand if I have more time and energy. Rule #1. Every thing gets a date (And more an aspiration than reality a time.) I am truely an oddball and have my own date format 2023_12Dec_26Tue_1312. Rational, when used a a corresponding file name, it sorts naturally assuming you used 0 prefix on one digit numbers. Day of week, just another memory clue to try to recall specific thought experiment. Rule #2. It is a process, not end product. Like the article mentions, no erasure. Point is re remember mistakes, not hide them. Rule #3. Take you time to write neatly. It is for your future you, and you will thank yourself. May feel like a debt due to slowness, but really an investment for your future self. Pros. reMarkable likes: feels like nice inkflow. Can get a fine line not quite comperable to a Pilot G-Tec-C 0.25 pen, but fine enought. I write small to increase information density per page. NeoSmart pen: doesn't just feel like ink flow, it is inflow Cons. NeoSmart pen: notebooks are expensive. Cheapskate in me, writes less than I should. Remarkable tablet: Miss the real estate of two pages open in a labnote book. Also, miss the leafthrough feel of physical notebook. Can't as easily binary seach to find something like one can in paper notebook Just saw Smart Plate at neosmartpen site, resisting urge for impulse purchase Edit - update to expand Additional cons or NeoSmartpen: Very limited number of pen widths. Note: When I worked for a living, it was much more likely to be greenfield project, not a repetition of a existing workflow. Even on repetitive projects, debugging is a good match for a notebook. Now my projects are for learning, never finished product. As always, your mileage may vary. Try to formalize your process and have repeatable actions done in repeatable ways. I have a collection of tokens. Before starting I write down a #GOAL, useful for climbing out of rabbit holes. Of course, the all important #ASSUMPTIONS. If the goal is not likely achievable in one session, write down a #PLAN. If need motivation to pursue a path, I write #BABYSTEPS for an action to start with. If I have a thought that would distract from present pursuit, I write a #STASH. If I do have a distraction I pursue, I note #DETOUR. End of day gets a few #LESSONSLEARNED and a #WRAPUP. I have a target audience of one, me. Thru trial and error I know what I'll want to know in the future, usually from stuff I regreted not having written down. From time to time, recall, revise, and document "process" currently in use in notebooks. Date_Time are links to other notes in notebook, Date_Time_Notebook are references to other notebooks. Date_Time and Date_Time_Notebook are also typed as comments in associated computer files, be they source code or what serves as documentation. To re-emphasize importance of Rule#1: Truth value of things changes with time. Time, usually, but not always, serves as a shortcut description of context, or at least provides enough hints to re-build the context

karencarits 3y ago

I wonder if we soon will get wearable lab cameras/microphones that can log what you do while keeping your hands free. More than once, I have been working with both hands and suddenly been distracted for a brief moment just to discover that I no longer remember if I really added stuff to that well. Also, having more exact timestamps at each stage of an experiment might be useful for later

BasilPH 3y ago
A former colleague of mine left to work at a company that builds a voice-controlled lab assistant[^0]. The idea is that you can record data and ideas while your hands are busy dealing with samples or a microscope. I don't know how well it works, but I've always found it to be an interesting niche. Pharmaceutical companies have deep pockets and security, compliance and specific requirements let you build a moat from more generic voice assistants. [^0]: https://www.labtwin.com/ https://www.labtwin.com/

atrettel 3y ago

One advantage of lab notebooks that the article does not mention is that by design, the pages are firmly bound to the book, so you cannot lose them. I started to use lab notebooks because I would otherwise just write thoughts on loose-leaf paper and have a difficult time keeping track of the pages and their order. The lab notebook (or any notebook with firmly bound pages) solves that problem. That said, I don't use lab notebooks for coding/software development. I tend to use Markdown files in a Git repository for that. That setup serves the same purpose, since I write down what I intend to do and whether I was able to achieve it or not, but it is much easier to use when coding in my experience.

stanford_labrat 3y ago
As a grad student I have an “official” lab notebook which sits on my lab bench and has all my technical details etc. I also have what I affectionately call my “real” lab notebook which is a stack of loose printer paper at my office desk, full of scribbles, designs, math, random planning and thoughts. I plan to get it bound and call it my “real” thesis.

robertlacok 3y ago

A huge downside of writing this down on paper is team collaboration. Imagine that instead this is kept in a tool like Notion which the entire team can access. At first this feels incredibly uncomfortable - your notes are there for everyone to see. However it’s a massive force multiplier because often the work is picked up by someone else later on, and instead of asking you they can self-serve. It’s pretty common that by the time they need it you would no longer be at the company. I don’t have experience writing in a lab journal format, but for documents like growth experiments and how they worked, or RFCs, this is a godsend. It takes a lot of work to keep it tidy, but it’s worth it.

oslem 3y ago
I agree that it’s useful for the rest of the team to have access to your notes, but I find that it’s far more useful to write my notes with a pen and paper rather than typing them out. It forces me to slow down and think through what I’m doing, in addition to aiding in memorizing the parts of my notes that I find important.
- nolamark 3y ago
  Discussion prompted me to find NeoSmartpen accessory, seems like it could help with the sharing. https://shop.neosmartpen.com/collections/accessories/products/smart-plate https://shop.neosmartpen.com/collections/accessories/product... an impulse purchase I am fighting not make, not actual experience with the tablet. Do use Neo Smartpens
- cratermoon 3y ago
  The important point is that the raw notes are for the note-taker. They not only document what you thought, they are themselves a part of and tool for the thinking process. Writing is thinking[1], and the initial results aren't going to be very useful for someone else, because they'll be rambling and messy, with lots of false starts, errors, and just wrong ideas. They're for the author to remember the path to the outcome[2], they are not the outcome itself. For the rest of the team, those raw notes are source material. That can be the input into lightweight documents like technical memos and decision records. 1. https://www.insidehighered.com/blogs/higher-ed-gamma/writing-thinking https://www.insidehighered.com/blogs/higher-ed-gamma/writing... 2. https://posts.oztamir.com/the-opposite-of-forgetting-is-writing/ https://posts.oztamir.com/the-opposite-of-forgetting-is-writ...

distortionfield 3y ago

I recommend the Leuchtturm 1917 Notebook for this kind of stuff. It comes with spine labels and a table of contents pages in the notebook already, and the paper is extremely high quality. They're the perfect daily-use notebook, IMO.

wduquette 3y ago
Agreed.
hodgesrm 3y ago
> I recommend the Leuchtturm 1917 Notebook for this kind of stuff. Just bought my first one last week. My previous notebook is full and I'm eager to try it out tomorrow!

bayindirh 3y ago

I do this for my personal software projects, research or otherwise. I have more stricter rules about paper and pen I use though. A single fountain pen, filled with the same permanent, pigment or cellulose reactive ink, on a hardcover, ink-safe notebook. However, the rest “at the speed of thought” rules are almost the same. Theorize, design and detail on paper, implement, document results, GOTO 10. It’s a greet speed regulator, clarifier and keeps everything documented. As a bonus, you get “That weird guy” merit badge, which I don’t care.

shcheklein 3y ago

It reminds me an approach to data science and ML. It's way more common and essential for ML teams to log their attempts (experiments). There is a whole set of tools for this - experiments trackers. Primarily to even being able to compare and pick the best direction, but also to ensure reproducibility (in some areas it might be required). ML/DS always seemed to me closer to science in its nature vs software engineering. Can be because of its nature as well - in a lot of cases it's a process of incremental improvements (vs - simplifying this a lot - let's say in SE we do a button that just works or not).

rramadass 3y ago
> ML/DS always seemed to me closer to science in its nature vs software engineering. Can be because of its nature as well - in a lot of cases it's a process of incremental improvements Right, i have felt this way too (even though i am just a noob at ML/DS). For me it is the use of Statistics/Probability/Mathematics in driving understanding/intuition about the problem.

cratermoon 3y ago

I know of two places to buy genuine engineering lab notebooks, for anyone interested. 1. Scientific Notebook Company <https://snco.com/ https://snco.com/> 2. BookFactory <https://www.bookfactory.com/ https://www.bookfactory.com/> Of the two, I slightly prefer BookFactory's offerings. Note that there are many, many variants. Unless you are employed somewhere that mandates a particular type – and if you do, they are likely available from your office supply closet – pick one you like. Do go with the bound hardcover style, though. They're much more durable than spiral or softbound.

jcpst 3y ago

I always have one of more pages of loose-leaf on my desk. It’s messy, but basically the lowest barrier to entry. And if I’m somewhere else and write something down, it goes in the pile with the other notes when I get back. A few times a year, I open an org-mode file and start summarizing my notes. Those sheets then get put away, to help keep things a little less messy.

charred_patina 3y ago

I started working at a lab where I am the only SWE and everyone else is a scientist. I have a lab notebook, but I don't use it properly (thankfully nobody gets on my case about it). I usually just use it as a short-term working memory, but it helps me keep track of the progress I've made. It's nice to look back on something I was working on two months ago and see all my thoughts written down about which library to use or architecture to use. Looking back at my past struggles gives me a sense of accomplishment and progress. That said, I usually try to preserve the results of my "experimentation" in documentation with the software, so I can share it with others. It's a good sign when people never talk about X to me, and I end up asking them if they ever used X and they tell me "I just used your documentation". Best feeling ever.

devsda 3y ago

Does anybody have a reference/example of a lab notebook for software engg ? If I'm forced to log my thoughts/actions it comes out as either too terse(everthing feels obvious) or it's too verbose and it's often difficult to find the right balance.

rramadass 3y ago
Don't over think it; just trust your intuition and do what seems natural. I also have the same problem but find that when it is too terse it acts like a "bookmark" into my "Memory Library" where i can recall the details. The process of "writing it down" is an aid to understanding/remembering/sorting/connecting ideas/concepts in one's head and it doesn't matter what system you follow.

iancmceachern 3y ago

I get these, modified to be regular notebooks, from https://snco.com/ https://snco.com/ You can get them custom printed with whatever paper you like, custom names, titles and numbering on the books, for no more cost than a regular notebook.

Fomite 3y ago

A good reminder to get back to keeping lab notebooks - they are extremely useful for being able to look back in time and go "What was I thinking and where did that number come from?"

photochemsyn 3y ago

One issue is whether to organize lab notebooks by project, or by time. In the latter case, this means the notebook is a record of your day-to-day activity, which has certain advantages (like having a single record to refer to), and disadvantages (reconstructing a project's timeline can be time-consuming if you're working concurrently on different projects). A decent compromise is to have a daily notebook, but never put work on two projects on the same page. Then if you really want to reconstruct a project's history from the daily journal, you can at least just copy pages. The primary reason for having a lab bench notebook is that, if properly used, you can always reconstruct the process you were documenting. A lot of bench research involves protocol development, for example. In comparison with software engineering, a protocol is like an algorithm, only you're applying it to physical samples. In a well-run lab, once someone has developed a protocol, they 'publish' it (stick it in the lab's manual of protocols, where other people can have access to it). So, you don't share your lab notebooks with others, you share the completed protocol, and then if there's some question about if the protocol can be improved, you can go back to the lab notebooks to see what was already tried (this seems similar to software library or device driver development). Some people don't like lab notebooks much, because properly recording what you did in enough detail that you can go back and repeat it eats up time and requires a certain discipline, and it often takes some unfortunate catastrophe (comparable to losing all your data backups) to learn the value of the practice. Incidentally, if a research lab, public or private, has no guidelines at all on lab notebook practices, it's an indication they may be engaging in shady behavior (e.g. Theranos) - it's comparable to a crypto exchange outfit like FTX not having an accounting division, and should be a red flag for investors. This is also a reason for the ink-only rule, with pencil you can go back and fudge data more easily.

carrozo 3y ago

Highly recommend Muji’s gridded notebooks (€1.95 for A5-size, 30 sheets): https://www.muji.eu/product/recycling-paper-notebook-dark-grey-a5-7160.muji https://www.muji.eu/product/recycling-paper-notebook-dark-gr...

crispyambulance 3y ago

I have seen meticulous, beautiful lab notebooks before. Paper ones. With numbered oversize greenish pages, faintly quadrille ruled, filled with exceptional INK penmanship, paragraph after paragraph of cogent technical observations, theoretical musings, responses to papers, thought-experiments and real experiments, taped graphs and images. Everything dated and signed. I've seen stacks of them spanning YEARS of work. I tried. Mine look like shit. If it's in ink, it's peppered ALL OVER with correction after correction after correction. Perhaps it's a matter of practice, but then I've not been at one place long enough where projects weren't constantly re-shuffled, killed, or lasted long enough to get a grip (sometimes the job and the company itself doesn't even last long enough). Perhaps these are becoming a relic of a lost era? Paper is too much of an "air-gap" when everything is done on computer. Moreover, the standard 2-week Agile "sprint" leaves no time for careful thinking and writing. One can't put epiphanies gained from writing up notes into "the deliverable" for a freaking Jira Ticket.

PNewling 3y ago
>Perhaps these are becoming a relic of a lost era? Paper is too much of an "air-gap" when everything is done on computer. Moreover, the standard 2-week Agile "sprint" leaves no time for careful thinking and writing. One can't put epiphanies gained from writing up notes into "the deliverable" for a freaking Jira Ticket So, I get what you're getting at here (I think), especially as someone who came from research labs and then moved into computer science, but teams should be building doco into the tickets and pushing back on anything that doesn't have some meaningful documentation built in. Easier said than done in some organizations, I know, but I still think we need to be the ones to push back.
- crispyambulance 3y ago
  Yeah, it depends on the team. FWIW, some do structure a "data review" state prior to closure of a ticket where the deliverable is some type of report (often with a presentation). Obviously, this can't be done for every ticket but it's appropriate for periodic "capstone" tickets.

bobek 3y ago

I use lab notebook inspired process to share information across our engineering organization. It is like your daily standup, but written throughout the day. https://bobek.cz/blog/2021/written-standup/ https://bobek.cz/blog/2021/written-standup/

bobek 3y ago

jstanley 3y ago

I've started writing a markdown file every day, with a separate heading for each project, and a tool that automatically creates an index page that says what topics (based on headings) were present each day. Apart from the format of the headings, it is deliberately informal and unstructured. I've found it very useful.

plg 3y ago
It would be great for you to share the code
- jstanley 3y ago
  It's in https://github.com/jes/notes/tree/master/mkjesnotes https://github.com/jes/notes/tree/master/mkjesnotes but it's not particularly user-friendly for anyone but me. Nor is it supposed to be, it is very much a "home-cooked app".

ac50hz 3y ago

100% yes. And a new page per day (paper is cheap), with the date always on top outer corner of the page. Never share a page across dates. Thoughts, however bizarre, around the topics are very useful. Elaborate where possible. Names, keywords and classifications will be useful later, so do consider the future as you write.

physicles 3y ago

The main problem with paper is that it's not searchable, so I've been using [0] since 2018. Each invocation just appends a timestamped line of text. I have a crontab that pops up a box every hour and I type in what I'm working on. Sometimes I'll add extra notes if I'm working through a particularly thorny problem. It's always right there on the terminal. I'd say it's saved my ass at least a dozen times over the years, where I hit a problem that I've seen before but can't quite remember how to solve it. I also page through it quarterly to get a good summary of what I've worked on for that quarterly 1:1 with the boss. [0]: https://gist.github.com/mfischr/f1ee2967ec0181b934639c30f4e68f17 https://gist.github.com/mfischr/f1ee2967ec0181b934639c30f4e6...

ac50hz 3y ago
Whilst i agree with all of this, I like paper, and to set time aside (with coffee) to review batches of notes.
hodgesrm 3y ago
You need highlighting pens! I have a couple of them to circle anything significant--a bright circle means "this is probably interesting!" It makes searching vastly easier. The other thing is to date every page accurately. Notes are a time series.

rramadass 3y ago

Tangential: I find the website design to be particularly nice !