6 ms·

> I tried something new for the first time, which was to keep a journal during development. DO THIS!!! I cannot stress this enough! If you work in a professi

> I tried something new for the first time, which was to keep a journal during development. DO THIS!!! I cannot stress this enough! If you work in a professional science lab, say, physics, biology, chemistry, you are expected to keep an experiment journal. It provides more help to you than the company too (knowledge dump, liability, etc). I can't tell you how many times some stupid ass seemingly benign comment saved my behind. They're worth their weight in gold. For ML experiments I use wandb and hydra[0]. Put all your configs into hydra. Be fucking pedantic. You should log your seeds, versions, the date, and I mean everything. It only takes a few extra minutes to set this up but the one time you need it it'll save you hours. Dump all that into wandb AND your model checkpoints. You will forget what that checkpoint corresponds to. Make liberal use of wandb tags and comments (through hydra you can make these cli arguments to automate even if launching from slurm scripts). Turn on wandb's code saving. Most importantly, use those notebooks wandb gives you. Don't worry if it gets messy. It's a experiment notebook, it'll get messy. You'll get better with experience and as you find your style. It sounds like a lot of work but it really isn't. You can get this all done under 20 minutes and if you write it right you can just copy paste it moving forward (i.e. yeah, make a personal library). I can PROMISE you that one mishap will far outweigh this extra work. You look like a pretentious perfectionist but really I'm a lazy piece of shit rust doesn't want to spend hours or days debugging some stupid mistake I'm too dumb or tired to catch. The extra benefit is when shit world you can spin up some (wandb) sweeps and go do some other thing that's always behind. (On topic, stop using personal wandb accounts for your work experiments. They're like the best company out there, get your boss to pay. They provide an amazing service and are a delight to work with. I cannot speak highly enough about them. They're not the company you want to mooch from. I've literally seen this happen while working for a top 3 market cap which was already paying for seats and you just needed to send a slack message to one dude... not cool guys... not cool...) [0] https://hydra.cc/docs/intro/ https://hydra.cc/docs/intro/

0cf8612b2e1e 1y ago
From my two minute skim of the docs, not encouraging that hydra only officially supports up to Python 3.11.
- godelski 1y ago
  I use it in python 3.12, and 3.12 just got out of bug fix. I haven't tried 3.13 but I would be surprised if there was a break. Most of it works through OmegaConf[0]. Idk why they haven't pushed an update in 2 years but neither has this been a problem. FWIW, they're still updating the repo[1] [0] https://omegaconf.readthedocs.io https://omegaconf.readthedocs.io [1] https://github.com/facebookresearch/hydra https://github.com/facebookresearch/hydra
nine_k 1y ago
The problem with keeping a journal is that the distraction of doing so may break the state of flow. OTOH there are natural breaks in the process of working; writing things down during these works fine. The fidelity is a bit lower, but it's still much better than nothing.
- isaacimagine 1y ago
  Agree, it's much better to write up a journal at times when your colleagues would be https://xkcd.com/303 https://xkcd.com/303
- godelski 1y ago
  > the distraction of doing so may break the state of flow. Sure, but like you said, don't do it when in the state of flow. Or better, make it part of your flow state. To me, it is part of my flow state, so not a real issue. I mean whatever works for you. You gotta time manage and I can't manage for you. I'm sure your boss is asking for more writeups than I am and just send them your notes. They don't care whats in it half the time, they just don't know how to figure out if you're working or not and just want something. Hell, we're on HN on a workday... I can guarantee you aren't in a flow state the whole time and can't be bothered with a few minutes to write some stuff down. I mean you have to eat and go to the bathroom, right?
isaacimagine 1y ago
Chaotic energy haha, I like it. Thanks for the tips re: keeping a journal, I will do this more in the future. I usually keep development notes, though normally in markdown files scattered across the codebase or in comments, never by date in the README. In the future, I might make JOURNAL.md a standard practice in my projects? re:w&b, I used w&b when it first came out and I liked it but I'm sure it's come a lot further in the time since then. I will have to take a look! Also lol "pretentious perfectionist" I'm glad to finally have some words to describe my design aesthetic. I like crisp fonts, what can I say.
- godelski 1y ago
  > Chaotic energy haha, I like it My boss says I'm eccentric. I say that's just a nice word for crazy lol > normally in markdown files scattered across the codebase or in comments I used to do that too but they didn't end up helping because I could never find them. So I moved back to using a physical book. The wandb reports was the first time I really had something where I felt like I got more out of it than a physical book. Even my iPad just results in a lot of lost stuff and more time trying to figure out why I can't just zoom in on the notes app. I mean what is an iPad even for if it isn't really good for writing? But the most important part of the process I talked about is the logging of all the parameters and options. Those are the details you tend to lose and go hunting for. So even if you never write a word you'll see huge benefits from this. > re:w&b Wandb's best feature is that you can email them requesting a feature and they'll implement it or help you implement it. It's literally their business model. I love it. I swear, they have a support agent assigned to me (thanks Art! And if wandb sees this, give the man a raise. Just look at what crazy people he has to deal with) > lol "pretentious perfectionist" I'm glad to finally have some words to describe my design aesthetic To be clear, I'm actually not. Too chaotic lol. Besides, perfectionism doesn't even exist. It's more a question about personal tastes and where we draw the line for what is good enough. I wish we'd stop saying "don't let perfectionism get in the way of good" because it assumes like there's universal agreement about what good enough is.
  - isaacimagine 1y ago
    Parameters and options, got it. I try to keep all configuration declarative and make building and running as deterministic as possible. Then I can commit whenever I do something interesting, that I can just checkout to revisit.
rvz 1y ago
All of this agreed. Now in the age of AI, many students entering into CS need do this NOW, otherwise any answer they come up in the interview, will be assumed that it was from an AI and they need to show that they something useful came out of their blogpost or research. It is what it now means to know how to experiment, understand and build knowledge, rather than spitting out the answer because it it from stack overflow or ChatGPT. The mistakes are raw, all the learnings in a blog post which is what makes us human yet, 90% of candidates do not do this which is why most of them cannot explain an AI's mistakes an interview if they use it.
- godelski 1y ago
  I actually really like this idea. I've often found it odd we don't show off reports or how we run experiments during interviews. Certainly this has far greater influence over your aptitude than leetcode. > 90% of candidates do not do this or cannot explain AI's mistakes an interview. I have a growing concern that people do not see mistakes. This seems to be a bigger divide than "uses AI to code" vs "doesn't".
mr_toad 1y ago
Same goes for something as simple as setting up a server. You will forget, and if you don’t write it down you’ll have to figure it out again.
- godelski 1y ago
  Where did I put those configs again? Where did Bob put that script? Fuck, why didn't I write an ansible script. It's never a one off, and it serves as documentation. I'll remember after I make the same mistake next time. Also, environment modules for the win https://modules.readthedocs.io/en/latest/ https://modules.readthedocs.io/en/latest/
- stirfish 1y ago
  One time my boss asked me to upgrade some servers and was surprised I put all my work in a script in version control. Then there was a second batch of servers.
- eptcyka 1y ago
  Use nix and you’ll at least will be able to deploy the same again.
  - genewitch 1y ago
    So what happens in nix if a package version is found to have some huge vulnerability? And my app expects that exact version? Because this is literally what docker, venv, and nix claim to fix but after getting burned by three other systems I'm not willing to invest the time in getting in to nix. I keep machines powered off that have a working configuration of older AI and other software tools because there is no other way to run them, regardless of the code being available on github.
    - godelski 1y ago
      There's other solutions too, this is a big part of systemd. Which also has nspawn and vmspawn do more explicitly. But everything has some containerization capabilities and ideally you'd give your program access to only what it needs. Privatetmp should always be on. But if versions are vulnerable you usually want to remove those versions, not put them in containers
crubier 1y ago
Work logs are generalized at my company and they are AWESOME
- rurban 1y ago
  In my company also. We put everything into our github issues. Some things also into the README.md or the github wiki, but it's usually lost there
heavyset_go 1y ago
My current workflow is to keep a wiki, would you say hydra would replace/complement especially that if you're used to note keeping the wiki way?
- godelski 1y ago
  Hydra is part of the documentation process imo. Truthfully, the most important stuff that goes in your experiment journal is all those pesky parameters and things that can surprisingly change results. So I love that hydra uses OmegaConf and I essentially get 3 copies: the experiment config yaml, the wandb log, a dictionary in the checkpoint. Multiple times my dumbass has had to try to match the checkpoint to the wandb log, so the redundancy is incredibly helpful. Sometimes just a library version has unexpected changes on performance and this makes it trivial to trace. The yaml file is more helpful when passing off the code to someone else or releasing to public. So yeah, I would say that it'll benefit no matter how you document. Use whatever documentation method works for you. Reports can still offer some benefits in just throwing some charts together quickly and organizing but I think you'd still benefit from hydra. It's too easy to lose track of those little things and this helps me automate. But you can also just straight up use OmegaConf or even dictionaries. Whatever works for you. The real help is logging. So whatever tools help you log, use them. This is just what I benefit from (there's a lot I can talk about too and I'd love to see what others do as well)
  - heavyset_go 1y ago
    Thanks, appreciate the detailed response. I'm going to have to check it out.
lairv 1y ago
Converged to something similar after spending 2 days bissecting a repo to reproduce a training run, having to wait 3hr on each commit before conclusive results. I couldn't get myself to use hydra though, it felt like a lot of bloat vs loading a yaml with pydantic
tomcam 1y ago
OK so I try to do that. But then I'll have some big problem or add too many big features and just give up. (My sleep is nearly nonexistent so I don't have a lot of time for logging things anyway.)
- godelski 1y ago
  You might be doing it wrong. Make your code more modular. Think about how powerful functions actually are. But also make them simple and self contained. All those programming books suggest this not because "it's pretty", "good form", or whatever. You do this because you know that things never work out like you expect them to. Only a naive programmer or a literal god thinks they'll get the program right on the first go. Even just the fact that the world changes underneath our feat means it will change. So you write expecting things to change. You write so a change can be added quickly and not break everything else. You write so you don't have to pull apart a bunch of tangled mess. There's a lot of complexity no matter what, so even a little goes a long way. > My sleep is nearly nonexistent so I don't have a lot of time for logging things anyway I'll make a bet. I'll bet that if you log you'll get more sleep. This is a classic negative feedback loop and is honestly why I started doing this in the first place. A little extra work upfront saves me a lot of work down the line. You need to be concerned with today but that doesn't mean you can ignore tomorrow. The point of my strategy is in how positive things compound. A little here, a little there, do this for a bit and you got something beautiful while it seems like you did no extra work (because you spread it out) But the negative effects compound too. They create more work. The less sleep you get the more mistakes you make. Worse, the more subtle hard to catch mistakes you make! You just end up missing sleep chasing down bugs and issues introduced because you wrote while being sleep deprived. We all do this! But we need to recognize it and try to break this cycle as soon as we recognize it happening. My bet is you are the one creating most of the work that is keeping you up and making you feel over burdened. My bet is if you take a break you'll actually get more done. My bet is you're caught in a destructive loop. I'll make this bet because I have so much experience with this same self-destructive behavior. Been there. Done that. I don't want to be there nor do I want you to be. But to get out, you have to fight that impulse that got you there in the first place.
  - 1y ago
chairmansteve 1y ago
Anyone use a digital notebook, like the reMarkable, for this kind of thing?
- thyristan 1y ago
  Started but stopped. most of the things are commands, code snippets, urls, all of which are tedious to hand-write and just easier to copy&paste. Often I do 'typescript' for shell sessions, asciinema or stuff like that, and file those. Also, use git, commit everything, never care about doing tidy commits, just commit commit commit and use tons of branches to try out stuff. if you need clean history later on, you can always do interactive rebase, squash merge or whatever. but having a documentation of all the things tried and failed is far more important.
  - Xss3 1y ago
    I thought the entire point was that it syncs cross platform really quickly and lets you have the best of both worlds?
    - thyristan 1y ago
      Only handwriting in a proprietary format. It isn't at all like one would wish for. It works as a replacement for a paper notebook. But it largely ignores the things one could do when adding more digital embeddings. In that way it is even worse than OneNote. If you want to do something like that, my recommendation would actually be something like OneNote on some Windows tablet.
rolandhvar 1y ago
So here's the thing I struggle with. I do a lot of work in jupyter notebooks. I come up with a new model or approach to some problem, and I want to fork out and test a hypothesis in the background (which might be some set of hyperparameters, and might take several minutes, or hours; call it Run A) while continuing to work down some other path in the same notebook, and maybe kick off a Run B that explores some other change (like a restructure of the code that's not "compatible" with the hyperparameter search of Run A). Then at some point when Run A finishes, I want to incorporate the changes I made in Run B and kick off Run C, and so on. The hard/important things are: 1) Being able to do this while staying in a Jupyter notebook context the whole time. Even something as simple as multiprocessing sucks because I've found it's too hard to manage in a Jupyter context (e.g. how do you handle where stdout and stderr go?). It's easier if you move to scripts where you have full support for this sort of thing and you are expecting to look at multiple log files on disk and whatnot. Also the sequential nature of notebooks doesn't help when you want to occasionally fork out or conditionally run stuff. 2) Keeping track of all these changes and hypotheses and merging the results/code together as you learn. It's like you need a VCS for your hypotheses. Maybe hydra & wandb help with that, I haven't used them. But this idea of keeping track of hypotheses seems like the more fundamental thing. 3) The main reason I prefer to stay in a notebook context is because I have all my objects easily accessible. My models, all my dataframes, functions to do some ad-hoc charting etc, all super easy to access in a REPL-like form. That is invaluable for doing ad-hoc sanity checks or digging/drilling down. So a big part of the workflow is you basically have this in-memory database of a bunch of relevant objects and you're querying it and constructing new objects & visualisations using Python as your tool, without having to load things from disk or build up the context from scratch. It's all "just there". 4) And then sometimes you want to take the results X1 of that notebook and plot them against some entirely different set of data X2 that requires a whole bunch of other code that you've defined in some other notebook somewhere, or maybe even as a real Python module. Like maybe that data lives in a database and you transform it or something. So OK, you call some functions to load X2 within your original notebook, but BOOM you get an OOM and you're like ok now I have to write some code to serialise X1 to disk, and make YET ANOTHER notebook so I can go analyze X1 and X2. It all just seems so... unnecessary, if only the right tooling existed. My current best approach is to use semantic versioning on the filename, just copy the whole notebook each time I make a fundamental change, and try to keep track of my hypotheses, preconditions, learnings etc within comments and have a few of those on the go running, but it's often hard to engage in critical thinking when everything you know is sprawled across multiple notebooks. Maybe a simple global journal is the only thing for this sort of use case. And that doesn't even address (4) which is often a huge pain point. Can anyone think of something better?
hy4000days 1y ago
[dead]
neilv 1y ago
If your work is tracked in an issue-tracking system, you can put your in-progress thinking notes as comments there (if they don't go in the code or some other artifact). It helps to have a safe environment, among whomever might access that issue comment history. If people don't feel safe exposing their thought process, then they won't do it, or they'll be stressed by presenting vulnerability, and even modify their problem-solving for appearances. (I have some more complicated options involving a wiki, but explaining requires too much context. The issue-tracking comments solution is obvious.) What I try not to do is to introduce new places that important information goes. If you don't rein this in, there will be an explosion of employees plastering your IP all over a bunch of random SaaSes, to be undiscovered or even lost to your company entirely (also, those other SaaS companies and hackers might get more use out of stealing your IP than you do).
- zeroCalories 1y ago
  Blame -> pr -> issue is a great way to learn a codebase if your team is good about keeping a log of their work, which they generally should be.
xico 1y ago
Hydra and omegaconf are almost officially abandonware, you shouldn't really depend on them.
deleted 1y ago
[deleted]

0cf8612b2e1e 1y ago

From my two minute skim of the docs, not encouraging that hydra only officially supports up to Python 3.11.

godelski 1y ago
I use it in python 3.12, and 3.12 just got out of bug fix. I haven't tried 3.13 but I would be surprised if there was a break. Most of it works through OmegaConf[0]. Idk why they haven't pushed an update in 2 years but neither has this been a problem. FWIW, they're still updating the repo[1] [0] https://omegaconf.readthedocs.io https://omegaconf.readthedocs.io [1] https://github.com/facebookresearch/hydra https://github.com/facebookresearch/hydra

nine_k 1y ago

The problem with keeping a journal is that the distraction of doing so may break the state of flow. OTOH there are natural breaks in the process of working; writing things down during these works fine. The fidelity is a bit lower, but it's still much better than nothing.

isaacimagine 1y ago
Agree, it's much better to write up a journal at times when your colleagues would be https://xkcd.com/303 https://xkcd.com/303
godelski 1y ago
> the distraction of doing so may break the state of flow. Sure, but like you said, don't do it when in the state of flow. Or better, make it part of your flow state. To me, it is part of my flow state, so not a real issue. I mean whatever works for you. You gotta time manage and I can't manage for you. I'm sure your boss is asking for more writeups than I am and just send them your notes. They don't care whats in it half the time, they just don't know how to figure out if you're working or not and just want something. Hell, we're on HN on a workday... I can guarantee you aren't in a flow state the whole time and can't be bothered with a few minutes to write some stuff down. I mean you have to eat and go to the bathroom, right?

isaacimagine 1y ago

Chaotic energy haha, I like it. Thanks for the tips re: keeping a journal, I will do this more in the future. I usually keep development notes, though normally in markdown files scattered across the codebase or in comments, never by date in the README. In the future, I might make JOURNAL.md a standard practice in my projects? re:w&b, I used w&b when it first came out and I liked it but I'm sure it's come a lot further in the time since then. I will have to take a look! Also lol "pretentious perfectionist" I'm glad to finally have some words to describe my design aesthetic. I like crisp fonts, what can I say.

godelski 1y ago
> Chaotic energy haha, I like it My boss says I'm eccentric. I say that's just a nice word for crazy lol > normally in markdown files scattered across the codebase or in comments I used to do that too but they didn't end up helping because I could never find them. So I moved back to using a physical book. The wandb reports was the first time I really had something where I felt like I got more out of it than a physical book. Even my iPad just results in a lot of lost stuff and more time trying to figure out why I can't just zoom in on the notes app. I mean what is an iPad even for if it isn't really good for writing? But the most important part of the process I talked about is the logging of all the parameters and options. Those are the details you tend to lose and go hunting for. So even if you never write a word you'll see huge benefits from this. > re:w&b Wandb's best feature is that you can email them requesting a feature and they'll implement it or help you implement it. It's literally their business model. I love it. I swear, they have a support agent assigned to me (thanks Art! And if wandb sees this, give the man a raise. Just look at what crazy people he has to deal with) > lol "pretentious perfectionist" I'm glad to finally have some words to describe my design aesthetic To be clear, I'm actually not. Too chaotic lol. Besides, perfectionism doesn't even exist. It's more a question about personal tastes and where we draw the line for what is good enough. I wish we'd stop saying "don't let perfectionism get in the way of good" because it assumes like there's universal agreement about what good enough is.
- isaacimagine 1y ago
  Parameters and options, got it. I try to keep all configuration declarative and make building and running as deterministic as possible. Then I can commit whenever I do something interesting, that I can just checkout to revisit.

rvz 1y ago

All of this agreed. Now in the age of AI, many students entering into CS need do this NOW, otherwise any answer they come up in the interview, will be assumed that it was from an AI and they need to show that they something useful came out of their blogpost or research. It is what it now means to know how to experiment, understand and build knowledge, rather than spitting out the answer because it it from stack overflow or ChatGPT. The mistakes are raw, all the learnings in a blog post which is what makes us human yet, 90% of candidates do not do this which is why most of them cannot explain an AI's mistakes an interview if they use it.

godelski 1y ago
I actually really like this idea. I've often found it odd we don't show off reports or how we run experiments during interviews. Certainly this has far greater influence over your aptitude than leetcode. > 90% of candidates do not do this or cannot explain AI's mistakes an interview. I have a growing concern that people do not see mistakes. This seems to be a bigger divide than "uses AI to code" vs "doesn't".

mr_toad 1y ago

Same goes for something as simple as setting up a server. You will forget, and if you don’t write it down you’ll have to figure it out again.

godelski 1y ago
Where did I put those configs again? Where did Bob put that script? Fuck, why didn't I write an ansible script. It's never a one off, and it serves as documentation. I'll remember after I make the same mistake next time. Also, environment modules for the win https://modules.readthedocs.io/en/latest/ https://modules.readthedocs.io/en/latest/
stirfish 1y ago
One time my boss asked me to upgrade some servers and was surprised I put all my work in a script in version control. Then there was a second batch of servers.
eptcyka 1y ago
Use nix and you’ll at least will be able to deploy the same again.
- genewitch 1y ago
  So what happens in nix if a package version is found to have some huge vulnerability? And my app expects that exact version? Because this is literally what docker, venv, and nix claim to fix but after getting burned by three other systems I'm not willing to invest the time in getting in to nix. I keep machines powered off that have a working configuration of older AI and other software tools because there is no other way to run them, regardless of the code being available on github.
  - godelski 1y ago
    There's other solutions too, this is a big part of systemd. Which also has nspawn and vmspawn do more explicitly. But everything has some containerization capabilities and ideally you'd give your program access to only what it needs. Privatetmp should always be on. But if versions are vulnerable you usually want to remove those versions, not put them in containers

crubier 1y ago

Work logs are generalized at my company and they are AWESOME

rurban 1y ago
In my company also. We put everything into our github issues. Some things also into the README.md or the github wiki, but it's usually lost there

heavyset_go 1y ago

My current workflow is to keep a wiki, would you say hydra would replace/complement especially that if you're used to note keeping the wiki way?

godelski 1y ago
Hydra is part of the documentation process imo. Truthfully, the most important stuff that goes in your experiment journal is all those pesky parameters and things that can surprisingly change results. So I love that hydra uses OmegaConf and I essentially get 3 copies: the experiment config yaml, the wandb log, a dictionary in the checkpoint. Multiple times my dumbass has had to try to match the checkpoint to the wandb log, so the redundancy is incredibly helpful. Sometimes just a library version has unexpected changes on performance and this makes it trivial to trace. The yaml file is more helpful when passing off the code to someone else or releasing to public. So yeah, I would say that it'll benefit no matter how you document. Use whatever documentation method works for you. Reports can still offer some benefits in just throwing some charts together quickly and organizing but I think you'd still benefit from hydra. It's too easy to lose track of those little things and this helps me automate. But you can also just straight up use OmegaConf or even dictionaries. Whatever works for you. The real help is logging. So whatever tools help you log, use them. This is just what I benefit from (there's a lot I can talk about too and I'd love to see what others do as well)
- heavyset_go 1y ago
  Thanks, appreciate the detailed response. I'm going to have to check it out.

lairv 1y ago

Converged to something similar after spending 2 days bissecting a repo to reproduce a training run, having to wait 3hr on each commit before conclusive results. I couldn't get myself to use hydra though, it felt like a lot of bloat vs loading a yaml with pydantic

tomcam 1y ago

OK so I try to do that. But then I'll have some big problem or add too many big features and just give up. (My sleep is nearly nonexistent so I don't have a lot of time for logging things anyway.)

godelski 1y ago
You might be doing it wrong. Make your code more modular. Think about how powerful functions actually are. But also make them simple and self contained. All those programming books suggest this not because "it's pretty", "good form", or whatever. You do this because you know that things never work out like you expect them to. Only a naive programmer or a literal god thinks they'll get the program right on the first go. Even just the fact that the world changes underneath our feat means it will change. So you write expecting things to change. You write so a change can be added quickly and not break everything else. You write so you don't have to pull apart a bunch of tangled mess. There's a lot of complexity no matter what, so even a little goes a long way. > My sleep is nearly nonexistent so I don't have a lot of time for logging things anyway I'll make a bet. I'll bet that if you log you'll get more sleep. This is a classic negative feedback loop and is honestly why I started doing this in the first place. A little extra work upfront saves me a lot of work down the line. You need to be concerned with today but that doesn't mean you can ignore tomorrow. The point of my strategy is in how positive things compound. A little here, a little there, do this for a bit and you got something beautiful while it seems like you did no extra work (because you spread it out) But the negative effects compound too. They create more work. The less sleep you get the more mistakes you make. Worse, the more subtle hard to catch mistakes you make! You just end up missing sleep chasing down bugs and issues introduced because you wrote while being sleep deprived. We all do this! But we need to recognize it and try to break this cycle as soon as we recognize it happening. My bet is you are the one creating most of the work that is keeping you up and making you feel over burdened. My bet is if you take a break you'll actually get more done. My bet is you're caught in a destructive loop. I'll make this bet because I have so much experience with this same self-destructive behavior. Been there. Done that. I don't want to be there nor do I want you to be. But to get out, you have to fight that impulse that got you there in the first place.
- 1y ago

chairmansteve 1y ago

Anyone use a digital notebook, like the reMarkable, for this kind of thing?

thyristan 1y ago
Started but stopped. most of the things are commands, code snippets, urls, all of which are tedious to hand-write and just easier to copy&paste. Often I do 'typescript' for shell sessions, asciinema or stuff like that, and file those. Also, use git, commit everything, never care about doing tidy commits, just commit commit commit and use tons of branches to try out stuff. if you need clean history later on, you can always do interactive rebase, squash merge or whatever. but having a documentation of all the things tried and failed is far more important.
- Xss3 1y ago
  I thought the entire point was that it syncs cross platform really quickly and lets you have the best of both worlds?
  - thyristan 1y ago
    Only handwriting in a proprietary format. It isn't at all like one would wish for. It works as a replacement for a paper notebook. But it largely ignores the things one could do when adding more digital embeddings. In that way it is even worse than OneNote. If you want to do something like that, my recommendation would actually be something like OneNote on some Windows tablet.

rolandhvar 1y ago

So here's the thing I struggle with. I do a lot of work in jupyter notebooks. I come up with a new model or approach to some problem, and I want to fork out and test a hypothesis in the background (which might be some set of hyperparameters, and might take several minutes, or hours; call it Run A) while continuing to work down some other path in the same notebook, and maybe kick off a Run B that explores some other change (like a restructure of the code that's not "compatible" with the hyperparameter search of Run A). Then at some point when Run A finishes, I want to incorporate the changes I made in Run B and kick off Run C, and so on. The hard/important things are: 1) Being able to do this while staying in a Jupyter notebook context the whole time. Even something as simple as multiprocessing sucks because I've found it's too hard to manage in a Jupyter context (e.g. how do you handle where stdout and stderr go?). It's easier if you move to scripts where you have full support for this sort of thing and you are expecting to look at multiple log files on disk and whatnot. Also the sequential nature of notebooks doesn't help when you want to occasionally fork out or conditionally run stuff. 2) Keeping track of all these changes and hypotheses and merging the results/code together as you learn. It's like you need a VCS for your hypotheses. Maybe hydra & wandb help with that, I haven't used them. But this idea of keeping track of hypotheses seems like the more fundamental thing. 3) The main reason I prefer to stay in a notebook context is because I have all my objects easily accessible. My models, all my dataframes, functions to do some ad-hoc charting etc, all super easy to access in a REPL-like form. That is invaluable for doing ad-hoc sanity checks or digging/drilling down. So a big part of the workflow is you basically have this in-memory database of a bunch of relevant objects and you're querying it and constructing new objects & visualisations using Python as your tool, without having to load things from disk or build up the context from scratch. It's all "just there". 4) And then sometimes you want to take the results X1 of that notebook and plot them against some entirely different set of data X2 that requires a whole bunch of other code that you've defined in some other notebook somewhere, or maybe even as a real Python module. Like maybe that data lives in a database and you transform it or something. So OK, you call some functions to load X2 within your original notebook, but BOOM you get an OOM and you're like ok now I have to write some code to serialise X1 to disk, and make YET ANOTHER notebook so I can go analyze X1 and X2. It all just seems so... unnecessary, if only the right tooling existed. My current best approach is to use semantic versioning on the filename, just copy the whole notebook each time I make a fundamental change, and try to keep track of my hypotheses, preconditions, learnings etc within comments and have a few of those on the go running, but it's often hard to engage in critical thinking when everything you know is sprawled across multiple notebooks. Maybe a simple global journal is the only thing for this sort of use case. And that doesn't even address (4) which is often a huge pain point. Can anyone think of something better?

hy4000days 1y ago

[dead]

neilv 1y ago

If your work is tracked in an issue-tracking system, you can put your in-progress thinking notes as comments there (if they don't go in the code or some other artifact). It helps to have a safe environment, among whomever might access that issue comment history. If people don't feel safe exposing their thought process, then they won't do it, or they'll be stressed by presenting vulnerability, and even modify their problem-solving for appearances. (I have some more complicated options involving a wiki, but explaining requires too much context. The issue-tracking comments solution is obvious.) What I try not to do is to introduce new places that important information goes. If you don't rein this in, there will be an explosion of employees plastering your IP all over a bunch of random SaaSes, to be undiscovered or even lost to your company entirely (also, those other SaaS companies and hackers might get more use out of stealing your IP than you do).

zeroCalories 1y ago
Blame -> pr -> issue is a great way to learn a codebase if your team is good about keeping a log of their work, which they generally should be.

xico 1y ago

Hydra and omegaconf are almost officially abandonware, you shouldn't really depend on them.

deleted 1y ago

[deleted]