3 ms·
Hyperparam tuning that has better intuition and can incorporate architecture changes automatically. It won't invent something completely new though.
by ipsum2 7mo ago
Hyperparam tuning that has better intuition and can incorporate architecture changes automatically. It won't invent something completely new though.
- kraddypatties 7mo agoHm, that's fair. It does feel like there's low hanging fruit in combining "old school" methods for conducting a hyperparameter sweep efficiently _with_ the higher level architecture edit ability of Autoresearch. Probably would cut the number of runs down by a significant number (as far as I can tell it's doing a grid search once it decides to mess with a knob or section of the architecture).
- falcor84 7mo ago> It won't invent something completely new though. I don't necessarily disagree, but am wondering whether you have any particular reason/intuition driving you to claim this. I have seen AI agents be quite creative in other tasks; do you think there's a particular reason why we shouldn't see creativity in architecture research, given enough time and resources?