4 ms·

"Generative AI and large language models (LLMs), deep learning techniques, including LSTMs, GRUs, RNNs, neural networks, unsupervised learning, Seq2Seq model, t

by paxys 3y ago

"Generative AI and large language models (LLMs), deep learning techniques, including LSTMs, GRUs, RNNs, neural networks, unsupervised learning, Seq2Seq model, the SciPy ecosystem, Jupyter Notebooks, Keras, TensorFlow". No, every programmer does not need to know these things. Not even close. Looking over the table of contents it is maybe 20% relevant topics and 80% soup of current hot buzzwords strung together to sound appealing and sell the latest edition of the book. And of course you can hit a button to get a monthly subscription and read all of it! If you want some actually helpful algorithms-focused material look up the website and YouTube channel of any good university. They all have great foundational courses covering data structures and algorithms available for free online that will be much more concise and useful.

tonyarkles 3y ago
I generally agree with you. If you want classical algorithms (which are, imo, fundamental), you can't go wrong start with Skiena's "Algorithm Design Manual" or lecture notes: https://www3.cs.stonybrook.edu/~skiena/ https://www3.cs.stonybrook.edu/~skiena/ > the SciPy ecosystem, Jupyter Notebooks These, though, are gems that can be dramatic quality-of-life improvements for all kinds of data analysis. I resisted for a while (writing code in a web browser? BARF), but have gradually come to have a deep affection for pandas, numpy, scipy, seaborn, etc. They're quirky, their APIs are sometimes a bit strange and unintuitive, but they make data analysis tasks go so much quicker. I'm not a data scientist; most days I'm more of a systems programmer. Being able to take log files, write a bit of Python glue to transform them into Pandas Dataframes, and then slice-and-dice-and-plot different facets of the data rapidly is a game changer.
- vips7L 3y ago
  I really like David Kopec's Classic Computer Science Problems series. He has it implemented in multiple languages too. https://classicproblems.com/ https://classicproblems.com/
tikhonj 3y ago
Frankly, the classic algorithms courses at universities aren't great either. The set of "classic" algorithms covered in an intro course is arbitrary and comes down to history more than pedagogy. There are some useful concepts that all programmers should know like dynamic programming or divide-and-conquer, but spending a bunch of time on relatively specialized and narrow algorithms like the simplex algorithm or FFT is not a great way to illustrate them. The core algorithms sequence ends up feeling more like a grab bag of "historically significant" algorithms rather than a coherently structured introduction to the subject. The set of algorithms taught is not a great practical foundation for solving problems, but also aren't especially effective at teaching how to generalize the ideas and design algorithms for your own problems. You can use a classic algorithms course as a starting point to learn more, but it's not an obviously great starting point. One grab-bag of algorithms might be better than another, but it isn't fundamentally better.
- jerf 3y ago
  I think of a lot of the school algorithms as stepping stones and good practice. It's easy for established programmers to forget the days when a pointer was beyond mindblowing and, sure, OK, I get the idea of a function, but a function that calls a function? A function that takes as a parameter a function to call? What is this? A linked list is useless in 2024. But it is also just about the simplest possible non-trivial data structure, and while you may not ever implement one ever again the pointer practice is good, and so I support keeping it in the curriculum, albeit I also support being very clear and up front about the fact that it is not actually useful anymore as well. Even something like a compilers course, odds are decent you're not going to ever use those exact algorithms ever again, but the general overview and practice is very valuable and the difference between those who understand those concepts and those who don't is often pretty clear to me. You've got to bootstrap off of something and I think a lot of people fall into the unexamined trap of defeating the curriculum in detail ("well, this thing is useless, and this next thing is useless, and the next thing is useless" and therefore the entire thing is useless) and in the process cutting off any way of getting enough experience with the space in the first place to get to the point that they don't need the bootstrap anymore. There is no scenario where we sit a student down in a seat, say a few words over a couple of hours, and presto they're as fluent as any other programmer with decades of experience with the basic tools of data structure design.
ametrau 3y ago
Thanks, saved me some time.
knightoffaith 3y ago
It's quite strange to include LSTMs/GRUs/RNNs here. I can maybe see why a programmer should be familiar with generative AI, and how a neural network works at a high level, but people not working in AI (that is, people who are at least touching a tensor library - making OpenAI API calls doesn't count) really have little reason to study RNNs, let alone GRUs. Looking at this again - the book description is "Delve into the realm of generative AI and large language models (LLMs) while exploring modern deep learning techniques, including LSTMs, GRUs, RNNs". This doesn't sound like 50 algorithms every programmer should know at all. A very strangely put together book. It's definitely not a generic book for the average programmer as the title seems to suggest (just way too much machine learning for that), but even we interpret it as a machine learning book, it's really odd to include content on the TLS handshake and dynamic programming.

by paxys 3y ago

tonyarkles 3y ago

I generally agree with you. If you want classical algorithms (which are, imo, fundamental), you can't go wrong start with Skiena's "Algorithm Design Manual" or lecture notes: https://www3.cs.stonybrook.edu/~skiena/ https://www3.cs.stonybrook.edu/~skiena/ > the SciPy ecosystem, Jupyter Notebooks These, though, are gems that can be dramatic quality-of-life improvements for all kinds of data analysis. I resisted for a while (writing code in a web browser? BARF), but have gradually come to have a deep affection for pandas, numpy, scipy, seaborn, etc. They're quirky, their APIs are sometimes a bit strange and unintuitive, but they make data analysis tasks go so much quicker. I'm not a data scientist; most days I'm more of a systems programmer. Being able to take log files, write a bit of Python glue to transform them into Pandas Dataframes, and then slice-and-dice-and-plot different facets of the data rapidly is a game changer.

vips7L 3y ago
I really like David Kopec's Classic Computer Science Problems series. He has it implemented in multiple languages too. https://classicproblems.com/ https://classicproblems.com/

tikhonj 3y ago

Frankly, the classic algorithms courses at universities aren't great either. The set of "classic" algorithms covered in an intro course is arbitrary and comes down to history more than pedagogy. There are some useful concepts that all programmers should know like dynamic programming or divide-and-conquer, but spending a bunch of time on relatively specialized and narrow algorithms like the simplex algorithm or FFT is not a great way to illustrate them. The core algorithms sequence ends up feeling more like a grab bag of "historically significant" algorithms rather than a coherently structured introduction to the subject. The set of algorithms taught is not a great practical foundation for solving problems, but also aren't especially effective at teaching how to generalize the ideas and design algorithms for your own problems. You can use a classic algorithms course as a starting point to learn more, but it's not an obviously great starting point. One grab-bag of algorithms might be better than another, but it isn't fundamentally better.

jerf 3y ago
I think of a lot of the school algorithms as stepping stones and good practice. It's easy for established programmers to forget the days when a pointer was beyond mindblowing and, sure, OK, I get the idea of a function, but a function that calls a function? A function that takes as a parameter a function to call? What is this? A linked list is useless in 2024. But it is also just about the simplest possible non-trivial data structure, and while you may not ever implement one ever again the pointer practice is good, and so I support keeping it in the curriculum, albeit I also support being very clear and up front about the fact that it is not actually useful anymore as well. Even something like a compilers course, odds are decent you're not going to ever use those exact algorithms ever again, but the general overview and practice is very valuable and the difference between those who understand those concepts and those who don't is often pretty clear to me. You've got to bootstrap off of something and I think a lot of people fall into the unexamined trap of defeating the curriculum in detail ("well, this thing is useless, and this next thing is useless, and the next thing is useless" and therefore the entire thing is useless) and in the process cutting off any way of getting enough experience with the space in the first place to get to the point that they don't need the bootstrap anymore. There is no scenario where we sit a student down in a seat, say a few words over a couple of hours, and presto they're as fluent as any other programmer with decades of experience with the basic tools of data structure design.

ametrau 3y ago

Thanks, saved me some time.

knightoffaith 3y ago

It's quite strange to include LSTMs/GRUs/RNNs here. I can maybe see why a programmer should be familiar with generative AI, and how a neural network works at a high level, but people not working in AI (that is, people who are at least touching a tensor library - making OpenAI API calls doesn't count) really have little reason to study RNNs, let alone GRUs. Looking at this again - the book description is "Delve into the realm of generative AI and large language models (LLMs) while exploring modern deep learning techniques, including LSTMs, GRUs, RNNs". This doesn't sound like 50 algorithms every programmer should know at all. A very strangely put together book. It's definitely not a generic book for the average programmer as the title seems to suggest (just way too much machine learning for that), but even we interpret it as a machine learning book, it's really odd to include content on the TLS handshake and dynamic programming.