3 ms·

In the age of agentic programming and the ever increasing pressure to ship faster, I'm afraid this kind of knowledge will become more and more fringe, even more

In the age of agentic programming and the ever increasing pressure to ship faster, I'm afraid this kind of knowledge will become more and more fringe, even moreso than it is today. Who has the time to think through the intricacies of parallel data structures? Clearly we'll just throw more hardware at problems, write yet another service/api/http endpoint and move on to the next hype. The LLM figures out the algorithms and we soon lose the skills to develop new ones. And tell each other the scifi BS myth that "AI" will invent new data structures in the future so we don't even beed humans in the loop.

dgb23 6mo ago
I think the opposite is the case. We increasingly need to care more about performance and know how to leverage hardware better. The market is telling us that through increased hardware prices. LLMs being very powerful means that we need to start being smarter about allocating resources. Should chat apps really eat up gigabytes of RAM and be entitled to cores, when we could use that for inference?
boricj 6mo ago
AI is like a genie: be careful what you wish for or you'll get what you asked for. Lately at work I've done C++ optimization tricks like inplace_map, inplace_string, placement new to inline map-like iterators inside a view adapter's iterators and putting that byte buffer as the first member of the class to not incur std::max_align_t padding with the other members. At a higher architecture level, I wrote a data model binding library that can serialize JSON, YAML and CBOR documents to an output iterator one byte at a time without incurring heap allocation in most cases. This is because I work on an embedded system with 640 KiB of SRAM and given the sheer amount of run-time data it will have to handle and produce, I'm wary not only about heap usage, but also heap fragmentation. AI will readily identify such tricks, it can even help implement them, but unless constrained otherwise AI will pick the most expedient solution that answers the question (note that I didn't say answers the problem).
- hakanderyal 6mo ago
  This is also the reason why we have two polar opposite views on AI. “Slop generator” vs “Next best thing since sliced bread”. With SOTA models it all depends on how you drive them.
  - timeinput 6mo ago
    All my old software before AI was self documenting and didn't need comments -- it just was obvious. Today my prompts never make slop. I'm a really good driver.
ozgrakkurt 6mo ago
People underestimate the effect of knowledge accumulation that happens when learning from high quality sources imo. LLMs aren't even close to the level of knowledge distillation capacity a human has yet.
Andys 6mo ago
Or..? A golden era for people who want to think of new things and test out their ideas quickly by having AI code it up.

dgb23 6mo ago

I think the opposite is the case. We increasingly need to care more about performance and know how to leverage hardware better. The market is telling us that through increased hardware prices. LLMs being very powerful means that we need to start being smarter about allocating resources. Should chat apps really eat up gigabytes of RAM and be entitled to cores, when we could use that for inference?

boricj 6mo ago

AI is like a genie: be careful what you wish for or you'll get what you asked for. Lately at work I've done C++ optimization tricks like inplace_map, inplace_string, placement new to inline map-like iterators inside a view adapter's iterators and putting that byte buffer as the first member of the class to not incur std::max_align_t padding with the other members. At a higher architecture level, I wrote a data model binding library that can serialize JSON, YAML and CBOR documents to an output iterator one byte at a time without incurring heap allocation in most cases. This is because I work on an embedded system with 640 KiB of SRAM and given the sheer amount of run-time data it will have to handle and produce, I'm wary not only about heap usage, but also heap fragmentation. AI will readily identify such tricks, it can even help implement them, but unless constrained otherwise AI will pick the most expedient solution that answers the question (note that I didn't say answers the problem).

hakanderyal 6mo ago
This is also the reason why we have two polar opposite views on AI. “Slop generator” vs “Next best thing since sliced bread”. With SOTA models it all depends on how you drive them.
- timeinput 6mo ago
  All my old software before AI was self documenting and didn't need comments -- it just was obvious. Today my prompts never make slop. I'm a really good driver.

ozgrakkurt 6mo ago

People underestimate the effect of knowledge accumulation that happens when learning from high quality sources imo. LLMs aren't even close to the level of knowledge distillation capacity a human has yet.

Andys 6mo ago

Or..? A golden era for people who want to think of new things and test out their ideas quickly by having AI code it up.