7 ms·

I love this >Data dominates. If you've chosen the right data structures and organized things well, the algorithms will almost always be self-evident. Data stru

• dzonga 3y ago
  my algorithms D/S professor told me one trick. always use a 'map'. works for most cases.
• bitwize 3y ago
  "Show me your flowcharts, but keep your tables hidden, and I shall continue to be mystified. Show me your tables, and I won't need to see your flowcharts, they'll be obvious."
  • kwhitefoot 3y ago
    You forgot the credit: Fred Brooks, Mythical Man Month,
  • jrpelkonen 3y ago
    Great quote and so true in the general case. Unfortunately, I have seen some database designs that have left me more mystified than I was before.
    • capableweb 3y ago
      I've seen programs that got more mystical the more the employees showed me how it worked internally.
• karmakaze 3y ago
  This should be Rule #1.
• lawn 3y ago
  I'm not a fan of leetcode, but when I did some competitive programming it was quite common that you first had to transform the input data into the proper data structure, and then you can apply some algorithm to produce the answer.
• jasode 3y ago
  >Which is why all the LeetCode interviews always struck me as odd. They focus on algorithms, not data structures, The typical leetcode questions also focus on "data structures" because the candidate needs to have the mental catalog of data structures in his brain to pull from when he "pattern matches" on the problem/solution. The interviewer isn't going to volunteer whether the candidate needs a "priority queue", or "adjacency matrix", or "trie", etc to solve various algorithm questions. If the candidate is stuck, then the interviewer might give a hint to the data structure to use. But of course, too much hand-holding will not make the candidate come across as a strong hire signal.
  • no_wizard 3y ago
    Its been a time since I interviewed in such a manner, perhaps things have shifted. When I was interviewing a few years back, it was algorithm heavy, often a basic data structure was provided and there was alot of focus on traversal (hence the memes around reversing binary trees and such)
    • Icathian 3y ago
      To add another point of anecdata, my experience very recently at tech companies aligns with the parent comment.
• Swizec 3y ago
  > Which is why all the LeetCode interviews always struck me as odd 90% of leetcode is choosing the right data structure. If your algorithm doesn’t wanna come together, you probably missed a better way to model the data.
  • danielmarkbruce 3y ago
    I had a checklist one time for interviews. One of the items was: "consider data structures x, y, z" - only maybe 4 or 5 data structures. It worked 95% of the time.
    • hdlothia 3y ago
      Which data structures were those?
      • danielmarkbruce 3y ago
        Oh, I don't have the list I used to use, but it was very basic stuff. hash table, stack, queue, linked list, sorted array, binary tree, graph. Many problems seemed difficult until I just considered shoving the data into one of those.
• dagw 3y ago
  Which is why all the LeetCode interviews always struck me as odd. They focus on algorithms, not data structures I helped my nephew preparing some competitive coding event, and transforming the data into the right data structure was a huge part of solving most problems. For example most of the code you might end up writing is for finding the longest path through a weighted DAG, but the trick was realising that you could represent the problem as a weighted DAG, and if you did then the length of the longest path through that graph was your answer. If you didn't spot that then you might very well still be able to solve the problem, but your solution would end up being much slower and more complicated.
  • pfisherman 3y ago
    How much of this is finding the right data structure (graph) vs translating the problem into a new domain? I think maybe the former follows from the latter?
  • mikhailfranco 3y ago
    This is the low-hanging fruit algorithm: - build the DAG with heavy nodes (e.g. lead split-shot) - use string or fishing line for edges, with length proportional to 'weights' - hold the root node(s) high in the air, if more than one, hold them together, at the same height - the lowest node, or the length of taught line (drop distance below your hand), is the answer
    • roughly 3y ago
      What’s great is that it’s constant time, once you’ve prepared the data structure. Never underestimate the parallelism of physical systems.
    • a1369209993 3y ago
      That doesn't actually work, because a DAG only lacks directed cycles. Consider: /-3-> D A -1-> B -1-> C \------4------^ In this case, the longest path is A-C, but C will be held at level 2 by B, leaving D lowest at level 3. This works for a fully-acyclic directed graph (eg a tree), but not for a directed-acyclic directed graph. It can also find the node whose shortest path is the longest (so longest shortest path, minimax style). (It also works for a acyclic undirected graph, as illustrated at [0], but that's a bit tangential.) 0: https://www.youtube.com/watch?v=wGrOPSBPpyk https://www.youtube.com/watch?v=wGrOPSBPpyk
• mikebenfield 3y ago
  I don't really see how you can choose one without at least a rough idea of the other. How do you know what data structure to use if you have no idea how you'll be accessing the data?
  • softfalcon 3y ago
    You're not wrong, data structures and an algorithm for data retrieval are often connected and developed together. You often see someone come up with a novel way of traversing data, then they model an elegant data structure to match it and enable said traversal. What isn't covered in this flow is how composition of multiple data-structure/algorithm pairings work together amidst many others. It's often not an algorithm on its own that defines an overall solution. It's the connecting of many of them together, to build a whole software stack, domain language, etc. When you understand the algorithms, the corresponding data structures, when to use them, similarities in various frameworks in your chosen domain, how that analogues to your current problem set, etc you really start getting somewhere.
    • eska 3y ago
      * data structures and an algorithm for data retrieval are often connected and developed together* Often? Always! An algorithm always only works on a certain data data structure.
      • nonrandomstring 3y ago
        Or as Fred Brooks put it: " Show me your tables, and I won't usually need your flowcharts; they'll be obvious." But to be fair, you need to be a pretty damn good comp sci person (and actively thinking in algs and DS) to quickly look at DS and see the "obvious" processing implied.
        • sfn42 3y ago
          A problem I see a lot is when the flowcharts should be obvious, but aren't because whoever wrote the code didn't write the obvious solution. Instead they wrote a horrible buggy mess that tries (and usually fails at least a little) to do the same thing as the obvious solution but with much more, and more convoluted, code.
• gigatexal 3y ago
  This point I really resonated with as well. Really cool. A lot to think bout here.
• softfalcon 3y ago
  I can also anecdotally agree with your analysis. I've seen for myself how much more some folks will respect you during interviews if you "get past" the part about fizzbuzz (algorithms) and move onto immediately talking about data structures, architecture, and how that analogues to the domain in question. All of the sudden, the interviewer relaxes and you can just see on their face the, "oh, we've got an actual senior engineer on the call here." They open up about their technical problems and stop caring about "proving if you can even code". Similarly, the teams I've had the most problems creating positive change, meeting milestones, or collaborating together properly as a team with is where no one seems to have a good grasp of data structures and code architecture. It seems to have become more common too. Many folks are quite used to having a framework that can just do everything, and if it can't, someone "smarter than them" has made a plugin or middleware to save them from having to think too hard. I personally find engineers who avoid data structures to be shooting themselves in the foot, they're giving up on one of the most useful tools in their arsenal and I witness how it limits them day-to-day. It's tough to watch.
  • pklausler 3y ago
    Asking "FizzBuzz" is not done to test algorithmic knowledge.
• yxhuvud 3y ago
  But the distinction is meaningless. A data structure is just a collection of algorithms for working on underlying memory. More complex algorithms are built with simpler algorithms as building blocks. With that in mind, it essentially boils down to picking algorithms which when put together allows solving the problem in an as easy way as possible. Which of course is self evident. Though I suppose it doesn't hurt to remind people to step back and look at the full picture.
  • e12e 3y ago
    You need to do data modeling and (de) normalization even if your data is in "data structures", not an SQL DBMS.
• jacobgorm 3y ago
  See also Peter Naur's letter to the editor in https://dl.acm.org/doi/pdf/10.1145/365719.366510 https://dl.acm.org/doi/pdf/10.1145/365719.366510 . In Denmark, CS is not called CS but Datalogy.
  • asalahli 3y ago
    I prefer the term Informatics over Computer Science for this reason.
• TeMPOraL 3y ago
  > >Data dominates. If you've chosen the right data structures and organized things well, the algorithms will almost always be self-evident. Data structures, not algorithms, are central to programming. I've been thinking about it a lot. I want to still believe it, but then my experience tells me that the "right data structure" is usually "the data structure most convenient for the operations/algorithms you want to run". Which makes this principle either backward, or circular.
  • nyssos 3y ago
    The right data structure is the one that's produced and consumed in a way that matches the domain you're trying to model. That domain can be inputs to a particular algorithm, but it can also be things like "states of this system" (constructors correspond to state transitions), "witnesses to this property" (constructors correspond to valid inferences), etc.
    • TeMPOraL 3y ago
      Leaving aside the unusual, to me, meaning of "domain" in this context, your description matches what I wrote above: the right data structure is determined by what you're planning to do with the data. It only convinces me further that Pike's take is wrong. Specifically: - "Data dominates" and "Data structures, not algorithms, are central to programming" are just plain wrong; - "If you've chosen the right data structures and organized things well, the algorithms will almost always be self-evident" is correct only in the descriptive sense, not in the prescriptive sense. That is, it only works when you read it as restatement of Brooks's "tables vs. flowcharts" line (quoted elsewhere in this thread): it's easier to guess what your code is doing from how you organized your data, than it is to guess how you organized your data by looking at the code. I.e. it's an advice on where to start understanding someone's existing solution; it has nothing to say about solving new problemms / writing new code.
• samhuk 3y ago
  Although LeetCode does have a strong algo slant, choosing the optimal data structure is almost always a key part in solving the problems. If you think leetcode-like problems is always about the algo, "BFS or DFS" etc., then at best you are not realizing the data structure choices you are making, at worst you may not be so good at solving them or haven't progress that much through leetcode-like challenges.
• galaxyLogic 3y ago
  If you want a better data-structure you must ask: Better for whom? Better for the algorithm that manipulates the data! So I don't think that "data dominates". You may need to adapt your data to the algorithm or vice versa. What dominates is what we want to do. Without algorithm we are doing nothing. In a sense data is part of the algorithm, it is implicitly coded into the assumptions the algorithm makes about its arguments and results.
  • namaria 3y ago
    On the other hand, algorithms are just data. In the end it's just a set of bytes that determines what happens to other sets of bytes. Data structures and algorithms are two sides of the same thing. In the abstract meaning space in our minds, data structures are an extension of the algorithm's logic. In the memory space, algorithms are just data structures that change other data structures. Seeing the connection from both sides is insightful.
    • galaxyLogic 3y ago
      Yes. But there is a difference between data and program. Program is data as you say but it must be data that can be interpreted as a set of possibly conditional steps. It is often pointed out that Lisp programs are "just data" but not every list can be executed as a function. Further programs are what interpret data, by making branching decisions based on the data. Program must be interpreted by the language interpreter, and then on the 2nd level the interpreted program interprets the data.
• bob1029 3y ago
  All software is effectively a set of ETL jobs with a user-friendly interface wrapped around. Those fancy algorithms, languages, frameworks, etc are simply a very ceremonious way to get data from A to B. The schema/data are completely agnostic to the computation. You don't even need a computer. 100% of schemas are possible to print out on sheets of physical paper and iterate offline.
  • gridspy 3y ago
    Computer used to be the job title for people who DID do this by hand. It was co-opted by the technology which replaced the job. Of course modern computation is often impractical to do by hand. It might even be so complex that humans would make too many errors and take to long to ever complete correctly.
    • esafak 3y ago
      Common use cases were abstracted into libraries and services because nobody should have to reinvent the wheel. This let people operate at a higher level of abstraction and concentrate on more complicated tasks. These too became abstracted, and the cycle repeated.
  • sirsinsalot 3y ago
    While true, I think this is reductive. When you're dealing with most modern software, there's so many of these ETL-like pipelines, abstractions, black-boxes and just sprawling complexity that the job becomes much more about complexity management than anything. No component exists in a vacuum, I would argue software is as much about managing the complexity of the data and algorithms towards a purpose as it is about the purpose, data or algorithms. Entropy comes for us all, after all.
• usrbinbash 3y ago
  > Which is why all the LeetCode interviews always struck me as odd. They focus on algorithms, not data structures, which is exactly what you don't want to do out of the gate, most of the time. Which is one of the reasons why I will never use that style of problems as an interview question. And the other reason is: that style of problems also don't teach anything about architecture.
• tabtab 3y ago
  Indeed! I've long been a fan of "table oriented programming" where the common CRUD objects are specified mostly as data. You could also create them in code by calling a RAM-table constructor, so it's not either/or. Most the data fields, navigation structure, and event handlers (or handler stubs) could be readily table-ized. Code is a lousy place to store boat-loads of attributes. You can do data-oriented transformations and filters on them if in a data structure. Hierarchical file systems are limiting and messy, because grouping by one factor de-groups another. I want to be able to "query" dev code units via SQL or similar. CRUD patterns are pretty consistent across industries such that most CRUD idioms shouldn't need custom re-invention, and thus should be attribute-tized. You will still need occasional code-based tweaking, and this can be accomplished by having the attributes generate "runtime draft" UI markup and/or SQL clauses, which can then be tweaked via code as needed. I'm building a proof-of-concept that uses what I call "fractal rendering" to be able to intercept the "draft" construction of screens and SQL clauses at a fine level or course level, depending on need. This avoids the all-or-nothing problem of prior attempts per attribute-vs-code debate. Ya git both! (Dynamic SQL generation should probably be avoided for public-facing sites, per injection risk, but limits the wonderful power of query-by-example.) I don't claim it will be as performant as the code-centric approaches, but if it catches on, performance tweakers will find a way to scale it. (For CRUD apps, the performance bottleneck should usually be the database, not app code anyhow, unless you doing something wrong or special.) The CASE tools of the 1980's and 90's started going in this direction, but were too expensive & clunky, and then the OOP push ended the idea by favoring attributes-in-code, back to square one, sigh. (CASE tools don't have to be proprietary.) It's ripe area for groundbreaking R&D. CRUD may not be sexy, but it runs the world.