4 ms·

I tried to finish this quiz but just can’t. Every question I got was a very big, “it depends on context…” “Do you prefer strong static or dynamic or a mix?” We

I tried to finish this quiz but just can’t. Every question I got was a very big, “it depends on context…” “Do you prefer strong static or dynamic or a mix?” Well… are we teaching 9th graders an intro to coding, writing a quick script to answer a bespoke data question, or writing a data processing library? “On algorithms I focus on…” Okay, well… do we care about performance? Where is it running? How often will it run? Will the code be disposed of soon or live a decade? Do we need it working today or next week? I just don’t understand how to even begin formulating an opinion on any of these questions without any context. To use the compass analogy: shouldn’t you want to best know how to use a compass? What value is there in saying, “my favourite bearing is east-northeast”? That is, the substance in any of this is the “it depends…” portion. Any answers to this quiz are really just a proxy for the kinds of contexts people are solving problems in.

arjonagelhout 1y ago
I have the same issue. For many of the questions my answer is "all of the above, but A in context A, B in context B, etc.". Many are also not mutually exclusive. Take this example: "When debugging I typically:" > Write tests to isolate the problem In the case of math functions, or more primitive building blocks, writing tests can help ensure correctness of the underlying functions, to exclude them from the problem search. > Reason about the code logically first This is always useful. > Use a debugger to step through code systematically Useful when dealing with a larger codebase and the control flow is hard to follow. The call stack can give quick guidance over trying to manually decipher the control flow. > Add print statements to understand data flow: Useful when debugging continuous data streams or events, e.g. mouse input, where you don't want to interrupt the user interaction that needs to be debugged.
- thierrydamiba 1y ago
  Is the fact that the answer isn’t easy a feature or bug?
  - teapot7 1y ago
    Oh a bug, surely. How is the quiz supposed to give you insights when it's unanswerable?
    - thierrydamiba 1y ago
      By forcing you to make a decision without context. Similar to how when presented with the trolley problem some will ask many follow up questions about the individuals on each track, the train, etc. That’s not the point.
      - shadowfox 1y ago
        > By forcing you to make a decision without context. Not the OP, but what would be the point to that? In any practical scenario there is always context, isnt it? I guess I don't quite get what we are trying to measure here.
      - mannykannot 1y ago
        But here, the context is relevant, and the analogy does not change this.
  - arjonagelhout 1y ago
    I think it’s a bug in getting a useful outcome, but it’s a feature in creating engagement with the post.
  - mannykannot 1y ago
    It is not primarily a matter of difficulty. The problem with choices like these is that they seem to be predicated on a rather simplistic, formulaic view of software development - a view that experienced developers will recognize as flawed. In this case, one tacit assumption is that a given developer will typically adopt just one of these approaches. Another is that they can meaningfully and objectively be ordered along either of the axes purportedly being measured.
IgorPartola 1y ago
I took it as “when working in the kinds of things things I prefer to work on using my preferred tools.” For example, I prefer a mix of static and dynamic typing. Even for performance optimization where technically I do all four of the options, trying to write performant code from the start is what I prefer to do when possible. This isn’t about the right tool for the job as much as what kinds of tools to you prefer to work with when given the choice.
- fuzztester 1y ago
  yes, the mutually exclusive answers (radio buttons) are a quiz smell. they could have used a mix of radio buttons and check boxes, per question, as appropriate, i.e. check boxes for questions where more than one answer is applicable, and radio buttons for the rest.
  - Waterluvian 1y ago
    And that I can see being an interesting survey. It becomes more about “what’s kinds of problem solving tools do you enjoy using?” I could easily answer that. I love solving the kinds of problems that call for strong static typing and careful specifications and unit testing. I also love the opportunity to “whip up a script” in what feels like a hacker speedrun. I absolutely adore teaching newcomers intro to programming. Holy crap the glimmer in their eyes when they grok what this opens them up to… I skip past all the “computer science” and jump into making a small game and sharing it with friends via web. Or even just Autohotkey to show them how they can become a hacker of their own computer habits.
rk06 1y ago
pick what you would choose by default. in case of multiple options, pick the first option you would try. it is not a DSA test, but more like intuition test
jll29 1y ago
That's why I hate (and never use in exams) multiple choice questions.
- xigoi 1y ago
  These questions would be better if they were multiple choice. The main problem is that they are single choice.
keepamovin 1y ago
I’m that way with personality tests haha but this one i screwed down: Abstract ↔ Concrete: -13 Concrete Human ↔ Computer Friendly: +7 Human-Friendly
mhh__ 1y ago
"it depends on context" is a personality trait
ethan_smith 1y ago
The "it depends" instinct is actually the hallmark of an experienced engineer - context-awareness is precisely what separates dogmatic programming from effective problem-solving.
hiAndrewQuinn 1y ago
Avoiding questions just because it depends on context might be a valuable way to signal to other people that you're careful and considerate, and that definitely has value. But quizzes like this are explicitly designed to be contextless. You're supposed to answer with your gut feeling, the first step in the random walk. Someone who actually depends heavily on context and doesn't have a strong preference either way will answer quickly, and end up near the center on all dimensions of interest - that's where I landed: """ You value clarity and directness in code. You prefer explicit, step-by-step solutions that are easy to understand and debug, even if they require more lines of code. Abstract ↔ Concrete: -2 Concrete Human ↔ Computer Friendly: +5 Human-Friendly """
- baq 1y ago
  Experienced engineers will have a gut feeling of ‘it depends’ and if there is no extra context given the question is pointless and any answers are useless.
  - hiAndrewQuinn 1y ago
    It's not useless at all. We expect people whose answer to everything is "it depends" to answer more or less at random, when they don't have enough information to say otherwise. That is an extremely telltale signature. Put another way, if your assertion is right, then a simple 5 minute quiz like this should be enough to rule out people who are claiming to be senior, but who actually arrive at extreme answers. A 5 minute quiz like that would be worth tens of millions in improving hiring practices. So either we're all leaving millions of the floor by not building a company around this, stat, or your assertion is just wrong. There can indeed be senior engineers who are nonetheless very principled even in low-context situations.
    - baq 1y ago
      It’s an easy 5 min screen at most and it’s common knowledge anyway which you can easily fake by answering ‘it depends’ and asking basically random questions. Good enough additional context turns the quiz into a systems design interview, which serves your purpose obviously.
hoistbypetard 1y ago
I had that feeling about several questions. The one that stood out to me was """ When debugging, I typically: * Write tests to isolate the problem * Reason about the code logically first * Add print statements to understand data flow * Use a debugger to step through code systematically """ and I typically do all 4 of those things. If I don't understand the dataflow yet, I'll start with either print statements or the debugger to understand that. If it's code where I already understand the dataflow, and I can reason about the code already, I'll do that. Otherwise I might first write tests to help me reason about the code. But I generally do all of these things and the order depends on my specific problem.
- bogomog 1y ago
  Yeah, that stood out to me as a virtual tie also. When I got pretty good at debugging was when I began to recognize which of these approaches would likely be most effective in a given situation, and when to back out and use another approach, or mix approaches.
mannykannot 1y ago
I agree, but would add that it is just one of its problems. Even before we get to the questions, we are told that one of the outcomes will be a score on a “friendliness” axis. What would it mean to be computer-friendly, and why would it matter?

arjonagelhout 1y ago

I have the same issue. For many of the questions my answer is "all of the above, but A in context A, B in context B, etc.". Many are also not mutually exclusive. Take this example: "When debugging I typically:" > Write tests to isolate the problem In the case of math functions, or more primitive building blocks, writing tests can help ensure correctness of the underlying functions, to exclude them from the problem search. > Reason about the code logically first This is always useful. > Use a debugger to step through code systematically Useful when dealing with a larger codebase and the control flow is hard to follow. The call stack can give quick guidance over trying to manually decipher the control flow. > Add print statements to understand data flow: Useful when debugging continuous data streams or events, e.g. mouse input, where you don't want to interrupt the user interaction that needs to be debugged.

thierrydamiba 1y ago
Is the fact that the answer isn’t easy a feature or bug?
- teapot7 1y ago
  Oh a bug, surely. How is the quiz supposed to give you insights when it's unanswerable?
  - thierrydamiba 1y ago
    By forcing you to make a decision without context. Similar to how when presented with the trolley problem some will ask many follow up questions about the individuals on each track, the train, etc. That’s not the point.
    - shadowfox 1y ago
      > By forcing you to make a decision without context. Not the OP, but what would be the point to that? In any practical scenario there is always context, isnt it? I guess I don't quite get what we are trying to measure here.
    - mannykannot 1y ago
      But here, the context is relevant, and the analogy does not change this.
- arjonagelhout 1y ago
  I think it’s a bug in getting a useful outcome, but it’s a feature in creating engagement with the post.
- mannykannot 1y ago
  It is not primarily a matter of difficulty. The problem with choices like these is that they seem to be predicated on a rather simplistic, formulaic view of software development - a view that experienced developers will recognize as flawed. In this case, one tacit assumption is that a given developer will typically adopt just one of these approaches. Another is that they can meaningfully and objectively be ordered along either of the axes purportedly being measured.

IgorPartola 1y ago

I took it as “when working in the kinds of things things I prefer to work on using my preferred tools.” For example, I prefer a mix of static and dynamic typing. Even for performance optimization where technically I do all four of the options, trying to write performant code from the start is what I prefer to do when possible. This isn’t about the right tool for the job as much as what kinds of tools to you prefer to work with when given the choice.

fuzztester 1y ago
yes, the mutually exclusive answers (radio buttons) are a quiz smell. they could have used a mix of radio buttons and check boxes, per question, as appropriate, i.e. check boxes for questions where more than one answer is applicable, and radio buttons for the rest.
- Waterluvian 1y ago
  And that I can see being an interesting survey. It becomes more about “what’s kinds of problem solving tools do you enjoy using?” I could easily answer that. I love solving the kinds of problems that call for strong static typing and careful specifications and unit testing. I also love the opportunity to “whip up a script” in what feels like a hacker speedrun. I absolutely adore teaching newcomers intro to programming. Holy crap the glimmer in their eyes when they grok what this opens them up to… I skip past all the “computer science” and jump into making a small game and sharing it with friends via web. Or even just Autohotkey to show them how they can become a hacker of their own computer habits.

rk06 1y ago

pick what you would choose by default. in case of multiple options, pick the first option you would try. it is not a DSA test, but more like intuition test

jll29 1y ago

That's why I hate (and never use in exams) multiple choice questions.

xigoi 1y ago
These questions would be better if they were multiple choice. The main problem is that they are single choice.

keepamovin 1y ago

I’m that way with personality tests haha but this one i screwed down: Abstract ↔ Concrete: -13 Concrete Human ↔ Computer Friendly: +7 Human-Friendly

mhh__ 1y ago

"it depends on context" is a personality trait

ethan_smith 1y ago

The "it depends" instinct is actually the hallmark of an experienced engineer - context-awareness is precisely what separates dogmatic programming from effective problem-solving.

hiAndrewQuinn 1y ago

Avoiding questions just because it depends on context might be a valuable way to signal to other people that you're careful and considerate, and that definitely has value. But quizzes like this are explicitly designed to be contextless. You're supposed to answer with your gut feeling, the first step in the random walk. Someone who actually depends heavily on context and doesn't have a strong preference either way will answer quickly, and end up near the center on all dimensions of interest - that's where I landed: """ You value clarity and directness in code. You prefer explicit, step-by-step solutions that are easy to understand and debug, even if they require more lines of code. Abstract ↔ Concrete: -2 Concrete Human ↔ Computer Friendly: +5 Human-Friendly """

baq 1y ago
Experienced engineers will have a gut feeling of ‘it depends’ and if there is no extra context given the question is pointless and any answers are useless.
- hiAndrewQuinn 1y ago
  It's not useless at all. We expect people whose answer to everything is "it depends" to answer more or less at random, when they don't have enough information to say otherwise. That is an extremely telltale signature. Put another way, if your assertion is right, then a simple 5 minute quiz like this should be enough to rule out people who are claiming to be senior, but who actually arrive at extreme answers. A 5 minute quiz like that would be worth tens of millions in improving hiring practices. So either we're all leaving millions of the floor by not building a company around this, stat, or your assertion is just wrong. There can indeed be senior engineers who are nonetheless very principled even in low-context situations.
  - baq 1y ago
    It’s an easy 5 min screen at most and it’s common knowledge anyway which you can easily fake by answering ‘it depends’ and asking basically random questions. Good enough additional context turns the quiz into a systems design interview, which serves your purpose obviously.

hoistbypetard 1y ago

I had that feeling about several questions. The one that stood out to me was """ When debugging, I typically: * Write tests to isolate the problem * Reason about the code logically first * Add print statements to understand data flow * Use a debugger to step through code systematically """ and I typically do all 4 of those things. If I don't understand the dataflow yet, I'll start with either print statements or the debugger to understand that. If it's code where I already understand the dataflow, and I can reason about the code already, I'll do that. Otherwise I might first write tests to help me reason about the code. But I generally do all of these things and the order depends on my specific problem.

bogomog 1y ago
Yeah, that stood out to me as a virtual tie also. When I got pretty good at debugging was when I began to recognize which of these approaches would likely be most effective in a given situation, and when to back out and use another approach, or mix approaches.

mannykannot 1y ago

I agree, but would add that it is just one of its problems. Even before we get to the questions, we are told that one of the outcomes will be a score on a “friendliness” axis. What would it mean to be computer-friendly, and why would it matter?