4 ms·

Sure, the ability to form generative models is important, but it's at least as important to be able to change or adapt them when faced with contradictory eviden

Sure, the ability to form generative models is important, but it's at least as important to be able to change or adapt them when faced with contradictory evidence. The "CS IQ" test in the article bothered me because it rewarded premature optimization (forming a generative model and sticking to it even when you have no reason to believe it's right) instead of what I would consider a more optimal strategy (recognize that you don't know how assignment works, hypothesize about the possible ways it could work, and use a different model of assignment for each question, thus hedging your bets for partial credit).

deleted 18y ago
[deleted]
LogicHoleFlaw 18y ago
Adapting your model in the face of new evidence is part of the definition of a model-seeking approach to problem-solving. I don't think that the article is implying that sticking to a particular model when confronted with contradictions is beneficial. I think you are also using a different definition of "optimal strategy" than the author. The test was not measuring test-taking strategy - it may not even have been scored in a student-visible way. In the absence of a score-optimizing approach the natural inclination of the student's analytic style should come to the forefront. Additionally, the test had predictive power regardless of a root cause. Students who used a consistent model in selecting answers went on to succeed in the programming course, and the others did not. The underlying cause may actually be different but the concrete result stands.
- seregine 18y ago
  You're right, I'm not arguing against the article so much as trying to continue its thought. I agree that generative models are useful. Among people who use them, I suspect that the next major predictor of success would be adaptability. Confirmation bias is the opposing force: once we form a model, we are biased towards it, and tend to overlook contradictory evidence. One tactic that increases adaptability is not forming a model until you actually have to; this avoids confirmation bias. You're also right about the nature of the test: as an experiment, it does show that presence of generative models predicts success. I'm interested in education, so I'm voicing the reason why this wouldn't be a practical test of qualification.
  - shevertalov 18y ago
    The bias point is well made. I don't remember the quote exactly, but i think it's something like "...when all you know are hammers, everything looks like a nail...". Though I think his point is that you have to at least be able to form models about the problems you tackle and stick to those models. I feel like you need some commitment or vision to be able to move on. Development tends to have two phases, exploration and exploitation. Balancing the two is a real problem. If you exploit too soon you may miss a much better solution, but if you do too much exploring you may just be wasting time. Balancing the two contributes to writing good code it seems.

deleted 18y ago

[deleted]

LogicHoleFlaw 18y ago

Adapting your model in the face of new evidence is part of the definition of a model-seeking approach to problem-solving. I don't think that the article is implying that sticking to a particular model when confronted with contradictions is beneficial. I think you are also using a different definition of "optimal strategy" than the author. The test was not measuring test-taking strategy - it may not even have been scored in a student-visible way. In the absence of a score-optimizing approach the natural inclination of the student's analytic style should come to the forefront. Additionally, the test had predictive power regardless of a root cause. Students who used a consistent model in selecting answers went on to succeed in the programming course, and the others did not. The underlying cause may actually be different but the concrete result stands.

seregine 18y ago
You're right, I'm not arguing against the article so much as trying to continue its thought. I agree that generative models are useful. Among people who use them, I suspect that the next major predictor of success would be adaptability. Confirmation bias is the opposing force: once we form a model, we are biased towards it, and tend to overlook contradictory evidence. One tactic that increases adaptability is not forming a model until you actually have to; this avoids confirmation bias. You're also right about the nature of the test: as an experiment, it does show that presence of generative models predicts success. I'm interested in education, so I'm voicing the reason why this wouldn't be a practical test of qualification.
- shevertalov 18y ago
  The bias point is well made. I don't remember the quote exactly, but i think it's something like "...when all you know are hammers, everything looks like a nail...". Though I think his point is that you have to at least be able to form models about the problems you tackle and stick to those models. I feel like you need some commitment or vision to be able to move on. Development tends to have two phases, exploration and exploitation. Balancing the two is a real problem. If you exploit too soon you may miss a much better solution, but if you do too much exploring you may just be wasting time. Balancing the two contributes to writing good code it seems.