3 ms·
Here's the study: http://chrisparnin.me/pdf/stress_FSE_20.pdf http://chrisparnin.me/pdf/stress_FSE_20.pdf They asked students to find the "Longest Substring Wi
by jefftk 6y ago
Here's the study: http://chrisparnin.me/pdf/stress_FSE_20.pdf http://chrisparnin.me/pdf/stress_FSE_20.pdf
They asked students to find the "Longest Substring Without Repeating Characters".
For the formats: "We created two interview settings: public setting and private setting (see Figure 1). In the public setting,
wearing an eye-tracker, participants solved the technical interview problem in the presence of an experimenter. The participant was
instructed to talk-aloud while solving the task, which is a standard
practice in technical interviews [28, 47]. The lab space contained a
whiteboard at the front of the room. An experimenter was situated
near the participant for the entire session without interrupting
participants’ thought process. If a participant asked for clarifica-
tion, the response was brief. For example, if a participant asked,
“Does this look correct”, the experimenter was instructed to reply,
“Complete to the best of your ability”.
In the private setting, participants solved the technical interview
problem in isolation. Participant were provided a private room with
a whiteboard. Participants were informed that the experimenter
will step out the room and will not return while they are solving
the problem. Thus, they had to make sure that there is no questions
left for them. When the participant was ready to begin the task,
they wear the eye-trackers, the door was closed, and the participant
worked in privacy, until the task was completed within allotted
time. After the experiment, we clarified with participants if they
had any uncertainty about the problem they just solved."
They ran a randomized controlled trial on the effect of having another person in the room, which is interesting. It does not, however, show that these interviews fail to assess software skills.
This is also essentially the worst case for having another person in the room: they are completely superfluous, just watching you, not helping, not communicating, not collaborating. Not even saying things that might make you more at ease.
- PragmaticPulp 6y ago> They ran a randomized controlled trial on the effect of having another person in the room, which is interesting. It does not, however, show that these interviews fail to assess software skills. The headline of the linked article is indeed heavily editorialized. There were differences between the two groups, but the study didn't conclude that the interview was only assessing anxiety. > This is also essentially the worst case for having another person in the room: they are completely superfluous, just watching you, not helping, not communicating, not collaborating. Not even saying things that might make you more at ease. That stood out to me, too. In an attempt to remove the influence of the observer from the equation, they simply put a silent observer standing over the person's shoulder. In the example photo, they have an observer standing almost literally over the person's shoulder, silently, observing their every move. It's not clear if that's how the experiment was conducted or if it's just an example photo, but it's not how any real world whiteboard interviews are performed. Even the study participants noted how strange the situation was: "P25 felt unnerved that someone was “watching me and they are not commenting on my progress”
- 1qaz2wsx3edc 6y agoWearing an eye tracker creates one hell of a different context. Being ask to solve a problem without your typical tools does as well. I assume these tests don't actually get to the root of the issue but only represent a portion of behaviour.