It seems to me that the input data for the study is "just" the final report from the hackers to the customer. The academic researchers (who are presumbably nowh

It seems to me that the input data for the study is "just" the final report from the hackers to the customer. The academic researchers (who are presumbably nowhere near the level of expertise of the hackers) then annotate and categorize the conceptual tasks behind each word and sentences in the report. It seems to me that a lot of bias on the input data, based on the annotators knowledge. They try to account for this by using multiple (n=7) annotators, but I doubt if that is enough. Two questions come to my mind: 1) What level of detail do the final reports contain? I have procured and read a few pen testing reports myself, and the level of technical detail seemed too low to infer the hour-by-hour activities of the hackers in any meaningful way. Would be nice if the paper explained what those reports actually contained 2) I wonder what it would take to get the hackers themselves to keep a diary/journal of the hour by hour activities. That would remove a lot of noise from the input data.