5 ms·

"I for one have no idea how to a) identify the innovations the definitively WOULDN'T have been generated by an alternative system, nor do I - b) know how to mea

"I for one have no idea how to a) identify the innovations the definitively WOULDN'T have been generated by an alternative system, nor do I - b) know how to measure the costs." Exactly. It can't be measured. The situation is political, and the academics are right not to engage in a dialog that uses the language of market economics and 'value'. The private sector market experts are going to want to use their models and supply-demand curves to justify resource allocation to obvious industry-relevant subjects. The academics are going to use history, ethics, etc to defend breadth, freedom, and security of non-obvious inquiry. The faculties need to take back control of the university. They're being bent over a barrel by their administrations.

grovulent 13y ago
Well - I'm not sure I'm as pessimistic as that... But look - if your default position on a topic is X, and it happens to be the case that there is no rational way to explain why you are wrong about X, then you can hardly be blamed for continuing to carry on believing X. So if you're right that there is nothing that can be said on this issue - then the powers that be will just steam-roll academia. How can they even play political? They won't be able to convince anyone to support them! My point is that while my intuition is that academics ARE valuable and should be given large amounts of freedom - still we don't have a choice but to believe that we can find some justification for this. We certainly won't figure it out by being defeatist about it. My position is the only rationale one on this topic - but lordy convincing a single academic of its reason is beyond me. The irony of this is painful.
yummyfajitas 13y ago
If the benefits can't be measured, why are we wasting money on them? I can also give you oodles of unmeasureable and unquantifiable benefits. Just send me a big check and then don't try to box me in with oversight or accountability. (There are of course some measurable benefits from the current system. For example: http://www.opensecrets.org/industries/indus.php?ind=W04 http://www.opensecrets.org/industries/indus.php?ind=W04 )
- rmk2 13y ago
  Excellent, alongside the Humanities, then, we can also close down physics, because, clearly, if benefits can't be measured, why waste money? Many of the "impact" arguments assume that the only victims will be the humanities, a sacrifice lots of people are very willing to make. (The original article talks, albeit not quite as strongly, of a hemorrhaging that is slowly happening in terms of funding, jobs and perspective.) However, there is some other, less obvious, collateral damage: physics, mathematics (except for applied maths), economics all share similar problems. And here are some choice quotes about the UK's REF and its "Impact" pilot study, with "Impact" being an integral part of the future score (20%, which is going to go up to 25% for the next one) that decides about research spending from Research Councils over the next couple years, due in 2014: > "Physicists had some particular concerns about the challenge of assessing impact in the manner foreseen by HEFCE. 1. The difficulty in attributing UoA-specific ‘ownership’ of impact that arises from the work of a large collaboration, either experimental (in our case, a ten-institute international collaboration working on experiments at CERN) or theoretical (an eight-institute collaboration using High Performance Computing facilities). 2. The blurry distinction between the ‘use of’ new high performance computers and the ‘stimulation of’ the development of these machines. At what point does close collaboration with a company such as IBM in the development of teraflop or petaflop computers become ‘impact’? 4. Commercial confidentiality became an issue. Even in this pilot exercise, and making use of the ‘Confidential’ tag on the submissions, we still had to remove some of the strongest material concerning cooperation with industrial companies because of confidentiality agreements. In REF itself, this would be a huge problem. It should be noted that this also contributed to the Institute of Physics giving up on a recent exercise to try to quantify the impact of Physics. It is impossible to quantify the impact of public understanding of science initiatives such as TV and radio appearances, popular lectures, books, etc at the scale of individuals or groups. Across the community, such work clearly has an important impact, but this cannot be sensibly quantified." (p. 29) > "Several respondents noted that research impact was most readily observed and measured where the distance between the two realms, research and impact, was relatively short and the pathways rather direct: for example, applied scientists working in collaboration with a private business seeking expressly to translate the generic insights into commercial advantage. The socio-economic impact of more fundamental research might go unnoticed and unremarked most of the time, reflecting the diffuse, cumulative and rather unpredictable nature of intellectual advances. Indeed, one might imagine that the occasion where a fundamental breakthrough produces a relatively immediate and direct social improvement is rare, even anomalous. In practical terms, this means the impact case studies track back to particular types of people and types of work. It might also mean that some of the most consequential socio-economic contributions will be excluded by virtue of their rather indirect link with cutting-edge research. There was a suggestion that institutions had taken the low hanging fruit, and that in some sense the portfolio of case studies was not a good representation of the breadth of research undertaken. This sentiment appears to be rooted in a general sense that impact is most obvious in some narrow areas, within subjects rather than between them, and largely unknowable, at least in practical terms, in most instances. There were also suggestions made that a proportion of the more significant, evident impacts was linked to people and activities outside the very best academic research, albeit dependent upon that work, and which would be overlooked as a result. There is always going to be an over-reliance on a small number of sub-groups, even individuals, whose research happens to lend itself to non-academic impact. The impact submission is not necessarily representative of the UoA as a whole." (p. 30) > "The biggest challenge was the need to acquire evidence to reveal the reach and significance of a given impact. There were many practical issues that stood in the way, and the response of the majority, for the pilot exercise at least, was to firstly focus on the more obvious cases and, secondly, to use whatever narratives, references and statistics that came to hand readily." (p. 31) Source: REF Research Impact Pilot Exercise Lessons-Learned Project Feedback on Pilot Submissions, November 2010 (http://www.ref.ac.uk/pubs/refimpactpilotlessons-learnedfeedbackonpilotsubmissions/ http://www.ref.ac.uk/pubs/refimpactpilotlessons-learnedfeedb...)

grovulent 13y ago

Well - I'm not sure I'm as pessimistic as that... But look - if your default position on a topic is X, and it happens to be the case that there is no rational way to explain why you are wrong about X, then you can hardly be blamed for continuing to carry on believing X. So if you're right that there is nothing that can be said on this issue - then the powers that be will just steam-roll academia. How can they even play political? They won't be able to convince anyone to support them! My point is that while my intuition is that academics ARE valuable and should be given large amounts of freedom - still we don't have a choice but to believe that we can find some justification for this. We certainly won't figure it out by being defeatist about it. My position is the only rationale one on this topic - but lordy convincing a single academic of its reason is beyond me. The irony of this is painful.

yummyfajitas 13y ago

If the benefits can't be measured, why are we wasting money on them? I can also give you oodles of unmeasureable and unquantifiable benefits. Just send me a big check and then don't try to box me in with oversight or accountability. (There are of course some measurable benefits from the current system. For example: http://www.opensecrets.org/industries/indus.php?ind=W04 http://www.opensecrets.org/industries/indus.php?ind=W04 )

rmk2 13y ago
Excellent, alongside the Humanities, then, we can also close down physics, because, clearly, if benefits can't be measured, why waste money? Many of the "impact" arguments assume that the only victims will be the humanities, a sacrifice lots of people are very willing to make. (The original article talks, albeit not quite as strongly, of a hemorrhaging that is slowly happening in terms of funding, jobs and perspective.) However, there is some other, less obvious, collateral damage: physics, mathematics (except for applied maths), economics all share similar problems. And here are some choice quotes about the UK's REF and its "Impact" pilot study, with "Impact" being an integral part of the future score (20%, which is going to go up to 25% for the next one) that decides about research spending from Research Councils over the next couple years, due in 2014: > "Physicists had some particular concerns about the challenge of assessing impact in the manner foreseen by HEFCE. 1. The difficulty in attributing UoA-specific ‘ownership’ of impact that arises from the work of a large collaboration, either experimental (in our case, a ten-institute international collaboration working on experiments at CERN) or theoretical (an eight-institute collaboration using High Performance Computing facilities). 2. The blurry distinction between the ‘use of’ new high performance computers and the ‘stimulation of’ the development of these machines. At what point does close collaboration with a company such as IBM in the development of teraflop or petaflop computers become ‘impact’? 4. Commercial confidentiality became an issue. Even in this pilot exercise, and making use of the ‘Confidential’ tag on the submissions, we still had to remove some of the strongest material concerning cooperation with industrial companies because of confidentiality agreements. In REF itself, this would be a huge problem. It should be noted that this also contributed to the Institute of Physics giving up on a recent exercise to try to quantify the impact of Physics. It is impossible to quantify the impact of public understanding of science initiatives such as TV and radio appearances, popular lectures, books, etc at the scale of individuals or groups. Across the community, such work clearly has an important impact, but this cannot be sensibly quantified." (p. 29) > "Several respondents noted that research impact was most readily observed and measured where the distance between the two realms, research and impact, was relatively short and the pathways rather direct: for example, applied scientists working in collaboration with a private business seeking expressly to translate the generic insights into commercial advantage. The socio-economic impact of more fundamental research might go unnoticed and unremarked most of the time, reflecting the diffuse, cumulative and rather unpredictable nature of intellectual advances. Indeed, one might imagine that the occasion where a fundamental breakthrough produces a relatively immediate and direct social improvement is rare, even anomalous. In practical terms, this means the impact case studies track back to particular types of people and types of work. It might also mean that some of the most consequential socio-economic contributions will be excluded by virtue of their rather indirect link with cutting-edge research. There was a suggestion that institutions had taken the low hanging fruit, and that in some sense the portfolio of case studies was not a good representation of the breadth of research undertaken. This sentiment appears to be rooted in a general sense that impact is most obvious in some narrow areas, within subjects rather than between them, and largely unknowable, at least in practical terms, in most instances. There were also suggestions made that a proportion of the more significant, evident impacts was linked to people and activities outside the very best academic research, albeit dependent upon that work, and which would be overlooked as a result. There is always going to be an over-reliance on a small number of sub-groups, even individuals, whose research happens to lend itself to non-academic impact. The impact submission is not necessarily representative of the UoA as a whole." (p. 30) > "The biggest challenge was the need to acquire evidence to reveal the reach and significance of a given impact. There were many practical issues that stood in the way, and the response of the majority, for the pilot exercise at least, was to firstly focus on the more obvious cases and, secondly, to use whatever narratives, references and statistics that came to hand readily." (p. 31) Source: REF Research Impact Pilot Exercise Lessons-Learned Project Feedback on Pilot Submissions, November 2010 (http://www.ref.ac.uk/pubs/refimpactpilotlessons-learnedfeedbackonpilotsubmissions/ http://www.ref.ac.uk/pubs/refimpactpilotlessons-learnedfeedb...)