Is psychological assessment coming to a school near you?
The Every Student Succeeds Act (recently replacing No Child Left Behind as Federal law in the US) provides a number of changes. Perhaps most interestingly, it requires schools to assess at least one non-academic outcome.
Some schools will soon begin testing their students for desirable personality traits such as optimism, empathy, and self-control.
Why are some of the most prominent researchers in the field warning us that this is a bad idea?
It’s not because they’re against psychological assessment.
It’s because we already know 4 big mistakes that are likely to be made.
(Not coincidentally, these are the very same mistakes we already see far too often with some pre-employment assessments.)
1. Circular reasoning
It is easy to confuse what something is supposed to measure, with what it actually measures.
For example, this is a notorious problem with IQ tests.
- What do IQ tests measure? Intelligence.
- What is intelligence? Your IQ score.
Even though many of these tests have been scientifically validated, we can’t just take the results at face value.
Having test anxiety doesn’t literally mean you’re less intelligent (at least not in any other context). Taking a short practice test to boost your score doesn’t literally cause an increase in intelligence.
We need to understand the context. For example, research has already shown that students in more challenging school environments rate themselves more negatively on items such as “I am a hard worker” and “I can do anything if I try.”
Comparison between students in the same challenging environment are still valid, but comparing them to students in a different environment probably isn’t meaningful.
2. Confusing assessment with testing
When previous evidence has shown that something is generally “good” to have, it can be tempting to take that at face value.
We want our students to have optimism, empathy, and self-control. It sounds right, and seems to make sense.
Unlike the scientific research it’s based on, an assessment can become a test with right and wrong answers. More is better.
But it would be foolish to just assume that more is better. Even when research has established that people vary in meaningful ways—and perhaps some enjoy better outcomes than others—it rarely tells us exactly how or why.
This problem follows directly from testing instead of assessing.
When an assessment has right or wrong answers, it’s usually pretty obvious which are which.
Imagine being asked to agree or disagree with statements such as “I am a hard worker” and “I can do anything if I try.”
It is incredibly easy to cheat this type of assessment.
Whenever results are tied to a valuable outcome such as school funding, students and teachers will probably be motivated to provide the “right” answers, regardless of how anyone really thinks or feels.
These responses wouldn’t be particularly meaningful or useful for anyone.
4. Uninformed decisions, unclear outcomes
When we make too many assumptions:
- We don’t know if we’re really measuring what we think we’re measuring
- We don’t know if more is really better
- We don’t know if people are responding in a meaningful way
None of this prevents bad decisions from being made.
For example, we mentioned earlier that students in more challenging schools disagree more with statements such as “I am a hard worker.”
If we didn’t know any better, we might conclude that we need to make these already hard-working students work even harder.
Or, for example, imagine we wanted to make kids more optimistic. Even if we assume that more optimism is better, we have no idea what interventions might work, or what outcomes might result. Could it actually make things worse? If we make kids too optimistic, will it also increase risky behaviors or make negative events in their lives seem extremely devastating?
And if people are motivated to cheat the assessment, we might end up making any number of decisions based on misinformation. This is perhaps especially bad if the results mask a real problem that could be fixed.
We have seen all these mistakes before
It happens whenever people use assessments with untested assumptions.
As we have already discussed, these problems can be avoided by using a data-driven approach to assessment.
Hopefully our schools will learn this lesson sooner than later.