At Omnicor we assess people every day. Assessment batteries vary, but for everyone we always assess some measure of personality and some measure of cognition. We also commonly look at complexity, skills, integrity and anything else that could determine whether the person is a good fit for the role. Over time we aggregate the results of thousands of people and this allows for comparisons and correlations across tests. We were interested in what the assessment data told us if we did some analyses of the data we have.
Cognition and Complexity
We wondered what happened to Cognition and Complexity with age. It is often the case that we are assessing middle and senior managers who are middle aged. What happens to their performance on the standard tests?
Complexity (levels 1–5; Operational — Strategic)
Cognition (sten 1–10)
As is evident, age has devastating effects on performance of cognitive tests. The older you are the lower you are likely to score. Complexity levels also drop, which is less easy to explain as complexity is associated with strategic thinking which should develop with experience and wisdom. While this is probably true, the assessments used to measure complexity are only an analogue for strategic thinking and thus prone to the usual noises found in psychometric tools, the decline in processing agility being one of them. It does of course raise difficult issues, for example the manner in which we interpret cognitive scores of older candidates. Should they be normed differently ? Should we be more lenient in the acceptable ranges of good fit ? Does work experience at a senior level mitigate against the slowing of analytical sharpness? Another explanation derives from the Flynn effect. Could it be that younger generation are always going to be ahead of the curve compared to the older generation, and that the instruments themselves have a limited shelf-life? These are tricky questions and are key interpretive components that any good psychometrist will take into account.
Cognition and Social Disposition.
Who scores better on cognitive tests, Introverts or Extraverts? Think of your answer before looking at the graph below. Also think why you would think that is the case. Why should social disposition have any impact on cognition ? Well, here is what we found using the Concept Formation Test (Intermediate Level), a standard test of abstract reasoning:
The sample was just over 1000 candidates who had come for assessments, yet only 58 of these identified as Introvert (from a Personality Questionnaire), compared to 363 Extraverts (yes, most people are Ambiverts, in the middle!). This might already point to an aberration that many less introverts make it through the initial interviews. Its unclear. But one way or another the Introverts outperformed the extraverts, but just over a sten. Solidly significant. There must be some literature on that somewhere, but I trawled through the journals and didn’t find anything convincing. Given the relationship between cognition and job performance, it suggests that introverts may have the edge in this area. Not just because they are smarter, but also because they are less distracted by socialising.
Cognition and Qualification.
Most people would expect that people with higher qualifications do better on cognitive tests. The reasoning being that cleverer people are more likely to pursue tertiary education. We used the Concept Formation Test Intermediate Level, a standard test of abstract reasoning, to interrogate cognition. Interestingly, while there was a cognitive trendline that followed the predicted trajectory, the differences were not as stark as one might expect. The numbers of people in each group and the results follow:
Undergraduates are still studying, and graduates have a degree. There is almost no difference in problem solving for the 3 graduate types, and barely a difference between Grade 12 and Post-Graduates — less than a sten separates them. Clearly there is not a profound intellectual difference between the least and most qualified, though there is arguably some range restriction going on, in that those coming for assessments are intrinsically more successful — just in being selected for assessments. One might expect that in a random sample of school leavers who got no tertiary education compared to those who did, that there would be a greater demonstrated difference in cognitive score. There are almost certainly other possible explanations.
Cognition and Personality
Firstly, a word about correlations. They are a widely used statistic, but of course provide no indication of causality. Also even if a correlation is statistically significant, that just indicates that the relationship between the variables is unlikely to exist by chance, but it is the effect size,
that tells us more, and this is the proportion of one variable explained by another. In general, with correlations we can use the following guidelines:
After squaring the correlation, even relatively high correlations have a somewhat modest effect size. For example, if I know that there is a .69 correlation in the IQ score of twins raised apart, it means that genes appear to play a role of around 50%. Well that accounts for a lot of variance, but there is still a lot left to explain.
All of the findings between cognition and personality dimensions were weak. Look at the table below:
The data was based on over 1400 candidates. It is possible that despite being significant these significant results are arbitrary. The small percentage of variance explained is one indication of spurious effects, but nevertheless one could make a convincing rationale for almost any of them. Of course, you could say, more intelligent people are always going to challenge the rules. They are more alert to inconsistencies and better at joining the dots to find flaws in regulations. Similarly it makes sense that they are less conscientious. Clever people are keen to find short-cuts, they can get bored with detail, they need diversity, so they look for ways to make their jobs easier. Clever lazy people might be the best candidates to employ in roles where innovation is required. The strongest result (a mere 3% of the variance) — the more intelligent you are, the more likely you are to trust others. I don’t know why that is, maybe because when you trust people you can delegate, and what better way is there of getting someone else to do the work for you.
Conclusion
Science has been emphatic about the importance of cognition predicting job performance. No argument there. Yet cognition (as measured by psychometric tools) has a life of its own, it is higher in younger, introverted graduates, and that makes some sense. Yet we see that it changes over the course of a lifetime, much more than we might think. For all the effort goes into becoming a graduate, it doesn’t seem to dramatically improve problem solving, even though that might be expected. The results when correlating cognition and personality are mildly curious and we will continue to trawl through the ever increasing pile of data that we have, in the hope of mining for treasure and finding useful nuggets of information that improve our knowledge of the world of work.
