Whilst we continue to collect live candidate data on the results, our theoretical position and original drivers were very clear:
- Multi-construct – why we assess more than one construct in one assessment
First, face validity, that is, the extent to which the assessment feels like a true reflection of the job itself.
Our view is that the nature of work is multi-faceted. We rarely work on one thing in isolation or use one skill at a time to achieve a task. Work itself is not one dimensional, so why would an assessment of workplace performance be?
Our aim with blended assessment was to get to the heart of multi-faceted human behavior and simulate work in the most real way possible.
Bringing together multiple psychological constructs and task requirements, in the same way we utilize a blend of strengths, cognitive abilities and emotion in the workplace, makes the assessment ‘true’ to real life. This also emphazises the authenticity that is at the core of our assessment design and candidate experience.
Second, incremental validity, that is, the extent to which different criteria contribute to the prediction of long-term job performance.
All meta-analyses show that we obtain increases in incremental validity when additional constructs are added to a prediction model.
This has been validated through our learning of the hiring tool Koru7 and our wider analysis working with blended assessment in Capptivate. Less, in fact, is not always more, especially in the case of predictive assessment: combining assessment constructs and assessment types leads to greater predictive validity for the specific requirements of a role.
Third, fairness. It is well documented that minority demographic groups typically perform lower on specific types of assessment, that is, cognitive aptitude tests (Cottrell et al., 2015).
It follows that in mainstream recruitment processes, employers report a barrier to diversity through including cognitive assessments in their assessment flow, whilst needing to assess for these constructs as part of the role specification.
By replacing sequential, multi-stage processes with one-time measurement of all criteria, we mitigate losing minority groups at a specific “stage” (such as cognitive testing).
Blended assessment enables employers to take a more holistic view of a candidate’s suitability based on all relevant job-criteria, and with robust test design, whilst alleviating levels of adverse impact due to testing error or the adverse impact seen in traditionally designed cognitive aptitude tests.
- Multi-method – why we include a combination or blend of different assessment types in one assessment
First, construct validity, that is, the robustness with which the assessment measures particular constructs. In the same way that we apply multi-method techniques to other areas of assessment (such as job analysis or assessment centers), the more varied the opportunities to gather evidence of a construct, the more thorough the overall measurement of that construct.
Second, criterion-related validity, that is, the extent to which the assessment goes on to relate to future job performance. It is well acknowledged that certain constructs are best assessed with specific item-types, and that this increases validity (Lievens and Sackett, 2017).
Our work with blended assessment utilizes a wide range of item-types to ensure the constructs within a success framework are measured in the most appropriate and accurate way possible. This is also the focus of ongoing research, as we continue to explore, validate and improve the best ways to assess specific constructs for predicting future job performance.
Finally, face validity. Consistent with our belief that work is multi-dimensional, using a variety of item-types within an assessment increases the authenticity and face validity of the test to the candidate. In real life, we work with a multitude of information sources, question types and challenges. Mixed assessment items best reflect this reality and provide an authentic assessment-taking experience to the candidate.
What impact is this having with clients?
We originally released our early research into the blended assessment approach in 2017, discussing in Assessment and Development Matters, that the ‘Integration of multiple assessment types can increase fairness and candidate experience.’
We now have a further three years of data into the impact of this approach.
In summary, our aggregated data shows that utilizing a blended approach to assessment:
- Increases diversity – We have observed effect sizes between demographic groups decrease using a blended approach compared to a single-method approach
- Increases face validity – Candidates consistently tell us their assessments feel more relevant, true to life and increase autonomy compared to traditional forms of assessment
- Increases efficiency whilst maintaining quality – Blended assessments are typically time-compressed, and our data show still deliver the same size of predictive relationships with outcome measures of interest
We will release further validation studies of our work in this new method of blended assessment this year and beyond, with a particular focus on longitudinal outcomes and prediction of job performance over time.
What more do we still need to learn?
There is still a lot to be explored and discovered about this new style of assessment.
Cappfinity’s ambition is to ensure the expected metrics of validity and reliability, traditionally captured for single-construct instruments, are demonstrated for blended assessment. This will require new algorithms and statistical methods to be developed. This will require our industry and clients to commit even harder to robust data validation and longitudinal investigation of job performance outcomes over time – the ‘Holy Grail’ of recruitment predictive analytics.
Blended assessment presents an opportunity to dramatically enhance the robustness of the solutions we provide as psychologists, as well as challenging the pitfalls that still exist with traditional single-construct assessment. Assessment innovation is in a rich vein, and the opportunities presented by advances in data science, AI and machine learning will all help shape this predictive people analytics world of the future.
