Our open access paper published in The Leadership Quarterly explores how novel machine learning techniques can help advance knowledge about leadership. The techniques we propose make up two simple steps: (1) create a black-box machine learning algorithm to predict leadership emergence and (2) open the black-box to explain the outcomes. Using bleeding edge analytical techniques, our goal was to reveal which personality traits (and combinations of traits) are important for becoming a leader. Here are some important tips for those who want to emerge as a leader while avoiding black-box pitfalls:
1) Deep thinking and openness
The machine learning techniques suggest that 'Need for Cognition' - how much someone enjoys deep thinking - is an important trait. The algorithm predicted leader emergence more often for individuals scoring above average on this trait. 'Openness to Experience' is also crucial. According to the algorithm, emergent leaders are very open to experience or very consistent/cautious, but not average on this trait. Finally, the combination of Need for Cognition and Openness to Experience is important for becoming a leader. If you are high on openness, then also strive to think deeply and vice versa.
We then added gender and age as predictor variables. After adding gender and age, the black-box algorithm became more accurate at predicting leadership emergence. However, when we opened the black-box, we found the algorithm used sexist decision-rules. We still found a similar role for personality traits, but the algorithm predicted consistently lower leadership emergence for younger women. Simply put, the algorithm learned society’s 'real' bias against selecting younger women for leadership roles.
This magnification of social bias gone unchecked is extremely dangerous. If organizations blindly use black-box machine learning algorithms to select leaders without opening the black-box (which some are), then they can reinforce existing inequalities and overlook suitable leader candidates such as younger women.
The techniques we present in our paper are therefore not only important to increase knowledge about leadership, but also to prevent discrimination. This level of complexity and interpretability in terms of leadership insights is completely new territory. It is not a matter of "computer says yes or no." Leaders, instead, have control over black-box output for human-centered decision support rather than AI-centered decision making. In this eyes wide open world, AI is a valuable co-worker not a draconian overlord.