The final installment
This post is the final part of a three-part blog-post about the BEM and how Argyle Analytics is using the BEM to evaluate employee perceptions about the drivers of performance in their workplace.
In the first two posts, I introduce some of the basic ideas behind performance consulting as well the theoretical foundations of the BEM. In this post, we're going to spend some time talking about Roger Chevalier's work that helped move the BEM forward as a performance consulting tool.
Roger Chevallier, may you rest in peace
Roger Chevalier departed us earlier this year on April 30th, 2018. While his loss saddens me, I'm also grateful the world had an opportunity to benefit from his contributions. A scholar, a U. S. Coast Guard officer, and a well-respected business consultant, Roger was a real Renaissance man. He was also a nice guy. I've met Roger a hand full of times, and the last time I saw him before his passing was in Philadelphia. I was a committee chair of the 2017 ISPI conference in Philadelphia, and I was listening to him, and Roger Addison (one of his contemporaries) talk about their careers. As more highly tenured members of any profession are wont to do, the Rogers were reflecting on their past accomplishments, trials, and some hard lessons learned. Roger Chevallier had been instrumental in standing up a large portion of the U. S. Coast Guard training system...like, a lot of it. One of my first jobs after graduating from Boise State University was evaluating one part of the system he helped build in the form of the recruit training programs at the U. S. Coast Guard Training Center in Cape May, NJ. In listening to him talk about the work he did around helping design parts of the program I was able to quip that he got to do a lot of the fun stuff while I just got to monitor what he built.
Roger's legacy was a big step forward for training consulting
Roger was a smart guy, and he was one of the authors my professors assigned for reading in graduate school. One of his thoughts that struck me as foundational when I was in school was the notion that Gilbert's BEM factors could be prioritized to help identify solutions that produce more value relative to others. Throughout his career, Roger theorized that the first most important factor to address is the ability for employees to access information and feedback on their work. For example, in the Coast Guard, there is a standard operating procedure (SOP) for just about everything. These procedures are written down, so they are referenceable, but before the advent of the Internet, ensuring access to the latest version of an SOP was a real challenge. These SOPs govern everything from how to properly tie down a ship to how to refit a security cutter.
In many cases, the Uniform Code of Military Justice requires adherence to these SOPs, so access isn't just a performance issue; it's one that may carry legal consequences. Even with the Internet, accessing SOPs can still be a challenge since there are dozens upon dozens of SOPs, many over 100 pages long, and at the time I worked with the Coast Guard in 2009 these PDFs were not searchable in a manner we would call "searchable" today. You had to know what the contents of a PDF were from graduating from one of several Coast Guard training programs and use the system as a means of downloading an electronic copy of the SOP. It just goes to show that there will always be another hurdle even after we make it over the one right in front of us.
How did Roger prioritize BEM factors?
Roger had a theory that the factors of the BEM were not all weighted equally. Context plays a big part, but there are also some practical things to consider when looking for cost-effective learning interventions. For Roger, the BEM was more of a tool to think with, a taxonomy for organizing data as you analyze a client's situation and taxonomy for organizing performance improvement interventions (e.g., better job aids). However, even after you have all the analysis to tell you what the root cause of a performance problem is, and some sense of what solution will address the issue, we all must make trade-offs to accomplish what we can with the resources we have. Make no mistake, the challenges with prioritizing where to invest your training budget are real. Where do you spend your limited training budget first? Invest in computer skills or leadership development, eLearning vs. classroom training, new employee orientation vs. or training to bolster the capabilities of tenured employees? The people making these decisions also face potential career consequences if they misstep. I agree with Roger's assessment that the BEM needed some refinement to make it more actionable and when I read his take on Gilbert's work it was something of a revelation.
Roger Chevalier suggested that if you think of performance results as a see-saw with the BEM factors on one side and "performance results" on the other you'll have a good metaphor to explain the relative weight of each factor in driving performance. Roger arranged the BEM factors according to their impact and cost or effort.
Roger posits that the performance improvement solutions associated with factors furthest from the fulcrum would be the most impactful and cost-effective. Remember, the closer you are to the fulcrum of a see-saw, the more effort is required to raise the other side. The metaphor is a good one since some training interventions will be more impactful than others depending on the context of the performance problem and anything that improves performance for less cost is a no-brainer business decision.
Roger's rank ordering of the BEM's factors is as follows with "Information & Feedback" interventions being the most impactful and cost-effective.
Information & Feedback
Tools & Resources
Incentives & Benefits
Skills & Knowledge
Much to my surprise when I read Roger's work, training interventions were not the first place we should look to deploy an intervention, but the last. From Roger's perspective, one should consider the applicability of the interventions associated with these other factors before one should consider training. I'm sure many of my readers will agree that this seems counter-intuitive advice, but only if you think of Roger as a "training consultant" rather than a management consultant. This is where the line between training consulting and management consulting start to blur. In many cases, root cause with access to information or feedback stems from management decisions and practices with overlap into investments in the tools and resources their employees need. This is one of the reasons under-investing in management and leadership training is a sure sign of bad things to come. It is a well-known phenomenon in the training world that management is often the single biggest hurdle to performance improvement for any number of reasons. These hurdles are often due to a lack of follow-through from managers after training to hold their people accountable for using the training. In some rare and unfortunate cases, bosses will even play the role of a bad actor and actively block the application of new skills and knowledge. Any business looking to invest in the professional development of their employees should seek to invest in their management teams first in order to make the training investment is fully realized.
The Workplace Probe Study
Roger's work was a game changer for me. Not just because it helped me see a way forward when working with clients struggling to prioritize their training investments, but because of Roger also provided something of a scorecard along with his theory to help performance technologists in evaluating workplace performance issues with the BEM. Over the tenure of Roger's career, during which he performed many a performance analysis, he started recognizing patterns and commonalities with the work environments he analyzed. He distilled those insights into a series of questions which we at Argyle Analytics turned into a self-report survey and are using to collect data.
We intend to continue collecting this data in perpetuity so we can try and explore connections between some of the demographic dimensions we're studying and the specific elements of the performance environment Roger advises training consultants to evaluate in their work. We're asking all our survey participants to kindly consider their working situation and play amateur analyst when it comes to their work environment. We expect participation to be modest at first, but we hope to see it pick-up over time.
Why should I participate in the Workplace Study Probe?
Well, that's a personal question, but to try and sweeten the deal I will send you an updated report a couple of times a year (the goal is to get up to quarterly). We believe this report will yield some impressive results over time about the degree to which employees perceive different factors influencing their on-the-job performance. It's important to note there are some inherent limitations with this study. First, we're using a convenience sample so we should be cautious about any conclusions we draw. Second, we are asking people for self-perceptions. Self-report bias is a well-known phenomenon where survey takers evaluate themselves a bit more favorably than is warranted. We are also using a simplified scale with only four points and/or no mid-points. I'm collecting data in this format since we are not conducting any parametric analysis looking for correlations or regressions. Since frequency counts and percentages are the most complicated stats we will run, we don't need scales that allow for robust measures of variability. In short, we want to use this survey to understand how employees perceive their work environment through the lens of the BEM and without the life and work of Roger Chevalier, we wouldn't have this opportunity to do so.