Transform Your Workplace with Agile Performance Management Powered by Data

In the digital age of the twenty-first century, everything can be more data-driven, and agile thrives on data. However, when data includes feedback, which is notoriously subjective, care must be taken to avoid amplifying systematic errors inherent to measuring feedback. Performance management software is invaluable in using data but comes with the caveat of understanding where its analytics may fall short.

The Continual Aspect of Everything Agile

It’s important to remember how the agile approach focuses on continual feedback. I’ve previously explained why the operative word here should be “continual” as opposed to “continuous,” but most everything you read about the agile philosophy uses the word “continuous” even though it’s technically not the right word. Still, I do like the way one writer explains how important this concept is to the agile approach, in this case, to project management:

“…in Agile management, the password is ‘continuous.’ Continuous integration, continuous delivery, continuous improvement, continuous testing, continuous-whatever-you-want. In other words, the iterative organization of work and the incremental construction of the result characterize a whole range of elements in a ‘continuous’ or ‘quasi-continuous’ way, from planning to monitoring and control of progress, from moments of dialogue and communication to the collection and application of lessons learned… This principle also applies to the collection and analysis cycle of data to support decisions, which is carried out on two loops grafted into one another… Data collection, analysis, and possible adjustment actions are performed and discussed by the agile team daily (Daily Stand-up Meeting) and at the end of each work iteration (Review and Retrospective Meeting). The actions are restricted to a more limited scope (aka, only tasks processed during the iteration) but with a much higher “sampling rate,” which is what ultimately determines the true “agile” nature of this type of approach” (source).

How does this kind of continual data collection apply to agile performance management? It doesn’t seem practical or feasible to collect, analyze, or discuss performance feedback data for individual employees every day. Still, it could happen within the timeframe of the shortest goal timeline in use. Our previous article about goal setting for agile performance management suggested thinking of short-term, medium-term, and long-term goals in buckets. Short-term performance goals at many places will likely be monthly, although I think an argument could also be made for weekly performance goals in the short-term bucket. In this sense, the daily stand-up meeting in an agile project could become a weekly stand-up meeting related to individual employee performance.

Deloitte conducted exhaustive research on their performance management scheme and landed on weekly stand-up meetings between team leaders and individual team members: “These brief conversations allow leaders to set expectations for the upcoming week, review priorities, comment on recent work, and provide course correction, coaching, or important new information. The conversations provide clarity regarding what is expected of each team member and why, what great work looks like, and how each can do his or her best work in the upcoming days—in other words, exactly the trinity of purpose, expectations, and strengths that characterizes our best teams” (source). Note that these weekly check-ins aren’t “in addition to” the other work of a team leader, they are the work.

Whichever way it works out in more frequent performance management meetings with employees, it will only be effective if the goals are spelled out so that they can be easily translated into metrics for measurement, data collection, and data analysis. This should bring to mind the “measurable” piece of the SMART goal puzzle.

Beware the Dangers of Data

Anyone who was around in the early days of personal computers and software coding becoming more widely available may remember this simple mantra: Garbage in, garbage out, or GIGO. It means the quality of any analytical output is directly tied to the quality of the input. In computer programming, flawed input results in flawed output. In data analytics, if bad data is used, a bad analysis will result, leading further to bad decisions based on the lousy analysis. This has become even more important in recent years with the increasing reliance on machine learning and artificial intelligence (AI). A machine learning or AI algorithm must be trained on data. If the data used to train the algorithm is of poor quality, then its performance will also be poor or result in unintended outcomes.

This becomes an issue of significant concern if the training data used is biased. And there are many instances of showing how data with baked-in biases perpetuates those biases in machine learning and AI algorithms trained on biased data. Amazon’s resume filtering tool was biased against women because it was trained on a data set of primarily men’s resumes (source), and predictive policing algorithms can perpetuate and even amplify bias (source) in policing. I raise these concerns because in arguing for greater reliance upon data and data analytics for data-driven agile performance management, care must be taken to understand the quality of performance data and where issues might arise.

Performance Management Data: Risks and Opportunities

One problematic use of performance data is ranking employees in a forced distribution stacked ranking scheme. Despite how often this has backfired and has been shown to do more damage than good, it is still used by an unfortunate number of companies and organizations. It inevitably results in labeling some employees as underperforming when they’re doing just fine.

For a recent discussion of these issues, I’d like you to please take a look at Why Ranking Employees by Performance Backfires. Another useful related discussion about how people find higher fairness in temporal comparisons (comparing an individual’s current performance with their own past performance) rather than social comparisons (comparing an individual’s performance to that of their peers) can be found in the Harvard Business Review article, People Don’t Want to Be Compared with Others in Performance Reviews. They Want to Be Compared with Themselves.

Another area of dangerous waters is the foundational concept of rating or scoring employee performance or skills. Scientific research on this reaches an interesting conclusion: “Although it is implicitly assumed that the ratings measure the performance of the ratee, most of what is being measured by the ratings is the unique rating tendencies of the rater. Thus ratings reveal more about the rater than they do about the ratee” (source).

Deloitte solves for this “idiosyncratic rater effect” by asking managers or team leaders to weigh in on future-oriented actions toward an employee rather than what they think of the employee. It results in a less-biased assessment, the call a “performance snapshot,” than the typical ratings approach (source). Here are the four performance assessment questions Deloitte uses to get their project-based performance snapshots:

1. Given what I know of this person’s performance, and if it were my money, I would award this person the highest possible compensation increase and bonus [measures overall performance and unique value to the organization on a five-point scale from “strongly agree” to “strongly disagree”].
2. Given what I know of this person’s performance, I would always want him or her on my team [measures ability to work well with others on the same five-point scale].
3. This person is at risk for low performance [identifies problems that might harm the customer or the team on a yes-or-no basis].
4. This person is ready for promotion today [measures potential on a yes-or-no basis].

What works for Deloitte won’t necessarily work at every organization (not all companies have such a direct and robust tie-in to annual compensation). Still, it does illustrate a more thoughtful approach to collecting performance management data compared to more traditional methods by simply shifting to “…asking our team leaders what they would do with each team member rather than what they think of that individual.”

I also encourage every company and organization to spend time digging into their performance management practices and systems to discover ways in which they may be biased and get help correcting those biases. One of the best case studies I’ve read about making this happen is How One Company Worked to Root Out Bias from Performance Reviews. It’s a great illustration of the GIGO principle.

Another “required reading” about performance data, especially feedback, is The Feedback Fallacy. There’s a magnification effect that happens in data analysis when feedback inevitably falls prey to the previously mentioned idiosyncratic rater effect:

“And because your feedback to others is always more you than them, it leads to systematic error, which is magnified when ratings are considered in aggregate. There are only two sorts of measurement error in the world: random error, which you can reduce by averaging many readings, and systematic error, which you can’t. Unfortunately, we all seem to have left math class remembering the former and not the latter. We’ve built all our performance and leadership feedback tools as though assessment errors are random, and they’re not. They’re systematic.”

Following the GIGO principle again, aggregating a bunch of unreliable feedback ratings takes you even further away from learning anything reliable about the individual’s performance! In other words, relying on the ratings supplied by others is literally all noise and no signal. Any rater’s rating is only their perception unique to them, not a universal truth about the person being rated, and yet most performance management practices treat those ratings as truth. They can only be subjective, and yet we treat them as objective.

The final point to make here about feedback as it relates to performance management is to dismantle the assumption that feedback about where an employee is lacking will teach them how to improve. “Again, the research points in the opposite direction. Learning is less a function of adding something that isn’t there than it is of recognizing, reinforcing, and refining what already is.” This confirms something I’ve stated in previous articles: how important it is for performance management to play to people’s strengths, not their weaknesses, and scientific evidence about how people learn backs this point up. Focusing on a person’s shortcomings impairs their ability to learn.

Is Data-Driven Agile Performance Management Even Possible?

At this point, you may think that data-driven performance management is impossible, given all the different ways it can go terribly wrong. True, but wouldn’t you rather engage your agile performance management efforts with your eyes wide open to those dangers so you can at least be aware of them and try to address them than go into it blindly and potentially do more harm than good?

With the big data that is now generated by nearly any performance management system or app, it’s up to each company and organization to use it wisely by respecting its limitations. As you utilize technology-based tools such as a learning management system (LMS) or performance management system (PMS) for data-driven agile performance management, look for easy-to-use apps that still offer the power and flexibility you need to tailor them to your specific and ever-evolving context. eLeaP is one such LMS, and when paired with the upcoming release of an accompanying agile performance management app, it will position your company to make the most of both worlds.