In episode six of the Talent Risk Assessor series of Defensible Decisions, Scott Kelly (Birmingham/Washington) and Cam Ellis (Atlanta) are again joined by Pete Bell of the firm’s data analytics team to examine how compensation data can be analyzed for potential pay disparities. Building on the performance rating patterns discussed in episode five, the speakers walk through a layered approach, starting with average pay comparisons among similarly situated employees, moving to a review of outliers on a scatterplot based on tenure and time in role, and then, finally to statistical significance testing. The discussion continues with consideration for factors such as prior experience, education, and geographic differentials that may act to explain observed differences. The episode emphasizes that this work must be conducted under attorney-client privilege, and the need for analysis refresh given expected employment movements, and that a more robust, regression-based analysis should be considered where flags remain.
Home > Insights & Resources > Podcasts > Talent Risk Assessor, 6: Pay Analysis Tools for Assessing Risk and Building a Defense
Talent Risk Assessor, 6: Pay Analysis Tools for Assessing Risk and Building a Defense
By T. Scott Kelly and Cameron W. Ellis
0:00:00
0:00:00
Transcript
Announcer: Welcome to the Ogletree Deakins podcast, where we provide listeners with brief discussions about important workplace legal issues. Our podcasts are for informational purposes only and should not be construed as legal advice. You can subscribe through your favorite podcast service. Please consider rating this podcast so we can get your feedback and improve our programs. Please enjoy the podcast.
Scott Kelly: Hey everyone. Welcome back to Defensible Decisions. I’m your host, Scott Kelly, and this is episode six in the Talent Risk Assessor series. Today we’re going to cover compensation and pay.
Here’s the question we’re going to be asking. If two similarly situated employees are paid differently, how does an employer know whether that’s a legitimate business reason or a legal risk in the making? Joining me again today, we’ve got Pete Bell and Cam Ellis. We’re continuing with the same multi-state employer we’ve been following throughout this series. Last episode, we examined performance ratings, and today we’re going to see what happens when the Talent Risk Assessor turns its attention to compensation data.
A quick note, of course, everything we discuss is informational and a dashboard trend or a statistical flag isn’t necessary enough to establish a legal violation on its own. Before we dive into the compensation analysis, I want to connect some dots from episode five. In it, we talked about performance ratings and the risks that arise when ratings aren’t consistently applied. How does that relate to what we’re looking at today, Cam?
Cam Ellis: It’s a foundation, Scott. It seems like the majority of our clients tell us they’re a pay for performance culture. So, in most organizations, performance ratings drive merit increases, bonus payouts, promotional opportunities. And if that rating process itself has adverse impact or inconsistency, or even just poor documentation, all of which we talked about in the last episode, those issues don’t stay contained. They infect compensation outcomes and then get perpetuated year after year.
Pete Bell: That’s exactly right, Cam. So, if we’re going to model pay, we know that performance ratings are frequently one of those key explanatory factors. If those ratings are skewed, they can be masking or creating pay disparities, depending on what’s impacting the data. So, we like to look at these two types of analyses in sequence, and the Talent Risk Assessor does that by flagging rating distribution concerns for the employer, and then we can see any potential consequences or echoes in the compensation data.
Scott Kelly: So, really what we’re trying to do here is to give an employer the tools and the feedback and the input through using this Talent Risk Assessor product along with our legal advice to figure out if they have an issue, and maybe they need to go, if it’s in pay, maybe it could have a connection to those upstream rating process issues. Is that right, Cam?
Cam Ellis: That’s right, Scott. If an employer wants to rely on performance to explain pay, the employer needs to be understanding how performance is documented, calibrated, and applied manager to manager. And all of that creates an advantage for a proactive approach. You can identify the root cause on the front end rather than just treating symptoms or playing whack-a-mole as issues arise. A plaintiff’s attorney is definitely going to connect dots in litigation between any issues in performance ratings and outcomes in pay. And it’s better that you, the employer, get there first and under privilege.
Scott Kelly: Got it. Very important caveat there for the privilege. Let’s kind of jump into the analysis itself. We’re going to say for the purposes of our discussion today that a client has given us data, the talent, we’ve put it in our Talent Risk Assessor, Pete. And what’s the first thing that our analysis is going to be looking at?
Pete Bell: Well, Scott, we’re going to start at a very high level. So, really the first layer here is just average pay. And we really haven’t even dug into the statistics yet. The concept is simple. We’ll group employees who are similarly situated, so work of a similar skill, similar effort, and similar level of responsibility. That may be title, grade, some other level of advancement or other criteria. And then we’re just going to look and compare average pay across demographic groups within those groupings. We have not conducted any statistical test at this point. It’s just a high level of review of where averages by eye may look off.
Scott Kelly: All right. And so, what does “similarly situated” mean in practice? Anybody want to take a stab at that?
Pete Bell: I’ll start, but Cam, definitely feel free to jump in. And it’s really that apples to apples comparison. So similar role, similar skill, similar levels of responsibility. And if we have those within a group, are we seeing meaningful differences in that average pay when we segment it by protected characteristic?
Cam Ellis: And just to emphasize, all of this needs to be done under attorney-client privilege. We are performing a legally informed analysis and inquiry. In order to protect that and to prevent legal risk and to be able to assess legal risk without creating more of it, the lawyer needs to be driving the process. The employer needs to have engaged counsel and that counsel needs to direct the analytics being performed that are being performed for the purpose of providing legal advice. And that’s the framework that is essential for protecting this.
Scott Kelly: Let’s talk about what the analysis showed for our hypothetical employer, Pete.
Pete Bell: Sure. So, when we did that average pay comparison, we identified a few groups where the differences in average pay for several demographic groups exceeded what our threshold would be for further review. It’s worth noting these are not necessarily violations, just something that tells us we might want to do a little bit of deeper investigation.
So, the next couple steps from our view would be to say, well, are these differences in averages that we’ve observed, are they meaningful once a statistical test is applied? Something like the t-test or Fisher’s exact. Basically, are these results statistically significant? And if we see flags in the talent assessor at that level, we can consider taking it further to a pay equity analysis where we’d start to evaluate if the differences can be explained by legitimate non-discriminatory factors.
Cam Ellis: And Pete, I want to dig in on both the explained by and legitimate non-discriminatory there. Whether the employer thinks something is non-discriminatory, as we’ve talked about with performance, may or may not play out under analysis. And separate question, what’s explained versus unexplained, that’s also something a plaintiff’s expert is going to be asking in litigation, or a regulator’s going to be asking during investigations or enforcement proceedings.
Those are questions we want to ask ourselves first under privilege so we can develop the factual record and determine next steps. We need to note here that an averaging analysis, like Pete said, it’s high level. It’s not a substitute for the regression or the rigorous pay equity analysis like a regression that can control for factors and pay decisions. This averaging analysis is kind of like a dashboard light to see if more thorough analysis or research and investigation can identify particular issues.
Scott Kelly: All right. So, beyond these averages that we’ve been talking about, I know that our Talent Risk Assessor also can look at individual outliers, run different type of statistical tests. Pete, will you walk us through that a little bit?
Pete Bell: Yeah, that’s right, Scott. And as Cam said, this isn’t a replacement for our robust pay equity analysis that we may do, but it can indicate where we have hot spots. And just because at the average level where we’re seeing differences, that doesn’t mean we should stop investigation and immediately pivot to pay equity. We do still have a couple of tools on our tool belt within the assessor that we can review.
So, that second layer is really a pay outlier analysis. So simple plots where we compare salary against timing variables. So, tenure, or how long you’ve been in the current role. We can plot salary against those timing variables and what you would expect to see is a positive correlation. So, the longer you’ve been in your position, the more your salary is. And where that isn’t the case or where folks are way off from that line that we post on a scatter plot, those are identified as outliers and something we may want to investigate.
Scott Kelly: So, just to be clear for our listeners, outliers are going to be the ones that fall outside the expected patterns. Is that right, Pete?
Pete Bell: Exactly. So, the outlier is someone, so let’s say they have significant tenure, but they’re paid well below their peers that are, let’s say, similarly tenured or a newer employee who’s paid significantly above. Maybe it was a market thing, or some other decision was made that wasn’t considerate of our pay philosophies. We might see that new employee is paid well above, and that’s an outlier as well. All of these are individuals we want to flag for further review under privilege so that we can determine whether there’s a legitimate explanation.
Cam Ellis: And Pete, from a legal perspective, this is where I really like the tool. Because individual outliers represent potential individual claims, and the tool allows us to easily see those people above and below the scatter plot that aren’t paid what we would expect them to be paid based on the quantifiably or statistically accessible or available job related factors that are in the analysis. An employee who discovers that they’ve been underpaid relative to peers with less tenure has a compelling narrative and potentially a viable claim just depending on the facts. And so these outliers are really an ideal place to first focus our attention.
Scott Kelly: All right. So, can we layer in the statistical analysis here too?
Pete Bell: We do. So, when we have large enough groups, groups of sufficient size, we can run a t-test analysis, which is a fairly standard statistical method where we can determine if the differences between two groups, let’s say men and women, if those differences in mean salary are statistically significant or more likely just due to chance. What we want to see is whether or not those pay differences between demographics groups is both large enough and given the variations and variability in pay that we’re seeing, if it’s likely to be maybe more of a systemic issue or if it’s likely to have just occurred randomly and by chance.
Cam Ellis: And regulators and courts pay attention to these kind of statistically significant results, sometimes too much attention. In plain terms, when a pay gap is statistically significant enough, it means the gap is large enough, unexpected enough, and consistent enough that it’s very unlikely to be random noise. That doesn’t mean it’s discrimination. It just means the employer needs to be ready to explain it with legitimate factors.
Pete Bell: And Cam, I want to add a comment here where we’re dealing with sensitive information. This is compensation data. We have demographic information. This can be highly sensitive. So, I just want to highlight that we do take that data security very seriously. And the Talent Risk Assessor does a good job. It maintains access controls. We’re encrypting the data. We’re logging who is accessing that. And before we would give access to clients, we want to make sure we understand who are our authorized users and make sure they understand the privilege framework before they access any of these results.
Scott Kelly: Thanks for that clarification, Pete. I think that’s really important. So, we’re kind of at the point where we’ve got an analysis that has flagged some potential disparities. I’m really curious in y’all’s take on what would happen next. What are the factors that might explain some of these differences?
Cam Ellis: Scott, there’s several recognized legitimate factors. Prior relevant experience is a big one. Two employees in the same role can have very different backgrounds. Now quantifying that down to a data point can be challenging. Education certifications are similar and they can also justify pay differences. Market adjustments are common ones. If you have to pay a premium to attract a candidate from a competitive market, that’s generally defensible. Geographic differentials for multi-state employers with location-based pay structures. And of course, performance, which brings us back to our last episode. All of those can be integrated into more robust statistical tests. And the idea is that this pass can help us direct where and what work needs to be considered next.
Scott Kelly: All right. So, those are all really good examples that hopefully can explain some of the differences that our analysis is flagging. But what happens if we’re kind of scratching our head and we don’t know how to explain after talking with the client, what happens if you can’t explain some of these disparities?
Cam Ellis: You need to keep digging. That’s where privileged review with counsel becomes critical. Counsel works with HR and the business to investigate root causes. Maybe there was a flawed hiring negotiation. Starting pay can often have knock on effects for years, if not decades. Was there a missed market adjustment cycle? Is there a manager problem who systematically rates certain groups lower, which suppresses merit? Then it could be any of those things. And the goal is to identify the operating mechanism, not just the outcome.
But I also want to emphasize that this isn’t a one and done kind of exercise. Pay equity is dynamic. We’re looking at similarly situated groups, and groups are dynamic. You have new hires, you have departures, you have promotions, you have market adjustments, reorganizations. In no two years is the group going to look completely the same, and all of those changes can introduce new disparities.
Scott Kelly: All right. Well, I think we’ve covered a lot today, so let’s try to recap for our listeners. Today we explored how proactive pay analysis works from looking at an average pay comparison among similarly situated employees to then doing a bit of an outlier investigation based on tenure and time and role to statistical significance testing for some of the larger groups. We discussed legitimate factors that may be able to explain pay differences and the critical importance of conducting all of this work under the attorney-client privilege.
I think the key message or the takeaway is that compensation risk doesn’t exist in isolation. It connects to performance management, to hiring, to promotion, and the Talent Risk Assessor allows employers to see those connections and address them systematically, ideally before a plaintiff’s attorney would do it for them. And we’re definitely fans of using a more robust or regression model for more robust pay analytic purposes. That’s not necessarily included in the Talent Risk Assessor, but we do that type of work with clients all the time.
That’s where you can really use the computer to test a lot of the different factors. And if you can get it set up right with the right subject matter experts, Pete is a statistician and helps lead these, along with other members of his team, for our clients, that really can knock out a lot of the manual work that some of these teams are having to do.
So, in our next episode, and believe it or not, it’ll be the final episode of our Talent Risk Assessor series, episode seven, we’re going to talk about identifying these trends, and then coming up with some defensible actions. So, we’re going to bring everything together we’ve covered across this series and look at how a comprehensive, proactive workforce analytics program is going to create a defensible posture across the full employee life cycle.
And hopefully our clients will see how valuable that can be in this kind of ever-changing regulatory environment that we’re finding ourselves in. So, Pete, Cam, thanks for joining me today. For our listeners, thank you all for being here. I’m Scott Kelly, and we’ll see you all next time.
Announcer: Thank you for joining us on the Ogletree Deakins podcast. You can subscribe to our podcast on Apple Podcasts, or through your favorite podcast service. Please consider rating and reviewing so that we may continue to provide the content that covers your needs. And remember, the information in this podcast is for informational purposes only and is not to be construed as legal advice.
Speakers
Share Podcast
Practice Group
Workforce Analytics and Compliance
Ogletree Deakins’ Workforce Analytics and Compliance Practice Group provides tailored guidance and legal recommendations for a myriad of workforce issues, informed by data-driven, state-of-the-art compliance and risk assessment services. Our services encompass all stages of the employment life cycle, such as selections, career advancement, compensation and benefits, and retention, which enables employers to make informed decisions […]