In this seventh and final episode of the Talent Risk Assessor series on Defensible Decisions, shareholders Scott Kelly (Birmingham/Washington, D.C.) and Lauren Hicks (Indianapolis/Atlanta) are joined by Pete Bell and Chrissy Blantz of the firm’s data analytics team to bring together the six workforce dimensions covered across the series—(1) selections and hiring, (2) dispositions, (3) separations, (4) talent progression, (5) performance, and (6) compensation—into a single, repeatable seven step framework. The speakers walk through each step in sequence, protecting privilege, validating the data, identifying the right comparison group, drilling down to investigate a flag, documenting a legitimate explanation, evaluating whether change is warranted, and monitoring over time, while emphasizing that these dimensions are interconnected windows into the same workforce rather than isolated analyses. The episode closes with a broader point that the Talent Risk Assessor’s value depends on integrating statistical rigor with legal judgment under privilege, and that this framework is designed to remain stable even as the regulatory landscape and underlying data continue to evolve.
Home > Insights & Resources > Podcasts > Talent Risk Assessor, 7: Seven Steps From Trend to Defensible Action
Talent Risk Assessor, 7: Seven Steps From Trend to Defensible Action
By T. Scott Kelly and Lauren B. Hicks
0:00:00
0:00:00
Transcript
Announcer: Welcome to the Ogletree Deakins podcast, where we provide listeners with brief discussions about important workplace legal issues. Our podcasts are for informational purposes only and should not be construed as legal advice. You can subscribe through your favorite podcast service. Please consider rating this podcast so we can get your feedback and improve our programs. Please enjoy the podcast.
Scott Kelly: Hello everyone. Welcome back to Defensible Decisions in our series on the Talent Risk Assessor. I’m Scott Kelly, and this is our capstone, episode seven. Today, we’re not just recapping. We’re going to hand you a single repeatable framework for moving from a workforce trend to a defensible action built from everything this series has covered. So, over the past several episodes, we have walked through workforce dimensions, selections and hiring, dispositions and merit-based hiring, separations, talent progression, performance, compensation and pay issues. And today we bring it all together into one framework. Joining me again are Chrissy Blantz, Pete Bell, and Lauren Hicks. Welcome back everybody.
Pete Bell: Thanks, Scott. Excited to be here for the capstone.
Chrissy Blantz: Yes, we’ve covered a lot of ground.
Lauren Hicks: Thanks again, Scott. And I think hopefully this will be the episode where the whole framework comes together and sort of clicks.
Scott Kelly: Me too. So, let’s start with a little bit of a quick look back. We’ve covered a lot of ground, as y’all said, in these six episodes. Can you each give us a 30-second view of what we’ve explored? Chrissy, you want to start us off?
Chrissy Blantz: Sure. Selections and hiring looked at applicant flow for adverse impact. Dispositions examined whether merit-based hiring decisions hold up under statistical scrutiny. Separations analyzed involuntary terminations by protected class. Talent progression looked at promotions and access to growth opportunities. Performance, examined rating distributions and calibration, and then pay equity used T-test analyses to identify meaningful pay gaps that may warrant a deeper client review.
Lauren Hicks: That’s substantial coverage, Chrissy. And underneath every one of those episodes was kind of the same skeleton, right? Protect privilege, validate the data, find the right comparison or comparators that are legally defensible, then drill down and document the legitimate explanations and decide whether change is warranted and finally monitor. So, today we’re going to kind of walk through that skeleton pretty explicitly.
Scott Kelly: That’s right. And what we’re really trying to do, and I know it’s kind of hard in a podcast because you can’t see things and I’m a visual person, so what we’re really trying to do is give you this key insight for our listeners, that these aren’t six isolated analyses, they’re six windows into the same workforce. So, a problem that shows up in one of the dimensions often has roots in another. So, in talking about that interconnection, Pete, tell us how do you see these six dimensions actually relating to each other in practice?
Pete Bell: Yeah, Scott. And that’s where the dashboard and just the talent risk assessor itself kind of earns its name. It takes those 15 to 16 different ways of doing analyses and looking at it and puts it into a single dashboard. So, then you’re able to quickly compare hiring outcomes against later progression for that same population. So, we might have no issues in the hiring bar, we’re hitting expected rates, but then later on we’re stalling in the promotion data a few years later. So, that tells us it’s a progression issue, not a hiring issue. And so, we’re able to find that root cause at any point in the employment life cycle. You would miss that if you only looked at maybe one of any of the individual dimensions.
Lauren Hicks: Yeah, and that’s kind of the key point. Legally, that connection really matters because an enforcement agency, Department of Labor, EEOC, DOJ, they’re all going to look broadly during an investigation, especially if there are allegations of systemic or pattern and practice violations. So, if you’ve already run an analysis on all of your employment decisions, the entire life cycle underprivileged, you’re in a far stronger position to respond.
Scott Kelly: So, can you give us another example of how these dimensions connect one another?
Chrissy Blantz: Sure. If we look at separations and performance, for example, if involuntary separations are disproportionate for a particular group, the natural next question to ask is what their performance ratings look like. If ratings are lower, you have to ask whether the performance system itself is producing the outcome. You can’t answer that separations question without performance data and you can’t trust performance data without understanding the calibration process that goes in behind it.
Lauren Hicks: I agree, Chrissy. I think that’s why a holistic approach matters because an employer who only looks at pay equity might close a compensation gap this cycle without realizing the gap will simply regenerate next cycle if the underlying promotion criteria are the issue producing the disparity. And so, you really have to look at the totality of your employment system collectively.
Scott Kelly: All right. Thanks, y’all. So, I think we could really build this out step by step so our listeners can really walk away with something that they can actually use. So, seems like over the course of the series, we’ve really talked about seven distinct steps. We’ve talked about protecting privilege, about validating the data, about finding the right comparison group, about drilling down, documenting what the legitimate explanation is, decide whether change is warranted and then monitoring it. Let’s start with step one, a flag that appears in the data. What happens first?
Chrissy Blantz: I’ll actually let Lauren, our counsel, take that one because it actually has to happen before we touch anything else.
Lauren Hicks: Yeah. So, step one, protect privilege from the outset. This is so critical, and it’s a very often misstep, but before a single number gets pulled, council should be directing the engagement. So, the validation, the analysis, the interpretation, all are conducted under attorney-client privilege or work product doctrine. Get that structure wrong or try and jump in midstream in the process and everything that’s happened, however rigorous, could end up being discoverable and that is just not a risk that’s worth taking.
Scott Kelly: Yeah, I totally agree, Lauren. And I know we covered this in a recent webinar that we did on AI bias and the importance of looking at all the things related to that type of technology and making sure that you get that done under privilege. There’s a recent decision out of the Northern District of California that we discussed in that, that was really helpful. And I would suggest if our listeners weren’t able to join us for that, if they’d like to reach out, we certainly can get them a copy of that webinar. But we’re assuming now that we’ve taken the right steps and we’ve privileged this analysis. What’s step two?
Pete Bell: So, step two, Scott, is really validating the underlying data. And you think about it in the dashboard, we’re going to see the end result of all of our work, right? The standard deviation, where the flags are, what that is. If we haven’t done our work to validate the data, we’re not sure if that flag is real. So, we have to do a lot of work on the front end to make sure we’ve done that correctly. Do we have the right groups? Are we looking at appropriate date ranges? Have we defined the population appropriately? Any kind of coding error or issue in the underlying data could create a false signal and that’s going to impact our legal response to a data problem.
Scott Kelly: All right. So, data’s validated, what about step three?
Pete Bell: And this one’s really going to tie very closely to step two, and that’s our comparison group. We’re going to run a lot of analyses, but if we are looking at the wrong population, we don’t have the right job grouping mechanism, we’re looking at inconsistent time periods, we’re not looking at a consistent labor market. These are all things that may manufacture a flag that isn’t real or worse, hide a flag from our purview that is real. So, when we apply these statistical tests, we want to see what the results are. That’s the part that’s maybe more exciting, but we need to make sure that the comparison is appropriate and that the situation is appropriate as well, or we’re not going to be as confident that we’re looking at like to like.
Lauren Hicks: Yeah. And if that comparison confirms a meaningful disparity, then I think the next step, which would be step four, is to drill down by stage, category, business unit, reason, location, whatever breakdown is relevant to the particular analysis. And I of course want to emphasize this one more time because it’s the principle this whole series has been built on, a flag is not a conclusion. An initial statistical indicator is not a conclusion. It can be, that can happen, but it is more often than not a prompt for a legally informed inquiry, right? So further investigation, drilling down, is how we figure out what’s actually driving the numbers. A facially neutral policy with a disparate effect, a single decision maker is sometimes the explanation, or it could be a structural issue in how opportunities are allocated.
Scott Kelly: All right. So once the drill down turns up an explanation, what’s step five?
Lauren Hicks: So, Scott, if we get an explanation, if we’ve got a legitimate business explanation that we have found and we’re confident it’s rational, it’s comprehensive and underprivileged, sometimes the drill down turns up a defensible reason, seniority, a shift differential, a business necessity qualification. It could be any number of things. We’re always looking for neutral rational explanations. Once we’ve found it, we write that down clearly, under privilege with the supporting data while it’s fresh, and that document is what turns, we looked into it into a legally defensible record. If there’s not a clean and legitimate explanation, which does happen on occasion, step six is to evaluate whether a policy or practice change is warranted. And sometimes we do need to make those structural changes. So that can be updated criteria, manager training, a revised process, or where pay is involved, sometimes a compensation adjustment.
So every change will tie back to the specific finding that prompted it. We always talk about between us and the team that there’s no one formulaic answer, no one right answer that’s always going to appear in every case. So, making that remedy, whatever it might be, shows good faith rather than kind of a generic gesture.
Scott Kelly: All right, that brings us to our seventh step.
Lauren Hicks: I would say it’s monitor over time, right? You rerun the analysis after any change to confirm it’s actually working, and you keep watching the trend as it goes forward. That’s what turns a one-time fix into a closed loop, something that’s very well monitored. Of course, you got to protect the privilege, validate, compare it, drill down, document, all those things. And that’s what makes the whole process defensible, not just at one moment, but over time.
Scott Kelly: So, there’s the framework, start to finish, protect the privilege, validate the data, identify the right comparison group, do that drill down, hopefully find a legitimate explanation that you can document, evaluate whether change is warranted and then monitor. So, we’ve given the seven steps and it works the same way whether you’re looking at hiring, separations or pay.
Chrissy Blantz: Exactly. And with those 15 to 16 different analyses that we talked about, in one dashboard, you can run that framework across dimensions at the same time and see how a change in one area moves the numbers in another.
Scott Kelly: That framework sounds pretty simple on a whiteboard, but it takes some real skill to run well. What makes this run well in practice, Lauren?
Lauren Hicks: Honestly, it’s the people, not just the platform. The four of us on this podcast, for example, this is very natural and easy for us because we work together all the time, the lawyers and our practice analytics team, the data scientists, data analysts. It’s never Pete and Chrissy look at something in isolation or Scott, and I look at something in isolation. We are big here on brainstorming, really thinking about things, looking at things from a very rigorous point of view on the statistical side and from the legal and practical operational knowledge, putting all of these things together to talk about what types of things we should be investigating, what sort of avenues have we not thought of, are there potential interesting remedies we haven’t thought of? And that’s where looking at this holistically before enforcement proceedings by the government or we’re sitting in a systemic discrimination investigation are really important.
We know what the investigators look for because we’ve spent our careers responding to them and most organizations have kind of more of a separation between legal and data or you might be using a third party, it might be an outside law firm working with an outside data vendor and it’s best to really communicate closely and everyone have their hands and their eyes on the totality of the situation. And I think if you do those things, if you’re all in the room together at the same time, that’s really where you see the most benefit. And I think that’s what our bench here at Ogletree really does.
Scott Kelly: That integration means the analytics are always legally informed. So, we’re really not just running numbers here in a vacuum. We’re designing our analysis with legal defensibility in mind, the methodologies, the documentation standards and the interpretation frameworks. All of it is built to withstand the scrutiny. What about data security? Pete, I know you talked about that earlier, but that’s something that certain clients have asked us about before. Do you have anything you want to share there?
Pete Bell: You’re right, Scott, and it’s foundational, right? I think I mentioned on episode six, we’re dealing with really sensitive data, whether that’s compensation data or reduction force termination type data. We want to make sure that data is secure. So, the Talent Risk Assessor with that in mind, it runs with encryption at every level. We have limited access controls on who can see this dashboard. We have permissions at the individual level, so that sort of governs who sees the data and what they’re able to do with it. We know we’re handling sensitive information under privilege. Having that security is a pre-req really before we do anything meaningful.
Lauren Hicks: And when it comes to statistical analysis, we don’t like formulaic, we don’t like standardization, but when it comes to process and procedure, I think that standardization of the process is very important. So, when every client engagement uses validated methodology in the platform, you get consistency and that consistency is what allows us to kind of stand behind the analysis in whether it’s litigation or another regulatory context or just for investigative purposes under privilege. So, it’s not one random person’s view or thoughts. It’s a rigorous, repeatable process. And we kind of support that with ongoing client training and webinars. And the goal isn’t just to hand over a report, right? We want to build internal capacity so the HR and the legal teams understand the data, they can interpret it appropriately with our help and know when to escalate. So, we want the clients to be empowered to know what’s going on within their workforce and to understand how best to remedy any issues that do arise.
Scott Kelly: All right. Thanks for that. As we’re looking ahead, the legal landscape seems to be constantly evolving. That’s probably an understatement in the current environment. How does this framework adapt, Lauren?
Lauren Hicks: Yeah, it is definitely evolving all the time. And I think that maybe the strength of building on kind of, again, statistical rigor around industry norms and legal principle that is sort of broader than any one single regulation or executive order or enforcement activity. So, whether the regulatory emphasis shifts or new protected classes emerge or get more focus, the seven steps that we’ve talked about here today, Scott, they stay the same because it’s about, again, sound and consistent process rather than one specific rule or desired outcome.
Pete Bell: And to that point, the platform’s going to evolve in the same way, right? We anticipate we will see new data sources. We’ll learn and inherit new analytical techniques. There will be changes to workforce structures, consideration of additional demographic characteristics. We can take all of those things and fold them into the dashboard without necessarily having to rebuild the framework from the ground up.
Scott Kelly: All right. So, Lauren, what would you say if a client, you could be somebody in HR, somebody like an employment counsel, a general counsel, a business owner who’s listening to this and thinking about where to start, what would you say?
Lauren Hicks: Start with understanding that knowledge under privilege is power. Scott, I think you and I discuss all the time that you cannot fix what you don’t measure. You can’t defend what you haven’t identified and ideally have documented. So, the worst position to be in, the one we hope to avoid for most of our clients is about learning that there is a disparity for the first time because a plaintiff’s attorney or a government auditor, some enforcement agency investigation found it and identified it and is now questioning you about it. The best position is having already identified it, investigated it, and either resolved it or developed that rational, legally defensible explanation under privilege.
Chrissy Blantz: And I would add here too that we don’t want to let perfect be the enemy of the good. You don’t need perfect data to start. We’re very good at cleaning data and scrubbing data. You just need a commitment to the process and to be willing to validate what you have, analyze it rigorously, act on what you find and improve your data quality over time. The talent risk assessor gives you the framework to do exactly that.
Scott Kelly: All right. As we close out the Talent Risk Assessor series, I’ve been thinking about it, and this is something I want to leave our listeners with, defensible decisions and the decision making isn’t about avoiding risk. It’s really about understanding your risk, documenting your response to it, building some type of repeatable process. The seven steps we walked through today would be real good ones to consider, I think, or you might want to, that you can really stand behind those at every stage of the employment life cycle. This series has explored selections, dispositions, separations, talent progression, performance and compensation, all running on one privileged, secure, consolidated platform, six different dimensions for this one workforce and now one framework that ties them all together. We’ve gone through our seven steps and the goal here one final time is that the talent risk assessor exists and really was created by us at Ogletree to help HR and legal teams identify workforce trends, diagnose what’s driving them, manage the legal risk and do their part to remove EEO or barriers to EEO rather. And that’s something to be thinking about and is important under Title VII.
So, it’s a legal risk and workforce analytics tool. This is important to remember that data alone isn’t going to create defensibility. We really think that it’s important to work with counsel that are skilled, but doing that alone isn’t enough. So, hopefully it’s working with the team here at Ogletree because we’ve got both the data analytic skills you’ve heard from some of the folks directly on this series that have those, as well as some of the employment lawyer skills. And so, everything we’ve discussed across the series is informational. Of course, as you’ve heard almost in every episode that a dashboard trend on its own isn’t going to establish any kind of legal violation or prove compliance that works both ways. You’ve got to be really dependent and have a determination on the specific facts and make sure for privilege purposes that you’re getting that legal advice associated with all of that.
So, with that, I really want to thank you Pete and Chrissy and Lauren, not just today, but really for the entire series and frankly for leading our teams of lawyers and data analytic professionals to kind of bring this tool together. I know we had a lot of fun building it over several months. A lot of work was put into it, but I think we enjoyed that work. So, thank you and thank you to our listeners for staying here with all of us for all of these episodes.
Pete Bell: Thanks, Scott. Really enjoyed it.
Lauren Hicks: Yep. Thank you all. This has been great.
Scott Kelly: All right. This has been Defensible Decisions and concludes our Talent Risk Assessor series. We hope you found it useful and stay tuned for more episodes from this podcast very soon.
Announcer: Thank you for joining us on the Ogletree Deakins podcast. You can subscribe to our podcast on Apple Podcasts or through your favorite podcast service. Please consider rating and reviewing so that we may continue to provide the content that covers your needs. And remember, the information in this podcast is for informational purposes only and is not to be construed as legal advice.
Speakers
Share Podcast
Practice Group
Workforce Analytics and Compliance
Ogletree Deakins’ Workforce Analytics and Compliance Practice Group provides tailored guidance and legal recommendations for a myriad of workforce issues, informed by data-driven, state-of-the-art compliance and risk assessment services. Our services encompass all stages of the employment life cycle, such as selections, career advancement, compensation and benefits, and retention, which enables employers to make informed decisions […]