Talent Risk Assessor, 1: What Data Reveals About Selection and Hiring Processes

Announcer: Welcome to the Ogletree Deakins podcast, where we provide listeners with brief discussions about important workplace legal issues. Our podcasts are for informational purposes only and should not be construed as legal advice. You can subscribe through your favorite podcast service. Please consider rating this podcast so we can get your feedback and improve our programs. Please enjoy the podcast.

Scott Kelly: Welcome back to Defensible Decisions. I am Scott Kelly, your host and a shareholder in the Birmingham and Washington, D.C. offices of Ogletree Deakins. I’m chair of the Workforce Analytics and Compliance Practice Group and also co-chair our Pay Equity and Government Contracting and Compliance Practice Groups. I’m excited to share more with the listeners today about the Talent Risk Assessor. This is the first episode in a series where we’re going to dig into some of these offerings that the Talent Risk Assessor will have for our clients. If you caught us in the introductory episode, you heard me, Chrissy Blantz, and Pete Bell from our data analytics team at Ogletree lay the groundwork. We talked about what the Talent Risk Assessor is, we talked about why we built it, and really the philosophy behind proactive legal risk management.

Today, we’re going to go a bit deeper, and we’re going to take on selections and hiring, which we think of as the front door of the employment relationship. And we’re going to walk through some real-world hypotheticals that illustrate how this tool will work in practice. I’m excited to be joined today by two of my colleagues, Chrissy Blantz. She’s bringing the quantitative rigor, the ability to look at hiring funnels and identify where the numbers tell a story. She’s a manager in our data analytics group here at Ogletree. And we’re also joined by Chris Near, a shareholder in our Columbia, South Carolina office who brings more than a decade of experience advising employers on the hiring process and how to respond when data raises questions during the selection process. Chrissy, welcome.

Chrissy Blantz: Happy to be here. Hiring data is where a lot of the most actionable insights live, so this is a great place to start.

Scott Kelly: Chris, we’re excited you’re here too.

Chris Near: Yeah. Thanks so much, Scott. And thank you, Chrissy, I’m glad to be here. And from a legal standpoint, selections are very often the first place that a regulator might look, so the stakes here are real, so this is a really good topic for us to discuss. Often an area also where we find the largest volume of data. So good to talk through this.

Scott Kelly: Great. Let’s go ahead and set the scene. But before we get into the hypothetical, I want to ground our listeners in what we mean when we talk about this service offering, the Talent Risk Assessor. It’s not a single metric or a one-off report. It’s a comprehensive dashboard that brings together data from the full employment life cycle. So, we’re going to be able to look at hiring, separations, talent progression, and some pay issues. The idea here is to give legal teams and HR leadership a unified view of where legal risk may be accumulating. Of course, we also have the ability to conduct the separate analysis instead of the comprehensive one. We can provide those separate analysis and dashboards as well. Is that right, Chrissy?

Chrissy Blantz: Yep, that’s exactly right. We can meet the client wherever they’re interested. Behind the dashboard is Ogletree’s team. We have our workplace analytics and compliance attorneys and the data analysts. We have statisticians from our data analytics team as well. We’re not just running queries, we’re building a framework that lets employers see patterns they would otherwise miss until an EEOC systemic investigation or a plaintiff’s discovery request surfaces them.

Chris Near: And Chrissy, that’s a real key understanding with this tool. The Talent Risk Assessor tool is designed for legal risk management. We’re not framing this as a diversity initiative or a check the box type thing for compliance. It’s a proactive evaluation. We’re asking where might an employer face legal exposure based on how current systems or processes exist, and what can they do about it now while there are still potentially some options?

Scott Kelly: Right, so we’re really looking at being proactive, not reactive. And when we look at selection specifically, Chrissy, what are the major components that the Talent Risk Assessor covers?

Chrissy Blantz: At a high level, we’re looking at external selections. These are the people coming in from outside the organization. And then internal selections, which are the promotions, the transfers, the lateral moves within the organization. Then we look at offers and hires, because there can sometimes be a meaningful drop off between who gets offered that role and then who actually accepts the offer and who actually joins the company. And then, this is where it gets really powerful, we do stage analyses. We break the hiring process funnel into its component parts and examine each transaction point along the way.

Scott Kelly: Great. So, let’s talk about why the stage analysis really matters. Why not just look at the overall hire rate?

Chris Near: Well, let me jump in here because overall rates can typically mask exactly where a barrier is occurring. Rarely is there just a single step in the selection process, so each step is important to look at individually. So, if you’re looking at like a top line hire rate, it might look perfectly fine even if there is maybe a real drop off for one group at a particular stage later on or even earlier offset by a spike somewhere else in the process. So, averaging across the full process without considering really the how you got that from start to finish may end up hiding the very thing that you need to see.

Chrissy Blantz: Exactly. And from an analytics perspective, a stage analysis lets you pinpoint whether a barrier is tied to a specific decision maker, maybe it’s a recruiter or hiring manager, a specific criterion, or a specific location. That specificity is what makes a response actually actionable.

Scott Kelly: All right. Let’s bring this to life with a continuing hypothetical, and we’re going to build on this scenario across the Talent Risk Assessor series. We’re going to say we have a multi-state employer with a high applicant volume. They’re hiring across dozens of locations, lots of multiple job families, they’ve invested in their recruiting infrastructure, and by all appearances, they’re doing all things right. But when the Talent Risk Assessor, when we get their data, we run it through the analysis. It identifies a significant drop off at a particular stage of the hiring funnel for a particular demographic group. The overall rate looks acceptable, but at this one transition point, the numbers start to diverge.

Chrissy Blantz: So, if we think a hiring process of five different stages, we refer to it as the funnel, think about application and then there’s a screening, there could be an assessment, interview, and then ultimately an offer. Walking it through stage by stage, we’ll say four of the five look consistent across groups. Candidates move forward at similar rates, but at the assessment stage, one group is passing through at a meaningfully lower rate than the others, and it’s not just noise. Underneath, we do run statistical testing to confirm it as a real pattern, not a fluke of small numbers or just bad data. But the plain English version is this is the one place in the process where the numbers genuinely diverge.

Chris Near: And from a legal perspective, that raises an immediate question. What is happening at that stage? Is the criterion that’s being applied at that stage, is it job related? Is it consistent with business necessity? Very important questions to ask, because if it’s not, and there is adverse impact, the employer has some potential exposure under Title VII, potentially under Executive Order 14398, the FAR clause 52.222-90, particularly for federal contractors. So, these are very important things to consider.

Scott Kelly: Yeah, Chris, I’m glad you mentioned the FAR clause and the new executive order from the spring of 2026 that we’ve seen focus on race and ethnicity discrimination. There’s really a lot to unpack there. So, for the listeners that are federal contractors or subcontractors, I suggest you might want to look at some of our webinar materials or articles on that. We’re seeing a lot of those FAR clauses being inserted in contracts for our clients, and we’re getting close to the implementation date set by the FAR Council of July 24th. So, lots to unpack there. But back to the Talent Risk Assessor and when it’s surfacing this pattern in the hiring activity, it brings this pattern to light. But can you, Chris, tell us a little bit about what this doesn’t do, what this analysis at this stage does not do?

Chris Near: Yeah, I mean, this is a good level set right here because first and foremost, it’s important to note that this Talent Risk Assessor does not, quote unquote, “conclude” that the employer is violating the law. But I’ll also say it’s not also concluding that the employer is in compliance either. Something that you’re going to hear throughout this series of episodes where we’re talking about the Talent Risk Assessor is that a statistical flag is not a conclusion. It’s a prompt for a legally informed inquiry. So, this distinction is really critical here. The tool is telling us where we should be looking. It’s then the legal team’s next step to determine what that actually means.

Chrissy Blantz: Exactly. And the tool doesn’t replace judgment. It gives you the quantitative foundation to ask better questions. Maybe the assessment at that stage is perfectly okay and the difference reflects genuine variation in qualifications. Or maybe the assessment has a design flaw that introduces unnecessary adverse impact. You cannot know until you investigate deeper.

Scott Kelly: Okay, so let’s do that. Let’s go one level deeper on the stage analysis then. When the Talent Risk Assessor breaks down the funnel, Chrissy, what exactly is it measuring?

Chrissy Blantz: To put it simply, we track what percentage of candidates make it from one stage to the next. And then we compare that rate across groups at every single stage. If a gap shows up, we check whether it holds up once you account for the size of the applicant pool. Sometimes a small group of candidates can produce a swing that looks dramatic, but it’s not actually meaningful. So, once we’re confident the gap is real, that’s what gets flagged.

Scott Kelly: And when this is all happening, are you telling me that it’s going to happen across all locations, all job families? Another way to say this: is it customizable?

Chrissy Blantz: It definitely is customizable. And that’s one of our favorite aspects. The dashboard allows you as the employer to slice the data at different levels of aggregation. You can look at the enterprise level. You can look at the regional level, the facility level, or the individual requisition level. And that flexibility matters because a pattern that looks benign at the enterprise level might be concentrated in one location or the decision from a hiring manager. Our clients can tailor all of this from the start of the engagement. We ask that question, how do you want to see your data presented at the very beginning, and then throughout the process, build it according to their needs.

Chris Near: And Scott, you and I have spent a lot of years with enforcement agencies dealing with these types of issues and this is exactly how enforcement agencies, whether federal or state, have typically approached this. They’re looking at the data based on location, how decisions are actually made or where claims might be arising. I think more so than the company as a whole, because that’s typically just not how employers usually make a hiring decision. The caveat though here though is I do think there’s always that potential for scope creep in those requests for information. So it may be that, yes, really the correct way to look at things is a location or a smaller subset versus a company as a whole, but that is something that you do have to keep in mind. The nice thing is, is the Talent Risk Assessor is in a sense, letting the employer see the data in these various ways that an auditor or a plaintiff might before they actually arrive on the scene.

Scott Kelly: That’s pretty powerful framing there, Chris. So essentially this tool is going to allow you to preview your own audit experience.

Chris Near: Yeah, exactly, and I think that preview gives you time to investigate, to document your analyses, and really, if warranted, to make any particular changes that need to be made. All of that is going to strengthen your legal position down the road.

Chrissy Blantz: And I would add it’s not just as…or it is just as useful for confirming where the process is working well. Four clean stages out of five means legal and HR can put their limited time into the one stage that actually needs it.

Scott Kelly: All right, so let’s say that an employer sees a flag in the dashboard. Chris, walk me through what happens next. I’m assuming there are a couple of different informed responses from a legal standpoint, and probably a lot of it would depend on what these flags actually might show, but walk us through that.

Chris Near: Yeah, and this really can vary from project to project. And let me say what I’m about to say with a few assumptions. Presumably before an analysis begins, there has been some type of discussion about how the hiring process actually occurs and the analysis is set up in a way that matches those steps. If not, though, if that hasn’t been done, then that’s a good place to start. Outside of that, though, the next or maybe even the potentially first step is to understand what is being applied at the flagged stage. So, in our hypothetical, what’s been flagged is an assessment. So, the legal team needs to ask a couple of things, at least in our hypothetical here. For example, what is this assessment measuring? How is it being offered? Why is it being used? What are the possible implications?

And then really after you go through these questions and you talk through some of these issues, then you devise a plan from there, weighing both the statistical and the practical significance considerations. So again, there’s not a one-size-fits-all approach, and really the different factors to look at and how to maneuver the analyses become unique to each company and its processes. So again, it’s important for the company and the council to have these discussions really from start to finish.

Chrissy Blantz: And from the analytics side, we want to understand the data quality. Are candidates being properly coded at each stage? Is there a stage where disposition codes are inconsistent or where candidates are falling out of the system without clear documentation? Because sometimes what looks as a barrier is actually a data integrity issue.

Chris Near: And that’s such a great point, Chrissy, and I know one that with disposition codes, you, me, and Scott are going to talk about in our next episode. But really before you can draw any conclusions about whether a process is creating a legal risk, you need to confirm that the data accurately reflects what is actually happening. And that’s where the HR team in particular plays a critical role here.

Scott Kelly: Super important. But what if the data is clean and the barrier is real?

Chris Near: Well, then you get to move on to that legal analysis. You start to ask yourself some of these questions. That’s how we get through this process sometimes. Sometimes it’s not just the answers, but the questions that are so important. Is there a true legal problem here to solve? That’s something we’ve got to figure out. Is this creating risk? And if so, how might we mitigate this in the future? Should or even can the company consider an equally effective alternative with less or no adverse impact? These are just some of the questions you got to walk through to determine exactly what the position is and is it defensible or what steps need to be taken.

Scott Kelly: Okay. And that word defensible, Chris, that’s really the heart of the matter, right? We’re not telling employers what decisions to make. We’re helping them understand and make decisions that they can defend.

Chris Near: Yeah, exactly. Important to remember, the employer retains that full autonomy over its hiring process. And we, as counsel, what we’re doing is we’re providing information and the legal framework to ensure that whatever choices are made ultimately are grounded in some form of evidence and documented appropriately.

Scott Kelly: And that really gives our listeners a solid foundation for how we’re using the Talent Risk Assessor and approaching the selection and hiring analysis. But quickly, here’s what we’ve covered today. We talked about the Talent Risk Assessor being a comprehensive dashboard. It covers things like hiring, separations, talent progression, and pay issues. We walked through a hypothetical where stage analysis revealed a demographic drop off at a specific point in the hiring funnel. We discussed why a flag is just a starting point for the inquiry and not a conclusion. And we outline what a defensible response looks like. Understanding the criterion, confirming the data quality, and conducting the legal analysis.

Knowing where the drop-off happens is only half the story. Once you have located the stage, you still need the record showing why each candidate advanced or did not advance, and this is exactly where we’re going to pick up in episode two. Dispositions and merit-based hiring documentation is what we’re going to be talking about then. We will stay with this same hypothetical employer and dig into the documentation that turns a well diagnosed flag into a defensible decision.

Chrissy Blantz: I’m really looking forward to it. The documentation piece is where we see a lot of employers have gaps and it is one of the most addressable risk factors we see.

Chris Near: Agree. And episode two, it’s going to tie directly back to what we’ve discussed here today. Really, if you can’t explain why a candidate did not advance in the selection process, you’re likely going to have a tough time defending those decisions.

Scott Kelly: Well said, both of you. Thank you, Chris and Chrissy for joining me today, and thank you to our listeners for joining us also. This is Defensible Decisions, the Talent Risk Assessor series, and we’re going to see you next time.

Announcer: Thank you for joining us on the Ogletree Deakins podcast. You can subscribe to our podcast on Apple Podcasts or through your favorite podcast service. Please consider rating and reviewing so that we may continue to provide the content that covers your needs. And remember, the information in this podcast is for informational purposes only and is not to be construed as legal advice.