Talent Risk Assessor, 2: Auditing Disposition Codes for Legal Risk

Announcer: Welcome to the Ogletree Deakins podcast, where we provide listeners with brief discussions about important workplace legal issues. Our podcasts are for informational purposes only and should not be construed as legal advice. You can subscribe through your favorite podcast service. Please consider rating this podcast so we can get your feedback and improve our programs. Please enjoy the podcast.

Scott Kelly: Hi everyone, welcome back to Defensible Decisions. I’m your host, Scott Kelly, a shareholder in the Birmingham and Washington, D.C. offices of Ogletree Deakins, and chair of the Workforce Analytics and Compliance Practice Group, and I’m a co-chair of the Pay Equity and then also of the Government Contracting and Compliance Practice Group. Today is episode two for the Talent Risk Assessor series where we will discuss dispositions and merit-based hiring documentation. Joining me again today is Chrissy Blantz, a manager in our data analytics group here at Ogletree, as well as Chris Near, a shareholder in our Columbia, South Carolina office.
Behind the three of us, it’s Ogletree’s deep and knowledgeable bents of employment lawyers and data analytic professionals who work on these data related employment issues together every day. So, before we get started, this discussion is informational, it’s not legal advice, and just want to note that any dashboard trend on its own never establishes a violation or a compliance failure. Chrissy, welcome today.

Chrissy Blantz: Thanks, Scott. Glad to be here.

Scott Kelly: Chris, glad to have you too.

Chris Near: Yeah. Thanks, Scott. And looking forward to building on what we started in episode one.

Scott Kelly: All right. So today we’re going to go a little bit deeper into what the Talent Risk Assessor showed us last time. We’re going to look into dispositions, documentation, and what it actually looks like to demonstrate merit-based hiring, which has been a very prominent emphasis placed on by the current administration. So, I’m going to set the stage. In the prior episode, we discussed selections and hiring. We walked through a scenario with a hypothetical multi-state employer that was using Ogletree to work through the Talent Risk Assessor’s dashboard to monitor its selections processes. We talked about what flags might appear and what they mean and critically what they don’t mean. So, let’s pick up with that thread. Where does the disposition analysis come in, Chrissy?

Chrissy Blantz: The dashboard is a monitoring tool. It surfaces statistical patterns across stages of the hiring process funnel. It doesn’t prove or disprove merit-based hiring on its own. It directs attention to specific stages or requisitions that warrant closer examination.

Chris Near: Yeah, and Chrissy, that examination is where dispositions become essential. When we say disposition, what we’re meaning is the recorded reason that a candidate moves forward in the process, gets screened out of the process, or withdraws from the process. And the quality and the consistency of those records is often the difference between understanding decisions that actually happened through a defensible process and one that’s difficult or maybe even at times impossible to explain when reviewed under scrutiny.

Scott Kelly: All right. So, the dashboard points us to a spot that looks different, and then we look at what was documented about each decision in that spot.

Chrissy Blantz: That’s exactly right. The dashboard is the signal and then the dispositions are the substance behind that signal.

Scott Kelly: Perfect. Let’s return to this hypothetical employer. They posted a role that requires a specific professional certification, at least two years of relevant experience, and availability for a defined work schedule. The dashboard is now flagging stage level differences in screen-outs for a particular demographic group. Walk us through what happens next.

Chrissy Blantz: The first thing to understand is that the flag itself is informational. It tells us that at a given stage, the pass-through rate differs across groups at a level that warrants more attention. It does not tell us why. The why lives in the disposition data, the codes, the notes, the documentation trail.

Chris Near: Yeah and let me underscore something that I think both Scott and Chrissy have said here today and that I said in episode one as well, and that you’re going to hear throughout this series, that a statistical flag by itself is not a conclusion. It’s a prompt for a legally informed inquiry. The central legal question here is this. Can the employer show with data and documentation that a candidate advanced or did not advance based on a stated job-related criteria that’s tied to and posted in the requirements of a particular job advertisement or a posting?

Scott Kelly: That’s really key framing there, Chris. Chrissy, can you walk us through what a disposition analysis actually would look like in practice?

Chrissy Blantz: Of course. Disposition analysis is a review of the systems that an organization uses for tracking, analyzing, and reporting applicant outcomes. We look at how candidates are coded at each stage. Did they lack that certification? Did they not meet the experience threshold? Were they unavailable for the posted schedule? Or did they withdraw on their own? We’re looking at whether those codes are consistently applied, whether they map back to the posted job requirements, and whether there are gaps. Places where the candidates were screened out, but the reason wasn’t documented or doesn’t correspond to an identifiable job-related standard.

Chris Near: Yeah, good analytical points, Chrissy. On the legal side, what is so important is the nexus between the posted requirements and the disposition that’s actually used. If the job ad says must hold X certification and a candidate gets screened out with a code indicating did not hold required certification, that’s straightforward, right? The decision ties directly to a published job-related criteria. But let’s say an organization that runs into difficulty is where maybe a disposition code is vague or just maybe is used in batches to close out an applicant pool. Something like a kind of a nondescript disposition of not a fit or other or something like that. Or maybe there’s no note at all and the disposition field is just left blank.

So, another layer of difficulty might arise where a company may have different recruiters or hiring managers that use maybe the same disposition, but for different reasons. And those reasons may not always line up neatly. So that’s another area you got to pay attention to. So, these gaps don’t necessarily mean, again, that anything improper is occurring, but what they do is they make it harder to support why a decision was made and that the process was merit based. And remember, not every disposition may be relevant for every employer. So again, we talk about different things being unique to different companies. That applies again here. So, make sure that you review what your company has available to use on the disposition side. Is it working? Does it need to be revised? Along with making sure that the dispositions that are being used are relevant and that the recruiters and the hiring managers understand how they should be captured and used.

Scott Kelly: Got it, Chris. So, a key part about this disposition analysis is really asking, does our documentation tell the story of a process that’s grounded in job related criteria, which would be another way to test the merit-based decision-making.

Chrissy Blantz: Yes, that’s exactly right. And one of the most valuable outcomes of this work is refining dispositions, helping organizations develop codes and documentation practices that more precisely capture why candidates are accepted, rejected, or withdraw. When you refine that tracking, you gain a much clearer picture of your process, and you strengthen your ability to demonstrate its integrity.

Scott Kelly: All right. Thanks, Chrissy. So, let’s spend a little bit more time on this concept of merit-based decision making. It sounds simple on its face, but in practice, it seems to require real discipline around documentation. What does it look like when an employer gets this right, Chris?

Chris Near: So, merit-based decision-making from a legal risk management standpoint means that every selection decision, meaning who advances and who does not, is traceable to criteria that is job related and that is announced in advance. So, we’re not dealing with surprises and figuring out what we want to have on the backside. The job advertisement is really our anchor here. If the ad says you have to have two years of experience, specific certification, and certain schedule availability, then every candidate that applies for that job should be evaluated against those same criteria and not changed throughout the process. And then the documentation itself should also reflect that evaluation.

Chrissy Blantz: And that’s exactly what we want to see when we look at the data. We want to see that consistency. So if 10 candidates lack the required certification and all 10 are screened out with the same disposition code, regardless of demographic group, that’s a strong indicator of a merit-based process. Where we see variance in how that criterion is applied or when the documentation doesn’t capture the criterion at all, that’s where questions arise about did inconsistencies in the merit-based process impact equal employment opportunity and create legal risk?

Scott Kelly: So, the documentation here is really doing double duty. It’s guiding the decision makers in real time, and it’s also creating a record that can demonstrate consistency after the fact.

Chris Near: And you know what I’ll add? Maybe not double duty, but maybe triple duty. I think a third function here is that it creates accountability. When recruiters and hiring managers know that their decisions need to be documented against specific posted criteria, it really encourages that rigor in the moment. It’s not just about being able to look back and explain. It’s about being able to build a process that by design produces that defensible outcome. And keep in mind too, particularly for employers with high turnover, and I say that, but really, I mean, this does apply for any employer really hiring any number of employees, but most often, this is significant with high turnover, the odds are you will not be doing an analysis of that data immediately upon receipt of a resume and when it’s reviewed and a disposition is entered. There’s going to be a time gap there. So having good, accurate, accountable data of the details and what happened is very, very important because you’re likely not going to remember all of those point-by-point issues later on into the future when it becomes so important.

Chrissy Blantz: And I’ll add that this is where the dashboard and disposition analysis work together. The dashboard can show you where the outcomes differ, and the disposition analysis tells you whether those differences are explained by the consistent application of job-related criteria. Neither tool alone completes the picture. It’s the combination that gives organizations the ability to assess their own practices proactively. To be clear, the dashboard alone doesn’t prove merit-based hiring. The proof lives in the documentation, the disposition codes, the recruiter notes, and how well they align with the posted job requirements that we’ve been talking about.

Scott Kelly: All right. Let’s bring this home with some practical advice and guidance that we really, I think, pride ourselves giving here at Ogletree. If an employer is listening to this and thinking about their own processes, what are the steps they may want to or wish to consider?

Chris Near: Scott, how about we start at the start, right? Job advertisements. Where I would suggest starting is considering a review of those job advertisements. It’s important that the requirements that are in those ads are genuinely job related and that you can articulate why each one of those listed job duties is necessary for the role. Similarly, but maybe also conversely, make sure that you have listed in the job advertisement all the important aspects that are being considered in the selection process. So keep in mind, the ad is the foundation. Everything else flows from it. And plaintiffs, counsels, enforcement agencies, it’s not uncommon for them to typically ask for this type of information in discovery as they’re trying to build a case.

Chrissy Blantz: And then second, once there are applicants who have applied to these job advertisements, we find value in auditing the disposition codes. Are they specific enough to map back to those posted requirements? If you have a code that says did not meet qualifications, can you tell which qualification? Ideally, your system should allow you to document which specific requirement a candidate did not meet, ultimately tying it even closer to those job advertisements.

Scott Kelly: So, I think it’s important to pause here because I know we’re all very familiar from our experiences assisting federal contractors and others with really robust applicant tracking systems. Those have long captured why candidates didn’t make it into the next stage of selection process. But we see an opportunity to build on that now and add reasons or disposition codes of applicants that may be selected for an interview or perhaps why a finalist was chosen over other applicants that were interviewed. This type of documentation can also be used in a more robust statistical analysis as we can actually do analyses to test if you’re hiring based on merit-based decisions without looking at protected classifications. That could be very telling, and for some employers who might be feeling some pressure, given a lot of the signals coming from this administration about making demographic or race, gender, or ethnicity-based decision making and doing any analytics about that, perhaps doing a merit-based analysis would give some employers some comfort.

I would just counter that and highlight that the EEOC, both the chair and the acting general counsel have been pretty consistent in their public statements if you thread them together over time, particularly over the last year, that it is lawful and they’re somewhat encouraging employers to engage in these privileged workforce analytics to minimize risk. They just talk about the importance of keeping those decisions away and that information, the results away from people that would have impact or input on employment decisions. So, I think all of this could be very telling. Chris, do you have anything to add?

Chris Near: Yeah, and I’ve got a third thing to add, but let me also add something to what you had mentioned. You were talking about federal contractors are probably used to seeing this type of issue and hearing this discussion because they’ve had to deal with for a number of years kind of this robust applicant tracking system obligation. And one thing I’ll mention to that is even for contractors who truly may have been doing this and been doing it fantastically for years in the past, it’s always a good idea to give dispositions a fresh look every now and then because processes can change, who is using the dispositions can change, and generally what is job related or not can change, which could again affect the dispositions.

But kind of a third kind of takeaway here to mention that I think really goes hand in hand with that point also is training is crucial. Making sure to train your recruiters and your hiring managers. They need to understand that every screen-out decision should be documented with some reason tied to the job posting. Not a fit really isn’t a defensible disposition most times, at least not an easy one to defend. Did not hold required certification. That’s much more detailed and specific and very often an easier defensible disposition. So, Chrissy, any other takeaways?

Chrissy Blantz: Yeah. I would add that we would encourage employers to use the data proactively and not to wait for an external audit or a complaint. Run regular disposition analyses to find documentation gaps and inconsistent application of criteria and have those analyses reviewed through a legal lens. That’s where Ogletree’s combined bench comes in. The data team surfaces the patterns and then the legal team assesses what that might mean for regulatory exposure and litigation posture. That pairing is where the proactive risk management lives.

Scott Kelly: That’s really powerful. You start with the job ad, you build documentation that ties back to it and then use data and legal review together to verify the process to make sure it’s working as intended.

Chrissy Blantz: Exactly. And over time, as you refine your dispositions and documentation practices, the data becomes a stronger and stronger asset. It moves from being something you review reactively to something that continuously affirms the integrity of your process.

Scott Kelly: All right. Well, this has been another incredibly practical discussion. To summarize today’s episode, disposition analysis is about reviewing the systems organizations use for tracking, analyzing, and reporting applicant outcomes, and it asks whether the documentation shows the candidates advanced or were screened out based on job related criteria tied to the posted requirements. The dashboard is a tool that helps direct that review. It’s going to surface signals, but the substance really lives in the documentation. In our next episode, episode three, Chris and Chrissy, you’re going to join me again as we’ll shift from hiring into separations and terminations. We’re going to look at how similar principles of documentation, data monitoring, and merit-based decision making apply when an employment relationship ends. That’s a whole different set of considerations and we’re looking forward to digging in. So, thanks Chris and Chrissy for another outstanding conversation, and thank you to all of our listeners. This is Defensible Decisions and our series on the Talent Risk Assessor.

Announcer: Thank you for joining us on the Ogletree Deakins podcast. You can subscribe to our podcast on Apple Podcasts or through your favorite podcast service. Please consider rating and reviewing so that we may continue to provide the content that covers your needs. And remember, the information in this podcast is for informational purposes only and is not to be construed as legal advice.