top of page
Search

Evidently Talented w/ Josh Millet



Karen Mangia

Today on Success from Anywhere, we will meet a medieval French history student whose dream is to make résumés into ancient history, and he has 25 million data points to prove it. Please join me in welcoming to the show Josh Millet, founder and CEO of Criteria.

Josh Millet

Thanks for having me.

Karen Mangia

One question I like to start the interview with, for every guest, because we talk about work, is what was your first paying job and how did that job inform or inspire your career trajectory?

Josh Millet

That is a great question. I am laughing because this is sometimes a question we love asking candidates in our own interviews. That criteria, especially for sales roles, is a great question to start off with; it's always a concern if someone's first job was at age 27. So, my first job growing up, I believe, was working on my uncle's peach farm picking and packing peaches. It was a job that I could not do at my current age because, in the Georgia peach orchards, it's pretty hot. I don't think I would survive in that role today.

Karen Mangia

I guess you could say it was the pits. I know you had to have heard that before.

I mentioned in the opening your degree in medieval French history. What were you planning to do with that? And do you use any aspect of that now in what you do?

Josh Millet

I always had a kind of passion for history and I was thinking I was going to be a professor or teacher of history somewhere. In undergrad, I got hooked when I did a trip to France. So, that got me into it. As grad school progressed, I realized that there's not that great a demand for this role in the world. It was a little bit of a circuitous route, but I found my way to software entrepreneurship.

Karen Mangia

And you have been a serial entrepreneur; take us through your career to date before we talk about the business that you're in now.

Josh Millet

So right out of school, I did have a small startup. It was actually in the test preparation space. The idea behind it was trying to make test prep free for things like the LSAT and GRE. We launched the company in the year 2000. That was a while ago, right in the middle of the internet bubble. I ended up selling the company after a couple of years to a company out in California, which is how I got to be out here in California. I was originally on the East Coast.

Karen Mangia

And now you have a business called Criteria. Tell us more about the concept of the business and what inspired you to start it.

Josh Millet

Criteria is a software business. We like to call ourselves a talent success company, meaning what we do is all about helping companies make better talent decisions, especially around hiring. So specifically, a lot of the tools are around how to hire the right people. Assessments are a big part of the platform as are interviewing solutions and video interviewing, which is something that has really taken off during the pandemic, as obviously a lot more companies are doing their interviews remotely or virtually. There is a whole suite of tools oriented around helping companies hire better.

The way I got into it was actually the startup I mentioned. After the acquisition, I moved out to LA to work for them. And somehow, after a couple of years, I got involved in hiring, which I was totally unqualified for at the time.

There were a couple of experiences, but one in particular, where I was in an interview with a candidate; I distinctly remember looking up at the clock on the wall about seven minutes into the hour and realizing that it wasn't a fit on their side. That kind of caused me to think about how we can prevent bad interviews. How could you avoid all that waste of time in interviews where the fit wasn't really good, probably on either side? That was the kernel of the idea that became Criteria.

Karen Mangia

Say more about how your screening process works. We probably all hear you talk about this and picture some horrendous assessment we've taken that was either inaccurate or so dated we did not believe it could possibly create an accurate picture of who we are and our potential. Say more about what kinds of screening criteria are important now.

Josh Millet

Many people are familiar with assessments like the Myers–Briggs and it can be an engaging experience to see your results, but it's based on 80-year-old science so it shouldn't be used in the modern world for hiring or anything like that. Our assessments, in particular, are pretty varied as there are all sorts of different types of assessments. There are aptitude assessments; there are personality assessments. There are also more kinds of straightforward assessments like skill assessments. Think about computer literacy or the ability to use Microsoft Excel or something similar. Recently, we acquired a company in Australia that had a great emotional intelligence assessment and have gotten into emotional intelligence as an area as well.


As to the kinds of screening available now — the key to all assessments is to develop and validate them in a way that makes them reliable, and predictive of outcomes based on sound science, most of which is drawn from decades of work in Organizational Psychology. As an employer, as long as you're comfortable and have asked the partner or the vendor that you're using about the science behind their assessments, you’ll find they are a great way to predict outcomes, especially when you compare them to the more subjective things that companies often use. And I'm sure we'll maybe talk about those later.

Karen Mangia

Some people would challenge the idea of using an assessment to screen for emotional intelligence. What would you say to those skeptics?

Josh Millet

Emotional intelligence is a really interesting area. It's relatively new in terms of the science behind it. It's also not universally applicable to all jobs as there are some jobs where it's very important; there are others where you really shouldn't be assessing it because it's not that related to job performance.

The emotional intelligence assessment we have is actually game-based. It involves reacting to images of people's faces, making sure that you're emotionally aware, and recognizing emotions properly because that’s the first leg of emotional intelligence is being aware of other people's emotions and reacting appropriately.

Karen Mangia

It might surprise our audience to discover your bold statement that resumes are the enemy of candidate diversity. Say more about that.

Josh Millet

That's right. I think there's a pretty widespread recognition in the world of HR that in the world of hiring today resumes are limited in terms of their utility. I would go further and say they're actively often the problem in hiring today. There are a couple of reasons. One is that they don't work very well and—to your point—they're really bad for diversity. If you think about it, the resume is entirely backward-looking. Think about what a recruiter gets out of a resume. You hear all these terrifying stats about the average hiring manager or recruiter spending six or seven seconds reading a resume. So, in that short amount of time, what are you getting from a resume?

Often, you are getting two things: one is some sense of the amount of experience in a given field that someone might have or might not have; the second is an educational pedigree, like where you went to school, where you got a degree from if you got a degree, etc. If you think about those two things, there are a couple of problems. One is that much research has shown pretty definitively that the amount of experience you have in a given field for most jobs is not a very good predictor of future performance. The second thing is that educational level or whether you have a degree from a good school or not has some correlation to how you're going to do in the workplace, but it's a pretty weak correlation. It's what statisticians would call a weak signal. There are many better signals out there. For those two reasons, we don't think they work very well.

On the diversity front, they’re also very backward-looking. They reflect the opportunity you've had in the past and what you've done in the past; they don't really say anything about your future potential. And, if you're trying to build a workforce that is more diverse, basing it off of a look backward is quite the wrong approach. To use an extreme example, I would argue that if you're looking for ten years of experience in a field, and you're trying to build a more diverse workforce than you had ten years ago, modeling your future workforce after what the workforce looked like ten years ago is not a good idea. We prefer selection criteria and looking at things that are forward-looking and highlight the potential ability to learn and grow to develop that kind of thing.

Karen Mangia

I'm a big fan of the book, What Got You Here Won't Get You There. So many of the concepts you are highlighting show up for me. Because what we're missing when we only look at someone's past performance is the context for realizing their potential in the future. How do organizations get more proactive? I mean, what should we all be screening for now in this pseudo-post-pandemic world of talent?

Josh Millet

Whether you use assessments or other techniques, we think the things you should look for are pretty, pretty constant. We know that certain things are predictive almost across the entire job universe, certainly across a broad swath of it in terms of predicting long-term performance and potential, because too often, when people talk about potential, they're using that as an excuse for making subjective decisions, like I think this person has potential. We don't want that either as that's going to create as many problems as what we had. I think you should focus on what we call long-term talent signals.

We know—and this is not going to sound like rocket science—there's a ton of research that shows that people who are smart, who are good problem solvers, who are good learners who learn, digest, and apply new information quickly, as well as having qualities like a great work ethic, being persistent, having grit—those things all tend to correlate with long term success across a lot of different jobs. So those are some of the things you can do to really make sure you're predicting long-term success is look for those kinds of qualities and try to measure or assess them in an objective way.

Karen Mangia

We all read these headlines about the war on talent or in the world that I live in sometimes it's the struggle for tech talent. There's this belief that talent is a scarce resource for which we're all battling. Is that true? Or how do we need to think about this war on talent differently?

Josh Millet

It’s a war metaphor, so obviously, it has limitations, but I think that's right.

You mentioned the tech sector and where we stand now in 2023. The narrative that is so prominent in the media, based largely on the tech sector, is that there are layoffs, there's downsizing. That is all very true in the tech sector. But you look beyond—and I run a technology company, so it's kind of my world as well and it's my employees' world they see the layoffs on LinkedIn—it is certainly true that after a period of incredible growth, this is probably a pretty sensible, if painful, correction in technology, but look beyond technology. We have customers who are in technology, but 90% of our customers are not. They're in a whole host of other industries outside of technology, and maybe a few other fields that are technology adjacent like media, etc. There is a fundamental labor shortage still going on, and it's actually getting worse.

I believe we're going to be faced with a labor scarcity that's going to be very long-term and that we're already well into it. Obviously, even with all the layoffs and downsizing in the technology sector, we still get sub-4% unemployment. And, when we talk to our customers outside of the technology industry, their primary dilemma is they can't find enough qualified talent. One of the primary breaking mechanisms on their growth is they can't find enough people to do the roles they have open. If you look out on sort of a 10-year time horizon, and you look beyond tech, that is the biggest challenge for companies in terms of optimizing their talent strategy.

Karen Mangia

What I'm experiencing with a number of guests on the show and organizations that I work with—and I'd be curious to get your perspective—is that thinking about the talent attraction and development process is becoming a longer life cycle. It's now how do we go not just back to universities and become a hiring brand of choice? It's going back to the thought of internships and even apprenticeships to say, how do we cultivate our own talent pipeline? What's your perspective and are you doing any breakthrough work in that space?

Josh Millet

It's a great point. I think the focus on things like apprenticeship, which we haven't talked about much until recently in the economy, are all a function of that basic skills gap as some people would call it. I think of it in sort of more straightforward terms as a supply and demand imbalance in the labor market. I think all of those solutions, some of which are old and some of which are newer, are a reaction based fundamentally on the fact that we don't have enough people who have the right skills to do the jobs that we have open.

Another area that's very much topical for our customers, especially the enterprise customers that really do a ton of hiring, is this concept of talent mobility. It's got a lot in common with apprenticeships and that kind of thing, because it's about having someone in an organization who maybe doesn't have a path up in their current role, but based on their transferable skills, they might be a great fit for some other roles in the organization. That's a conversation I was never having with our customers five years ago. I didn't even know what that term meant five years ago, and now it's very much top of mind for chief people officers and people who run HR at companies. The thinking is: I have this people asset and rather than thinking of it as linear in all cases, how do I optimize it by giving people multiple routes up? You hear a lot about upskilling, but you also hear about redeploying people with skills; maybe their path upward is blocked in one area or is not realistic in one area. How do you redeploy them to another that might work for them, as well as the organization?

Karen Mangia

Your comments about talent mobility remind me of a leader I used to work for who said to me that one of the ways that I'm measuring you is whether or not you are a net exporter of talent from your team, meaning do you find a diamond in the rough, a high potential person, then sort of hoard them and therefore force them to be stuck or hitting a ceiling? The other side of it is exactly what you describe, which is: do you see that someone has a skill where they have the opportunity to be more successful or have the next step in their career somewhere else? And I thought, what a great guiding principle related to talent mobility as leaders — are you a net exporter of talent?

Josh Millet

A great way to think of the success of a leader is whether he or she is developing talent in such a way that it becomes in demand. I had a head of marketing once who was really good at that as well. Sometimes, it meant people left the org, but, ultimately, for the people involved it was really good. You know, he was maximizing there. He was helping them develop them in a way that would make them desirable within and without the organization.

Karen Mangia

You're talking about measurements and metrics of winning and that called to mind for me your 2023 candidate experience report. You've got a whole study where you've studied the candidate experience. Tell us more about what you've discovered. People always like to know more about benchmarks and real data as opposed to just theoretical conversations. Tell us about the study and what you’re discovering.

Josh Millet

It's relatively new. We've always, or for a long time, measured employer sentiment around hiring. Recently, we said let's look at the other side of the picture as well; it's a candidates’ market. This is relevant for many employers. They want to know how to attract, retain, and construct their hiring process in a way that's appealing to candidates. I believe this is the second annual one where we look at candidate experience, and what's important to candidates.

Some of the findings are pretty straightforward and maybe not earth-shattering. Candidates like transparency in the process; they like timely responses. It is not surprising in this market that having an overly long process where the candidate isn't given very frequent insights into where they are is really a death knell for recruiting practice. I think it's really telling when you look at even some of the top employers now—this started in tech, which is now softening—and it's kind of spread out to other areas. Some well-known employers that have great employer brands have had systems where they do six or seven rounds of interviews. And that's going away, I think pretty definitively because candidates aren't putting up with it. They'll take another offer if you're taking that long to make a decision on them. That's one thing that definitely came out is accelerating your process, keeping the candidate at the center of it.

Another finding that echoes what we were beginning to see last year is that candidates like the transparency piece. Candidates really appreciate knowing how they're evaluated. One of the core parts of our product is assessments. Some HR people who we talk to sometimes raise transparency as one of their objections thinking that they don't know if candidates are going to like this. It's not an outrageous thought. We realized at Criteria a long time ago that we love assessments a little more than some of the candidates that are out there. I think what came out is that the current legacy hiring process gives you very little feedback as a candidate on how you're doing. Sometimes you just submit your resume or your application. You never hear back or you hear back intermittently that you've made one stage and there's another ask for you to do something.

Some of our assessments, for example, will come with feedback for the candidate. They get a readout that's different from what the employer gets, obviously, but it's oriented around growth and development and strengths and opportunities. We saw on the survey that a growing number of people prefer a transparently objective technique for measuring them, whether it was assessments or something else, like a structured interview, versus submitting their resume to the ATS abyss, and hoping for the best and not hearing much back. I think there are a lot of candidates now who know a little bit about how the technology works in a lot of companies around sort of keywords and resumes and things like that. And, some of them do not want to play that game. There are not very many winners in that game, in my view. I think that that was one of the trends too, candidates will put up with a rigorous sort of evaluation process if they have an idea of how it works, and why it's being done. Because ultimately, they want to be on a level playing field. And so they appreciate that.

Karen Mangia

Imagine going to take the LSAT test, and then you get there and find out you have to do a painting. I mean, you would be so shocked, right? The reality is, in that you're set up for success by knowing what you're going to be assessed on. What you said there is so powerful in terms of transparency and trust in that process. People want to know, what are the criteria? Because outside of that, you start to wonder, is it a likability contest? Are we mysteriously labeled as cultural fit? We're screening for cultural fit. What you're describing is so critical, it's—share the criteria. And I think everyone can relate to that experience of going through even an intense interview process with multiple rounds, not getting the job, and not getting any feedback. And you're left wondering: Why?

Josh Millet

That's right. And don't get me started on cultural fit because I think you're exactly right that it's going to be subjective. And it's not saying that culture is not incredibly important at companies. It is. It's a key barometer of the health of an organization and all those things. But you don't build your culture solely by looking for cultural fit. The analogy I like is in the US presidential election where they sometimes give surveys of which candidate would you rather have a beer with. It’s a proxy for which one's the most likable and which one's the most down to earth and you could sit at a picnic table with. But ultimately, which one you want to have a beer with should not be the criteria for running the free world. There are more important things there than who is the most likable. I like that you picked up on cultural fit because it's a conversation customers often come to us with. We try to redirect the conversation to yes, you should certainly avoid people who will make your culture toxic or, you know, be counterproductive, but let's reframe that question.

Karen Mangia

Speaking of toxic cultures, managers, and leaders, I have to believe you have a different approach to screening great leadership candidates. Anyone who's ever had a bad boss, which is everyone, would love to know: What are the criteria on which we should all be evaluating great leaders and people leaders of organizations given we've had such a significant context shift?

Josh Millet

There has been a big shift over the last few years for managerial roles. There are some things we know that work very well, critical thinking and problem-solving being one of them. In general, when you are thinking about aptitude and intellectual ability, it becomes more and more relevant the higher up you get in an organization. There’s a minimum bar there. There's also a series of personality traits that tend to correlate with success in managerial roles. That has sort of been an interesting thing to watch as, as the paradigm of what makes a good manager culturally is shifting.

If you look at a 20-year timeframe of how a good manager was portrayed in the press or movies, 25 years ago might be considerably different than now. Things like assertiveness, while still important, faded and gave way to a little bit more of an emphasis on empathy and emotional intelligence. When we were six to nine months into COVID, and a lot of folks were working from home, we got a lot of questions from new prospects and existing customers. There was a lot more interest in emotional intelligence because emotional intelligence for managers has always been a part of the picture; it's always been important to know when your team is strained or when an individual team member is suffering. Recognizing that can obviously be helpful. I think the bar was raised there, I think it is harder to do that through Zoom. It's a flatter medium. Knowing when the team dynamic, or an individual, is off or struggling or needing support can be more difficult when you are going through video versus in person. In a sense, maybe some of those abilities that I think have always been a part of what makes a successful manager are coming to the forefront more in the last couple of years.

Karen Mangia

When I think about your twenty-five million assessments and growing that you've administered—we have members of our audience who aren't ready to invest yet and would probably benefit from knowing what is even one question they could ask that's new, different, refreshed, more insightful in the interview process as soon as their next interview?

Josh Millet

We developed structured interviewing tools as well. Our big emphasis there is making sure that the questions you ask are job-related. Most employers know a few key qualities, even if they haven't taken the time to formally do a job requirements analysis rigorously. They know a couple of the qualities that are really important. Asking any questions you can that demonstrates real kinds of scenarios. We call it behavioral interviewing, which is designed to measure those qualities and how they'd be applied in practice. That will obviously differ from one job to another.

Our big message around interviewing—because there's so much evidence and my story that I told you upfront is almost an example of it—is that a lot of interviewers make up their minds about an interviewee in the first five minutes. It's something like 30% have already formed an impression about whether that person's a fit or not. I took seven minutes, so I wasn't quite as guilty, but it's the same kind of thing. If you think about what happens in the first 5-10 minutes of an interview, it's not terribly substantive. You're judging the appearance, the demeanor, maybe making small talk and discovering some superficial things you might have in common or not.

Our big advice with interviewing is to do it in a structured way and make it more like an assessment. Ask the same questions of all candidates. It doesn't mean you have to be stiff and robotic. You can ask follow-up questions that aren't scripted but have three or four core questions that you're going to ask everyone and try to grade them in a rigorous way. If you can have one amazing quality—like you're super charismatic, for example—you don't want that to overweigh the other four things you were supposed to be looking for in the candidate. So you want to come up with a rigorous way to grade them and weight it and say if there are four things you're looking for, let's grade all the questions at five points or whatever. Doing it in that way helps remove a lot of bias from the process,

Karen Mangia

Removing bias is critical to a more diverse talent pipeline and workforce. What do you want your legacy from this work you’re doing to be?

Josh Millet

Besides helping employers get better at hiring and helping them build better businesses, we really feel that the approach we're taking, which is evidence-based and science-based, is really helping to make the world of work a little bit fairer and more equitable for all and I mean that on both sides—employer and the job seeker. My mission is kind of aligned with the companies, which is helping to make the world of work more fair.

Karen Mangia

Speaking of the world of work, people say that they miss spontaneous water cooler interactions. That's why I have a segment on the show to replicate that experience. I am going to ask you five quick questions. You just say the first thing that comes to mind. It is as if we're at a virtual water cooler together. Are you ready?

Josh Millet

All right.

Karen Mangia

What time of day do you do your best creative work?

Josh Millet

11:00 a.m.

Karen Mangia

Speaking of time, imagine miraculously, every day now has twenty-five hours instead of twenty-four? How do you spend your extra hour?

Josh Millet

Hopefully with my family.

Karen Mangia

That’s awesome. If they’re reading, you get huge kudos. We hope they are. If you had to eat one meal every day for the rest of your life, what would it be?

Josh Millet

I would have a weight issue if I did this, but Cacio e Pepe is my favorite kind of pasta.

Karen Mangia

Now imagine the zombie apocalypse is coming. Who are three people you want on your team?

Josh Millet

I am laughing because my wife has admitted that, if in that scenario, I would not be one of the ones for her. I don’t tend to be too handy around the house, which she feels probably rightly is incredibly important in that event. My in-laws are pretty good in that way, I would probably take one of them. My son is very . . . my 11-year-old is very handy. I would probably take him, but we don't want to get into hard family choices, so I'm going to have to think about this more. That’s a really good question

Karen Mangia

When you can make someone with a degree like you have in French medieval history think, I feel like it’s a winning day for me.

Josh Millet

Let me tell you the recipe for who you want in a zombie apocalypse is not the medieval historian.

Karen Mangia

It is not the podcast hosts either, believe me. Last question: How can our audience stay in touch with you and keep track of your thought leadership and what you are doing?

Josh Millet

The best way to find me is on LinkedIn. They can check me out, Josh Millet on LinkedIn. There is also a ton of information on our website if you are interested in exploring assessments or the types of tools we have been talking about today. You can go to criteriacorp.com, which is our main website.

Karen Mangia

Thanks to Josh Millet, founder, and CEO of Criteria for joining us today on Success from Anywhere. Because success is not a destination. Success is not a location. Success is available to anyone, anytime, anywhere.


1 view0 comments

Recent Posts

See All
bottom of page