Sabremetrics for Software Teams
“He’s got the look of a ballplayer.” That phrase was enough to make a purchasing decision in the millions. That changed with Sabremetrics. Sabremetrics is a series of a statistics designed by Bill James in the 70s. It has transformed baseball but was ignored for 40 years. James figured out that statistics we used to rank baseball players were inaccurate. For example, he found that batting average didn’t capture player’s impact. It didn’t include walks or differentiate between singles and home runs. James instead leveraged On Base Percentage and Slugging Percentage to value hitters. This led to a more accurate read on hitters.
The cost of misjudging a player is high. Baseball teams are enterprises. Teams compete for talent. The richest teams have budgets several times larger than their competitors. Picking up a dud can cost teams millions and getting a superstar is out of reach for small market teams. Scouts, the people charged with finding and ranking potential players relied on intuition. The top tier of players was easy to recognize and thus were expensive and out of reach for small market teams. In response to mounting costs, teams began to leverage Sabremetrics. Soon, great players who traditional scouts ignored were then discovered. Traditional scouts ignored David Ortiz. But, he had high on-base and slugging percentages early on his career. For the Red Sox, these were better indicators of his potential than the fact that he was chubby. Decisions like signing Ortiz are how the Red Sox won the World Series in 2004, 2007 and 2013.
Software teams are just as guilty for misjudging talent and missing the diamonds in the rough. Delivering working software that people will want to use is the goal of most SaaS software teams. Yet, what kind of traits are software teams looking for in candidates? If you look at lists of interview questions like Awesome-interview-questions, you’ll find only technology questions. Technical questions have value and you should be doing a technical screening. But we shouldn’t miss the other elements of working on a software team. How did they approach a problem they had never solved before? How would they? How do they ensure quality in their codebase? How do they interact with nontechnical team members? How do they leverage open source technologies? Do they mentor and pair program? How do they react to conflict? I’ve had candidates who were almost seething when I mentioned tabs and spaces. Do you think that person will keep their cool when something goes wrong?
With a competitive market, smaller teams need to find the undervalued players. But they also need to avoid talented professionals who are difficult to work with. Build a team of all stars without competing for the types of talent Google are fighting for.