We Salute You, Founding Fathers of the NFL’s Analytics Movement – The Ringer

The NFL is valuing youth and innovation more than ever before. A year after the Rams made Sean McVay the youngest head coach in league history, Patrick Mahomes became the youngest MVP winner since Dan Marino. This offseason, an avalanche followed: The Cardinals threw caution to the wind and paired Kliff Kingsbury with Kyler Murray, the Packers ended the Mike McCarthy era, and the Bengals poached the Rams’ quarterbacks coach to be their new head coach. When did the NFL begin to resemble Silicon Valley? Welcome to Wunderkind Week, where we’ll dive deep into how the NFL became a young man’s league.

---

Brian Burke realized something was wrong when he noticed all of the commotion in the lobby of the Marriott Hotel in Karachi, Pakistan. It was September 2008, and almost 900 miles away, a dump truck carrying more than 1,300 pounds of explosives blew up at the entrance of the Marriott Hotel in Islamabad. The terrorist attack left more than 50 dead, at least 260 wounded, and a crater 25 feet deep. Burke, a former Navy fighter pilot turned weapons and tactics expert, was working in Pakistan for a U.S. government contractor based in Reston, Virginia. When he returned to his room, he received a phone call from his sponsors who explained the situation and ordered him to leave the country. He was told that next day, before sunrise, guards armed with AK-47s and shotguns, riding in flatbed trucks, would escort Burke and his colleagues to the Karachi airport. As he waited in his hotel room for the convoy to arrive, Burke got antsy and needed something to occupy his mind.

“I could either watch a bunch of House reruns, or I could do something to exercise my brain,” Burke said.

He chose to return to the passion project he’d been tinkering with during his trip: win-probability charts for NFL games. Burke declined to elaborate on the nature of his work as a contractor, but said it required a lot of overseas travel, which meant long flights and a lot of downtime. During flights to Singapore, Taiwan, Salalah, and the high deserts of Oman, he analyzed play-by-play data from NFL games to evaluate how different decisions affected the likelihood of winning, and how to make the information easily consumable. He drifted back to this work whenever he found quiet moments (and sometimes not-so-quiet moments). You may have seen these charts before. Here is the one from the New England’s infamous 34-28 overtime victory over Atlanta in Super Bowl LI.

Burke’s win-probability charts are useful in other ways besides making fun of the Falcons. What started as a hobby in 2008 eventually became an essential tool in the first wave of football analytics. Whereas Moneyball, Michael Lewis’s seminal 2003 book about statistical analysis overtaking conventional wisdom in baseball, was famous for changing how MLB front-office executives build teams, win probability changed how NFL coaches make decisions during games. For all of NFL history, decisions like punting on fourth down or going for it, kicking an extra point or going for two, and whether to be aggressive or conservative in overtime were a mix of received wisdom, gut feeling, and unscientific charts. Burke’s win-probability model put a percentage on whether a decision like going for it on fourth down from midfield would help or hurt a team’s chances of winning and by how much. It could account for every conceivable scenario involving score, time, down, distance, and field position.

“I did it for fun,” Burke, who joined ESPN as a senior analytics specialist in 2015, said. “I didn’t think this would become a thing or a career or make money. It was just for fun.”

Burke was one of a handful of people who, at the time, were creating databases and analyzing them. Their findings have changed football. Data analysis has already altered baseball’s DNA, from on-base percentage to wins above replacement to spin rate. It’s been applied to basketball, where 3-point attempts have increased by 77 percent in the last 10 years while midrange jump shots have become an endangered species.

Advanced stats arrived late in football, but their effects on the sport are undeniable. The Philadelphia Eagles proudly attributed their trademark fourth-down aggressiveness during their Super Bowl run (highlighted by the “Philly Special”) to analytics. Teams copied that strategy in 2018, shattering the record for fourth-down attempts with 16.8 per team. In 2019, every NFL team is at least waist-deep in exploring these previously uncharted waters. But long before teams were aware of how spreadsheets could help their teams, Burke and a handful of other like-minded people recognized that the way we talk about football was limited. Fueled by love, passion, and a little too much time on their hands, they created a new language.

“The word ‘analytics’ didn’t even exist at the time,” Burke says. “If you get an old copy of [Microsoft] Word and you type in ‘analytics,’ it will tell you it’s a misspelling. It doesn’t exist in the dictionary. So I just called it ‘advanced stats.’ That’s what I thought of it at the time.”

Aaron Schatz, editor in chief of Football Outsiders and a forefather of the football analytics movement, discovered his passion for the sport while working as a disc jockey in Daytona Beach, Florida. Schatz spent his teenage years in Sharon, Massachusetts, five minutes from Foxboro Stadium. But the Patriots “were not a very big deal” at the time, and it wasn’t until he moved to Florida that he started following the team, mostly, he said, to stave off homesickness. It helped that the Patriots made the Super Bowl that season, 1996. New England lost to Green Bay, but Schatz was hooked.

It wasn’t until 2002, after Schatz had moved to Boston, that his interest in football analytics began. Schatz fumed at a Boston Globe column that insinuated the Patriots missed the playoffs that season because they did not establish the run. Schatz disagreed and wanted to refute this logic, but there was no database of NFL plays he could use to test his theory. So he made one by copying and pasting more than 30,000 lines of play-by-play data from ESPN.com box scores. Schatz, who by his own admission is not “a computer programmer,” used a week of vacation over Christmas to compile the data until he had the first 16 weeks of the season, and then added Week 17 after those games were completed.

Schatz disproved the establish-the-run theory, but his big breakthrough came when he attempted to determine which team had the best running game. He borrowed a concept called success rate, which was originally laid out in the seminal 1988 book The Hidden Game of Football. Success rate addresses one of football’s simplest truths: All yards are not created equal. Gaining 5 yards on third-and-4 is more valuable than gaining 5 yards on third-and-10. Context is crucial, but nobody accounted for those distinctions until Schatz. Using his play-by-play library, he awarded points for each play deemed “successful” per the guidelines in The Hidden Game of Football (e.g., a 1-yard run on first-and-10 is worth zero points, but a 4-yard run on first-and-10 is worth one point.)

Once Schatz crunched the numbers, his formula determined that the most successful running back in the NFL was … Tampa Bay fullback Mike Alstott.

“That’s weird,” Schatz remembers thinking. “That can’t be true.”

Alstott was a short-yardage back who got the ball in situations conducive to success (like third-and-1), making him a bad comparison to players who received twice as many carries across many different scenarios. To solve this, Schatz compared every running back against the league-average success rate in the same situations (for instance, how does Mike Alstott do on third-and-1 compared to how the average running back does on third-and-1?) and made an adjustment to his formula. This tweak produced a different running back as the best in the league—Kansas City’s Priest Holmes.

Schatz called this formula defense-adjusted value over average, or DVOA. Seventeen years later, DVOA is still the gold standard for football statistics. The formula is so effective that DVOA ratings are more accurate at predicting a team’s future win total than the same team’s win total from the previous season.

Schatz invented DVOA to prove a Boston Globe columnist wrong, but Neil Hornsby created Pro Football Focus because he thought only one writer was right. Hornsby, who was born and raised in Workington, England, got into American football after watching his first game as a student at the University of Liverpool in 1983. By the 1990s he was digesting every football magazine he could get his hands on, including Sports Illustrated, which he read when copies were brought to the office by Kenny King, one of Hornsby’s coworkers and the brother of Sports Illustrated writer Peter King. But Hornsby’s favorite writer at SI wasn’t Peter King. It was the legendary sportswriter Dr. Z, a.k.a. Paul Zimmerman, who published a unique All-Pro team each year along with his grades for each player on each play of the games he watched. Zimmerman’s process was so thorough—he had been charting players since 1947—that it made Hornsby doubt everyone else he read, especially when they made specific criticisms such as saying one player was a really good pull blocker.

“I’ve always been a skeptic a little bit,” Hornsby said. “I’m thinking to myself, why are you writing this? You are either doing a ton of work that nobody knows about, or you’re just making bullshit up.”

Hornsby felt Zimmerman’s scouting-based grading was the only way to analyze football, but Zimmerman was only one person. Hornsby wondered whether a few people could apply Zimmerman’s grading system to every player in every game. That idea—Every player and every play of every game—is still the motto for PFF, which now has 85 full-time employees, 550 part-time employees, and works with every NFL team and dozens of NCAA Division I programs. But the real reason Hornsby founded the company in 2007 was that he loved football and had nobody to talk to about it.

“I’d go to the gym and train with me buddies, and all they wanted to talk about was Liverpool against Chelsea,” Hornsby said. “Nobody gave a monkey’s about who was doing what in the NFL. So when I started PFF, it was really to try and have some interesting conversations with high-end fans in the U.S., and I think what we did was we sort of overshot our target market a little bit.”

Schatz, Hornsby, and Burke weren’t doing anything particularly unique—they were applying new technology to concepts originated by their predecessors. Dr. Z didn’t have the time or technology to watch every game. The authors of The Hidden Game of Football could not properly test success rate in the 1980s. Virgil Carter, who in 1971 created a model for expected points and is considered the founding father of this entire football analytics movement, was using punch cards to run his numbers on an IBM computer and could plug in only so many variables. When technology caught up to their concepts, everything changed.

“Really what happened was there was this confluence, this lucky moment in time where we had the internet, the data became freely available at the same time, computing horsepower got good enough, [and] you could do these sorts of things in a reasonable amount of time on your own PC,” Burke said. “I would equate it like [Leonardo] da Vinci and the Wright Brothers. They had these ideas for the flying machine, but they just did not have the technology, they didn’t have the engineering to make it work. And then the Wright Brothers came along and put it together.”

Creating the advanced stats was one challenge, but trying to explain a foreign language to NFL coaches and executives would generate a lot of miscommunication.

If Burke and Schatz were trying to write a new language, Frank Frigo was the traveling salesman trying to spread the gospel. In 2001, Frigo, an energy wholesaler and former backgammon world champion, teamed up with Chuck Bower, a physicist at Indiana University, to create a win-probability model called “game-winning chance” that could be customized to different opponents. (Their game-winning chance model would eventually become a pillar of the company they formed, EdjSports, which bought Football Outsiders from Schatz in 2018.) By 2004 they had arranged a meeting with the Cincinnati Bengals for their first presentation to an NFL team, which would include members of the Bengals front office as well as then-head coach Marvin Lewis. But Chuck, who drove to Cincinnati from Bloomington, did not pack shoes he could wear to the meeting, so they had to scramble to find him a pair.

“He’s an astrophysicist,” Frigo said. “He gets sidetracked.”

Even in their early, rudimentary models, it was clear teams were way too conservative on fourth down. Optimizing fourth-down aggressiveness alone could be worth one win per season. Entering the Bengals meeting, Frigo and Bower envisioned their business becoming the McKinsey for NFL teams.

“I’m walking in the door thinking, ‘If I can get you a win a season … ’” Frigo says. “Today, [NFL franchises are] worth over $2.5 billion. I mean, what’s a win worth, especially if you’re a midrange NFL team? It might be the difference between making the playoffs or not. That’s got to be worth a lot of money.”

The Bengals disagreed, according to Frigo and Bower. Their pitch was built on the assumption that coaches cared only about winning or losing, not how much they win or lose by. In backgammon, Frigo didn’t care whether he lost 17-16 or 17-0; he was focused on maximizing the odds that he won. But in their meeting with the Bengals and other teams in the NFL, they found that the score matters in football. Blowout losses are dangerous to their jobs, demoralizing to the team, and a hindrance to the hiring prospects of coordinators. That made coaches less willing to be aggressive, even if it would give them a slight edge to win. A percentage point here or there wasn’t worth the heat in the postgame press conference if the move failed.

“To come into this NFL culture where all of a sudden I’m finding out that there’s other motivations that aren’t necessarily perfectly aligned with win probability was a real eye-opener for me,” Frigo said. “First of all, there’s these other sort of risk-management considerations and biases that [coaches] have. Secondly, they just don’t think probabilistically. I mean, that’s the part that struck me.”

Frigo remembers former 49ers general manager Terry Donahue putting it bluntly:“The biggest challenge you guys are going to have,” Donahue told him, “is you’re dealing with a bunch of PE majors.”

Teams became more receptive, albeit at a glacial pace. Moneyball was released in 2003, and the ideas within it were slowly but surely catching on. Some coaches were ahead of the curve. In the summer of 2004, then–Titans defensive coordinator Jim Schwartz (now working in the same role in Philadelphia) invited Schatz to Nashville to discuss DVOA. After their debrief, Schatz admitted he had never played the game and lacked basic Xs and Os knowledge, so Schwartz and an assistant took Schatz into the film room and taught Schatz the basics of zone defense.

“At that point, it became clear that this might actually be a full-time job,” Schatz said. “I’m like, ‘Well I’ve got to throw myself into learning as much about football as I can.’”

Burke had an easier time breaking in. In 2007 he launched his own website, Advanced Football Analytics, and began posting his work. Within two years, he got an email from a The New York Times editor asking him to write a column.

“I was like, wow, The New York Times,” Burke says. “That seems like a pretty good place to start.”

While writing columns for the Times, he developed a special project called the 4th Down Bot, which spit out the probabilities of Burke’s win-probability model on the Times website and under its own Twitter account (both are defunct now that Burke works for ESPN). With his credibility from the Times and later The Washington Post and Sports Illustrated, it didn’t take long for NFL teams to start calling.

“If this guy writes for The New York Times, he’s not some guy in his pajamas in his basement,” Burke said. “Which I kind of was.”

His Times byline got him in the door, but it was his military background that made coaches and executives take him seriously.

“In the NFL there are very few people they look up to and have a lot of respect for,” Burke said. “Most of us, they look at us like we’re squirrels. … If you walk into a coach’s office and you give him a presentation, it didn’t hurt that I was a military veteran and not a professor.”

But even he could not crack the NFL’s obsession with secrecy.

“They’re open-minded to talking to outside contractors, but they would never let somebody come in as a consultant and completely run their analytics group, which is what we were hoping to do,” Frigo said. “They want you to be in-house. What they like to do, is they like to hire some young Ivy League kids and ask them to run certain kinds of analysis. That’s the current state of the game.”

With that in mind, it turned out that the best business model wasn’t data analysis, as Frigo or Burke were pitching, but creating new data and selling it. That’s how Pro Football Focus went from a ragtag operation of a few tape-grinders in the United Kingdom—who spent their time rewinding game broadcasts on DVDs for 40 hours per week on top of their full-time jobs—to an organization that charges a hefty fee from NFL teams and college football programs.

“I’ve never seen myself as anybody particularly doing analytics in football,” Hornsby said. “I’ve only ever seen us as doing data collection or production-based scouting.”

Hornsby received an email from the New York Giants in 2009 asking him to call them, which he thought was so absurd he assumed his friends were pranking him. They were not. Within five years of that phone call, PFF was working with 13 teams, and Hornsby sold a majority stake in the company to NBC’s Cris Collinsworth and remained on as the COO.

“I am not a mathematician,” Hornsby says. “I’m not a data scientist, and I don’t class myself as an analyst … I’m a business person.”

One of his most satisfying business acquisitions was finally acquiring the web domain name for PFF.com after years of being stuck with profootballfocus.com. The previous owner of PFF.com was the Pakistani Football Federation.

In Week 3 against the New York Giants in 2017, Philadelphia Eagles head coach Doug Pederson went for it on fourth-and-8 from the Giants’ 43-yard line with just over 2:30 to go in the first half. Quarterback Carson Wentz was sacked, but Pederson explained himself the next day by saying that the decision improved their chances of winning by 0.5 percent. Frigo and the people at EdjSports were floored. The head coach of the Eagles, one of their clients, was citing one of their metrics.

“I’ve never heard that language coming out of an NFL franchise,” Frigo told The New York Times.

When the Eagles marched to their Super Bowl LII victory over the New England Patriots, crediting analytics along the way, they also changed the language of football. The NFL is a copycat league, and teams began replicating Philadelphia’s analytically minded aggressiveness. The information is more accessible to fans and media members alike, and coaches are now criticized for being too conservative more than they are for being too aggressive. If baseball and basketball are any guides, this influence on in-game coaching is just the first frontier of what will likely be a sweeping change over the next two decades.

Nobody knows what the next breakthrough will be, but the NFL is so heavily invested in creating more data that somebody will surely figure it out. The league has inserted chips into the shoulder pads of every NFL player to let teams analyze player-tracking data. In February, the league hosted the “Big Data Bowl” at the NFL combine and invited students to submit papers assessing data on a range of topics including the best route combinations to get receivers open, how to lower the concussion rate on punts, and how to quantify the importance of speed on the field. Burke believes the next breakthrough will be quantifying how often quarterbacks make the correct decision on the field, and his current passion project is using machine learning to better understand player-tracking data. As with the first wave of breakthroughs, it’s an obsession for nerds, and it could develop a new language to discuss football that informs every play called and every decision made.

“If you love something or are passionate about something,” Hornsby said, “You will do things that most people would find ridiculous.”