Viewers who watched the game in Prime Vision with Next Gen Stats, one of Amazon’s three broadcast options, saw the unveiling of a feature called Defensive Alert that is powered by artificial intelligence to identify potential blitzes before the snap. The model highlights players it believes have a high probability of blitzing (crossing the line of scrimmage to rush the passer) with a red circle that appears under them.
As Sam Schwartzstein, one of the minds behind the model with Prime Vision and the Amazon Machine Learning team, eagerly watched his model work its magic on every snap, on one play, he became confused at why his program was highlighting a nickel corner who wasn’t giving any indication that he was blitzing. “Why are we highlighting this guy?” Schwartzstein yelled in fustration.
Was this a flaw? Did the machine get this prediction completely wrong? Schwartzstein, a former offensive lineman who played at Stanford with Andrew Luck, prided himself on being able to identify potential blitzes. His years of experience as a player and analyst told him the nickel wasn’t much of a threat.
Right before the ball was snapped, the inside linebacker dropped to the nickel’s side and the nickel finally moved toward the line of scrimmage. The program sniffed out this blitzer well before Schwartzstein, watching the game from a wide-angle camera shot. The weirdest part about this is no one really knows how Defensive Alert did it. It’s a self-learning program that has analyzed thousands of plays and movement patterns to understand how defenses move as a whole when certain players blitz.
The model is trained not to identify the usual four down linemen that typically rush the passer. It’s trained to identify unique players who rush the passer on 60 percent or less of snaps. It’s being fed tracking data from Next Gen Stats, which is derived from RFID chips in every player’s shoulder pads. The data includes the players’ acceleration, their orientation and where they are facing. From all that data, the machine starts to understand familiar movement patterns from the defense as a whole, which helps it predict which player is going to blitz.
“We’re highlighting things, starting at line set,” Schwartzstein said. “It’s happening in real time as information is coming in from the shoulder pads. And so you can see all this data coming in and (the model) gets more confident the closer we get to the snap because defenders have to more clearly define their roles the closer they are to the timing of the snap. One of the coolest features for me is we’re not just highlighting it at one time and sticking with it. It is on and off based on where players are moving throughout the play on both offense and defense.” (...)
The goal is to get viewers to see the game as the quarterback does. The quarterback isn’t certain who is going to blitz, especially early on when the defense is showing its initial disguise. But as the snap nears, players start to move around to get close to where they have to, to execute their assignments. The initial alignment and movement help the quarterback figure out who is blitzing or not. The best quarterbacks are coming to conclusions from their wealth of experience or film watching. The machine is processing information the same way, but it has an abundance of data that has been fed into it to pull from in an instant.
Some skeptics believe Amazon is using a delay to see the blitzes coming and highlighting the player on the live feed. Though the processing required for Prime Vision to paint visuals does add some delay (usually three seconds or less), the model that powers Defensive Alert does not use that delay. The team has spent considerable effort to produce predictions as fast as possible — even installing dedicated hardware in Amazon’s state-of-the-art production trucks. There is no person or program trying to trick the audience about prediction capabilities.
Again, Schwartzstein doesn’t know exactly how the model is making some of these predictions. It’s learning on its own as it keeps getting data, but don’t worry, football purists. The model is also getting input from a panel of actual football people that includes former players and coaches like Andrew Luck, Geoff Schwartz, David Shaw, David DeCastro, Ryan Fitzpatrick, Andrew Whitworth, Nate Tice and Andrew Phillips.
The panel of experts reviews the film of the model making predictions and makes sure it’s identifying legitimate threats and not looking at players who could not be rushers to the well-trained eye. Some of their feedback, along with that of Schwartzstein, who provides feedback on every play, is fed back into the system. (...)
The panel of experts reviews the film of the model making predictions and makes sure it’s identifying legitimate threats and not looking at players who could not be rushers to the well-trained eye. Some of their feedback, along with that of Schwartzstein, who provides feedback on every play, is fed back into the system. (...)
Just to ease the minds of concerned fans, teams cannot use this model to their advantage in games. Communication with the quarterback is cut off after 15 seconds of the play clock has expired and there’s no way to get information to the quarterback fast enough. Also, coaches in the booth don’t have access to the Amazon broadcast. Technology usage is extremely restricted for teams. They don’t have access to tracking data during games, and even when they are looking at their tablets, they are looking at stills, not video.
by Ted Nguyen, The Athletic | Read more:
Image: Amazon Prime
[ed. Feels like AI could transform football strategy into something similar to what we see with chess these days - more statistical probability, less imagination/intuition. Maybe there'll be transistors on every moveable part of a player's body at some point (soon?).]
