Is the Coach's Job Safe?
Net Newsledger
November 15, 2008
Bill Belichick’s job as head coach of the New England Patriots is probably safe for now, but check back in 15 or 20 years – some of the newest research on machine learning, artificial intelligence and computer vision is using football as a guinea pig, and who knows what new pass formation it might come up with?
At Oregon State University, experts in artificial intelligence are using a Digital Scout Project to develop computer programs that can “understand” video of football plays. They hope to push the limits of what computers can see, what action they can follow, what they can figure out and maybe even improve upon.
The studies, supported by the National Science Foundation and the Defense Advanced Research Projects Agency, are trying to make advances in the field of computer vision that could ultimately have applications in everything from surveillance to robotics, industrial operations or even natural science and ecology. And maybe even football.
“It’s really a huge challenge to go from the very low level, raw, sensory data we get from a video camera to a high level of understanding and interpretation of the visual world,” said Alan Fern, an assistant professor of computer science at OSU.
“In terms of vision and understanding, my three-year-old daughter can easily do things that our most advanced computer systems are not even close to achieving,” Fern said. “The visual world is a very messy place that humans are generally able to interpret without much effort; it is easy to forget just how remarkable the human visual system is.”
But with the continuing advances in artificial intelligence and computer vision, Fern said, it may indeed be possible for computers to significantly improve on what they “see,” how they interpret that data and what sense they can make of it. And the fast-paced, complex but somewhat structured game of football, researchers say, is a pretty good place to start.
A computer looking at film of a football game, Fern said, doesn’t even see people or yard lines – just pixels that seem to randomly change color as the film progresses. At first it can’t tell a player from an official or even a field logo, and even though it might be able to detect the football as a blip that moves across the screen, it has no clue that blip represented a successful forward pass.
“At the beginning, our computer vision systems would look at a football game the way a newborn baby might see it, just random images and motion with no understanding of what’s going on,” Fern said. “But even a baby learns very rapidly from what it sees, because we are very visual creatures. Unfortunately, computers don’t have the advantage of our brain or vision systems. So this research is about figuring out how to best capture those capabilities in computer systems.”
The best way for a computer to understand football and other visual content, Fern said, may be with systems that can learn. In other words, provide the computer with some basic rules of the game and then let it watch a lot of football, 24 hours a day if desired, and learn from all of that experience. A computer never gets bored, no matter how hopelessly the home team is losing.
And it never has to adjourn to the kitchen for refreshments.
“Initially the computer will have many holes and errors in its understanding, but over time, as it gets corrected by humans, it will become increasingly proficient,” Fern said. “Each play watched and every correction provides statistics about typical behaviors of football players and how those behaviors correlate to the pixels in the video. The computer learns by exploiting those statistics to do better on future videos.”
Research is still at early stages, Fern said, and it will be a significant accomplishment if computer systems are soon able to watch a video and reliably answer questions about a play, such as whether it was a run or a pass, what receivers ran what routes, what type of defense was being used or who blew a blocking assignment. Working with graduate student Robin Hess, it has taken almost a year to develop programs that can accurately recognize players and understand basic football formations. Studies now are at the stage of building programs that can reliably track individual players throughout a play and recognize their activities.
That won’t put any college or pro coaches out of work anytime soon, of course.
“These systems will be fairly simple at first and will make a lot of mistakes,” Fern said. “But to the extent they can learn from those mistakes, and make fewer and fewer of them all the time, we may be able to develop systems that are surprisingly effective and useful. And it’s possible that at some point computers may be able to automatically analyze games of an opponent and make useful recommendations about strategy.”
What about some revolutionary new approach?
“A computer might come up with the next West Coast offense, though I don’t expect that to happen in the near future,” he said. “If nothing else, this project has helped me to appreciate the sophistication and brilliance of these football strategies and the coaching minds that developed them.”
With further advances, researchers say, such vision systems might also be very useful in surveillance, such as in airport security. They could help a robot move around and make better sense of what it “sees.” Many industrial applications are possible, and uses could develop in environmental monitoring or other natural science research.
And some day, a human football coach and a computer may literally be able to watch some film and talk to each other about game strategy. And if the computer system is good enough, the machine might even win the argument.
|
