In BYU’s Perception, Control and Cognition Laboratory, our goal is to build agents that perform at human levels in complex tasks. To accomplish this, we are building abstract, cognitively inspired models of the world, implementing them in a new, scalable probabilistic programming language, and using them to create model-based deep RL algorithms.
Some of our recent work includes:
- Improved depth estimation for augmented reality
- Automated Affordance Extraction via Word Vectors
- Theory of Mind for Autonomous Agents
- Advanced simulators that stress a variety of high level cognitive functions
Deep learning is one of the most compelling advances in machine learning in recent memory. It has swept over both industry and academia, crushing benchmarks and generating impressive progress across fields as diverse as speech recognition, parsing of natural scenes, machine translation, robotics, machine vision, and even the game of Go.
However, there are problems. The sample complexity of deep RL algorithms is prohibitive: typical agents only learn after tens or hundreds of millions of interactions with an environment, making most algorithms unusable in anything but a fast simulator. This stands in stark contrast to humans attempting the same tasks, who can perform well after only a few minutes of practice (or even merely watching another human practice for a few minutes).
How can we fundamentally reduce the sample complexity of deep RL, and simultaneously improve deep RL’s ability to solve complex tasks? Building on ideas from cognitive science, we believe the solution is to more closely mimic three key human cognitive capabilities lacking in current deep RL algorithms: (1) our ability to build models of the world, which allows us to (2) transfer knowledge from previous experience via abstraction, and (3) reason explicitly about our own uncertainty.
To accomplish this, we argue that we must combine the strengths of two frameworks: deep neural networks and Bayesian models. DNNs are unarguably superior at processing low-level sensor data such as camera images, while Bayesian models have been shown to exhibit human-level performance on a wide range of reasoning, model-building and cognitive tasks. Furthermore, Bayesian methods have been shown to have excellent sample complexity because of strong inductive biases that come from rich, structured hypothesis spaces; appropriately hierarchical priors allow these biases to be learned from data (sometimes called the blessing of abstraction), resulting in single-shot or even zero-shot learning.
Is this approach ambitious? Absolutely. But we’d rather swing for the fences and take a real stride toward compelling, human-level AI than sit on the sidelines trying to decide if it will work. Great strides require great risk. We’re on our way.