While drones and driverless cars dominate the headlines, another breakthrough—robot dexterity—is likely to have an even greater impact in both business and everyday life.

“Robot manipulation is the next shoe to drop,” says Robert Platt, computer science professor and head of the Helping Hands robotics lab at Northeastern. “Imagine a robot that can do things with it’s hands in the real world—anything from defusing a bomb to doing your laundry. This has been a dream in the research community for decades, but now we’re finally getting to the point where it could actually happen.”

Recent advances in machine learning, Big Data, and robot perception have put us on the threshold of a quantum leap in the ability of robots to perform fine motor tasks and function in uncontrolled environments, says Platt.

It’s the difference between robots that can do repetitive tasks in a highly-structured factory environment and a new era of humanoid robots that can do meaningful work in the real world.

Why fine motor skills have lagged

There’s an irony in the field of robots and artificial intelligence known as Moravek’s Paradox: What’s hard for humans is relatively easy for robots, and what’s easy for humans is nearly impossible for robots.

We can program a robot with the computational skills to defeat an international chess champion, but struggle to give it the dexterity of a 2-year-old. Identifying and grabbing a pencil in a random pile of office supplies is nearly impossible for a robot, and opening a door and walking into a room—as demonstrated in this recent international robot competition—can look like a scene from a slapstick comedy. (See video below)

Because humans have been evolving their visual, sensory, and motor skills for millions of years, these complex skills are so deeply ingrained in human circuitry that we perform them unconsciously. In contrast, high-level endeavors such as mathematics, science, and stock analysis are relatively recent human endeavors—so they’re much easier for engineers to replicate.

Poised for a breakthrough

In spite of the enormous challenge, Platt says autonomous robots are poised to make huge leaps in their ability to manipulate unfamiliar objects.

For example, Platt and his team at the Helping Hands Lab have trained a robot to find, grab, and remove unfamiliar objects from a pile of clutter with 93 percent accuracy. Achieving this required significant advances in machine learning, perception, and control.

The researchers used a technique called reinforcement learning in which the robot learns via trial and error. They created a simulated world in which the robot could practice picking up and manipulating objects in virtual reality. When the robot did what the researchers wanted—grabbed an object from a pile—it was given a reward. This technique allows the robot to master skills in a virtual environment and then apply them to the real world.

A major advance in depth perception was also essential for robots to work in an uncontrolled environment. Previously, they could only see the world as a flat field of seemingly random colors. But with this new 3-D perception, they could identify individual objects in a crowded field.

Platt is building a power wheelchair with a robotic arm that can grasp items around the house and could enable elderly or people with disabilities to continue to live independently in their homes.

While vision is an excellent tool for guiding broad movements, fine motor skills require a sense of touch.

“Think of what you can do with gloves on,” explains Platt. “You can open the garage door, grab a shovel, and clear the driveway. But if you need to unlock the garage first, you need to take your gloves off to insert the key.”

As part of a NASA grant, Platt’s lab recently built a robotic hand equipped with tactile sensors and developed new algorithms for interpreting the tactile data.

“In order to insert a key into a lock, the robot needs to know exactly how it’s holding the key, down to the millimeter,” says Platt. “Our algorithms can localize these kinds of grapsed objects very accurately.”

Platt’s lab demonstrated these new capabilities by grasping a USB connector and plugging it into a port. While this may not sound like a big deal, it’s a critical step toward creating robots that can do precise manipulation tasks such as changing the battery in a cell phone.

What’s coming?

As with any fledgling advancement—radar, the telephone, the internet—the practical applications of robot dexterity are hard to predict. But here are a few:

Ÿ Home

Platt’s Helping Hands lab—in collaboration with the University of Massachusetts Lowell and the Crotched Mountain Rehabilitation Facility in New Hampshire—is building a power wheelchair with a robotic arm that can grasp items around the house or perform simple household tasks. This could enable elderly or people with disabilities to continue to live independently in their homes.

Platt is also interested in adapting this technology for everyday use. “We hear a lot about the Alexa-style assistants that can answer questions by accessing the internet. But these assistants can’t do anything physical,” says Platt. “We want to equip these devices with a robotic body so you can say, ‘Alexa, get the newspaper,’ or ‘Alexa, clean up Jimmy’s room.’”

Ÿ Outdoors

Engineering professor Hanumant Singh in collaboration with Platt are building a golf-cart sized mobile robot equipped with a robotic arm that can drive around Northeastern’s campus autonomously and perform simple manipulation tasks like taking out the trash.

Ÿ Military

Similar types of robots could take on similar duties in areas of intense conflict and be used for dangerous operations such as defusing mines. For example, Platt and his group recently completed a grant from the Office of Naval Research to develop fundamental manipulation technologies that will be used aboard Naval vessels.

Ÿ Hazardous waste

Engineering professor Taskin Padir and his team received a grant from the Department of Energy to adapt NASA’s Valkyrie robot for hazardous waste disposal. There are more than a dozen sites scattered around the U.S. where radioactive waste was buried in tunnels during the Cold War. For autonomous robots to locate, grasp, and place this waste in safe containers, they will need fine motor skills and an ability to operate in unfamiliar environments.

Ÿ Medicine

Funded by a grant from the National Science Foundation, engineering professor Peter Whitney is working with researchers at Stanford University to create a robot that can perform MRI-guided surgery.

Ÿ Space exploration

Platt is working with researchers at NASA to develop robotic manipulation capabilities for handling soft objects on future NASA space missions.

“Robots that work flawlessly in the lab break down quickly when they’re placed in unfamiliar situations,” says Platt. “Our goal is to develop the underlying algorithms that will allow them to be more reliable in the real world. Ultimately, this will fundamentally change the way we think about robots, allowing them to become partners with humans rather than just machines that work in far away factories.”

Allie Nicodemo contributed to this story