LONG BEACH, Calif.—The night before the final day of the ANA Avatar XPrize robotics competition at the Long Beach Convention Center, Team Northeastern was burning the midnight oil. The team of 10 made three prototypes for its mobile robot’s hand, repaired the left arm’s motor, and resolved software issues all before competing Saturday afternoon. 

On Friday, the team watched its robot, Robalto, fail to complete the third task in the competition, to operate a hand-held drill to open an access panel on the Mars-like obstacle course. The next day, after making enhancements and testing the robot from 6 a.m. until their final run at 1:30 p.m., Team Northeastern watched the robot complete the course in 21 mins and 8 seconds, winning them $1 million in third place among 17 finalists from around the globe.

“Throughout this entire competition, we’ve just been hoping that we’re able to showcase the robot for its full potential and make sure everyone realizes what we brought,” said David Nguyen, a mechanical engineering student and one of two undergraduate students on the team. “During the run we really hoped that everyone realized that, and finally the stars aligned and everything paid off. It was obviously very stressful to watch because it’s all your blood, sweat and tears, but it was really rewarding.”

After competing, the team triumphantly wheeled Robalto, a large black box on wheels with two hydraulically-powered arms and a small camera and screen where the operator’s face is shown, back to their “garage” where they were stationed during the two-day finals.

While the base platform was provided to the research lab by MassRobotics, the rest of the robot was built by the students, who ordered parts for the system as well as 3D printed parts for the arms and three-pronged grippers designed fully by the team.

Team Northeastern began its run in the ANA Avatar XPrize competition—developing physical, human-operated robotic “avatar” systems that can complete tasks and replicate a person’s senses, actions and presence to a remote location in real time — over a year ago, competing in the semifinals in Miami and winning $133,000.

Led by professors Taskin Padir and Peter Whitney, the team members spent nearly 60 hours a week working on Robalto in the professors’ research labs, studying haptics mechanics with Whitney and experiential robotics with Padir.

This year, the team received bare minimum instructions, knowing their robot would be controlled wirelessly by a judge who was evaluating the system’s comfort, ease of use and connectivity.

Each team’s robot traversed a straight course dotted with jagged rocks, completing five tasks to win: receive and respond to instructions given by a person standing in front of the robot, use one of its limbs to flick on a light switch, pick up canisters and place the heaviest one into a slot, operate a drill to unscrew an access panel, and finally reach behind a curtain and retrieve a rough rock, holding it up for the cheering audience.

While it may sound simple, more than half the teams failed to complete the course within the 25-minute limit.

“We had 45 minutes to train the operator beforehand,” Nguyen said. “We basically went through how the robot worked as well as the best way to approach tasks. Our system in the session specifically is designed to teleport your presence. It’s in the nature of the competition to just try to use your hands as you would normally, so that sort of intuition seemed natural.”

The judge operating Robalto sat in another room looking at a tall screen the team set up with a feed of the robot’s camera. Like a puppeteer, the operator had her arms in two “eco-gloves” wirelessly connected to the robot’s own two arms, and used her feet on a pedal to move the robot around the course.

Mechanical engineering Ph.D. student Evelyn Mendoza said the hardest task to complete was the drill, the task that stumped the team on day one. Building a robot that lacked VR capabilities, which several other teams employed, was also a challenge in the competition, she said.

“It was harder to be super immersive, so we found some tricks like having the lasers line up with wherever you’re moving the arms,” Mendoza said. “Training the operator to realize what they have to do was a bit hard, but once they had enough training they got it.”

Between Friday and Saturday’s run, the team stressed the features of the robot’s grippers to the operator to avoid the mistakes made the day before with the drill, Ph.D. student Rui Luo said.

“We kept emphasizing those techniques and made sure that if she had any questions to ask us so we can then get to that,” Luo said. “On the hardware part, we modified our hand a little bit to make it easier to integrate and pull the drill’s trigger.”

In real world operations, a system like Robalto can be used in the medical field, used for assisted living patients, or used for deep sea exploration or for handling hazardous materials or explosives.

What sets Team Northeastern’s Robalto apart from the other 17 robots at the competition is its hydraulic-based actuation, using air and water to create a precise hydraulic system to motor the machine, graduate computer science student Stephen Alt said. 

Alt said the robot would have performed better with more testing, but wouldn’t change anything else about Robalto.

“I think our system is very good and very unique,” Alt said. “It will just come down to more testing and more experience with it. I think we could have framed our robot to be perfect for the five tasks given to us, but I don’t think that’s the spirit of the competition.”

Team Northeastern was beaten only by the German robotics team NimbRo from the University of Bonn, who completed the course in a groundbreaking five minutes, and Pollen Robotics from Bordeaux, France. First place received a $5 million payout while second place won $2 million. 

After receiving the award for third place, Whitney said he was happy the team’s hard work paid off and could be showcased successfully.

“In each part of the challenge I can see the individual efforts of all the team members,” he said. “I’m really happy that we not only completed all of the tasks, but we had an operator that was skilled enough to run and learn how to operate the system at a very high level very quickly. She demonstrated many of the unique aspects of our system that were not required to do well in the competition, but are required for a much wider range of tasks. In our final run, what the most exciting part for me was to actually see, not just that we completed all of the tasks, but that we did it with a lot of style.”

Henry Mayne, an electrical engineering undergraduate student on the team, said the team all worked, and struggled, together to build a successful robot.

“When we were all out there on the course and we saw the robot grabbed the rock, we just hugged and all jumped up and down. We were just relieved that all of our work as a team had paid off,” Mayne said. “This wouldn’t be possible if any one person wasn’t here.”

For media inquiries, please contact media@northeastern.edu.