Kaiyu Hang, an assistant professor of computer science at Rice University, has won a National Science Foundation CAREER Award to develop robots that can manipulate unfamiliar objects in high-uncertainty situations.
The grants are awarded each year to a selective cohort of about 500 early career faculty across all disciplines engaged in pathbreaking research and committed to growing their field through outreach and education.
Hang’s project aims to develop general-purpose robots that can handle complex physical interactions in real-world settings without requiring perfect input from sensors or extensive instructions.
“My research is focused on robot manipulation,” Hang said. “And when we talk about robot manipulation, we’re referring to physically using the robot to change the configuration of the world.
“ ⎯Say you want to pick up an object from a densely cluttered space (like picking up a book off of a shelf, for instance) and then stably place it somewhere else ⎯ that’s one of the manipulation tasks we’re interested in,” Hang said. “We’re generally interested in physical interactions. That's why my lab is called, Robotics and Physical Interactions Lab.”
Making robots more dexterous in real time ⎯ i.e. better at manipulating unfamiliar objects and navigating complex, real-world situations and environments ⎯ requires improving their computational ability to carry out finely-tuned, fine-grained actions that are context-specific and self-correcting.
“Imagine having a robot that can clean surfaces in the home or in a hospital setting that is able to decide what cleaning motion or force to apply, depending on the type of object or area it encounters,” Hang said. “Instead of designing specific robots for specific tasks ⎯ which works well in an industrial setting where you actually have control over the working environment ⎯ I hope to develop robots that can perform daily tasks in new or unfamiliar environments that are constantly changing.”
The process of carrying out a physical task can be broken down into various sequences. In the context of structuring robot-world interactions, such sequences have traditionally followed a pattern that begins with sensing, is followed by planning, and culminates in action. Hang, however, argues that action should come first and serve as an occasion for gathering sense data, which in turn can inform an open-ended, evolving action plan.
“This is actually how we engage in the world ⎯ we learn by doing, and improve with practice,” Hang said. “If you’re trying to paint an object, it’s not enough to only look at it once. Instead, you make the first stroke with your brush on the canvas, then glance at the object, then make another brush stroke, and so on while proceeding with and adjusting the painting in terms of what you see each moment. My project is about using this understanding of embodied action to reframe how we structure robot-world interactions.”
Hang wants to design robots that can engage in a given task in an open-ended way that allows them to glean information about the environment in which they are acting ⎯ and acting upon ⎯ to improve their performance.
“You can think of this in terms of pouring some liquid into a bottle with a very narrow neck,” Hang said. “Even if you have steady hands to hold both containers and the flow of liquid is aimed right, you’ll probably end up spilling some due to fluid uncertainties. However, if you use a funnel, you can pour the liquid without worrying about spilling. What I’d like to do is reconfigure robotic manipulation tasks to essentially funnel action toward the desired goal in ways that minimize the likelihood of error.”
Hang’s project is poised at the intersection of two general strategies to manage robot manipulation.
“Currently, there are two operative frameworks in robotic manipulation: a modeling- or parameter-based approach where each action the robot takes is mapped out in detail in advance, and a newer, data- or deep learning-based approach that relies on vast datasets coupled with input from sensors to guide the robot into action,” Hang said. “I want to reduce the amount of assumptions both of these approaches make about how the robot acts in the real world.
“The model-based method, for instance, assumes it can come up with parameters to completely map out a given interaction, but the world is a place where things are constantly changing to produce uncertainties, and you want a robot that is adaptable to these changes rather than one that malfunctions each time your set parameters fail to align with real-world conditions.
“The data-driven or deep-learning approach is much newer, and it can work quite well, provided that you fulfill two main conditions: First, you need a huge amount of data to train your robots, and, second, you have to ensure that your robots are working in scenarios that are covered by your data. If one of these two conditions is not fully satisfied, then the robots are not going to work very well.
“For either framework, you have to precisely define the departure space and the destination of each task, which makes the whole system fragile to uncertainties. Any estimation of the environment that is incorrect or inaccurate, or any slight malfunction or noise in the data picked up by the sensors can make the whole system unreliable. The idea, then, is to provide a much broader entry to the problem space ⎯ hence the funnel analogy.”
Hang says recent advances in computational power and an increased supply of robots that incorporate compliant design enable him to carry out his vision.
“This is the right time to take on this project,” Hang said. “If we’d wanted to do this five or 10 years ago, we wouldn’t have been able to.”
Like all CAREER Awards, Hang’s project includes an educational and outreach component.
“This approach to robot manipulation is actually very new, and I want Rice students to learn and work on things that are happening at the frontier of the discipline,” Hang said. “I will plan to develop new courses that can teach students not only the theory but also provide hands on robot manipulation skills. I will also continue to provide research opportunities for students in my lab.”
Hang will leverage his role as faculty advisor for the Rice Robotics Club to provide learning and research opportunities for students from diverse educational backgrounds. He also plans to work with pre-college level students from underserved communities. Finally, he intends to create a series of hands-on robot manipulation tutorials and set of practical exercises that can serve as a resource for students who want to work on robotic manipulation projects but lack access to actual robots.
“If you don’t have access to robots, you can use simulators to develop projects, but they’re difficult to use,” Hang said. “I got inspired by practical, hands-on project guides developed by educators in the machine learning community, but, so far, I’ve not seen a similar resource for students looking to learn about robotic manipulation.
“I plan to develop and compile these tutorials into a book that will provide students with some good examples of hands-on projects they can implement even if they don’t have a robot,” Hang said.
- CAREER Award abstract:
-
https://www.nsf.gov/
awardsearch/showAward?AWD_ID= 2240040&HistoricalAwards=false - Image downloads:
-
https://news-network.rice.edu/news/files/2023/07/230705_Hang_Fitlow_012.jpg
CAPTION: Kaiyu Hang is an assistant professor of computer science at Rice University. (Photo by Jeff Fitlow/Rice University)
https://news-network.rice.edu/news/files/2023/07/230705_Hang_Fitlow_008.jpg
CAPTION: Kaiyu Hang, an assistant professor of computer science at Rice University, is the recipient of an NSF CAREER Award. (Photo by Jeff Fitlow/Rice University)
- Related stories:
-
Meet Rice CS’ New Faculty: Kaiyu Hang, Assistant Professor
https://kenkennedy.rice.edu/news/current-news/meet-rice-cs-new-faculty-kaiyu-hang-assistant-professor - Links:
-
Kaiyu Hang website: https://hangkaiyu.github.io/
Department of Computer Science: https://csweb.rice.edu/
George R. Brown School of Engineering: https://engineering.rice.edu
- About Rice:
-
Located on a 300-acre forested campus in Houston, Rice University is consistently ranked among the nation’s top 20 universities by U.S. News & World Report. Rice has highly respected schools of Architecture, Business, Continuing Studies, Engineering, Humanities, Music, Natural Sciences and Social Sciences and is home to the Baker Institute for Public Policy. With 4,552 undergraduates and 3,998 graduate students, Rice’s undergraduate student-to-faculty ratio is just under 6-to-1. Its residential college system builds close-knit communities and lifelong friendships, just one reason why Rice is ranked No. 1 for lots of race/class interaction and No. 4 for quality of life by the Princeton Review. Rice is also rated as a best value among private universities by Kiplinger’s Personal Finance.