Stefano Rosa

I do research in Robotics and Artificial Intelligence and I am working at Istituto Italiano di Tecnologia (IIT), in the Humanoid Sensing and Perception lab, with Lorenzo Natale.

My research interests include localization and mapping for mobile robotics, computer vision and ML applied to robot navigation, and human-robot interaction. I also have an interest in vision-based assistive technologies.

I was research fellow at University of Oxford (UK), under supervision of Prof. Niki Trigoni, working on the ESPRC Programme Grant "Mobile Robotics: Enabling a Pervasive Technology of the Future". I was assistant researcher at Politecnico di Torino (Italy), working in collaboration with Telecom Italia S.P.A. I was a PhD student in Robotics at Istituto Italiano di Tecnologia (IIT) and Politecnico di Torino, working on space robotics and service robotics.

Email  /  CV  /  Google Scholar  /  LinkedIn  /  GitHub

Spatial AI: Situational awareness requires autonomous agents to build and maintain a multi-layered model of the environment, including both a geometric model (useful for navigation and coordination) and a semantic level (useful to execute high-level tasks and to provide more succinct information to human operators). I work on using human experiences to improve semantic understanding of the environment for mobile assistive robots.

Generative Visual Models and Intuitive Physics Understanding: My research tries to bring common sense understanding to robotic perception. Interacting with the environment requires to perceive objects and understand how actions influence their movement ad shape. Generative perception models can make sense of partial and noisy observations and reconstruct their shape and semantics. On the other hand, understanding the intuitive physics of objects interacting with each other will provide next-generation AI agents with a common sense knowledge base that will enable human-level interaction with a complex, dynamical environment.

Interpretable Sensor Fusion: Recent developments in machine learning have made possible to learn end-to-end motion estimation from visual, inertial and ranging devices. I study reasoned ways to learn sensor fusion strategies in deep VIO frameworks. At the same time, I study how to integrate novel sensor modalities such as millimeter wave radar and thermal imaging into a single framework.

Self-improving object detection via disagreement reconciliation

This work studies how to automatically fine-tune a pre-trained off-the-shelf object detector while exploring a new environment.

Semantic Disagreement for Embodied Active Perception

We teach an embodied agent to look for disagreement in detected objects, in order to collect samples for fine-tuning an off-the-shelf detector. We analyze the zero-shot transfer of the learned policy.

Learning Selective Sensor Fusion for State Estimation

Learning Semantic Segmentation of Large-Scale Point Clouds with Random Sampling

RandLA-Net: Efficient Semantic Segmentation of Large-Scale Point Clouds

We show that random sampling combined with attention can achieve SOA performances in semantic segmentation while processing large point clouds in near real-time.

milliMap: Robust Indoor Mapping with Low-cost mmWave Radar

We show how to build dense occupancy grid maps of indoor environments from sparse, noisy mmWave measurements, with cross-modal training.

DeepTIO: A Deep Thermal-Inertial Odometry with Visual Hallucination

In this RA-L work we try to hallucinate visual features from thermal images that can help first responders to navigate visually-denied scenarios.

Selective Sensor Fusion for Neural Visual-Inertial Odometry

We show how data-learned sensor fusion strategies can improve accuracy and robustness in deep VIO when dealing with noisy/corrupted data, while adding interpretability.

3D Object Dense Reconstruction from a Single Depth View

We propose an end-to-end approach to high-resolution reconstruction of 3D objects from a single depth image. We also release a real-world dataset for 3D reconstruction. We argue that real-world benchmarks for shape reconstruction are necessary for a thorough validation of future approaches.

Learning the Intuitive Physics of Non-Rigid Object Deformations

Neural Allocentric Intuitive Physics Prediction from Real Videos

We learn how to predict future video of interacting objects by decoupling the problem into appearence and dynamics and leaning invertible transformations from real domain to simulation domain and from egocentric view to allocentric view and vice-versa.

Semantic Place Understanding for Human-Robot Coexistence - Towards Intelligent Workplaces

Robots and users can work synergistically by mutually learning over time, and benefitting from each other by exploiting each other's strengths. We show how detecting user activities can help robots to learn semantic understanding of the environment, while at the same time learning to better localise the user.

3D-PhysNet: Learning the Intuitive Physics of Non-Rigid Object Deformations

We show that conditioning a generative model that predicts soft object deformations on real physical properties can improve prediction accuracy as well as enabling generalisation abilities.

Defo-Net: Learning Body Deformation using Generative Adversarial Networks

We show that conditioning a generative model that predicts soft object deformations on real physical properties can improve prediction accuracy as well as enabling generalisation abilities.

Learning with Training Wheels: Speeding up Training with a Simple Controller for Deep Reinforcement Learning

We propose a way to embed a switchable, simple controller into a deep reinforcement learning algorithm, to speed up training of mobile robot navigation in simulated environments.

CommonSense: Collaborative learning of scene semantics by robots and humans

Assistant lecturer for Automatic Control, Politecnico di Torino, 2013
Assistant lecturer for Basics of Automatic Control, Politecnico di Torino, 2013

Introduction to ROS, Robotics, Politecnico di Torino, 2013-2015

Lecturer for Ph.D. course: Research topics in computer and control engineering, Politecnico di Torino, 2010-2012

• 5G-TOURS,Horizon 2020, 2020-2022
• HATFIL - InnovateUK, 2018-2020
• NIST - IPSER, 2018-2019
• ESPRC Programme Grant "Mobile Robotics: Enabling a Pervasive Technology of the Future", 2016-2018
• STEPS - Sistemi e Tecnologie per l'EsPlorazione Spaziale, 2011
• HExEC: Hand Exoskeleton Electromyographic Control, 2011
• MACP4Log - Mobile, autonomous and cooperating robotic platforms for supervision and monitoring of large logistic surfaces, 2008-2010

CSS credits: jonbarron