Research
Spatial AI:
Situational awareness requires autonomous agents to build and maintain a multi-layered model of the environment, including both a geometric model (useful for navigation and coordination) and a semantic level (useful to execute high-level tasks and to provide more succinct information to human operators).
I work on using human experiences to improve semantic understanding of the environment for mobile assistive robots.
Generative Visual Models and Intuitive Physics Understanding:
My research tries to bring common sense understanding to robotic perception.
Interacting with the environment requires to perceive objects and understand how actions influence their movement ad shape.
Generative perception models can make sense of partial and noisy observations and reconstruct their shape and semantics.
On the other hand, understanding the intuitive physics of objects interacting with each other will provide next-generation AI agents with a common sense knowledge base that will enable human-level interaction with a complex, dynamical environment.
Interpretable Sensor Fusion:
Recent developments in machine learning have made possible to learn end-to-end motion estimation from visual, inertial and ranging devices. I study reasoned ways to learn sensor fusion strategies in deep VIO frameworks. At the same time, I study how to integrate novel sensor modalities such as millimeter wave radar and thermal imaging into a single framework.
|
|
Self-improving object detection via disagreement reconciliation
Gianluca Scarpellini,
Stefano Rosa,
Pietro Morerio, Lorenzo Natale, Alessio del Bue
[ArXiv]
This work studies how to automatically fine-tune a pre-trained off-the-shelf object detector while exploring a new environment.
|
|
Semantic Disagreement for Embodied Active Perception
Gianluca Scarpellini,
Stefano Rosa,
Pietro Morerio, Lorenzo Natale, Alessio del Bue
ICCV-2023 Workshop on Out Of Distribution Generalization in Computer Vision, Paris, France
[Web]
We teach an embodied agent to look for disagreement in detected objects, in order to collect samples for fine-tuning an off-the-shelf detector. We analyze the zero-shot transfer of the learned policy.
|
|
Learning Semantic Segmentation of Large-Scale Point Clouds with Random Sampling
Quingyong Hu,
Bo Yang,
Linhai Xie,
Stefano Rosa,
Yulan Guo,
Zhihua Wang,
Niki Trigoni,
Andrew Markham,
Transactions on Pattern Analysis and Machine Intelligence (TPAMI), 2021, DOI: 10.1109/TPAMI.2021.3083288
[PDF]
|
|
RandLA-Net: Efficient Semantic Segmentation of Large-Scale Point Clouds
Quingyong Hu,
Bo Yang,
Linhai Xie,
Stefano Rosa,
Yulan Guo,
Zhihua Wang,
Niki Trigoni,
Andrew Markham,
CVPR-2020, Conference on Computer Vision and Pattern Recognition, Seattle, WA
[PDF]
[code]
[video]
We show that random sampling combined with attention can achieve SOA performances in semantic segmentation while processing large point clouds in near real-time.
|
|
milliMap: Robust Indoor Mapping with Low-cost mmWave Radar
Chris Xiaoxuan Lu,
Stefano Rosa,
Peijun Zhao,
Bing Wang,
Changhao Chen,
Niki Trigoni,
Andrew Markham,
MOBYSIS-2020, The 18th ACM International Conference on Mobile Systems, Applications, and Services, Toronto, Canada, June 2020
[PDF]
We show how to build dense occupancy grid maps of indoor environments from sparse, noisy mmWave measurements, with cross-modal training.
|
|
DeepTIO: A Deep Thermal-Inertial Odometry with Visual Hallucination
Muhamad Saputra,
Pedro Gusmao,
Chris Xiaoxuan Lu,
Yasin Almalioglu,
Stefano Rosa,
Changhao Chen,
Johan Wahlstrom,
Wei Wang,
Andrew Markham,
Niki Trigoni
RA-L, IEEE Robotics and Automation Letters
ICRA-2020, International Conference on Robotics and Automation, Paris, France, May 2020
[PDF]
In this RA-L work we try to hallucinate visual features from thermal images that can help first responders to navigate visually-denied scenarios.
|
|
Selective Sensor Fusion for Neural Visual-Inertial Odometry
Changhao Chen,
Stefano Rosa,
Yishu Miao,
Chris Xiaoxuan Lu,
Wei Wu,
Andrew Markham,
Niki Trigoni
CVPR-2019, Conference on Computer Vision and Pattern Recognition,
Long Beach, USA, June 2019
[PDF (2.6 MB)]
[Bibtex]
[Project Website]
We show how data-learned sensor fusion strategies can improve accuracy and robustness in deep VIO when dealing with noisy/corrupted data, while adding interpretability.
|
 |
3D Object Dense Reconstruction from a Single Depth View
Bo Yang,
Stefano Rosa,
Andrew Markham,
Niki Trigoni,
Hongkai Wen
Transactions on Pattern Analysis and Machine Intelligence (TPAMI), 2018, DOI: 10.1109/TPAMI.2018.2868195
[PDF]
[Bibtex]
We propose an end-to-end approach to high-resolution reconstruction of 3D objects from a single depth image.
We also release a real-world dataset for 3D reconstruction. We argue that real-world benchmarks for shape reconstruction are necessary for a thorough validation of future approaches.
|
|
Learning the Intuitive Physics of Non-Rigid Object Deformations
Stefano Rosa,
Zhihua Wang,
Andrew Markham
NeurIPS-2018 Workshops, Modeling the Physical World: Perception, Learning, and Control, Montreal, 2018
[PDF]
|
|
Neural Allocentric Intuitive Physics Prediction from Real Videos
Zhihua Wang,
Stefano Rosa,
Yishu Miao,
Zihang Lai,
Linhai Xie,
Niki Trigoni
arxiv
[PDF]
We learn how to predict future video of interacting objects by decoupling the problem into appearence and dynamics and leaning invertible transformations from real domain to simulation domain and from egocentric view to allocentric view and vice-versa.
|
|
Semantic Place Understanding for Human-Robot Coexistence - Towards Intelligent Workplaces
Stefano Rosa,
Andrea Patane',
Chris Xiaoxuan Lu,
Niki Trigoni
Transactions on Human-Machine Systems (THMS), 2018, DOI: 10.1109/THMS.2018.2875079
[PDF]
Robots and users can work synergistically by mutually learning over time, and benefitting from each other by exploiting each other's strengths. We show how detecting user activities can help robots to learn semantic understanding of the environment, while at the same time learning to better localise the user.
|

|
3D-PhysNet: Learning the Intuitive Physics of Non-Rigid Object Deformations
Zhihua Wang*,
Stefano Rosa*,
Bo Yang,
Sen Wang,
Niki Trigoni,
Andrew Markham
IJCAI-2018, 27th International Joint Conference on Artificial Intelligence, Stockholm, SWE
[PDF] [code]
[webpage]
We show that conditioning a generative model that predicts soft object deformations on real physical properties can improve prediction accuracy as well as enabling generalisation abilities.
|
|
Defo-Net: Learning Body Deformation using Generative Adversarial Networks
Zhihua Wang*,
Stefano Rosa*,
Bo Yang,
Linhai Xie,
Sen Wang,
Niki Trigoni,
Andrew Markham
ICRA-2018, IEEE International Conference on Robotics and Automation, Brisbane, AU
[PDF] [code]
[video]
[webpage]
We show that conditioning a generative model that predicts soft object deformations on real physical properties can improve prediction accuracy as well as enabling generalisation abilities.
|
|
Learning with Training Wheels: Speeding up Training with a Simple Controller for Deep Reinforcement Learning
Linhai Xie,
Sen Wang,
Stefano Rosa,
Andrew Markham,
Niki Trigoni
ICRA-2018, IEEE International Conference on Robotics and Automation, Brisbane, AU
[PDF]
[video]
We propose a way to embed a switchable, simple controller into a deep reinforcement learning algorithm, to speed up training of mobile robot navigation in simulated environments.
|
|
CommonSense: Collaborative learning of scene semantics by robots and humans
Stefano Rosa,
Andrea Patane',
Chris Xiaoxuan Lu,
Niki Trigoni
MOBISYS-2018 Workshops, 1st International Workshop on Internet of People, Assistive Robots and ThingS (IoPARTS), Munich, DE
[PDF]
|
Lu X., Kan X., Rosa S., Wen H., Markham A., Trigoni N., Towards Self-supervised Face Labeling via Cross-modality Association, poster, SenSys 2017, The Netherlands [PDF]
Rosa S., Lu X., Wen H., Trigoni N.,
Leveraging User Activities and Mobile Robots for Semantic Mapping and User Localization., HRI 2017 late break reports [PDF]
Rosa S., Toscana G., Bona B. Q-PSO: Fast Quaternion-based Pose Estimation From RGB-D Images, Journal of Intelligent and Robotic Systems, 2017, DOI: 10.1007/s10846-017-0714-3 [PDF] [code]
Anjum M.L., Rosa S., Bona B. Tracking a subset of skeleton joints - An effective approach towards complex human activity recognition, Journal of Robotics, vol. 2017, doi:10.1155/2017/7610417 [PDF] [code]
Rosa S., Toscana G.
Fast Feature-Less Quaternion-based Particle Swarm Optimization for Rigid and Articulated Pose Estimation From RGB-D Images, poster, ECCV 2016, Amsterdam, NL
Toscana G., Rosa S.,
Fast Feature-Less Quaternion-based Particle Swarm Optimization for Object Pose Estimation From RGB-D Images, BMVC 2016, York, UK [PDF]
[video]
GPU:[code]
CPU:[code]
Toscana G., Rosa S., Bona B.,
Fast Graph-Based Object Segmentation for RGB-D Images, Intellisys 2016, London, UK
[PDF]
[video]
[code]
Toscana G., Rosa S., Bona B.,
Vocal Interaction with a 7-DOF Robotic Arm for Object Detection, Learning and Grasping, HRI 2016 Late break reports [PDF]
[video]
Russo L.O., Rosa S., Maggiora M., Bona B. A Novel Cloud Based Service Robotics Application to Data Center
Environmental Monitoring, Sensors, 2016, DOI: 10.3390/s16081255 [PDF]
Ermacora G., Rosa S., Toma A. Fly4SmartCity: a Cloud Robotics Service for Smart City Applications, Journal of Ambient Intelligence and Smart Environments, 2016, DOI: 10.3233/AIS-160374 [PDF]
Rosa S., Russo L.O., Toscana G., Primatesta S., Kaouk Ng M., Bona B.,
Leveraging the Cloud for Connected Service Robotics Applications, Workshop on Robotics and Technology Transfer, ETFA 2015, Luxemburg, LU
B. de Gusmao P.B., Rosa S., Magli E., Lepsøy S., Francini L.,
Robotics Navigation Using MPEG CDVS, 17th International Workshop on Multimedia Signal Processing, MMSP 2015, Xiamen, China[PDF]
Lupetti M.L., Rosa S., Ermacora G.,
From a Robotic Vacuum Cleaner to Robot Companion: Acceptance and Engagement in Domestic Environments., HRI 2015 late break reports
Russo L.O., Farulla G., Pianu D., Salgarella A., Controzzi M., Cipriani C., Oddo C., Geraci C., Rosa S., Indaco M., A remote communication system for deafblind persons by means of gesture recognition, International Journal of Advanced Robotic Systems, 2014
[PDF]
Bona B., Carlone L., Indri M., Rosa S.,Supervision and monitoring of logistic spaces by a cooperative robotic team: methodologies, problems, and solutions, Intelligent Service Robotics, 2014, DOI: 10.1007/s11370-014-0151-0
[PDF]
Rosa S., Russo L.O., Bona B.,
Towards A ROS-Based Autonomous Cloud Robotics Platform for Data Center Monitoring. the 19th IEEE International Conference on Emerging Technologies and Factory Automation (ETFA), Barcelona, Spain, 2014
[PDF]
G. Ermacora, A. Toma, S. Rosa, B.
Bona, M. Chiaberge, M. Silvagni, M. Gaspardone, R. Antonini,
A cloud based service for management and planning of autonomous
UAV missions in smartcity scenarios, MESAS 2014, Rome, IT
Airo' Farulla G., Russo L.O., Pintor C., Pianu D., Micotti G., Salgarella A.R., Camboni D., Controzzi M., Cipriani C., Calogero M.O., Rosa S., Indaco M.,
Real-time single camera hand gesture recognition system for remote deaf-blind communication 1st International Conference on Augmented and Virtual Reality - Salento AVR 2014, Lecce, 17-20 September 2014
[PDF]
Ahmad O., Yin J., Bona B., Rosa S., Anjum M.L., Skeleton Tracking Based Complex Human Activity Recognition Using Kinect Camera, ICSR 2014, Syndey, AU
[PDF]
Ermacora G., Toma A., Rosa S., Antonini R.,
Leveraging open data for supporting a cloud robotics service in a smart city environment. at IAS-13, July 15 - 19, 2014, Padova, Italy
[PDF]
Rosa S., Russo L.O., AirĂ² Farulla G., Antonini R., Gaspardone M., Carlone L., Bona B.,
An Application of Laser-Based Autonomous Navigation for Data-Center Monitoring. at IAS-13, July 15 - 19, 2014, Padova, Italy
[PDF]
Yuan Z., Rosa S., Russo L.O., Bona B.,
A Kinect-based Front-end for Graph-SLAM Using Plane Matching in Planar Indoor Environments. at IAS-13, July 15 - 19, 2014, Padova, Italy
[PDF]
Yin J., Carlone L., Rosa S., Anjum M.L., Bona B.,
Scan Matching for Graph SLAM in Indoor Dynamic Scenarios.27th International FLAIRS Conference, May 21 - 23, 2014, Pensacola Beach, Florida, USA
Russo L.O., Rosa S., Matteucci M., Bona B.,
A ROS Implementation of the Mono-SLAM Algorithm. In: International Conference on Artificial Intelligence & Applications (ARIA-2014), 2014
[PDF] [code]
Abrate F., Bona B., Indri M., Rosa S., Tibaldi F.,Multi-robot map updating in dynamic environments, in Springer Tracts in Advanced Robotics, Volume 83, 2013, DOI: 10.1007/978-3-642-32723-0
[PDF]
Russo L.O., AirĂ² farulla G., Indaco M., Rosa S., Rolfo D., Bona B.,
Blurring prediction in Monocular SLAM, In: 8th IEEE International Design & Test Symposium 2013 (IDT), 2013
[PDF]
Abrate F., Bona B., Indri M., Rosa S., Tibaldi F., Multirobot Localization in Highly Symmetric Environments, Journal of Intelligent and Robotic Systems, 2012, DOI: 10.1007/s10846-012-9790-6
[PDF]
L. Carlone, J. Yin, S. Rosa, Z. Yuan, Graph optimization with unstructured covariance: fast, accurate, linear approximation. In: Simulation, Modeling, and Programming for Autonomous Robots (SIMPAR 2012), 2012.
[PDF] [code]
Rosa S., Paleari M., Ariano P., Bona B., Object Tracking with Adaptive HOG Detector and Adaptive Rao-Blackwellised Particle Filter. In: SPIE 8301, Intelligent Robots and Computer Vision XXIX: Algorithms and Techniques, 2012.
[PDF]
Paleari M., Margaria V., Rosa S., Ariano P., HExEC: hand exoskeleton electromyographic control, 4th International Workshop on Human-Friendly Robotics (HFR 2011) November 8th-9th, 2011, University of Twente, The Netherlands
[PDF]
Macchia V.; Rosa S; Carlone L; Bona B., An Application of Omnidirectional Vision to Grid-based SLAM in Indoor Environments. In: Workshop on Omnidirectional Robot Vision, International Conference on Robotics and Automation (ICRA 2010), 2010.
[PDF]
Abrate F; Bona B; Indri M; Rosa S.; Tibaldi F., Map updating in dynamic environments. In: ISR/ROBOTIK 2010, 2010.
[PDF]
Brevi D., Fileppo F. , Scopigno R. , Abrate F., Bona B., Rosa S., Tibaldi F., Hybrid localization solutions for robotic logistic applications. In: Technologies for Practical Robot Applications (TePRA), 2009.
[PDF]
Abrate F; Bona B.; Indri M; Rosa S; Tibaldi F., Three-State Multirobot Collaborative Localization in Symmetrical Environments. In: ROBOTICA 2009, 2009
[PDF]
Abrate F; Bona B; Indri M.; Rosa S; Tibaldi F.,Switching Multirobot Collaborative Localization in Symmetrical Environments. In: IROS 2008 2nd Workshop on Planning, Perception and Navigation for Intelligent Vehicles, 2008.
[PDF]
 |
Assistant lecturer for Automatic Control, Politecnico di Torino, 2013
Assistant lecturer for Basics of Automatic Control, Politecnico di Torino, 2013
Introduction to ROS, Robotics, Politecnico di Torino, 2013-2015
Lecturer for Ph.D. course: Research topics in computer and control engineering, Politecnico di Torino, 2010-2012
|
Past projects I worked on
|
|