Matteo Pirotta

Research Scientist at Facebook AI Research

About Me

I am research scientist at Facebook AI Research in Paris. Previously, I was postdoc at INRIA Lille - Nord Europe in the SequeL team for almost two years. Before I was postdoc at Politecnico di Milano. I have received my PhD in computer science at Politecnico di Milano, under the supervision of Luca Bascetta and Marcello Restelli.

My research interest is machine learning. In particular I am interested in reinforcement learning, transfer learning and online learning.

More details in my CV.

Contacts:
Email: matteo DOT pirotta AT gmail.com

Github:
https://github.com/teopir

News
  • I've been invited to give a talk at ARWL'18 in Beijing, China. I will talk about regret minimization (exploration-exploitation) in RL with prior knowledge (slides). I've been also invited to give the same talk at MSRA in Beijing. 6.11.2018
  • Going to NeurIPS! I've received a free registration as one of the "top" reviewers. Moreover, I have one paper accepted at NeurIPS'18. 29.9.2018
  • I'm really happy to announce that I've been selected for a research position (CR) at INRIA - Lille (link). I've even more happy to announce that I will join Facebook AI Research (Paris) in October 2018. 30.7.2018
  • Busy April! I have been giving several talks on exploration-exploitation in RL: Politecnico di Milano (Apr 03), Facebook Paris (Apr 17) and Google Zurich (Apr 27). 1.6.2018
  • 3 papers accepted at ICML'18.
  • I am organizing the 14th European workshop on reinforcement learning (EWRL 2018) .
  • ICML/IJCAI workshop on Prediction and Generative Modeling in Reinforcement Learning (PGMRL).
    Organizers: Me, Roberto Calandra (UC Berkeley), Sergey Levine (UC Berkeley), Martin Riedmiller (DeepMind), Alessandro Lazaric (Facebook).
  • Ronan Fruit and I are developping a Python library for Exploration-Exploitation in Reinforcement Learning.
    It is available on GitHub.
  • I'm going to visit Berlin and I'll give a talk at Amazon (Mar 19, 2018) on Efficient Exploration-Exploitation in RL. 2.3.2017
  • 3 papers accepted at NIPS 2017.
  • I'm going to spend a couple of weeks in California. I will visit UC Berkeley and I'll give a talk on Regret Minimization in MDPs with Options (Jul 14, 2017). I will then spend one week at Stanford University. 1.6.2017

Publications

Preprints
  • Jian Qian, Ronan Fruit, Matteo Pirotta and Alessandro Lazaric:
    Exploration Bonus for Regret Minimization in Undiscounted Discrete and Continuous Markov Decision Processes. Under review
  • Matteo Pirotta and Marcello Restelli:
    Cost-Sensitive Approach to Batch Size Adaptation for Gradient Descent. arXiv:1712.03428, 2017.
    • Journal Papers
      • Simone Parisi, Matteo Pirotta and Jan Peters:
        Manifold-based Multi-objective Policy Search with Sample Reuse. Neurocomputing 263, 2017. [Paper]
      • Giorgio Manganini, Matteo Pirotta, Marcello Restelli, Luigi Piroddi, and Maria Prandini:
        Policy search for the optimal control of Markov decision processes: a novel particle-based iterative scheme. IEEE Transactions on Cybernetics 46, 2016. [Paper]
      • Simone Parisi, Matteo Pirotta and Marcello Restelli:
        Multi-objective Reinforcement Learning through Continuous Pareto Manifold Approximation. Journal of Artificial Intelligence Research 57, 2016. [Paper]
      • Matteo Pirotta, Marcello Restelli and Luca Bascetta:
        Policy Gradient in Lipschitz Markov Decision Processes. Machine Learning 100, 2015. [Paper]
      Conference Papers
      • Ronan Fruit, Matteo Pirotta and Alessandro Lazaric:
        Near Optimal Exploration-Exploitation in Non-Communicating Markov Decision Processes. NeurIPS 2018, Montréal, Canada. [arXiv] [Paper]
      • Ronan Fruit, Matteo Pirotta, Alessandro Lazaric and Ronald Ortner:
        Efficient Bias-Span-Constrained Exploration-Exploitation in Reinforcement Learning. ICML 2018, Stockholm, Sweden. [arXiv]
      • Matteo Papini, Damiano Binaghi, Giuseppe Canonaco, Matteo Pirotta and Marcello Restelli:
        Stochastic Variance-Reduced Policy Gradient. ICML 2018, Stockholm, Sweden. [arXiv] [Paper]
      • Andrea Tirinzoni, Andrea Sessa, Matteo Pirotta and Marcello Restelli:
        Importance Weighted Transfer of Samples in Reinforcement Learning. ICML 2018, Stockholm, Sweden. [arXiv] [Paper]
      • Davide Di Febbo, Emilia Ambrosini, Matteo Pirotta, Eric Rojas, Marcello Restelli, Alessandra Pedrocchi and Simona Ferrante:
        Does Reinforcement Learning Outperform PID in the Control of FES Induced Elbow Flex-Extension? MeMeA 2018, Rome, Italy.
      • Ronan Fruit, Matteo Pirotta, Alessandro Lazaric, and Emma Brunskill:
        Regret Minimization in MDPs with Options without Prior Knowledge. NIPS 2017, Long Beach, California, USA. [Poster] [Full Paper]
      • Alberto Metelli, Matteo Pirotta, and Marcello Restelli:
        Compatible Reward Inverse Reinforcement Learning. NIPS 2017, Long Beach, California, USA. [Poster] [Paper]
      • Matteo Papini, Matteo Pirotta, and Marcello Restelli:
        Adaptive Batch Size for Safe Policy Gradients. NIPS 2017, Long Beach, California, USA. [Poster] [Paper]
      • Davide Tateo, Matteo Pirotta, Andrea Bonarini and Marcello Restelli:
        Gradient-Based Minimization for Multi-Expert Inverse Reinforcement Learning. IEEE SSCI 2017, Hawaii, USA.
      • Samuele Tosatto, Matteo Pirotta, Carlo D'Eramo, and Marcello Restelli:
        Boosted Fitted Q-Iteration. ICML 2017, Sydney, New South Wales, Australia.
      • Carlo D'Eramo, Alessandro Nuara, Matteo Pirotta, and Marcello Restelli:
        Estimating the Maximum Expected Value in Continuous Reinforcement Learning Problems. AAAI 2017, San Francisco, California, USA.
      • Matteo Pirotta, and Marcello Restelli:
        Inverse Reinforcement Learning through Policy Gradient Minimization. AAAI 2016, Phoenix, Arizona, USA.
      • Matteo Pirotta, Simone Parisi, and Marcello Restelli:
        Multi-Objective Reinforcement Learning with Continuous Pareto Frontier Approximation. AAAI 2015, Austin, Texas, USA.
      • Caporale Danilo, Luca Deori, Roberto Mura, Alessandro Falsone, Riccardo Vignali, Luca Giulioni, Matteo Pirotta and Giorgio Manganini:
        Optimal Control to Reduce Emissions in Gasoline Engines: An Iterative Learning Control Approach for ECU Calibration Maps Improvement. ECC 2015, Linz, Austria.
      • Giorgio Manganini, Matteo Pirotta, Marcello Restelli, Luca Bascetta:
        Following Newton Direction in Policy Gradient with Parameter Exploration. IJCNN 2015, Killarney, Ireland.
      • Simone Parisi, Matteo Pirotta, Nicola Smacchia, Luca Bascetta, Marcello Restelli:
        Policy Gradient Approaches for Multi-Objective Sequential Decision Making: A Comparison. ADPRL 2014, Orlando, Florida, United States.
      • Simone Parisi, Matteo Pirotta, Nicola Smacchia, Luca Bascetta and Marcello Restelli:
        Policy Gradient Approaches for Multi-Objective Sequential Decision Making. IJCNN 2014, Beijing, China.
      • Matteo Pirotta, Giorgio Manganini, Luigi Piroddi, Maria Prandini and Marcello Restelli:
        A particle-based policy for the optimal control of Markov decision processes. IFAC 2014, Cape Town, South Africa.
      • Matteo Pirotta, Marcello Restelli, Luca Bascetta:
        Adaptive Step-Size for Policy Gradient Methods. NIPS 2013, Lake Tahoe, Nevada, USA.
      • Matteo Pirotta, Marcello Restelli, Alessio Pecorino, and Daniele Calandriello:
        Safe policy iteration. ICML 2013, Atlanta, Georgia, USA. [Paper]
      • Martino Migliavacca, Alessio Pecorino, Matteo Pirotta, Marcello Restelli, and Andrea Bonarini:
        Fitted Policy Search. ADPRL 2011, Paris, France.
      • Martino Migliavacca, Alessio Pecorino, Matteo Pirotta, Marcello Restelli, and Andrea Bonarini:
        Fitted Policy Search: Direct Policy Search using a Batch Reinforcement Learning Approach. ERLARS 2010, Lisboa, Portugal.
      Workshops Papers

Teaching

Reinforcement Learning - Fall 2018 - MVA - ENS Paris-Saclay
  • Piazza: Registration (with your school email) and online class discussion on piazza
Previous Classes
  • Reinforcement Learning - Fall 2017 - MVA - ENS Paris-Saclay