Research

Interests

My current research interests regard the theory of deep learning. I am interested in various architectures, ranging from shallow networks to generative models, and try to understand their optimization and statistical properties. During my PhD, I studied in particular the connection between large-depth residual networks and neural differential equations. You're welcome to download my PhD thesis. More recently, I got interested on Transformer and denoising diffusions.

Publications

Peer-reviewed publications

  • Y.-H. Wu, P. Marion, G. Biau, C. Boyer, Taking a Big Step: Large Learning Rates in Denoising Score Matching Prevent Memorization, COLT 2025, July 2025. Online access
  • P. Marion, A. Korba, P. Bartlett, M. Blondel, V. De Bortoli, A. Doucet, F. Llinares-López, C. Paquette, Q. Berthet, Implicit Diffusion: Efficient Optimization through Stochastic Sampling, AISTATS 2025, oral presentation, May 2025. Online access
  • P. Marion, R. Berthier, G. Biau, C. Boyer, Attention layers provably solve single-location regression, ICLR 2025, April 2025. Online access
  • P. Marion, A.Fermanian, G. Biau, J.P. Vert, Scaling ResNets in the Large-depth Regime, Journal of Machine Learning Research, February 2025. Online access
  • P. Marion, L. Chizat, Deep linear networks for regression are implicitly regularized towards flat minima, NeurIPS 2024, December 2024. Online access
  • P. Marion*, Y.-H. Wu*, M. Sander, G. Biau, Implicit regularization of deep residual networks towards neural ODEs , ICLR 2024, spotlight presentation, May 2024. Online access
  • P. Marion, Generalization bounds for neural ordinary differential equations and deep residual networks, NeurIPS 2023, December 2023. Online access
  • P. Marion, R. Berthier, Leveraging the two-timescale regime to demonstrate convergence of neural networks, NeurIPS 2023, December 2023. Online access
  • A. Fermanian*, P. Marion*, J.P. Vert, G. Biau, Framing RNN as a kernel method: A neural ODE approach, NeurIPS 2021, oral presentation, December 2021. Online access
  • P. Marion, P. Nowak, F. Piccinno, Structured Context and High-Coverage Grammar for Conversational Question Answering over Knowledge Graphs, EMNLP 2021, November 2021. Online access
  • P. L’Ecuyer, P. Marion, M. Godin, and F. Puchhammer, A Tool for Custom Construction of QMC and RQMC Point Sets, Monte Carlo and Quasi-Monte Carlo Methods 2020, August 2020. Online access.
  • P. Marion, M. Godin, and P. L’Ecuyer, An algorithm to compute the t-value of a digital net and of its projections, Journal of Computational and Applied Mathematics, June 2020. Online access

(stars denote first co-authors.)

Technical reports

  • L. Phan et al., Humanity's Last Exam, arXiv:2501.14249, January 2025. Online access

Talks and posters

  • Theoretical insights into modern deep learning architectures, Simons Institute, Berkeley, USA, January 2025
  • Three stories on deep linear networks, séminaire de l'équipe MILES, LAMSADE, Université Paris-Dauphine, Paris, France, January 2025
  • Transformer: news from the front, FLAIR seminar, EPFL, Lausanne, Switzerland, December 2024
  • Deep linear networks for regression are implicitly regularized towards flat minima, poster at NeurIPS in Paris, Paris, France, December 2024
  • Two stories on deep linear networks, EPFL Postdoc days, Lausanne, Switzerland, November 2024
  • Three stories on deep linear networks, SIGMA 2024 Workshop, CIRM, Marseille, France, October 2024
  • Three stories on deep linear networks, Séminaire du département DATA, Laboratoire LJK, Grenoble, France, September 2024
  • Three stories on deep linear networks, Journées MAS 2024, Poitiers, France, August 2024
  • Implicit Diffusion: Efficient Optimization through Stochastic Sampling, FGS conference on optimization, Gijon, Spain, June 2024
  • Three stories on deep linear networks, Universitat Pompeu Fabra, Barcelona, Spain, June 2024
  • Three stories on deep linear networks, séminaire de l'équipe Inria Sierra, Paris, France, June 2024
  • Régularisation implicite des réseaux de neurones profonds vers des EDO neuronales, Journées de Statistique, Bordeaux, France, May 2024
  • Modèles de diffusion pour la génération d'images : algorithmes et (un peu de) théorie, Geolearning-RESSTE day, Paris, France, May 2024
  • Implicit regularization of deep residual networks towards neural ODEs, poster at ICLR 2024, Vienna, Austria, May 2024
  • Modèles de diffusion pour la génération d'images : algorithmes et (un peu de) théorie, ML Meetup, Nantes, France, April 2024
  • Implicit Diffusion: Efficient Optimization through Stochastic Sampling, Mostly Monte Carlo Seminar, Paris, France, March 2024
  • Implicit Diffusion: Efficient Optimization through Stochastic Sampling, Google, Paris, France, February 2024
  • Generalization bounds for neural ordinary differential equations and deep residual networks, NeurIPS@Paris 2023, Paris, France, December 2023
  • Leveraging the two-timescale regime to demonstrate convergence of neural networks, NeurIPS@Paris 2023, Paris, France, December 2023
  • Framing RNNs as a kernel method through the signature, Rencontre sur les signatures, applications et machine learning, Pau, France, December 2023
  • Neural ODEs and residual neural networks, séminaire de l'équipe Inria HeKA, Paris, France, September 2023
  • Generalization bounds for neural ordinary differential equations and deep residual networks, poster at StatMathAppli 2023, Fréjus, France, September 2023
  • Apprentissage par optimisation stochastique bi-échelle pour les réseaux de neurones, Journées de Statistique, Bruxelles, Belgium, July 2023
  • Leveraging the two-timescale regime to demonstrate convergence of neural networks, poster at FoCM 2023, Paris, France, June 2023
  • Neural network training with stochastic gradient descent in a two-timescale regime, PhD students seminar at LPSM, Sorbonne Université, Paris, France, March 2023
  • Framing RNN as a kernel method: a neural ODE approach, ICSDS 2022, Florence, Italy, December 2022
  • Recurrent neural networks are kernel methods: a neural ODE approach, EPFL, Switzerland, December 2022
  • Signatures and continuous-time neural networks for sequential data, séminaire de l'équipe Inria MIND, Saclay, France, November 2022
  • Limites profondes des réseaux de neurones, MathInnov Day, Paris, France, October 2022
  • Scaling ResNets in the Large-depth Regime, conférence LOL 2022, Marseille, France, October 2022
  • Framing RNN as a kernel method: a neural ODE approach, Summer Cluster on deep learning theory, Simons Institute, Berkeley, USA, July 2022
  • Scaling ResNets in the Large-depth Regime, Journées de Statistique, Lyon, France, June 2022
  • Towards understanding residual neural networks via the large-depth limit, Journée Équation d'évolution et Machine Learning, Paris, France, May 2022
  • Large-depth limit for residual neural networks, Rencontre des Jeunes Statisticiens, Porquerolles, France, April 2022
  • Framing RNN as a kernel method: A neural ODE approach, Séminaire Parisien de Statistique, Paris, France, January 2022
  • Framing RNN as a kernel method: A neural ODE approach, NeurIPS@Paris 2021, Paris, France, December 2021
  • Framing RNN as a kernel method: A neural ODE approach, oral at NeurIPS 2021 (online), December 2021 (Youtube video)
  • Framing RNN as a kernel method: A neural ODE approach, PhD students seminar at LPSM, Sorbonne Université, Paris, France, November 2021
  • Framing RNN as a kernel method: A neural ODE approach, DeepMath Working Group, Sorbonne Université, Paris, France, November 2021
  • A primer on Neural ODEs, Institut Mathématique de Toulouse, France, Septembre 2021
  • "Who plays Gandalf in LOTR?" - Natural Language Processing on structured data, PhD students seminar at LPSM, Sorbonne Université, Paris, France, May 2021
  • Algorithms and Software for Custom Digital Net Constructions, MCQMC 2020, online, August 2020 (Youtube video)
  • How to beat randomness? A Quasi-Monte Carlo methods overview, PROWLER.io, Cambridge, UK, November 2019
  • Deep learning beyond the hype, Corps des Mines, November 2019

Community service

  • 2024-2025:
    • Organizer of NeurIPS@Paris 2024, a 200-people meetup before NeurIPS 2024
  • 2022-2023:
    • Organizer of NeurIPS@Paris 2022, a 200-people meetup before NeurIPS 2022
    • Organizer of the session "Differential equations and machine learning" at ICSDS 2022
    • Organizer of the PhD seminar at LPSM and at Corps des Mines
  • 2021-2022:
    • Organizer of NeurIPS@Paris 2021, a 150-people meetup during NeurIPS 2021
    • Organizer of the PhD seminar at Corps des Mines
  • 2020-2021:
    • Organizer of the research seminar at Corps des Mines
  • Reviewer for journals (JASA, Annals of Statistics, Information and Inference, JMLR, SIMODS, Bernoulli) and conferences (NeurIPS 2023, ICLR 2024, NeurIPS 2024, ICLR 2025, AISTATS 2025). Top reviewer at NeurIPS 2023.