Research

Interests

My current research interests regard the theory of deep learning. I am interested in understanding the optimization and statistical properties of various architectures, ranging from shallow networks to generative models. I am also starting to work on AI for maths.

Previously, I worked on Natural Language Processing and on Quasi-Monte Carlo methods.

Publications

See also my Google Scholar. Stars denote first co-authors.

Preprints

  • J. Wu*, P. Marion*, P. Bartlett, Large Stepsizes Accelerate Gradient Descent for Regularized Logistic Regression, arXiv preprint, June 2025. Online access
  • R. Maulen-Soto, P. Marion, C. Boyer, Attention-based clustering, arXiv preprint, May 2025. Online access

Peer-reviewed Publications

  • Y.-H. Wu, P. Marion, G. Biau, C. Boyer, Taking a Big Step: Large Learning Rates in Denoising Score Matching Prevent Memorization, COLT 2025, July 2025. Online access
  • P. Marion, A. Korba, P. Bartlett, M. Blondel, V. De Bortoli, A. Doucet, F. Llinares-López, C. Paquette, Q. Berthet, Implicit Diffusion: Efficient Optimization through Stochastic Sampling, AISTATS 2025, oral presentation, May 2025. Online access
  • P. Marion, R. Berthier, G. Biau, C. Boyer, Attention layers provably solve single-location regression, ICLR 2025, April 2025. Online access
  • P. Marion, A.Fermanian, G. Biau, J.P. Vert, Scaling ResNets in the Large-depth Regime, Journal of Machine Learning Research, February 2025. Online access
  • P. Marion, L. Chizat, Deep linear networks for regression are implicitly regularized towards flat minima, NeurIPS 2024, December 2024. Online access
  • P. Marion*, Y.-H. Wu*, M. Sander, G. Biau, Implicit regularization of deep residual networks towards neural ODEs , ICLR 2024, spotlight presentation, May 2024. Online access
  • P. Marion, Generalization bounds for neural ordinary differential equations and deep residual networks, NeurIPS 2023, December 2023. Online access
  • P. Marion, R. Berthier, Leveraging the two-timescale regime to demonstrate convergence of neural networks, NeurIPS 2023, December 2023. Online access
  • A. Fermanian*, P. Marion*, J.P. Vert, G. Biau, Framing RNN as a kernel method: A neural ODE approach, NeurIPS 2021, oral presentation, December 2021. Online access
  • P. Marion, P. Nowak, F. Piccinno, Structured Context and High-Coverage Grammar for Conversational Question Answering over Knowledge Graphs, EMNLP 2021, November 2021. Online access
  • P. L’Ecuyer, P. Marion, M. Godin, and F. Puchhammer, A Tool for Custom Construction of QMC and RQMC Point Sets, Monte Carlo and Quasi-Monte Carlo Methods 2020, August 2020. Online access.
  • P. Marion, M. Godin, and P. L’Ecuyer, An algorithm to compute the t-value of a digital net and of its projections, Journal of Computational and Applied Mathematics, June 2020. Online access

Technical Reports

  • L. Phan et al., Humanity's Last Exam, arXiv:2501.14249, January 2025. Online access

PhD Thesis

  • P. Marion, Mathematics of deep learning: generalization, optimization, continuous-time models, November 2023. Download my thesis.