Research

Interests

My current research interests regard the theory of deep learning. I am interested in understanding the optimization and statistical properties of various architectures, ranging from shallow networks to generative models. I am also starting to work on AI for maths.

Previously, as part of master internships, I worked on Natural Language Processing and on Quasi-Monte Carlo methods.

Publications

See also my Google Scholar. Stars denote first co-authors.

AI for mathematics

  • Distilling LLM Feedback for Lean Theorem Proving G. Narozniak, G. Biau, R. Munos, A. Rammal, P. Marion NeurIPS 2026. arXiv
  • Humanity's Last Exam L. Phan et al. Technical report, January 2025. arXiv

Optimization and training dynamics

  • Edge Flow: A Tractable and Predictive Continuous-Time Model for Gradient Descent at the Edge of Stability P. Marion arXiv preprint, June 2026. arXiv
  • Exponential Convergence of (Stochastic) Gradient Descent for Separable Logistic Regression S. Kale, P. Manupriya, P. Marion, F. Bach, A. Raj TMLR, May 2026. arXiv
  • Large Stepsizes Accelerate Gradient Descent for Regularized Logistic Regression J. Wu*, P. Marion*, P. Bartlett NeurIPS 2025. arXiv
  • Phase Diagram of Dropout for Two-Layer Neural Networks in the Mean-Field Regime L. Chizat, P. Marion, Y. Yesbay arXiv preprint, October 2025. arXiv
  • Deep linear networks for regression are implicitly regularized towards flat minima P. Marion, L. Chizat NeurIPS 2024. arXiv
  • Leveraging the two-timescale regime to demonstrate convergence of neural networks P. Marion, R. Berthier NeurIPS 2023. arXiv

Diffusion and generative models

  • Understanding diffusion models requires rethinking (again) generalization P. Marion*, Y.-H. Wu* NeurIPS 2026, position paper. arXiv
  • Diffusion Removes Langevin's Conditioning Dependence: A Sharp Gaussian Analysis A. Perbost, F. Bach, P. Marion arXiv preprint, October 2026. arXiv
  • Optimal Stopping in Latent Diffusion Models Y.-H. Wu, Q. Berthet, G. Biau, C. Boyer, R. Elie, P. Marion ICML 2026. arXiv
  • Taking a Big Step: Large Learning Rates in Denoising Score Matching Prevent Memorization Y.-H. Wu, P. Marion, G. Biau, C. Boyer COLT 2025. arXiv
  • Implicit Diffusion: Efficient Optimization through Stochastic Sampling P. Marion, A. Korba, P. Bartlett, M. Blondel, V. De Bortoli, A. Doucet, F. Llinares-López, C. Paquette, Q. Berthet AISTATS 2025, oral presentation. arXiv

Attention and transformers

  • Clustering in Deep Stochastic Transformers L. Fedorov, M. Sander, P. Marion, R. Elie, M. Laurière ICML 2026, spotlight presentation. arXiv
  • Statistical Advantage of Softmax Attention: Insights from Single-Location Regression O. Duranthon, P. Marion, C. Boyer, B. Loureiro, L. Zdeborová ICLR 2026. arXiv
  • Attention-based clustering R. Maulen-Soto, P. Marion, C. Boyer NeurIPS 2025. arXiv
  • Attention layers provably solve single-location regression P. Marion, R. Berthier, G. Biau, C. Boyer ICLR 2025. arXiv

Deep residual networks and neural ODEs

  • Scaling ResNets in the Large-depth Regime P. Marion, A. Fermanian, G. Biau, J.P. Vert Journal of Machine Learning Research, February 2025. JMLR
  • Implicit regularization of deep residual networks towards neural ODEs P. Marion*, Y.-H. Wu*, M. Sander, G. Biau ICLR 2024, spotlight presentation. arXiv
  • Generalization bounds for neural ordinary differential equations and deep residual networks P. Marion NeurIPS 2023. arXiv
  • Framing RNN as a kernel method: A neural ODE approach A. Fermanian*, P. Marion*, J.P. Vert, G. Biau NeurIPS 2021, oral presentation. Proceedings

Earlier work: natural language processing and quasi-Monte Carlo

  • Structured Context and High-Coverage Grammar for Conversational Question Answering over Knowledge Graphs P. Marion, P. Nowak, F. Piccinno EMNLP 2021. ACL Anthology
  • A Tool for Custom Construction of QMC and RQMC Point Sets P. L’Ecuyer, P. Marion, M. Godin, and F. Puchhammer Monte Carlo and Quasi-Monte Carlo Methods 2020. Springer
  • An algorithm to compute the t-value of a digital net and of its projections P. Marion, M. Godin, and P. L’Ecuyer Journal of Computational and Applied Mathematics, June 2020. DOI

PhD thesis

  • Mathematics of deep learning: generalization, optimization, continuous-time models P. Marion PhD thesis, Sorbonne Université, November 2023. PDF