I was proud to receive my Ph.D. from from the Machine Learning Department of CMU. In the past few years, I have been very fortunate to be able to collaborate with different researchers on a variety of projects that provide insights on deep learning as well as its applications.

[Google Scholar] [Kaggle Profile] [GitHub Profile]

Index

Publications

  • Deep Equilibrium Optical Flow Estimation
    Shaojie Bai*, Zhengyang Geng*, Yash Savani and J. Zico Kolter (*equal contribution)
    In Conference on Computer Vision and Pattern Recognition (CVPR) 2022
    [BibTex] [PDF] [Code]

  • SHINE: SHaring the INverse Estimate from the forward pass for bi-level optimization and implicit models
    Zaccharie Ramzi, Florian Mannel, Shaojie Bai, Jean-Luc Starck, Philippe Ciuciu and Thomas Moreau
    In International Conference on Learning Representations (ICLR) 2022 (Spotlight Oral, 5.1% acceptance)
    [BibTex] [PDF] [Code]

  • Neural Deep Equilibrium Solvers
    Shaojie Bai, Vladlen Koltun and J. Zico Kolter
    In International Conference on Learning Representations (ICLR) 2022
    [BibTex] [PDF] [Code]

  • On Training Implicit Models
    Zhengyang Geng*, Xin-Yu Zhang*, Shaojie Bai, Yisen Wang and Zhouchen Lin (*equal contribution)
    In Neural Information Processing Systems (NeurIPS) 2021
    [BibTex] [PDF] [Code]

  • Implicit\( ^2 \): Implicit Layers for Implicit Representations
    Zhichun Huang, Shaojie Bai, and J. Zico Kolter
    In Neural Information Processing Systems (NeurIPS) 2021
    [BibTex] [PDF] [Code]

  • Joint Inference and Input Optimization in Equilibrium Networks
    Swaminathan Gurumurthy, Shaojie Bai, Zachary Manchester and J. Zico Kolter
    In Neural Information Processing Systems (NeurIPS) 2021
    [BibTex] [PDF] [Code]

  • Stabilizing Equilibrium Models by Jacobian Regularization
    Shaojie Bai, Vladlen Koltun and J. Zico Kolter
    In International Conference on Machine Learning (ICML) 2021
    [BibTex] [PDF] [Code]

  • A Community-powered Search of Machine Learning Strategy Space to Find NMR Property Prediction Models.
    Lars A Bratholm, Will Gerrard, Brandon Anderson, Shaojie Bai et al.
    PLOS ONE 16(7) 2021
    [Paper] [Code]

  • Multiscale Deep Equilibrium Models
    Shaojie Bai, Vladlen Koltun and J. Zico Kolter
    In Neural Information Processing Systems (NeurIPS) 2020 (Oral, 1.1% acceptance)
    [BibTex] [PDF] [Code]

  • Deep Equilibrium Models
    Shaojie Bai, J. Zico Kolter and Vladlen Koltun
    In Neural Information Processing Systems (NeurIPS) 2019 (Spotlight Oral, 3.0% acceptance)
    [BibTex] [PDF] [Code] [Poster] [Slides] [Zico’s Talk] [My NeurIPS Talk (at 37:20)]

  • Transformer Dissection: An Unified Understanding for Transformer’s Attention via the Lens of Kernel
    Yao-Hung Hubert Tsai, Shaojie Bai, Makoto Yamada, Louis-Philippe Morency and Ruslan Salakhutdinov
    In Conference on Empirical Methods in Natural Language Processing (EMNLP) 2019
    [BibTex] [PDF]

  • Multimodal Transformer for Unaligned Multimodal Language Sequences
    Yao-Hung Hubert Tsai*, Shaojie Bai*, Paul Pu Liang, J. Zico Kolter, Louis-Philippe Morency and Ruslan Salakhutdinov (*equal contribution)
    In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (ACL) 2019.
    [BibTex] [PDF] [Code]

  • Trellis Networks for Sequence Modeling
    Shaojie Bai, J. Zico Kolter and Vladlen Koltun
    In International Conference on Learning Representations (ICLR) 2019.
    [BibTex] [PDF] [Code] [Poster]

  • An Empirical Evaluation of Generic Convolutional and Recurrent Networks for Sequence Modeling
    Shaojie Bai, J. Zico Kolter and Vladlen Koltun
    In ArXiv: 1803.01271.
    [BibTex] [PDF] [Code]

  • The Effect of Pre-ReLU Input Distribution on Deep Neural Network’s Performance
    Shaojie Bai, J. Zico Kolter
    In CMU’s Undergraduate Senior Thesis (2017).
    [BibTex] [PDF] [Slides]

Competitions & Awards

Internships

  • Summer 2020: Facebook AI Research (FAIR). Supervisor: Michael Auli
  • Summer 2019: Bosch Center for AI (in both USA and Germany). Supervisor: Zico Kolter
  • Summer 2018: Intel Labs (Intelligent Systems Lab). Supervisor: Vladlen Koltun