Nima Shoghi
I’m a PhD student in Machine Learning at Georgia Tech, where I am focusing on Deep Learning for Scientific Applications under the guidance of Dr. Pan Li and Dr. Victor Fung. I earned my B.S. and M.S. degrees in Computer Science from Georgia Tech, during which I conducted research at the High Performance Computer Architecture Lab on accelerating ML training and inference. Prior to starting my PhD, I completed a two-year AI residency at Meta AI’s FAIR Chemistry team, where I worked on developing large pre-trained models, trained on a diverse mixture of chemical data across multiple domains, for general-purpose chemical property prediction. My research interests lie in the application of large-scale machine learning techniques to challenging problems in science and engineering. I am particularly excited about the potential for deep learning to accelerate discovery and understanding in fields like chemistry and climate science.
My CV is available here.
Recent Updates
- [Sep 2024] I gave an invited talk titled Unlocking the Potential of Pre-training for Accelerated Discovery in Chemistry at the AI for Science Institute (AISI) Beijing. [Slides]
- [Aug 2024] I gave an invited talk titled Unlocking the Potential of Pre-training for Accelerated Discovery in Chemistry at the 2024 Machine Learning for Materials and Molecular Discoveries Symposium in Gothenburg, Sweden. [Slides]
- [Aug 2024] I started my PhD in Machine Learning at Georgia Tech, where I will be focusing on Deep Learning for Scientific Applications under the guidance of Dr. Pan Li and Dr. Victor Fung.
- [Jul 2024] I gave an invited talk titled From Molecules to Materials: Pre-training Large Generalizable Models for Atomic Property Prediction at King Abdullah University of Science and Technology (KAUST). [Slides]
- [Jun 2024] I gave an invited talk titled From Molecules to Materials: Pre-training Large Generalizable Models for Atomic Property Prediction at SES AI. [Slides]
- [Jun 2024] I started a machine learning internship at ProcessMiner, where I will be developing novel pre-trained transformer models trained on manufacturing process data to predict process outcomes and detect anomalies.
- [May 2024] I wrote a blog post on From Molecules to Materials: Pre-training Large Generalizable Models for Atomic Property Prediction for Valence Labs.
- [Apr 2024] I gave an invited talk titled From Molecules to Materials: Pre-training Large Generalizable Models for Atomic Property Prediction at the Molecular ML Reading Group. [Slides] [Video]
- [Jan 2024] Our paper on large-scale diverse pre-training for chemical property prediction has been accepted to ICLR 2024! Please visit our webpage for more information, including an interactive visualization of its embeddings!
- [Dec 2023] I will be joining the High Performance Computer Architecture Lab at Georgia Tech as a Temporary Research Staff starting in December 2023.
- [Aug 2023] I gave a talk on From Molecules to Materials: Pre-training Large Generalizable Models for Atomic Property Prediction at the ACS Fall Meeting. [Slides] [Video]
Education
- Ph.D. in Machine Learning (School of Computational Science and Engineering), Georgia Institute of Technology, 2024 - Present
- Advisors: Dr. Pan Li and Dr. Victor Fung
- Research Focus: Deep Learning for Scientific Applications (e.g., Chemistry, Climate Science, etc.)
- M.S. with Highest Honors in Computer Science (Machine Learning Specialization), Georgia Institute of Technology, 2020 - 2021
- B.S. with High Honors in Computer Science (Machine Learning and Devices Threads), Georgia Institute of Technology, 2015 - 2019
- International Baccalaureate Diploma, Druid Hills High School, 2011 - 2015
Work experience
- Machine Learning Intern at ProcessMiner, Jun 2024 - Aug 2024
- Under the supervision of Dr. Kamran Paynabar, developed novel pre-trained transformer models trained on ~500,000 time-series data points from manufacturing processes to predict process outcomes and detect anomalies.
- Temporary Research Staff at the High Performance Computer Architecture Lab at Georgia Tech, Dec 2023 - May 2024
- Working on efficient inference strategies for pre-trained image diffusion models, with a focus on generating diverse, high-quality images.
- Advisors: Dr. Hyesoon Kim and Dr. Stefano Petrangeli
- AI Resident at Meta Fundamental AI Research (FAIR), Aug 2021 - Aug 2023
- Worked on the Open Catalyst Project on the FAIR Chemistry team, focusing on the development of large-scale pre-training methods for chemical property prediction.
- Advisors: Dr. Larry Zitnick and Dr. Abhishek Das
- Research Assistant at High Performance Computer Architecture Lab at Georgia Tech, May 2019 - May 2021
- Developed software-level and hardware-level techniques for accelerating deep learning training and inference.
- Advisors: Dr. Hyesoon Kim and Dr. Moinuddin Qureshi
- Graduate Teaching Assistant at Georgia Institute of Technology, Aug 2020 - May 2021
- CS 4510: Automata and Complexity, Spring 2021, Taught by Dr. Zvi Galil
- CS 4510: Automata and Complexity, Fall 2020, Taught by Dr. Merrick Furst
Publications
(* denotes equal contribution)
From molecules to materials: Pre-training large generalizable models for atomic property prediction
Nima Shoghi, Adeesh Kolluru, John R Kitchin, Zachary W Ulissi, C Lawrence Zitnick, Brandon M Wood, International Conference on Learning Representations, 2024
Distribution Learning for Molecular Regression
Nima Shoghi, Pooya Shoghi, Anuroop Sriram, Abhishek Das, arXiv preprint arXiv:2407.20475, 2024
The Open Catalyst 2022 (OC22) dataset and challenges for oxide electrocatalysts
Richard Tran, Janice Lan, Muhammed Shuaibi, Brandon M Wood, Siddharth Goyal, Abhishek Das, Javier Heras-Domingo, Adeesh Kolluru, Ammar Rizvi, Nima Shoghi, Anuroop Sriram, Félix Therrien, Jehad Abed, Oleksandr Voznyy, Edward H Sargent, Zachary Ulissi, C Lawrence Zitnick, ACS Catalysis 13 (5), 3066-3084, 2023
Context-Aware Task Handling in Resource-Constrained Robots with Virtualization
Ramyad Hadidi, Nima Shoghi Ghaleshahi, Bahar Asgari, Hyesoon Kim, 2023 IEEE International Conference on Edge Computing and Communications …, 2023
Transfer learning using attentions across atomic systems with graph neural networks (TAAG)
Adeesh Kolluru, Nima Shoghi, Muhammed Shuaibi, Siddharth Goyal, Abhishek Das, C Lawrence Zitnick, Zachary Ulissi, The Journal of Chemical Physics 156 (18), 2022
Open challenges in developing generalizable large-scale machine-learning models for catalyst discovery
Adeesh Kolluru, Muhammed Shuaibi, Aini Palizhati, Nima Shoghi, Abhishek Das, Brandon Wood, C Lawrence Zitnick, John R Kitchin, Zachary W Ulissi, ACS Catalysis 12 (14), 8572-8581, 2022
SmaQ: Smart quantization for DNN training by exploiting value clustering
Nima Shoghi, Andrei Bersatti, Moinuddin Qureshi, Hyesoon Kim, IEEE Computer Architecture Letters 20 (2), 126-129, 2021
Quantifying the design-space tradeoffs in autonomous drones
Ramyad Hadidi, Bahar Asgari, Sam Jijina, Adriana Amyette, Nima Shoghi, Hyesoon Kim, Proceedings of the 26th ACM International Conference on Architectural …, 2021
Understanding the software and hardware stacks of a general-purpose cognitive drone
Sam Jijina, Adriana Amyette, Nima Shoghi, Ramyad Hadidi, Hyesoon Kim, 2020 IEEE International Symposium on Performance Analysis of Systems and …, 2020
Secure Location-Aware Authentication and Communication for Intelligent Transportation Systems
Nima Shoghi Ghalehshahi, Ramyad Hadidi, Lee Jaewon, Jun Chen, Arthur Siqueria, Rahul Rajan, Shaan Dhawan, Pooya Shoghi Ghalehshahi, Hyesoon Kim, arXiv preprint arXiv:2011.08936, 2020
Pisces: power-aware implementation of slam by customizing efficient sparse algebra
Bahar Asgari, Ramyad Hadidi, Nima Shoghi Ghaleshahi, Hyesoon Kim, 2020 57th ACM/IEEE Design Automation Conference (DAC), 1-6, 2020
Neural network weight compression with nnw-bdi
Andrei Bersatti, Nima Shoghi Ghalehshahi, Hyesoon Kim, Proceedings of the International Symposium on Memory Systems, 335-340, 2020
Slam performance on embedded robots
Nima Shoghi Ghalehshahi, Ramyad Hadidi, Hyesoon Kim, Student Research Competition at Embedded System Week (SRC ESWEEK), 2019
Talks
[AI for Science Institute (AISI), Beijing] — Unlocking the Potential of Pre-training for Accelerated Discovery in Chemistry
Presented on unlocking the potential of large-scale pre-training methods to accelerate discovery in chemistry, highlighting key challenges and opportunities in this rapidly evolving field.
[2024 Machine Learning for Materials and Molecular Discoveries Symposium] — Unlocking the Potential of Pre-training for Accelerated Discovery in Chemistry
Presented on unlocking the potential of large-scale pre-training methods to accelerate discovery in chemistry, highlighting key challenges and opportunities in this rapidly evolving field.
[King Abdullah University of Science and Technology (KAUST)] — From Molecules to Materials: Pre-training Large Generalizable Models for Atomic Property Prediction
Introduced Joint Multi-Domain Pre-training (JMP), a robust supervised pre-training approach which demonstrates state-of-the-art results on key small molecule, large molecule, and materials datasets.
[SES AI] — From Molecules to Materials: Pre-training Large Generalizable Models for Atomic Property Prediction
Introduced Joint Multi-Domain Pre-training (JMP), a robust supervised pre-training approach which demonstrates state-of-the-art results on key small molecule, large molecule, and materials datasets.
[Molecular ML Reading Group] — From Molecules to Materials: Pre-training Large Generalizable Models for Atomic Property Prediction
Introduced Joint Multi-Domain Pre-training (JMP), a robust supervised pre-training approach which demonstrates state-of-the-art results on key small molecule, large molecule, and materials datasets.
[ACS Fall Meeting] — From Molecules to Materials: Pre-training Large Generalizable Models for Atomic Property Prediction
Introduced Joint Multi-Domain Pre-training (JMP), a robust supervised pre-training approach which demonstrates state-of-the-art results on key small molecule, large molecule, and materials datasets.
[Georgia Institute of Technology] — SmaQ: Smart Quantization for DNN Training by Exploiting Value Clustering
Introduced Smart Quantization (SmaQ) technique for DNN training, which exploits value clustering in DNNs to reduce memory usage during training by up to 6.7x with no loss in accuracy.
[Georgia Institute of Technology] — Legal Text Summarization Using Transformer Models
Presented work on a new transformer-based encoder-decoder architecture for abstractive legal text summarization, achieving state-of-the-art performance on the BIGPATENT dataset.
[Georgia Institute of Technology] — Attention is All You Need: The Transformer Architecture
Presented the seminal Transformer paper by Vaswani et al. (2017) and discussed its impact on the field of natural language processing.