Gobinda Saha
Deep learning optimization, Lifelong learning, Meta-learning, Online learning, Transfer and multitask learning
Machine Learning Researcher
I am a PhD candidate at the Department of Electrical and Computer Engineering at Purdue University where I am advised by Professor Kaushik Roy. I am also affiliated as a graduate student researcher at the Center for Brain-Inspired Computing (C-BRIC) at Purdue. During my PhD, I spent time at the Memory Solution Team at GlobalFoundries as a research intern.
During my PhD, I worked on deep learning optimizer and algorithm design, specifically within the application domains of computer vision and reinforcement learning. My expertise extends to various areas, including online learning, continual learning, meta-learning, and decentralized learning algorithms. Furthermore, I gained valuable industry experience through my involvement in hardware-software co-design for ML acceleration during my internship. I am deeply passionate about advancing the field of AI, particularly in the areas of computer vision, natural language processing, autonomous AI, and generative AI applications, with a strong emphasis on scalability and efficiency.
| Resume | Google Scholar | Github |
Work Experience
Graduate Research Assistant @ Purdue University (August 2017 - Present)
- Project 1: Efficient Continual Learning in Deep Neural Networks
- Project 2: Decentralized Learning with non-IID data and in Continual Learning setups
- Project 3: Machine Unlearning Algorithms for Removing Effect of Data Noise and Anomalies and to Preserve Privacy in Vision/Language/Multimodal Models
- Project 4: Hardware accelerator design with Photonic in-memory-computing primitives for Spiking Neural Networks
Research Intern, Memory Solution Team @ GlobalFoundries, USA (June 2019 - August 2019)
- Project: Software framework development for hardware-algorithm co-design for deep learning applications
News
- 01/10/2024: Served as a reviewer for CVPR 2024.
- 09/27/2023: Presented a collection of works done during my PhD at Boston Dynamics AI Institute, Cambridge, MA, USA.
- 02/11/2023: Presented our paper Continual Learning with Scaled Gradient Projection at AAAI 2023, Washington, DC, USA.
- 01/07/2023: Our paper Saliency Guided Experience Packing for Replay in Continual Learning made the WACV 2023 Award finalist list.
- 07/27/2022: Passed the PhD preliminary exam!
- 07/10/2022: Our perspective on A cross-layer approach to cognitive computing was presented at ACM/IEEE Design Automation Conference 2022.
- 10/5/2020: Attended and presented a poster at C-BRIC annual review at Purdue.
- 07/01/2021: Gave a talk on Continual Learning at C-BRIC Industry Meeting.
- 01/12/2021: Our paper Gradient Projection Memory for Continual Learning is selected for Oral presentation at ICLR 2021.
- 10/6/2020: Attended and presented a poster at C-BRIC annual review at Purdue.
Publications
Continual Learning with Scaled Gradient Projection
Gobinda Saha, Kaushik Roy
AAAI Conference on Artificial Intelligence (AAAI 2023)
[Paper] [Code] [Talk Video]
- A scaled gradient projection algorithm for balancing stability and plasticity during continual learning.
- Attained up to 2% higher accuracy in image classification and ~12% more reward in reinforcement learning (Atari games) tasks than SOTA with minimal forgetting.
Saliency Guided Experience Packing for Replay in Continual Learning
Gobinda Saha, Kaushik Roy
IEEE/CVF Winter Conference on Applications of Computer Vision (WACV 2023)
[Paper] [Code] [Talk Video]
- A new experience replay method for continual learning where explainable AI (XAI) tools such as saliency maps are used for memory selection.
- Attained up to 5% accuracy improvement over SOTA on online continual object classification benchmarks with tiny episodic memories.
Gradient Projection Memory for Continual Learning
Gobinda Saha, Isha Garg, Kaushik Roy
International Conference on Learning Representations (ICLR 2021) (Oral - top 1% paper)
[Paper] [Code] [Talk Video] [Poster]
- A novel orthogonal gradient descent algorithm for forget-free continual learning in deep neural networks.
- Obtained near zero forgetting on continual object classification tasks.
SPACE: Structured Compression and Sharing of Representational Space for Continual Learning
Gobinda Saha, Isha Garg, Aayush Ankit, Kaushik Roy
IEEE Access 2021
[Paper] [Code]
- A PCA-driven network pruning and growth method for forget-free continual learning.
- Achieved zero forgetting with up to 5x energy efficiency during inference due to emerging sparsity.
CoDeC: Communication-Efficient Decentralized Continual Learning
Sakshi Choudhary, Sai Aparna Aketi, Gobinda Saha, Kaushik Roy
Transactions on Machine Learning Research (TMLR), 2024
[Paper]
- A decentralized continual learning algorithm that combines orthogonal gradient projections with gossip-averaging among distributed agents.
- Achieved SOTA accuracy on image classification tasks with up to 4.8x reduced communication ensuring inter-agent data privacy.
Homogenizing Non-IID datasets via In-Distribution Knowledge Distillation for Decentralized Learning
Deepak Ravikumar, Gobinda Saha, Sai Aparna Aketi, Kaushik Roy
Preprint (arxiv), 2023 [Paper]
- A new distillation-based approach - IDKD to handle non-IID data distribution in a decentralized setting.
Deep Unlearning: Fast and Efficient Training-free Approach to Controlled Forgetting
Sangamesh Kodge, Gobinda Saha, Kaushik Roy
Preprint (arxiv), 2023 [Paper]
- A new Singular Value decomposition-based algorithm that unlearns requested classes of data from a pre-trained model without any retraining or finetuing!
An Energy-Efficient and High Throughput in-Memory Computing Bit-Cell With Excellent Robustness Under Process Variations for Binary Neural Network
Gobinda Saha, Zhewei Jiang, Sanjay Parihar, Cao Xi, Jack Higman, Muhammed Ahosan Ul Karim
IEEE Access, 2021
[Paper]
- Proposed a 10T bit-cell based IMC primitive for accelerating binary neural network inference.
- Developed a Python-HSPICE based software framework for hardware-algorithm co-design.
- Analysized performance of in-memory computing (IMC) arrays on DL workloads under nonidealities and process variations.
Photonic In-Memory Computing Primitive for Spiking Neural Networks Using Phase-Change Materials
Indranil Chakraborty, Gobinda Saha, Kaushik Roy
Physical Review Applied, 2019
[Paper]
- Designed optical spiking neural network based on photonic computing primitives to realize energy-efficient and fast computing.
- Developed a device-circuit-algorithm co-design framework to evaluate their performance as SNN inference engine.
Toward Fast Neural Computing using All-Photonic Phase Change Spiking Neurons
Indranil Chakraborty, Gobinda Saha, Abhronil Sengupta, Kaushik Roy
Scientific Reports, 2018
[Paper]
- Designed spiking neuron based on GST (phase change material) embedded optical micro-ring resonators.