Ferdous Alam

Md Ferdous Alam

I am a post-doc at MIT working with Faez Ahmed. Recently, I earned my Ph.D. from the Ohio State University where I worked on machine learning, representation learning and reinforcement learning for decision making in complex real-world tasks such as design, manufacturing and robotics. I was supervised by David Hoelzle. Previously, I also worked as a research intern and later a research collaborator at the Autodesk AI Lab where I worked with Rodger Luo on AI driven design approaches.

Email | Bio | Google Scholar | Academic CV

Research

My core research interest is to enable efficient decision making in complex systems. I draw from artificial intelligence (AI)/machine learning (ML) and control theory to develop learning algorithms for intelligent manufacturing systems, autonomous robots and generative/algorithmic engineering design tasks. Within AI/ML my interest lies in representation learning and reinforcement learning. I am also interested in the AI/ML systems integration for these critical application domains. During my Ph.D., I developed data-efficient reinforcement learning algorithms using knowledge transfer between tasks and probabilistic reward modeling. I was also part of the team that built a state-of-the-art autonomous manufacturing research robot that can take decisions in real-time to design and manufacture a complex geometry 3D artifact without any human intervention and require 10x less data samples than traditional approaches. I have designed and implemented the AI/ML software architecture for this autonomous robot. I have also worked as a research intern, and later a collaborator, at the Autodesk AI Lab where we developed transformer based representation learning and autoregressive generative models for voxel-based 3D design tasks.

Dissertation: Efficient Sequential Decision Making in Design, Manfuacturing and Robotics [overview]

	From Concept to Manufacturing: Evaluating Vision-Language Models for Engineering Design Cyril Picard, Kristen M. Edwards, Anna C. Doris, Brandon Man, Giorgio Giannone, Md Ferdous Alam, Faez Ahmed (under review), 2023 This paper presents a comprehensive evaluation of GPT-4V, a vision language model, across a wide spectrum of engineering design tasks
	An advantage based policy transfer algorithm for reinforcement learning with metrics of transferability Md Ferdous Alam, Parinaz Naghizadeh, David Hoelzle (under review), 2023 An advantage based policy transfer algorithm for reinforcement learning with metrics of transferability
	Representation learning for sequential volumetric design tasks Md Ferdous Alam, Yi Wang, Linh Tran, Chin-Yi Cheng, Jieliang Luo (under review), 2023 Models for learning representations of complex sequential design tasks with applications in volumetric design
	Reinforcement learning for autonomous manufacturing systems Md Ferdous Alam, Max Shtein, Kira Barton, David Hoelzle to be submitted, 2023 Successful demonstration of reinforcement learning on prototypical autonomous manufacturing robot hardware
	Autonomous manufacturing testbed to evaluate machine learning algorithm performance Zhi Zhang, Antony George, Md Ferdous Alam, Chaitanya Krishna Prasad Vallabh, Chris Eubel, Max Shtein, Kira Barton, David Hoelzle Journal of Manufacturing Science and Engineering, 2023 Hardware design for autonomous manufacturing systems
	Reinforcement learning enabled autonomous manufacturing using transfer learning and probabilistic reward modeling Md Ferdous Alam, Max Shtein, Kira Barton, David Hoelzle IEEE Control Systems Letters (L-CSS) , 2022, Conference on Decision and Control (CDC) , 2022 Transfer reinforcement learning on physical autonomous manufacturing data using reward modeling
	Sample efficient transfer in reinforcement learning for high variable cost environments with an inaccurate source reward model Md Ferdous Alam, Max Shtein, Kira Barton, David Hoelzle American Control Conference (ACC) , 2022 Creating useful temporal abstractions can help even when source reward model is wrong
	A physics guided reinforcement learning framework for an autonomous manufacturing system with expensive data Md Ferdous Alam, Max Shtein, Kira Barton, David Hoelzle American Control Conference (ACC), 2021 Temporal abstraction can be used to increase sample efficiency.
	Autonomous Manufacturing Using Machine Learning: A Computational Case Study With a Limited Manufacturing Budget [Best paper award], Md Ferdous Alam, Max Shtein, Kira Barton, David Hoelzle Manufacturing Science and Engineering Conference (MSEC), 2020 Decision making in manufacturing systems is extremely difficult with limited data

Projects

I have developed the open source autonomous manufacturing software architecture which is specifically built for our research robot. This software supports modular implementation of any learning algorithms for real-time decision making in the robot. Additionally I am developing another open source library for modular implementation of state-of-the-art AI algorithms. For example, here is the implementation of the vanilla transformer architecture from scratch and corresponding blog with implementation details.

Autonomous Manufacturing Software
Md Ferdous Alam,
under development, 2022
project page

Modular implementation learning algorithms for real-time deployment in autonomous manufacturing robot

zero-shot policy transfer, action-value transfer, PRM-TAPRL
Bayesian optimization algorithm

Applied Machine Learning tutorials
Md Ferdous Alam
under development, 2022
project page

Implementation of statistical machine learning, deep learning and reinforcement learning algorithms from scratch. Examples: Deep learing ⇒ Vanilla Transformer: implementaion in pure PyTorch, Reinforcement learing ⇒ SAC: implementaion in pure PyTorch, Classical machine learning ⇒ PCA, SVM: implementaion in pure python (numpy), Gaussian process: implementaion in pure python (numpy)

Misc

Reviewer: Transactions on Automatic Control, Mechatronics, Journal of Dynamic Systems, Measurement and Control, Conference on Decision and Control (CDC), American Control Conference (ACC), Manufacturing Science and Engineering Conference (MSEC),

The Graduate Student Perspective
Personal Blog Posts

Deep learning, Reinforcement learning, Optimal control theory

Adapted from this website.