Research & Publications
With rapid advancements in Cyber-Physical manufacturing, the Internet of Things, Simulation software,
and Machine Learning algorithms, the applicability of Industry 4.0 is gaining momentum. The demand
for real-time decision-making in the manufacturing industry has given significant attention to the field of
Digital Twin (DT). The whole idea revolves around creating a digital counterpart of the physical system
based on enterprise data to exploit the effects of numerous parameters and make informed decisions.
Based on that, this paper proposes a simulation-optimization framework for the DT model of a Beverage
Manufacturing Plant. A data-driven simulation model developed in Simio is integrated with Python to
perform Multi-Objective optimization. The framework explores optimal solutions by simulating multiple
scenarios by altering the availability of operators and dispatching/scheduling rules. The results show that
simulation optimization can be integrated into the Digital-Twin models as part of real-time production
planning and scheduling.
Detailed information unavilable due to NDA
-
Machine Learning based clustering algorithm for pick-path routing ( Patented and implemented).
-
AI-driven inventory placement algorithm - an innovative approach to wholistically rank SKUs (Patented).
-
Self-optimizing digital twin model based on a granular discrete event simulation platform. (Implemented)
-
Simulation toolkit for product, operations, and strategy planning. (Deployed)
Several studies have shown the success of Reinforcement Learning (RL) for solving sequential decision-
making problems in domains like robotics, autonomous vehicles, manufacturing, supply chain, and health
care. For such applications, uncertainty in real-life environments presents a significant challenge in training an RL agent. RL requires a large number of trials (training examples) to learn a good policy. One of the approaches to tackle these obstacles is augmenting RL with a Discrete Event Simulation (DES) model. Learning from a simulated environment, makes the training process of the RL agent more efficient, faster, and even safer by alleviating the need for expensive real-world trials. Therefore, integrating RL algorithms with simulation environments has inspired many researchers in recent years. In this paper, we analyze the existing literature on RL models using DES to put forward the benefits, application areas, challenges, and scope for future work in developing such models for industrial use cases.
Dispatching rules fundamentally dictate the performance of a job-shop problem. Heuristic approaches to solving Job Shop Scheduling(JSS) assumes JSS as a static environment and fails to incorporate the dynamic changes taking place in real-time. Such changes caused by in-coming jobs often alters the characteristics of an entire job queue, making static solution approaches unable to meet the real life requirements. In this study, we structured the JSS problem as a sequential decision-making process wherein a Reinforcement Learning (RL) agent outputs the policy of dispatching rule by adapting to the dynamic nature of incoming jobs. We modeled and tested an on-line dispatching rule framework that contains a Job shop simulator and a Deep Q-Learning Network(DQN) RL agent. A single machine environment was simulated to investigate the application of DQN for a multi-objective job shop scheduling problem. Three practical objectives including, total tardiness, total make-span, and percentage of jobs completed in time were evaluated. The experimental analysis indicates that the dispatching rule produced by RL agent achieves considerable gains when benchmarked with traditional heuristic approaches. The findings from this research are expected to be the basis for further investigations into applying reinforcement learning to more complex job shop environments in the future.
Paper Link - TBD (2024 WSC)
Unmanned Aerial Vehicles have proven to enhance customer service and increase efficiency in supply chain management. They offer greater flexibility, ease of operation, and bypass traffic congestions by flying directly between nodes. This paper presents an innovative version of the Team Orienteering Problem with Time Windows and Charging Stations. The proposed model integrates various optimization approaches, including heuristics, simheuristics, and AI-driven simheuristics. The primary objectives are to maximize service rewards, minimize total travel distance, and mitigate out-of-charge incidents. Experiments are conducted to demonstrate the competency of the applied AI-enabled simheuristic approach in various scenarios.
This graduate course level tutorial was developed for getting started with RL for engineering applications using MATLAB. Solutions are available upon instructor request - Please visit the link.
​
In Stage 1 we start with learning RL concepts by manually coding the RL problem. Later we see how the same thing can be done by using functions available in MathWorks RL toolbox.
In Stage 2, we deal with complex environments and learn how Deep Learning agents are modelled and trained. Additionally, we see how to custom build an environment in MATLAB.
In Stage 3 we introduce Simulink. We develop environments using Simulink RL blocks.
In Stage 4 brings us to additional environments of Mechanical and Industrial Engineering problems, that we will build using the concepts taught before.
In emergency cases the delay in receiving the necessary pre-hospital care results in a large number of deaths every year. Providing appropriate preliminary care, along with proper time management and pre-hospital management can contribute to a better survival rate. Here the authors propose a portable system which transmits the vital parameters to the health care center along with the images of the patient, also availing the patient's Personal Health Record to the doctor, thus bridging the gap between the hospital and the ambulance and “virtually” bringing the doctor to the ambulance, thereby allowing him to diagnose the patient remotely and as accurately as possible. The paper puts a glance on rapidly developing field of Tele-medicine while proposing a system to overcome infrastructure inadequacies