Projects

Game AI

Game AI is a subfield of AI that is concerned with the study of artificial intelligence methods and techniques in games. I am interested in how AI can play games and how games can be used to study AI.

General Video Game Playing (GVGP)

GVGP seeks to answer if an agent trained on some levels can work well on new levels. A novel reinforcement learning technique with dual-observation is proposed for general video game playing, assuming that it is more likely to observe similar local information in different levels rather than global information. Instead of directly inputting a single, raw pixel-based screenshot of the current game screen, our proposed general technique takes the encoded, transformed global and local observations of the game screen as two simultaneous inputs, aiming at learning local information for playing new levels.

Investigation on Game-based platforms

Games have been the perfect test-beds for artificial intelligence research for the characteristics that widely exist in real-world scenarios. Learning and optimisation, decision making in dynamic and uncertain environments, game theory, planning and scheduling, design and education are common research areas shared between games and real-world problems. Numerous open-source games or game-based environments have been implemented for studying artificial intelligence.

Games for AI education

Applying games in AI education is a promising approach to improve students' learning experience and outcomes. Beginners can learn AI concepts and techniques in a fun and interactive way, while advanced students can gain hands-on experience in solving real-world problems.

Safe Real-time Controlling

Decision making in real-time systems is challenging due to the uncertainty of the environment and the limited time for making decisions. Many work such as reinforcement learning have shown the capability of learning in real-time. However, the safety of the system may not be not guaranteed. I am working on how to make "safe" decisions in real-time systems.

Single-agent robot controlling via evolutionary contrained reinforcement learning

Evolutionary algorithms have been used to evolve a population of actors to generate diverse experiences for training reinforcement learning agents, which helps to tackle the temporal credit assignment problem and improves the exploration efficiency. However, when adapting this approach to address constrained problems, balancing the trade-off between the reward and constraint violation is hard. A novel evolutionary constrained reinforcement learning (ECRL) algorithm is proposed, which adaptively balances the reward and constraint violation with stochastic ranking, and at the same time, restricts the policy's behaviour by maintaining a set of Lagrange relaxation coefficients with a constraint buffer.

Safe Multi-agent system

Smart Logistics

Smart logistics is a promising application of AI in the real world. I am interested in how AI can be used to improve the efficiency of logistics, while considering the problem constraints.

Dynamic material handling

As one of the core parts of flexible manufacturing systems, material handling involves the storage and transportation of materials between workstations with automated vehicles. The improvement in material handling can impulse the overall efficiency of the manufacturing system. However, the occurrence of dynamic events during the optimisation of task arrangements poses a challenge that requires adaptability and effectiveness. Motivated by some real-world scenarios, unknown new tasks and unexpected vehicle breakdowns are regarded as dynamic events in our problem. We formulate the problem as a constrained Markov decision process which takes into account tardiness and available vehicles as cumulative and instantaneous constraints, respectively. An adaptive constrained reinforcement learning algorithm that combines Lagrangian relaxation and invalid action masking, named RCPOM, is proposed to address the problem with two hybrid constraints.

Split delivery vehicle routing problems with three-dimensional loading constraints