The goal was to develop an AI that learns to clean a room, from detecting trash to dumping it into a trashcan, all by itself using Reinforcement Learning.
The Agents/Bots will learn using Reinforcement Learning, therefore their actions are either rewarded or penalized. The future generation Bots will act upon these sets and create better and better behaviors.
The typical framing of a Reinforcement Learning scenario: an agent takes actions in an environment, which is interpreted into a reward and a representation of the state, which are fed back into the agent.
A typical Learning Environment contains three components:
Here the agent is keeping track of 8 observations. It will observe its distance and direction from the trashcan, its position vector, and whether it has picked up all trash. It uses five raycasts which cover up to a 120-degree field of vision. Each item is tagged and spawns randomly in the environment.
The Reward Function is an incentive mechanism that tells the agent what is correct and what is wrong using reward and punishment. The goal of agents in RL is to maximize the total rewards. These were the reward and punishment rules for the agent.
The first iteration was strange but expected because the 1st gen rewards system didn't counter the agent standing still. Adding penalties for standing still and hitting a wall solves this.
Also, training a single agent can be time-consuming and a slow process. Having multiple environments running parallel helps to lower the processing time significantly.
The results came out well. Though, I noticed that tweaking the reward system makes the bot adopt different strategies to achieve the goal.
Strategy #1: Collect all the litter first, then dump it.
*On increasing the 'Dump' reward*
Strategy #2: Dump after every pick up.
After all the modification, tweaking, and hours of training, I ended up with two different brains using two different strategies. Overall, this was a successful first attempt and an overwhelmingly enjoyable experience.