Share
With Amazon’s Deep Racer you can digitally control a small, real-life car, based on a Reinforcement Learning model. Sounds fun, right? Thijs de Vries and Diederik Greveling have been delivering workshops with the DeepRacer to educate participants about AWS services and Reinforcement Learning. Could DeepRacer be the accelerator for the application of Reinforcement Learning in the real world?
Reinforcement Learning has a lot of potential. However, despite the high expectations, it has been on the left side of Gartner’s Hype Cycle for AI for a few years now. Progress seems to be slow, for which there are several reasons, Diederik explains. He is a machine learning engineer at Xebia.
“First of all: Reinforcement Learning is pretty complex. Data scientists tend to get intimidated by the math behind it. The agent, that is, the algorithm, continuously has to calculate the best possible reward, given the state it is in. So if you want to implement it from scratch, you need to understand this math — at least a little bit."
But there’s more to Reinforcement Learning than ‘just’ complex math, cloud consultant Thijs adds. He is data engineer at Xebia.
“Reinforcement Learning requires a lot of data processing. And by a lot, I mean, really a lot. It, therefore, is most efficient to use cloud architecture. Then you can run, for example, 1,000 machines at the same time. Without cloud technology, the Google Go algorithm couldn’t have defeated the best Go players in the world. It needed enormous amounts of calculating power to continuously calculate the best option from millions of possibilities."
Reinforcement Learning Use Cases
RL works best in situations that can be simulated well and where a lot of data is available. A simulation is a closed environment with which the agent, the AI algorithm, can interact.
“It has to be pretty defined because you don’t want an infinite number of variables”, Thijs says. Therefore, board games are good use cases; as are intersections with traffic lights and the charging network for electric cars.
Real-World Data for Reinforcement Learning
You can simulate a situation to build a Reinforcement Learning model, or you can use real-world data.
Diederik: “Google now uses Reinforcement Learning for climate control in their data centers. That was feasible because they had huge amounts of data at their disposal.”
Now that companies have huge amounts of data at their disposal, reinforcement learning also becomes more accessible for them.
Thijs says: “More and more companies have large amounts of sensors. You can use the data from those to let your fleet drive more efficient routes, for example, or to perform smart maintenance. If you smartly design your cloud, you can apply edge computing for your Reinforcement Learning model. Then you can collect and use data relatively quickly and with relatively low bandwidth.”
Deep Racer as a Reinforcement Learning Booster?
Diederik and Thijs think that Amazon’s Deep Racer will give Reinforcement Learning yet another boost. Developers can experiment with Reinforcement Learning, using AWS Deep Racer. It uses cloud technology to take away the complexity of creating and training a Reinforcement Learning model. Users log in to the DeepRacer console to optimize the parameters of their model; AWS trains the model for them, thus making Reinforcement Learning much more accessible.
Developers can even compete in a global autonomous racing league. Diederik and Thijs use Deep Racer to show what Reinforcement Learning is and how it works. “It’s a very accessible tool for all kinds of skill levels”, Thijs says. “By making Reinforcement Learning accessible for software engineers, data scientists and other interested professionals in a fun yet competitive way, we hope to develop the skills around cloud and Reinforcement Learning. Ultimately, this could lead to more exciting Reinforcement Learning use cases.” “And more importantly”, Diederik adds.
“It’s fun. It’s fun to write an algorithm, and to see what the results of your efforts are immediately.”