The company miHoYo has introduced an AI agent named Gamer, capable of playing video games. It does not require access to the game code—the neural network relies on what is happening on the screen, just like a regular gamer.
Honkai: Star Rail
The operating principle of Gamer is quite simple. The AI agent analyzes the image on the screen at a rate of five frames per second, then independently decides which keyboard keys to press and how to move the mouse. Moreover, the agent can remember previous actions and plan future steps.
The capabilities of the AI agent were tested on miHoYo's game Honkai: Star Rail — and quite successfully. Gamer was able to navigate the game world, solve puzzles, and even defeat opponents in battles. However, not everything was perfect. As noted by miHoYo, the most advanced version of the AI agent showed an average efficiency of 49.7%.
miHoYo plans to soon release the engine that powers Gamer to the public. However, it requires powerful hardware to operate—the company used NVIDIA H200 processors.
Gamer is a joint project between miHoYo and researchers from Tsinghua University and the National University of Singapore. According to the developers, in the future, the AI agent may prove useful for studying human-computer interaction, automated game testing, and creating tools to help people with disabilities play games.
