Have anyone of you heard of a Pixel Bot? A pixel bot tracks the changes of game objects through image processing. AutoIt is a freeware BASIC-like scripting language designed for automating the Windows GUI and general scripting. It uses a combination of simulated keystrokes, mouse movement and window/control manipulation in order to automate tasks (e.g. VBScript and SendKeys).

For example, the top right corner of the game screen is my HP/MP bar where it is represented with unique colors. We can program a script to check the pixel located at specified X & Y position to track the changes in those bars. With the help of auto target function provided in some game (e.g. Tab button), a bot can help you to slay monsters automatically by sending simulated keyboard/mouse instruction to the game, even walking!

Interesting? However, this way of botting is so common that all the game guards have already blocked a few methods from User32.dll (User32.dll stores instructions for graphical elements such as dialog boxes and windows). However, you can try it using C# and call SendInput and GetPixel method by importing the method from user32.dll, you can go here to read more.

No.1 – Scan Pixel

Screenshooting! Since game guard has disallowed us to retrieve pixel info from the game window, we can just screenshoot the whole desktop and process the image. Even if it consumes a lot memory, but with the hardware specification we have nowadays, this shouldn’t be a problem. Since C# is a managed language, the image processing will not act fast unlike those unmanaged codes which deal directly with memory. To achieve almost the same speed a C++ program can do, we must first declare an unsafe method where it will allows us to access image pixel data with memory pointer and then lock the memory with Bitmap.LockBits(..) method to stop the bitmap data from moving around in memory.

This is a small example with source code (VS 2008 project) of how to locate monster’s hp bar with pixel detection. (Undocumented)

The difference between 2 ways of pixel retrieval is significant. In the example program attached above, scanning the hp bar with Bitmap.GetPixel(..) will take about 1.3 second + while lock bit method is less than 0.1 second.

Getting HP and MP info is not enough for to run a bot, we need to know what kind of monster we are killing (so that we won’t hit those strong ones / bot trap). Obviously, an OCR function is needed to scan text from image. You can make use of the Microsoft Office Document Imaging library (as a reference dll) to do this but for me, I did it in another way :D. By eliminating pixels which have both RGB value less than 30, we can eliminate all the extra pixels (Non-Black) and get a brief view of text patterns we are dealing with. We can compare them to predefined pattern of characters to determine which is what.

Before

After

For further details, you can refer to the ScreenDetector.cs source file in the RappelzBot source code.

No.2 – Bluetooth HID for Keyboard & Mouse

After executing a set of algorithms based on the screen input, you need something to control your game character. But too bad, the game guard blocks the SendInput command or anything which are considered as an emulated input. Game guard has also installed its own driver which will also block inputs from Virtual Drivers (VXD). For anyone who owns a sony ericsson phone, there is a small Remote Control which enables you to control your PC through Bluetooth. It is registered as a Bluetooth HID under Windows and most importantly, game guard never block input from a Bluetooth HID (HID = Human Interface Device e.g. keyboard & mouse). To modify the remote control role of each phone button, you can download the Bluetooth Remote Control tool, it can make your phone buttons to represent almost all keyboard and mouse inputs.

Our phone can now control the game, but we still need to send command to our phone as to emulate the press of phone buttons (which indirectly controlled the game). This can be done with AT&T command, a common ‘language’ for software to communicate with phone through a serial port (Bluetooth is considered as a serial port). For more detail on what AT&T command can do, download this (for Sony Ericsson phone only).

For further details, you can refer to the Keypad.cs source file in the RappelzBot source code (download link at below).

No.3 – Moving your mouse cursor

Somehow, the mouse move command from Bluetooth HID doesn’t work under game guard. We can’t move our cursor to the X and Y position we calculated (monster position) and click. There is a trick to do this, hide your game window, move your cursor to targeted point, and then show back your game window. There are a few user32.dll method you can use to achieve this, luckily, it is not blocked by game guard. Hide, Point and Show will take not more than 0.5 second, so it’s not a problem.

ShowWindowAsync(..);

ShowWindow((..);

SetForegroundWindow(..);

Again, check here to know more about the usage of those methods. (*Note: the mouse click is working fine)

Something to say…

For those of you who enjoyed reading what I’ve written above, thank you. But for those who think the bot is useless and memory hogging, I’m totally agree with you! Here are the reasons…

• When someone call your phone while you’re botting with your phone, say goodbye to your game character
• Not as responsive as a memory/packet editing bot, you need a PC with good spec to enjoy using it
• You’re out if you’re not using Sony Ericsson

*There is a bug with the Show/Hide implementation (should be user32.dll’s fault :P), sometimes the game window will not show up again after it is hidden, which at the end failing the pixel scanner.

Download the source here (Undocumented, View it at your own risk :P)