Logo: Max Planck Institute for Biological Cybernetics
Remote-controlling Lego Mindstorms NXT by using Visual C++




Remote-controlling Lego Mindstorms NXT by using Visual C++

Introduction

After the big success of the Lego Mindstorms RCX, the Lego NXT robotics kit provides an even more powerful system for simple robotic applications. Improvements include servo-controled motors, an ultrasound distance sensor, a more powerful computer brick and bluetooth communication.

However, again the standard way to program the NXT robot is to upload control programs to the NXT brick, after which they run autonomously without the need of a host computer. This has some drawbacks. The computational power and memory of the NXT brick is limited, which is an issue for complex control programs. Also in- and output are limited to the four sensor inputs and the three motor outputs of the NXT brick. This means that it is for example not possible for such stand-alone programs to use a signal of an overview camera, or a camera mounted on the robot for robot control.

We solved this problem previously, for the Mindstorms RCX robots, by using the infrared communication with the host computer to remote-control the robot. A host computer could then run computationally expensive applications, for example involving analysis of a camera image, read out sensor values from the RCX via infrared, and remote-control the motors of the robot, also via infrared communication. The design idea of this interface was also to provide a very simple solution which does not use proprietary libraries etc., with the source code available, so that the interface could also be modified/exdended or adapted to other compilers/languages/systems by an experienced programmer. Also the fact that the main program communicates with the interface by setting and reading variables, and the interface itself runs in a separate thread, should make it quite easy to use the interface.

The bluetooth communication capabilites of the Lego NXT robot allow for a similar way to remote-control the robot. The bluetooth interface has the advantage of higher reliability and larger range, compared to infrared communication. Also the data rate should be higher; however, polling sensor values is still very slow, because the bluetooth interface unfortunately needs approximately 50 ms to switch from receiving to sending, or vice versa.

Visual C++ remote control interface for the NXT

Fortunately, the NXT bluetooth interface can be accessed in almost the same fashion as the RCX infrared interface. For the host computer, the bluetooth connection looks just like a serial connection, and a C++ program can access it just like any other serial connection. Also fortunately, the direct control commands have not changed much from the RCX version, which made it quite simple to adapt the RCX remote control interface to the NXT. And, the NXT communication is much better documented by Lego than the RCX communication was. This interface uses the Lego Mindstorms NXT "Direct Commands".

However, there are some caveats. First, even though the bluetooth connection is more reliable and faster, polling sensors is still quite slow. Second, it is not straight-forward to access the very useful ultrasound distance sensor, for which some additional initialization commands are necessary. Third, the logic of the former RCX interface, which uses volatile variables which are simply set and read by the C++ application to remote-control the robot, is not capable of supporting the very accurate control of motors which is possible with the new servo motors of the NXT kit.

Currently supported features:

  • Remote-control communication runs in a separate thread, interfacing is done via volatile variables
  • Read-out of current NXT sensor values; all standard sensor types and modes supported (including the ultrasonic distance sensor!)
  • Motors can be switched on and off, and set to different speeds (using power regulation)

Currently NOT supported features:

  • Starting and stopping of programs on the NXT
  • Playing of audio sound files and tones
  • Accurate control of motor movements (for example, turning a motor for a specified angle)
  • Messages, motor position reset, battery level read-out, keepalive, uploading programs etc.

Here is the code (use on your own risk!):

Instructions to use the nxt_roam example program

This interface has been written to be compiled under Windows, using Visual Studio 2005. It may also work with other compilers. Here is a list of instructions how to get the program to run under Visual Studio 2005:
  • Open Visual Studio.
  • Go in the main menu of Visual Studio to "File/New/Project...".
  • In the "New Project" dialog, select "Win32" as project type, and "Win32 Console Application" in "Templates". Enter "nxt_roam" as a name. Press OK.
  • In the "Win32 Application Wizard", click "Next >". Then make sure that "Application type" is set to "Console Application", "Empty Project" is checked, and "Precompiled header", "ATL", "MFC" are NOT checked. Press "Finish".
  • Visual Studio 2005 should now have created a folder at the specified target location for your project; if you didn't change the default, it should be located in "My Documents\Visual Studio 2005\Projects\nxt_roam\". There you should find another subdirectory, again called "nxt_roam", where a file called "nxt_roam.vcproj" should be located. Copy the three files (nxt_remote.h, nxt_remote.cpp, nxt_roam.cpp) into that subdirectory (in the standard case, into "My Documents\Visual Studio 2005\Projects\nxt_roam\nxt_roam\", for example by using the windows explorer (hold down windows-key and press e).
  • Now back in Visual Studio, go in the main menu to "Project/Add Existing Item...". Select all three files (nxt_remote.cpp, nxt_remote.h, nxt_roam.cpp). Click "Add". In the solution explorer on the left side of the main window, you should see "nxt_remote.h" under "Header Files", and "nxt_remote.cpp" and "nxt_roam.cpp" under "Source Files".
  • Now go in the main menu to "Project/Properties". Open the "Configuration properties" on the left side and click on "General". in the "Project Defaults" on the right side, click on "Character Set" and switch from "Use Unicode Character Set" to "Use Multi-Byte Character Set". Press OK.
  • Now compile your program. It should compile with no errors (and no warnings), but will probably not run, because the robot communication link is missing.

To run the program, you first need to establish a link to your NXT robot by using the bluetooth interface. The robot should be set up as a TriBot with the switch and the ultrasound sensor mounted in front (see image). The left motor should be connected to output C, the right motor should be connected to output A. The ultrasound sensor should be connected to input 4, the touch sensor to input 1.

Then proceed as follows:
  • Set up a working bluetooth serial port (COM) connection between your host computer and your NXT. You can use the Lego Mindstorms NXT software to check whether the connection is working.
  • Find out which the number of the COM port that is used. Here on my system I get the bluetooth settings dialog by right-clicking on the bluetooth icon at the right in the task bar, or in the bluetooth device settings in the Windows Control Panel. For the bluetooth dongle on my notebook, for example, the COM port used to communicate with the RCX is "COM5".
  • In Visual Studio, open nxt_roam.cpp in the text editor window. Go to line 31, which reads: "if (nxtr->startcommunication("COM5")==0){" ..., and change the "COM5" to whatever COM port you are using on your system.

Now run the program. If everything is configured correctly, the robot should start to move forward. If it approaches obstacles ('seen' with the ultrasound sensor), it should slow down and veer off to the left. Hitting the touch sensor should stop the robot and the control program.

Here is a video of the NXT in remote-control action using this interface (MPEG-1, 835 kb).


Please remember that you use this software on your own risk.
Contact me for questions and comments at: daniel.berger@tuebingen.mpg.de
Last updated: March 15th, 2007