BrickGate is included in our extended leJOS SD card distribution.
Update SDCard (using WebStart).

1 Overview

When the Lego Mindstorms NXT brick is booted with the Lego or leJOS firmware, a server-like utility is started that listens for remote commands from a USB or Bluetooth connection. The remote device sends small byte-based command packets and the NXT interprets them according to a proprietary protocol (called direct command protocol). As an example, the byte sequence

0x06 0x00 0x80 0x03 0x0B 0x02 0xF4 0x01

will cause the NXT to emit a beep for half second.

Explanation:

0x06 0x00: Command length. 6 bytes (+2 length bytes) (0x0006, LSB first, little endian)
0x80: Prepare for Direct command with no reply. Every command can request a response message from NXT
0x03: Command type: Play a tone
0x0B 0x02: Frequency. Tone of 523 Hz (0x020B, LSB first, little endian)
0xF4 0x01: Duration in milliseconds. 500 ms (0x01F4, LSB first, little endian)

Compared to the autonomous mode, where a program is executed directly on the brick,  the direct mode has the advantage that robotics applications can be written on any remote device (PC, smartphones, microcontrollers, etc.) in many different programming languages. Most of the time Bluetooth communication is used, because the USB cable disturbs the movement of the robot.

The same scenario holds when the EV3 brick is booted with its original Lego firmware, but now the software is build around a Linux system that uses a kernel version 2.6.33 RC4. To access the hardware (motors, sensors, etc.) the system consists of multiple shared libraries written in C and compiled to machine code . A virtual machine interprets byte-code with a propriety protocol and also handles data communication from and to the outside. This virtual machine can execute byte-code based programs and also handles data communication to the outside. It is capable to interprete byte-code chunks from a remote client and could act as direct mode server (the Lego Mindstorms EV3 API based on Microsoft's .NET architecture uses this approach).

The EV3 system software from leJOS is also based on a Linux version 2.6 kernel and includes a Java Virtual Machine (Oracle Java SE Embedded).  Through its SSH server the Linux system can be accessed and managed from any SSH client. When booted, a EV3Menu program written in Java is started that displays a simple menu and handles requests from a TCP/IP link using Java RMI (Remote Method Invocation). RMI is elegant but the processing overhead is substantial. A test implementation showed response times in the order of 100 ms to 500 ms on a Bluetooth PAN based IP link. A simple IP socket client-server connection is faster by about a factor of 10. A response time of maximum 100 ms is fast enough for typical robotics applications where the robotics movement is controlled by data obtained from polling a sensor (e.g. a touch, light or ultrasonic sensor).

Our BrickGate is such a socket server written in Java that runs autonomously on the EV3 brick using the leJOS and EV3JLibA libraries. It is  started like any other leJOS programs from the leJOS menu. Similar to the direct server on NXT, it interprets commands from an remote client, but it uses well-known IP client-server technology and reflection to invoke the corresponding leJOS/EV3JLibA library methods (not RMI). The source of the former EV3DirectServer, a somewhat limited version of BrickGate, is part of the EV3JLib distribution and shows you the principles of operation.

BrickGate can handle direct commands from a remote PC client written in any programming language over a TCP/IP link using USB RNDIS, Bluetooth PAN or WLAN. But because the EV3 runs a multi-tasking Linux system, BrickGate also handles commands from a host program that resides on the brick itself through a TCP/IP localhost connection. Because the system runs now independently of any remote resource, it corresponds to an autonomous mode of operation. Again the program could be written in any programming language that is supported by the EV3, but our favorite is Python, because a full embedded Python interpreter can be easily installed on the brick and runs efficiently on the small embedded system.

To manage Python scripts, BrickGate displays a menu where the Python scripts in the /home/python/scripts directory are listed. Using the EV3 buttons, a script can be selected and executed autonomously.

2 BrickGate Command Protocol

BrickGate supposes that the classes if the EV3JLibA library are used locally on the EV3. The program behaves like an autonomous EV3 program with the addition of a socket server and a parser that interprets and executes the incoming commands.

All communications from the client to the BrickGate server (called commands) and back to the client (called responses) use a human readable text based protocol (strings). Commands consist of 2, 3 or 4 fields separated by a colon and must be terminated by a newline character (the linefeed character is used on the server side as end-of-command indicator). Examples:

Of course these commands may be sent to the server and the reply received by a socket client written in any programming language. But the Java based direct library EV3JLib applies exactly the same command API as the autonomous library EV3JLibA. In consequence, autonomous and direct mode programs look almost the same.

The general command format for motors and sensors is the following:

instance_name.method_name[.parameter_1][.parameter2]<lf>

Normally parameter_1 and parameter_2 are integers in string format. They are only present, if the method takes one resp. two parameters. instance_name is a predefined part-port identifier. Currently the following parts are available:

An unneeded parameter may be omitted or set to "n". For boolean parameters, "b0" stands for false and "b1" for true. To send a parameter that is recognized by the server as a string type, the leading tag character 's' must be inserted. The last character of the instance_name is used to designate the port. For sensors the characters '1','2','3','4' and for motors 'A', 'B', 'C', 'D' are legal.

As you have seen in the example above, there is no need to send a command to create the motor/sensor instance. The instance is automatically created when a addPart command is received. Because reflection is used to invoke the methods, consult the autonomous library documentation (EV3LibA) for the available methods and parameters, but only methods with no parameters and with one or two integer or boolean parameters are supported.

Every command execution will be confirmed by the server by sending back a response string. Until the response is received no other command should be sent to the server. This provides a simple handshaking between client and server. The response is normally an integer in string format with the following meaning:

If the command invokes a method with a boolean return type, the response is "0" for false and "1" for true.

Prior to send commands for motors or sensors a LegoRobot instance must be created by sending a robot.create command. All standard commands have the format

robot.method_name[.parameter_1][.parameter2]<lf>

with the following exceptions:

If you try to create a second instance of a device at the same port or the port is unavailable, an error is returned.

3 A Console For Testing Direct Commands

For testing purposes and during development of your own direct mode client library, a console program where you can send single commands and get the responses may be of great help. As a generic hint how to develop direct-mode programs in any programming language, refer to the following program written in good-old plain-C:

#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
#include <string.h>
#include <sys/types.h>
#include <sys/socket.h>
#include <netinet/in.h>
#include <netdb.h> 

void error(const char *msg)
{
    perror(msg);
    exit(0);
}

int main(int argc, char *argv[])
{
    int sockfd, portno, n;
    struct sockaddr_in serv_addr;
    struct hostent *server;

    char buf[256];
    if (argc < 3) 
    {
        fprintf(stderr, "usage %s hostname port\n", argv[0]);
        exit(0);
    }
    portno = atoi(argv[2]);
    sockfd = socket(AF_INET, SOCK_STREAM, 0);
    if (sockfd < 0) 
        error("ERROR opening socket");
    server = gethostbyname(argv[1]);
    if (server == NULL) 
    {
        fprintf(stderr, "ERROR, no such host\n");
        exit(0);
    }
    bzero((char *)&serv_addr, sizeof(serv_addr));
    serv_addr.sin_family = AF_INET;
    bcopy((char *)server->h_addr, 
         (char *)&serv_addr.sin_addr.s_addr,
         server->h_length);
    serv_addr.sin_port = htons(portno);
    if (connect(sockfd,(struct sockaddr *) &            
                serv_addr,sizeof(serv_addr)) < 0) 
        error("ERROR connecting");
    while (1)
    {
        printf("Please enter a command: (^C to quit): ");
        bzero(buf, 256);
        fgets(buf, 255, stdin);
        n = write(sockfd, buf, strlen(buf));
        if (n < 0) 
            error("ERROR writing to socket");
        bzero(buf, 256);
        int done = 0;
        char response[4096];
        bzero(response, 4096);
        int k = 0;    
        while (!done)
        {
            n = read(sockfd, buf, 255);
            if (n < 0) 
                 error("ERROR reading from socket");
           if (buf[n - 1] == 10)
               done = 1;
           int i;
           for (i = 0; i < n; i++)
               response[k + i] = buf[i];
           k += n;
        }
        printf("Response: %s\n", response);
    }
    close(sockfd);
    return 0;
}

Under Linux/MacOS you compile the program in a terminal using the built-in gcc command line compiler:

gcc client.c -o client

and execute it with

./client 10.0.1.1 1299

Here a version in Java using the convenient Console window from the ch.aplu.util package. Every command execution also reports the response time, so you can convince yourself that the system is fast enough to be used in Direct Mode EV3 applications.

import java.io.InputStream;
import java.io.OutputStream;
import java.net.Socket;
import ch.aplu.util.*;
import java.io.ByteArrayOutputStream;
import java.io.IOException;
import java.nio.charset.Charset;

public class EV3ClientConsole
{
  private String[] responseMsg =
  {
    "OK", "SEND_FAILED", "ILLEGAL_METHOD",
    "ILLEGAL_INSTANCE", "CMD_ERROR", "ILLEGAL_PORT", "CREATION_FAILED"
  };

  private String ipAddress = "10.0.1.1";
  private int port = 1299;
  private OutputStream os = null;
  private InputStream is = null;

  public EV3ClientConsole()
  {
    ch.aplu.util.Console c = new ch.aplu.util.Console();
    HiResTimer timer = new HiResTimer();

    try
    {
      System.out.println("Trying to connect to " + ipAddress);
      Socket s = new Socket(ipAddress, port);
      System.out.println("Connection established.");
      System.out.println("Enter command<cr>");
      os = s.getOutputStream();
      is = s.getInputStream();
      String command = "";
      while (true)
      {
        command = c.readLine();
        if (command.length() == 0)
        {
          System.out.println("Illegal command");
          continue;
        }
        timer.start();
        sendCommand(command);
        String response = readResponse();
        int rc = 0;
        try
        {
          rc = Integer.parseInt(response);
        }
        catch (NumberFormatException ex)
        {
        }
        if (rc >= 0)
          System.out.print("Response: " + response);
        else
          System.out.print("Response (error): " + responseMsg[-rc]);

        System.out.println(" in " + timer.getTime() / 1000 + " ms");
      }
    }
    catch (Exception ex)
    {
      ex.printStackTrace();
    }
  }

  private void sendCommand(String cmd) throws IOException
  {
    if (cmd == null || cmd.length() == 0 || os == null)
      throw new IOException("sendCommand failed.");
    cmd += "\n";  // Append \n
    byte[] ary = cmd.getBytes(Charset.forName("UTF-8"));
    os.write(ary);
    os.flush();
  }

  private String readResponse() throws IOException
  {
    if (is == null)
      return "";
    byte[] buf = new byte[4096];
    ByteArrayOutputStream baos = new ByteArrayOutputStream();
    boolean done = false;
    while (!done)
    {  
       int len = is.read(buf);
       if (len == -1)
         throw new IOException("Stream closed");
       baos.write(buf, 0, len);
       if (buf[len - 1] == 10)  // \n
         done = true;
    }
    String s = baos.toString("UTF-8");
    return s.substring(0, s.length() - 1);  // Remove \n
  }

  public static void main(String[] args)
  {
    new EV3ClientConsole();
  }
}

Execute the program locally using WebStart.

4 Client Connection/Deconnection Notification

When BrickGate is started, it emits a sound and waits for a client to connect to IP port 1299. (The port may be changed by editing ev3jlib.properties found in ch/aplu/ev3/properties. To edit the JAR file directly, use WinRAR or a equivalent utility.) The green EV3 LEDs flashes slowly as indication that the server is ready and listening for a client to connect.

When a client connects, the EV3 displays its IP address and the green LEDs are turned on constantly. When the client breaks the connection, the EV3 returns to the waiting/listening state, but does not terminate. To shutdown the server, the ESCAPE button of the EV3 must be pressed. (In emergency cases, you can kill the server by pressing DOWN+ENTER like you kill any other program started from the leJOS menu.) A client may be notified that the server died by using its own thread for receiving data. This thread hangs in the blocking read() of the InputStream to get data from the server. When the link is broken because the server closes the IP socket, a IOException is thrown. A client that tries to establish a connection to a BrickGate server that is not yet running is notified by a java.net.ConnectException.

The BrickGate may be defined as Default program. Since Run Default is the first option displayed in the leJOS menu, a simple click of the OK button is enough to start the server.

If you write a client program with these notifications in mind, you get a very stable client-server system.

5 Using the Dynamic Update Client (DUC) Of BrickGate

The EV3 may serve as front end processor for data acquisition using its sensors (temperature, humidity, luminosity, etc.) or for switching relays connected to the Prototype board by HiTechnic. It is also highly instructive to connect simple home brew electronic circuits to the EV3 to acquire data or control machines and devices. If you need to have access to the BrickGate server from a client outside the local area network, you can use a WLAN link to a router and use the router's address to connect from the remote. Because the router's address is set dynamically with DHCP by the internet provider, the address may change. To overcome this difficulty, BrickGate includes a Dynamic Update Client (DUC) that is turned off by default. Consult Install and Use of leJOS/Linux to get information how to configure it.

6 Running Autonomous Python Scripts With BrickGate

There are several approaches to use Python as programming language for the EV3. Because the EV3 runs a flavor of the Linux operating system, the solution a prima vista would be to install a Python interpreter for embedded systems and to write a Python library that accesses the C-routines for the EV3 hardware provided by the Lego development group (see the meritorious topikachu project). But this is reinventing the wheel because the leJOS group did exactly the same for the Java programming language. Since they put the source code to the public domain, have a look there and estimate how much work it takes to write a library that supports all the different EV3 specific hardware (display, sound, buttons, motors, sensors, etc.). To port the leJOS Java library to Python or to start from scratch was a no-go for us.

We realized another idea: Because the BrickGate server is a TCP/IP gateway that links commands from a IP client to the leJOS library, the client can also run on the brick itself and connect through a localhost link. With this design, a rather simple Python direct mode library must be written that can be used both on a remote PC or the EV3 itself. The port of the Java based EV3JLib classes to a Python class library is straightforward. The API naming is exactly the same, so porting the rich set of Lego robotics examples from Java to Python is simple. Moreover this design delegates different tasks to different Linux processes. There are three Linux processes involved: The leJOS menu and the BrickGate server run in a JRE process and the Python interpreter in another process. The socket communication over the local port link is fast enough for most robotics applications.

The BrickGate server is distributed together with the Python interpreter on the SD card installation of leJOS, so no additional installation is necessary. If the Python script directory /home/python/scripts contains at least one file, the BrickGate displays the scripts in a menu and you may run one of them by selecting it with the EV3 cursor buttons and pressing ENTER. As you see, no remote Linux SSH console is necessary to develop and run the Python scripts autonomously on the brick. Just copy the script to the EV3 via SCP and execute it in the BrickGate environment.

If you want to show some basic features of programming with Python to your audience, you may perform a sample session in a SSH terminal with Python on the EV3:

or start Python on a remote PC and connect via TCP/IP (over USB, Bluetooth or WLAN). The direct mode commands are exactly the same:

To understand what is a program, pack these commands in a text file MyFirst.py. Because the commands will be executed now by the machine automatically line-per-line, you must add a delay after the motor forward command. To execute the program, download it to the EV3 (with the Simple Copy Protocol (SCP) or WinSCP). Depending on what version of SCP you have, the command may slightly differ:

scp MyFirst.py root@10.0.1.1:/home/python/scripts/MyFirst.py

or with PuTTY pscp:

pscp -scp MyFirst.py root@10.0.1.1:/home/python/scripts

(password is empty).

The file appears immediately in the BrickGate menu. Tell the EV3 processor to execute it by selecting the file name entry with the EV3 Up-Down buttons and press Enter.

EV3 Python programming is smoothly integrated into our user-friendly TigerJython IDE. When you click the green Run button in the menu, the program is executed in direct mode. If you hit the EV3 button icon, the script is automatically downloaded and executed autonomously.