July 22, 2013

Hello, Mr. Bond; Hacking the Agent Watch with Xamarin.Android

By

Our friends over at Secret Labs (creator of Netduino, the .NET Micro Framework based microcontroller that shares the Arduino form factor) have created the coolest smart watch yet, called the Agent Watch:

Agent Watch

The Agent Watch runs .NET Micro Framework applications written in C# and sports an insane battery life, wireless charging, and a million other cool features. The response to it has been incredible. In fact, they ran a Kickstarter campaign to raise $100k, which they raised in 11 hours! And by the time the campaign was done, they raised well over a million dollars!

Not only can you write Agent Watch applications in C# that run on the watch, but the watch is equipped with Bluetooth LE 4.0, and supports the Serial Port Profile, which allows you to communicate with the watch over Bluetooth. As such, you can write mobile applications that run on Android and Windows Phones (iOS isn’t supported yet, at least in the emulator, but support may be coming in the future):

02 - Com between Android + Watch

The watch isn’t out yet, but a couple weeks ago, they released the Agent Watch SDK which includes an Agent Watch Emulator. Now, the awesome thing about this, is that we can communicate with the Emulator over Bluetooth, via the host machine running it. This means that we can start creating awesome mobile applications today that work with the watch!

03 - Com between Android + Watch Emulator

So, with the help of Chris Walker (CEO of Secret Labs), and Frank Krueger (Author of iCircuit and an awesome guy), I set out to create and document a sample application that does just that. It sends messages from an Android device to an Agent Watch application running in the emulator, and then sends back an acknowledgement:

04 - Side by Side

I know you’re excited, so, without further ado, let’s get to it.

Sample Download

The apps used here can be download here:

Step 1: Installation

Before we do anything, we need to get the the various software installed needed for our journey:

  1. Xamarin.Android – Make sure you’re setup and ready to build and deploy Xamarin.Android applications. You’ll also need a Bluetooth Capable Android device running at least Android 4.0.3. You can use either a Mac or Windows machine for this.

  2. Visual Studio (VS) 2012, or Visual Studio Express 2012 – To build and run our Agent Watch application in the emulator, you’ll need a Windows machine running VS or VS Express (which is free). Note, that in order to communicate with the emulator via bluetooth, VS can’t be run in a Virtual Machine due to a limitation in Bluetooth that doesn’t allow multiple hosts to share incoming Bluetooth connections. If you’re using a Mac to do all this, make sure to boot directly into Boot Camp, rather than running Windows in VMWare or Parallels.

  3. .NET Micro Framework (MF) 4.3 SDK – Our Agent Watch application is powered by the .NETMF, so this will need to be installed on the machine that you have VS installed on.

  4. Agent SDK – The Agent SDK includes the Agent Watch emulator that we’ll run the Agent Watch Application in. Make sure to install this on the same machine that you’re running VS on.

Step 2: Expose the Agent Emulator’s Serial over Bluetooth

In order to communicate with the Agent Watch Emulator, we need to pipe its Serial Bluetooth endpoint through the host machine’s Bluetooth Adapter. Once this is configured, the machine running the Emulator will get connected to from our Android device just like we would if we were connecting to the actual Agent Watch.

Note that this should be done on your Windows Machine.

  1. Open the Bluetooth Settings configuration panel by right-clicking on the Bluetooth Icon in the System Task Tray and selecting Bluetooth Settings. You may have to click the arrow to show the Bluetooth Icon:
    05 - Bluetooth Task Tray

  2. In the Bluetooth Settings dialog, click on the COM Ports tab and then the Add… button:
    06 - Bluetooth Settings

  3. In the Add COM Port dialog, choose Incoming, and click the OK button:
    07 - Add Com Port
    Your Bluetooth Settings should now show the Incoming Bluetooth COM port:
    08 - COM port added

  4. Make note of the COM port, e.g. “COM3,” as we’ll need this later when we configure the Agent Watch Emulator.

  5. Enable Bluetooth discovery by switching to the Options tab and checking the Allow Bluetooth devices to find this computer checkbox:
    09 - enable bluetooth discovery on windows

Step 3: Configure the Agent Watch Emulator

Now that we’re exposing a Serial Port (COM Port) over Bluetooth, we need to tell the Agent Watch emulator to use that port.

  1. Locate and open the AGENT Emulator.exe.emulatorconfig file in the agent install directory, probably: C:\Program Files (x86)\Secret Labs\AGENT SDK\Emulator\v4.3.

  2. Open the file in Notepad (running as an Administrator), or the text editor of your choice – it’s just an XML file, and near the end of the file, locate this section:

    <PhysicalSerialPort id="COM1">
      <ComPortHandle>Usart1</ComPortHandle>
      <PhysicalPortName>COM5</PhysicalPortName>
    </PhysicalSerialPort>
    
  3. Change the PhysicalPortName node to be the COM port we configured before (e.g. “COM3”):

    <PhysicalSerialPort id="COM1">
      <ComPortHandle>Usart1</ComPortHandle>
      <PhysicalPortName>COM3</PhysicalPortName>
    </PhysicalSerialPort>
    
  4. Save the file. Depending on whether you ran Notepad as an Administrator (oof, Windows), you may have to save the file somewhere else (like your desktop) first, and then copy it back over to the emulator config directory. Make sure that if you do so, to verify the file extension is not .txt.

Step 4: Create the Agent Watch Application

Now, let’s create the application that will actually run on the Agent Watch. Agent Watch Applications are written in C# and use the .NET Micro Framework, which was designed specifically to run on microcontrollers such as the Netduino and Agent Watch. I’m presenting this as walkthrough to explain how to create these apps, but if you’re already familiar with Agent Watch Applications, you can just grab the app here.

  1. Launch Visual Studio 2012 (or Visual Studio Express) and create a new AGENT Watch Application project (found under Visual C# > Micro Framework), and name it Hello_MrBond:
    10 - New Agent Watch Application

  2. Just to test to make sure the emulator is working and all that, run it by hitting the play button in the toolbar. The Emulator should start up and you should see:
    11 - Hello World Running on Emulator

Great, if that works, we’re good to go. Copy the following code into the Program.cs file:

 using System;
 using Microsoft.SPOT;
 using Microsoft.SPOT.Hardware;
 using Microsoft.SPOT.Presentation;
 using Microsoft.SPOT.Presentation.Media;
 using System.Threading;
 using System.IO.Ports;
 using System.Text;
 namespace Hello_MrBond
 {
     public class Program
     {
         static Bitmap _display;
         static SerialPort _serial = null;
         public static void Main()
         {
             // initialize display buffer
             _display = new Bitmap(Bitmap.MaxWidth, Bitmap.MaxHeight);
             // sample "hello world" code
             _display.Clear();
             Font fontNinaB = Resources.GetFont(Resources.FontResources.NinaB);
             _display.DrawText("Hello, Mr. Bond.", fontNinaB, Color.White, 10, 34);
             _display.DrawText("Waiting for HQ.", fontNinaB, Color.White, 10, 54);
             _display.Flush();
             _serial = new SerialPort("COM1");
             _serial.Open();
             // when we get data from the serial port, handle it
             _serial.DataReceived += DataReceived;
             // go to sleep; all further code should be timer-driven or event-driven
             Thread.Sleep(Timeout.Infinite);
         }
         /// <summary>
         /// When we get data, write it out to the screen
         /// </summary>
         /// <param name="sender"></param>
         /// <param name="e"></param>
         static void DataReceived(object sender, SerialDataReceivedEventArgs e)
         {
             // this is a workaround because of an SDK bug
             StringBuilder received = new StringBuilder();
             while (_serial.BytesToRead > 0)
                 received.Append(ReadString());
             //string received = ReadString();
             // write the text out to the screen
             _display.Clear();
             Font fontNinaB = Resources.GetFont(Resources.FontResources.NinaB);
             _display.DrawText("Message from HQ:", fontNinaB, Color.White, 10, 34);
             _display.DrawText(received.ToString(), fontNinaB, Color.White, 10, 54);
             _display.Flush();
             // send back an acknowledgement
             SendString("Agent 007 acknowledges message: " + received);
         }
         static string ReadString()
         {
             StringBuilder output = new StringBuilder();
             int bufferSize = _serial.BytesToRead;
             if (bufferSize > 0)
             {
                 byte[] buffer = new byte[bufferSize];
                 int read = _serial.Read(buffer, 0, buffer.Length);
                 output.Append(System.Text.Encoding.UTF8.GetChars(buffer));
                 return output.ToString();
             }
             return "";
         }
         static void SendString(string message)
         {
             char[] charArray = message.ToCharArray();
             byte[] byteArray = new byte[charArray.Length];
             for (int i = 0 ; i < message.Length ; i++)
             {
                 byteArray[i] = (byte)charArray[i];
             }
             _serial.Write(byteArray, 0, charArray.Length);
         }
     }
 }

If you run this now, you’ll get a screen like this, waiting for an incoming message:

12 - Hello, Waiting for HQ

In just a moment, we’ll take a look at the Xamarin.Android application that will talk to it, but first, let’s examine the important pieces here. The first is the following:

_serial = new SerialPort("COM1");
_serial.Open();
// when we get data from the serial port, handle it
_serial.DataReceived += DataReceived;</span>

The default serial port on the Agent Watch is COM1, so we create a new serial port at COM1, open the socket, and then listen for DataReceived events.

Then, in our DataReceived handler, we read in the data and draw the message to the screen:

 static void DataReceived(object sender, SerialDataReceivedEventArgs e)
 {
     // this is a workaround because of an SDK bug
     StringBuilder received = new StringBuilder();
     while (_serial.BytesToRead > 0)
         received.Append(ReadString());
     //string received = ReadString();
     // write the text out to the screen
     _display.Clear();
     Font fontNinaB = Resources.GetFont(Resources.FontResources.NinaB);
     _display.DrawText("Message from HQ:", fontNinaB, Color.White, 10, 34);
     _display.DrawText(received.ToString(), fontNinaB, Color.White, 10, 54);
     _display.Flush();
     // send back an acknowledgement
     SendString("Agent 007 acknowledges message: " + received);
 }

We use our ReadString method to read all the data in the serial port and then decode the bytes into a string:

 static string ReadString()
 {
     StringBuilder output = new StringBuilder();
     int bufferSize = _serial.BytesToRead;
     if (bufferSize > 0)
     {
         byte[] buffer = new byte[bufferSize];
         int read = _serial.Read(buffer, 0, buffer.Length);
         output.Append(System.Text.Encoding.UTF8.GetChars(buffer));
         return output.ToString();
     }
     return "";
 }

Finally, we call our SendString method to acknowledge the send a message back, acknowledging our receipt:

static void SendString(string message)
 {
     char[] charArray = message.ToCharArray();
     byte[] byteArray = new byte[charArray.Length];
     for (int i = 0 ; i < message.Length ; i++)
     {
         byteArray[i] = (byte)charArray[i];
     }
     _serial.Write(byteArray, 0, charArray.Length);
 }

You can imagine that if this were anything other than a test app, we could encode data from the watch’s sensors, such as the accelerometer, or any number of other pieces of data, to then be used in our mobile application.

Step 5: Pair Your Android Device with the Emulator Host Machine

So our Agent Watch Application is up and running in the emulator, but we need to pair our Android device to the host machine that is running that emulator so that we can talk to it:

  1. Open the Settings Application on your Android Device and select Bluetooth. Then, make sure that Bluetooth is turned:
    13 - Bluetooth Settings
  2. Select the host machine that your emulator is running in (for some reason only the MAC address of mine is showing here). And pair:
    14 - Pairing
    If you’re running Windows 8, a little box should show up allowing you to pair. Make sure to select it and click OK.

Step 6: Create the Android Application

Now we’ve got everything setup, and the emulator running our application, let’s run the Xamarin.Android app that talks to our Watch Application! The application has two screens, one that enumerates the paired (bound) bluetooth devices and allows you to choose which to connect to, and then a screen that actually allows you to send and receive data:

15 - Android App SidexSide

We can use this application to send a message to the Agent Watch Application:

16 - Emulator - Message from HQ

Let’s take a look at the app, you can download it here, and open it up in either Xamarin Studio or Visual Studio.

Let’s look at the relevant code that makes this possible. All of the relevant Bluetooth code can be found in the App.cs file.

Getting a Reference to the Bluetooth Adapter

The first step in working with Bluetooth is getting a reference to the default adapter:

this.BluetoothAdapter = BluetoothAdapter.DefaultAdapter;

Getting a List of Paired (Bonded Devices)

After we’ve got a reference to the adapter, we can access all of the paired devices via the BondedDevices property. In our case, we need a list of names to display, so we loop through and add the names to a list of them:

foreach (BluetoothDevice device in this.BluetoothAdapter.BondedDevices) {
this.BondedBluetoothDeviceNames.Add (device.Name);
}

Creating a Bluetooth Socket Connection

Once we have the device that we want to connect to, we need to create a Socket Connection. A socket connection is like a pipe that allows us to actually communicate with the Bluetooth device.

A Bluetooth device can have multiple Endpoints within a given socket and each endpoint is identified by a UUID (the Android version of a GUID). This allows devices to communicate in different ways. In our case, we want to use the Serial Port Profile (SPP), which allows serial port communications via a specific endpoint.

This is where things get tricky. Ordinarily, we would query the BluetoothDevice object via a GetUuids call to return the UUIDs of the various endpoints:

ParcelUuid[] uuids = this.CurrentBluetoothDevice.GetUuids ();

Unfortunately, this API is only available in Android 4.0.3 (Ice Cream Sandwich) and later, and it’s very broken on Google’s side even for that release (and all subsequent releases). The SPP profile endpoint should have it’s first numbers end with 1101, but if we actually look at what Android gives us, we’ll see that the first one starts with 1105:

17 - UUIDs

Fortunately for us, we can actually just hard code the UUID for now, when we create a connection, and that seems to work for now. Hopefully, in the future, Google will fix this bug.

To create a socket connection to the SPP endpoint, we need to call the CreateRfccommSocketToServiceRecord method and pass our hard coded SPP endpoint UUID:

this.BluetoothSocket = this.CurrentBluetoothDevice.CreateRfcommSocketToServiceRecord (Java.Util.UUID.FromString("00001101-0000-1000-8000-00805F9B34FB"));

Bluetooth supports a number of different protocols, but we want to use RFCOMM because it’s a simple serial communication protocol.

Once we’ve created the RFCOMM Socket, we need to open it, however, before we do, Google recommends we make a call to CancelDiscovery, which will stop the Bluetooth device from looking for other devices.

this.BluetoothAdapter.CancelDiscovery ();

If we don’t call this, opening a socket can take considerably longer, and it may even fail.

Finally, we can call the Connect method on our socket that we got earlier, and we’re ready for communication:

this.BluetoothSocket.Connect ();

Sending Data

Socket communication is all done via Byte arrays. So in order to send a string message to the device, we need to first encode the message into a byte array, and then call Write on the socket’s OutputStream:

byte[] byteArray = Encoding.UTF8.GetBytes (message);
this.BluetoothSocket.OutputStream.Write (byteArray, 0, byteArray.Length);

Listening for Data

Listening for data is slightly more complicated than sending. There are no events raised when data comes in on the socket input stream, so instead, we have to create a new thread that constantly listens for data, and then reads it in when it exists. To further complicate things, rather than just reading in from the stream until a 0 is encountered (as you would ordinarily do), we must call an extension method called IsDataAvailable, otherwise, the thread will silently crash:

if (this.BluetoothSocket.InputStream.CanRead) {
     new Thread ( new ThreadStart (() => {
         byte[] buffer;
         while (true) {
             if(this.BluetoothSocket.InputStream.IsDataAvailable()) {
                 Console.WriteLine ("Got a message!");
                 buffer = new byte[1024];
                 int i = 0;
                 for( i = 0 ; i < 1024; i++) {
                     // ordinarily, we'd just check to see of cur != 0, but that'll raise an exception
                     // here for some reason, so we have to call the IsDataAvaiable each time. :(
                     if(this.BluetoothSocket.InputStream.IsDataAvailable()) {
                         byte cur = (byte)this.BluetoothSocket.InputStream.ReadByte();
                         buffer[i] = cur;
                     } else {
                         break;
                     }
                 }
                 // copy the buffer to a new array
                 byte[] message = new byte[i];
                 message = buffer.Take (i).ToArray();
                 // raise our event
                 Console.WriteLine ("Raising the MessageReceived event, message: " + Encoding.UTF8.GetString (message));
                 this.MessageReceived (this, new MessageReceivedEventArgs (message));
             }
             // give it a moment
             System.Threading.Thread.Sleep (250);
         }
     })).Start();
 }

The app uses a custom event args for messages which takes a byte array and converts it to string upon access:

 public class MessageReceivedEventArgs : System.EventArgs
 {
     public MessageReceivedEventArgs(byte[] message)
     {
         this.MessageBytes = message;
     }
     public byte[] MessageBytes { get; set; }
     public string MessageString
     {
         get {
             return Encoding.UTF8.GetString(this.MessageBytes);
         }
     }
 }

When we run the app and send a message, we also get a message acknowledgement from the Agent Watch Application:

04 - Side by Side

Pretty cool!

Awesome, What’s Next?

This application is really just a proof of concept, showing the basics to getting this working, but, you can bet that I’ll be hacking more on this, and hope to create a strongly-typed API that just works with the watch, communicating with it’s sensors and allowing arbitrary messages passed to it. So stay tuned.

In the meantime, we look forward to seeing the awesome things that you, our community create!

TwitterFacebookGoogle+LinkedInEmail