The culmination of my Master’s project is a final report outlining the work that I completed. The report provides an in-depth look into the work that I conducted, and provides some insight into the future potential of the project. The latest version of the report can be found at the link below:

Master’s Project Report – Ken Beck, 2013

Master’s Project Report, Anonymized

The code for my Master’s project can be downloaded from GitHub.

I just wanted to drop a quick line to let you know where I’m at with my project at this point. I’ve been spending the past couple weeks tweaking my code, as I realized there were some timing bugs (the flapping wasn’t occurring consistently when I told the device to fly straight). This was partly due to transmitting and receiving in tandem, but it turns out the problem went a little beyond that: I had made some mistakes in my timer interrupts. I actually went ahead and removed one of my interrupts, and am doing all the work in a single interrupt. I’ll talk about the functionality more in-depth in my report.

In addition to tweaking the code, I purchased a case and battery pack for the Arduino, so I’m not tethered to a cord when I’m doing my demonstration at my defense. You can see the pictures of the case below. The case is rather large, which I didn’t entirely expect when I purchased it, but it should work for what I’m looking to accomplish. I’ve got to figure out a better way to mount the wires (I’m going to use some electrical tape to tape the IR receiver to the controller itself, I believe), but so far the case works pretty well.

At this point, I’m working on some further tweaks to the timing as well as starting on my final report. Hopefully, everything goes smoothly from here.

Arduino Case and Controller, Side

Arduino Case and Controller, Back

After over six months of working on this project, I finally managed to inflate the Air Swimmers blimp for the first time. Up until this point, I had been prototyping from my desk, using the raw components from the blimp (see my previous blog post for a video showing the setup), because I had not acquired any helium to fill the blimp.

My initial impressions are that a) the shark is huge, and b) once you get it set up, it’s actually kind of fun. Getting it up and running helped to better demonstrate the problem that I was solving. While the remote setup they utilize is good for turning (as you need to flap in the direction you want to turn, then wait a couple seconds, then flap again), the task of flying straight is a little more arduous. You have to toggle the controls rather quickly (left-right-left-right-left, etc) to get the blimp to fly in a straight line without wobbling significantly off-course. My Arduino setup will ensure that the commands are appropriately timed to ensure a mostly-straight flight path.

Getting the blimp set up did show me that I’ve got some adjustments to make for my command timing, but not very many. I do realize that waiting until so late in the game to get the blimp set up could have had dire consequences (for instance, I could have found out that the timing approach I took was way off, and would require significant reworking), but as I was working on a prototype platform, I felt it imperative that I work in an environment where I could test my functionality quickly and easily, without chasing down a shark every time a command goes astray. In the future, I think I would spend more time setting up an actual testing environment where I could more-easily test the platform as a whole, rather than just its component parts.

My last post regarding abstracting the control scheme away left off with me looking into ways to fix the problem I encountered when my device was both sending and receiving IR signals. I mentioned that the Air Swimmers blimp did not appear to be affected by the combination of the Gyropter remote’s IR signal and my device’s IR signal, so there must be a way to compensate for the overlapping signals. However, upon further analysis, I realized that the Air Swimmers blimp was, in fact, affected by the Gyropter remote’s signal. If I point the Gyropter remote at the IR receiver on the Air Swimmers blimp and turn it on, while also using the Air Swimmers remote, the Air Swimmers blimp appears to stutter, as seen in the video below:

Based on this analysis, I’ve determined that the best approach to the conflicting signals is to physically separate the IR receiver and IR transmitter on my device. This is not an ideal solution, but it would appear that an efficient code-based approach would not be feasible. The major problem is that, for every command the Gyropter remote send out, the Air Swimmers blimp expects two commands (the delay between signals on the Gyropter remote is ~120ms, while the delay between signals on the Air Swimmers remote is closer to 50ms). Thus, any further delays that would be introduced in whatever routine handles the IR signal conflict resolution would cause the Air Swimmers blimp to become far less responsive. Thus, for this project, I will cover up the Gyropter remote and my device’s IR receiver so that the commands from the Gyropter remote are only received by my device.

The ideal solution for this problem would be to utilize two different technologies for receiving commands and sending them back out. For instance, I could look into using a Bluetooth-based controller instead of the IR-based Gyropter remote. Of course, this would fall outside the realm of this project, but it could warrant further investigation in the future.

After implementing the control logic for the Gyropter remote, I started looking into abstracting away the control scheme. This proved to be slightly more complex than I initially anticipated, and is still a work-in-progress.

The initial problem that I had to address was that I needed to send multiple commands between each message from the Gyropter remote. This is because the Air Swimmers blimp expects commands once every 50ms, while the Gyropter remote only sends commands once every 120ms. Further, I wanted the ability to gradually decrease the blimp’s speed once a user turns off the remote, rather than immediately stopping all input. As a final requirement, I did not want to use the delay() methods built into the Arduino, as those methods block all operations until the timeout has expired.

In order to achieve my goals, I needed to make use of timer interrupts. The Arduino Uno comes with three timer interrupts built-in. Timer 0 and Timer 2 are 8-bit timers (meaning, they count from 0-255) and Timer 1 is a 16-bit timer. As Timer 0 is also used for the delay() methods and other timing functions, I decided not to use that timer at all. Timer 2 had been set up in the previous phase for sending out the appropriate PWM signals required by the IR transmitter. However, I could still add a  timer interrupt on Timer 2, to implement asynchronous packet sending, and this is the approach I took for transmitting packets asynchronously.

I first enabled Timer Overflow interrupts for Counter 1 on Timer 2 and then added an Interrupt Service Routine listening to that interrupt:

// Enable Timer Overflow interrupts
TIMSK2 = _BV(TOIE1);
IR *instance;
ISR(TIMER2_OVF_vect)
{
  // Handle TX logic here
  instance->handleTx();
}

void IR::handleTx()
{
  if (!this->txEnabled) return;

  if (this->currentPacketBit == 255) {
    this->txEnabled = 0;
    this->mark();
    return;
  }
  
  // Initialize start bit
  if (this->irConfig.startPulseDuration > 0 && !this->startPulseSent) {
    this->startPulseSent = 1;
    this->currentPulseDelay = this->irConfig.startPulseDuration;
    this->mark();
    
    return;
  }
  
  // Current pulse delay hasn't expired; continue previous operations
  if (this->lastBitTime > micros() - this->currentPulseDelay) {
    return;
  }
  
  // Handle pulse gap
  if (!this->inPulseGap) {
    this->inPulseGap = 1;
    this->currentPulseDelay = this->irConfig.pulseGapDuration;
    this->space();
  
    return;
  }
  
  if (bitRead(this->packetBuffer, this->currentPacketBit - 1)) {
    this->currentPulseDelay = this->irConfig.longPulseDuration;
  } else {
    this->currentPulseDelay = this->irConfig.shortPulseDuration;
  }
  
  this->mark();
  this->inPulseGap = 0;
  --this->currentPacketBit;
}

The ISR would call into the IR transmission functionality and handle toggling the PWM pin on and off for each packet, with the correct timing. Note that this method relies on outside input (namely, something has to set the packet buffer and set the txEnabled flag to true before anything will happen). This logic is handled in yet another timer interrupt, this time on Timer 1. Timer 1 is configured to trigger every gap duration for the IR packet (in the Air Swimmer’s case, that’s once every 50ms). When the trigger occurs, it builds the packet from the current state information, based on the current velocity and left/right up/down input readings. If the device is set to fly straight, logic is in place to toggle between flapping left and right after a set interval. At the end of the ISR, the IR::tx() method is called with the generated packet, which sets everything up for transmission:

void IR::tx(uint32_t *packet)
{
  if (!this->txEnabled) {
    this->txEnabled = 1;
    this->packetBuffer = *packet;
    this->currentPacketBit = this->irConfig.packetBits;
    this->startPulseSent = 0;
    this->inPulseGap = 0;
  }
}

This functionality all works fine with some initial testing without using the Gyropter remote. However, once I incorporated the remote, the transmission logic became sporadic. This is most likely due to the fact that, since we’re now transmitting asynchronously, we are both sending and receiving at the same time, which is most likely resulting in conflicts between the packets being read. This is one of the problems that I outlined in my project description, with a potential solution involving disabling transmission while we’re receiving and vice-versa. This isn’t a feasible approach, in hindsight, as we could be receiving at any time, and continuously at that, so there would never be a good time to disable transmission or reception.

I do know that a solution is possible, as the Air Swimmers blimp itself is not hindered when the Gyropter remote and its own remote (or my device) are transmitting to it at the same time. My task now is to find the appropriate workaround to get this functionality in place.

My proposal describes Phase 3 as abstracting away the control scheme for the Air Swimmers blimp. As I finished implementing the initial transmission logic, I realized that in order to perform the abstraction, it would be easier to first reverse-engineer the Gyropter remote. That way, I would have a consistent way of providing input to the system.

With that in mind, I started reverse-engineering the controller the same way I did for the Air Swimmers remote, using my DSO Nano oscilloscope. I took a systematic approach for figuring out the structure of the packet, working one joystick at a time. The initial state has the left joystick positioned fully at the bottom and the right joystick set in the center.

Before showing the breakdown, we should first review the controls on the remote:

The joystick on the left controls velocity. Its initial state is at the bottom (indicating zero velocity). The left joystick only moves up and down. In the middle of the controller is the on/off switch and the channel select. There are three channels: A, B and C. The right joystick moves up and down, left and right. The initial state is in the middle. The Left and Right Trim buttons are used to compensate for drag in the original remote (for instance, if the helicopter is rotating to the right when the right joystick is in the center, you could compensate by pressing the Left Trim button a few times). At the top-right of the controller (not shown) is a button that toggles the helicopter’s lights on and off.

// First step: Change channels
// Bits 0-1 represent the channel
CH
00 0000000111001110000: Initial state
00 0000000111001110000: Channel A
10 0000000111001110000: Channel B
01 0000000111001110000: Channel C

// Second step: Toggle light on and off
// Bit 23 represents the light
                     L
00000000011100111000 0: Initial state
00000000100100111000 1: Light On

// Third step: Left joystick fully up
// Bits 2-7 represent the left joystick
   LFTJOY
00 000000 0111001110000: Initial state
00 111111 0111001110000: Left joystick fully up

// Fourth step: Right joystick fully left and fully right, with trim
// Bits 8-15 represent left/right on the right joystick
         LFT/RGHT
00000000 01110011 10000: Initial state
00000000 11010111 10000: Full-Left Trim  & Right Joystick Full-Left
00000000 00000011 10000: Full-Right Trim & Right Joystick Full-Right

// Fifth step: Right joystick fully up and fully down
// Bits 16-18 represent up/down on the right joystick
                 U/D
0000000001110011 100 00: Initial state
0000000001110011 111 00: Right Joystick Full-Down
0000000001110011 000 00: Right Joystick Full-Up

// Packet Breakdown:
CH LFTJOY LFT/RGHT U/D 0 L

After staring at the breakdown above, I came up with the following ranges of values for each part of the packet:

CH:
  - 00: Channel A
  - 01: Channel C
  - 10: Channel B
  - 11: Unused
LEFTJOY (Left Joystick):
  - Minimum: 0
  - Maximum: 63
LFT/RGHT (Right Joystick Left/Right):
  - Full Left: 215
  - Center: 115
  - Full Right: 0
U/D (Right Joystick Up/Down):
  - Full Up: 0
  - Center: 4
  - Full Down: 7
L (Light):
  - 0: Button Released
  - 1: Button Depressed

With the information outlined above, it became possible for me to construct a method for parsing the IR signal from the Gyropter remote.

The IR transmitter for the Air Swimmers shark operates at 38KHz. What this means is that, when the infrared LED is transmitting a command, it flickers on and off roughly 38,000 times per second. This allows the IR receiver to better distinguish the light coming from the LED from naturally-occurring light, and it also increases the lifespan of the IR LED, since this flickering effect reduces the overall power consumption.

While researching how to handle IR transmission on the Arduino, I came across an article by Ken Shirriff entitled Secrets of Arduino PWM. This article provides a pretty in-depth discussion on how the Arduino handles PWM. Not surprisingly, Shirriff also details an IR Remote implementation, which I’ve used as the basis for my implementation.

Configuring the IR Signal

The code below uses phase-correct PWM, and is taken with slight modification (primarily, inlining of a macro) from IRremote.cpp.

void IR::enableIROut(int khz) {
 // Enables IR output. The khz value controls the modulation frequency in kilohertz.
 // The IR output will be on pin 3 (OC2B).
 // This routine is designed for 36-40KHz; if you use it for other values, it's up to you
 // to make sure it gives reasonable results. (Watch out for overflow / underflow / rounding.)
 // TIMER2 is used in phase-correct PWM mode, with OCR2A controlling the frequency and OCR2B
 // controlling the duty cycle.
 // There is no prescaling, so the output frequency is 16MHz / (2 * OCR2A)
 // To turn the output on and off, we leave the PWM running, but connect and disconnect the output pin.

 // Enable the PWM pin for output
 pinMode(3, OUTPUT);
 // Initialize the PWM pin to HIGH, defaulting the LED to off
 // (the LED is turned on when the pin goes LOW).
 digitalWrite(3, HIGH);

 // Define the PWM frequency
 const uint8_t pwmval = SYSCLOCK / 2000 / khz;

 // WGM2 = 101: phase-correct PWM with OCRA as top
 // CS2 = 001: no prescaling
 TCCR2A = _BV(WGM20);
 TCCR2B = _BV(WGM22) | _BV(CS20);

 // The top value for the timer. The modulation frequency will be SYSCLOCK / 2 / OCR2A.
 OCR2A = pwmval;

 // Set a 33% duty cycle
 OCR2B = pwmval / 3;
}

What this does is set up Timer 2 to count up from 0 to pwmval and then back down to 0. Whenever the timer value is between 0 and pwmval / 3, OC2B is set HIGH. For values between pwmval / 3 and pwmval, OC2B is set to LOW. With a modulation frequency of 38KHz, and a 16MHz CPU, pwmval is approximately equal to 210 (16000000 / 2000 / 38). Thus, pwmval / 3 = 70. With this configuration, after the initial iteration (which is only HIGH for half the time, and thus, not complete), OC2B is HIGH for 140 clock ticks and LOW for 280 clock ticks. With the 16MHz CPU, and no prescaling performed on the timer, a single clock tick takes ~62.5 nanoseconds. Thus, one PWM period lasts (140 + 280) * 62.5 nanoseconds, or approximately 26.25 microseconds. This results in a frequency of (1 / 0.00002625)Hz, or 38.095KHz, giving us our desired frequency.

Shirriff paints an easier-to-understand picture with the following image:

OCRnA represents pwmval and OCRnB represents pwmval / 3. OC2B is shown as OCnB at the bottom of the graph. For timer values below the orange OCRnB line, OCnB is HIGH. For values between the orange OCRnB line and the green OCRnA line, OCnB is LOW.

Transmitting an IR command

With the code outlined above, Timer 2 is configured to transmit the requisite 38KHz carrier wave required by the Air Swimmers IR receiver. However, we still need to incorporate logic for transmitting a series of 1′s and 0′s in the appropriate packet format for the Air Swimmers device. In Phase 1, I gathered the timing information for the IR packets transmitted by the Air Swimmers remote. Of note are the following:

• Gap Duration = 50000us
• Pulse Gap Duration = 340us
• Short Pulse Duration = 220us
• Long Pulse Duration = 720us

These define the four components of the IR packet shown below:

In this example, the Pulse Gap Duration refers to the length of time spent in the LOW setting between each bit transmitted, the Short Pulse Duration represents the length of the shorter pulse (represented as a 0 in the packet structure), and the Long Pulse Duration represents the length of the longer pulse (represented as a 1 in the packet structure). The Gap Duration is actually not shown, but refers to the minimum time to wait between sending two packets.

Given the above specifications, we can put together a method for transmitting an IR packet (here represented as a 32-bit integer):

void transmitIRPacket(uint32_t packet) {
  uint8_t packetLength = 24;
  uint16_t gapDuration = 50000ul;
  uint16_t pulseGapDuration = 340ul;
  uint16_t shortPulseDuration = 220ul;
  uint16_t longPulseDuration = 720ul;

  for(uint8_t i=packetLength;i>0;i--) {
    TCCR2A &= ~_BV(COM2B1); // Enables PWM on Pin 3
    if (bitRead(packet, i-1)) {
      delayMicroseconds(longPulseDuration);
    } else {
      delayMicroseconds(shortPulseDuration);
    }
    TCCR2A |= BV(COM2B1); // Disables PWM on Pin 3
    delayMicroseconds(pulseGapDuration);
  }
  delayMicroseconds(gapDuration);
}

Assuming the IR LED’s data pin is attached to Pin 3, this function should transmit an IR packet that the Air Swimmers IR receiver can interpret.

In Part 1, I broke down the packet structure of the Air Swimmers IR remote. This post will focus on parsing the IR commands with the Arduino.

The approach is rather straightforward. Arduino provides a pulseIn command, which returns the number of milliseconds a pulse lasts in a pulse-width-modulated signal. Using this function in a loop, one can read in all bytes from a particular IR signal. Following is the IR Receive command I utilized. Note the use of readPulse() instead of pulseIn. More on that later:

/**
 * Read an IR packet from an IR receiver
 *
 * packet: Pointer to integer which represent the packet
 * timeout: Program will return \0 if this timeout is reached.
 *          This prevents blocking, if no input is received
 *
 * Returns 1 on successful parsing of packet, 0 on failure
 */
uint8_t IR::rx(uint32_t *packet, uint32_t timeout)
{
  // Take initial timestamp
  uint32_t startTimeMillis = millis();
  // Bit mask applied to parsed packet, to 
  // ensure only the lower `packetBit` bits are read
  uint32_t packetMask = (1UL << this->irConfig.packetBits) - 1;

  // Number of bits read in the current iteration
  uint8_t currentPacketBits = 0;

  // Zero out the packet.
  *packet = 0;
  
  while(
          // Some IR packets have a specific checksum that can be
          // applied to verify the validity of the packet.
          // Subclasses can choose whether or not this 
          // checksum is implemented.
	  (!this->irConfig.hasChecksum || !this->checksum(packet))
          // Each IR packet typically is a fixed length. This check 
          // ensures that we have not yet reached that length.
	  && (this->irConfig.packetBits == 0 
              || this->irConfig.packetBits != currentPacketBits) 
          // Break out of the loop if the timeout has been reached
	  && (timeout == 0 || millis() - startTimeMillis < timeout)) {
    // Read in a pulse from the RX pin. `pulseInType` is one 
    // of HIGH or LOW, depending on the subclass's implementation.
    // The final parameter, `longPulseDuration`, determines the 
    // timeout for reading the pulse. We return after failing to
    // read a pulse after twice the length of the long pulse.
    uint32_t pulse = readPulse(
                       this->rxPin, 
                       this->irConfig.pulseInType, 
                       2 * this->irConfig.longPulseDuration
                     );

    // If a start pulse is specified, and the current pulse 
    // falls within the start pulse's parameters, we reset our packet
    // and bit count, as this means we've started a new packet.
    if (this->irConfig.startPulseDuration > 0 
      && pulse > this->irConfig.startPulseDuration - this->irConfig.pulseTolerance
      && pulse < this->irConfig.startPulseDuration + this->irConfig.pulseTolerance) {
      *packet = 0;
      currentPacketBits = 0;
    } else 
    // If the pulse falls within the parameters for the short pulse, 
    // we set the current bit value to '0' and increment the packet count
    if (pulse > this->irConfig.shortPulseDuration - this->irConfig.pulseTolerance 
      && pulse < this->irConfig.shortPulseDuration + this->irConfig.pulseTolerance) {
      *packet = (*packet << 1) & packetMask;
      currentPacketBits++;
    } else
    // If the pulse falls within the parameters for the long pulse, we 
    // set the current bit value to '1' and increment the packet count
    if (pulse > this->irConfig.longPulseDuration - this->irConfig.pulseTolerance
      && pulse < this->irConfig.longPulseDuration + this->irConfig.pulseTolerance) {
      *packet = ((*packet << 1) | 1) & packetMask;
      currentPacketBits++;
    }
  }
  
  // We perform one final check before leaving the loop,  
  // to verify the validity of the packet.
  if ((this->irConfig.hasChecksum && this->checksum(packet))
      || (!this->irConfig.hasChecksum && currentPacketBits 
           == this->irConfig.packetBits)) {
    return 1;
  }

  // A valid packet was not found; return false
  return 0;
}

The readPulse command is a rewrite of Arduino’s pulseIn command. When I was playing around with pulseIn, I found that I had to adjust the timing values I read from my oscilloscope so that they would work with the Arduino. At first, I thought it had to do with the crystal on the Arduino itself being configured improperly, but then I found out that it was a bug with the Arduino source. See this post by DaveMAA for details. The post also provides a replacement method for pulseIn which I’ve added to my codebase:

uint32_t IR::readPulse(
    uint16_t pin, 
    uint16_t signal, 
    uint32_t timeout
){
  uint32_t current, killTime, ptime;
  current = micros();
  killTime = current + timeout;
  
  while (digitalRead(pin) == signal) {
    delayMicroseconds(4); 
    current = micros();
    if(current >= killTime){
      return 0UL;
    }
  }
  
  while(digitalRead(pin) != signal){
    delayMicroseconds(4);
    current = micros(); 
    if(current >= killTime){
      return 0UL;
    }
  }
  
  ptime = micros();
  while (digitalRead(pin) == signal) {
    delayMicroseconds(4); 
  }
  return micros() - ptime;
}

I began working on my Master’s project this past week, and made significant headway in my first phase of reverse-engineering the Air Swimmers remote (by “significant headway”, I mean I successfully reverse-engineered it. What follows is

Arduino Uno Infrared Receiver Setup

The first step in this process is setting up an infrared receiver. I utilized the Arduino Uno, along with a basic IR receiver from SparkFun. To simplify my setup, I spent a little extra money on the breakout board, so all I needed to do was plug it into my breadboard and hook it up to the Uno. In the image above, you’ll also see my IR Transmitter setup which also utilizes a breakout board.

After hooking up the receiver, you want to make sure that you can actually receive messages. There’s an LED on the Uno that’s controlled by Pin 13. By hooking up the data line from the IR receiver (blue wire in the photo above) to Pin 13, and turning on the power (ensuring the IR receiver is also hooked to ground and power), you can then determine if a signal is being sent by holding an IR remote up to the receiver and pressing any button. The LED will flash rapidly (rather faintly). This ensures that the receiver is hooked up properly.

DSO Nano v2

The next step is to determine the actual message being sent. I utilized a pocket oscilloscope called the DSO Nano, a relatively cheap oscilloscope that provides the necessary functionality. I hooked up its two inputs, one to ground and the other to the data pin, pressed the Tail fin Right button on the controller and got the following output:

Right Button, Reading 1

Right Button, Reading 2

The above graphs show the voltage over time. Zero is at the bottom of the graph (the low points) and ~3.3v (the input voltage) is marked by the high points. As can be seen, there are two distinct waveforms. Infrared is simply a way of encoding binary data in what’s known as pulse-width modulation. The width of the pulse determines the bit being sent. For my purposes, I’ve deemed the longer pulses high bits, or 1′s and the shorter pulses low bits, or 0′s. From the graphs above, we can determine that the message being sent is:

011010101011110100011011

This message is repeated with a slight pause the entire time the button is held down. Doing the same for the other three buttons results in similar messages:

    7654 3210 FEDC BA98 7654 3210
U - 0110 1010 1011 1101 1000 0010
D - 0110 1010 1011 1101 0100 1110
L - 0110 1010 1011 1101 0010 1000
R - 0110 1010 1011 1101 0001 1011

Just as you can more easily break the Caesar cipher when you’re given more data, with all four message, we can easily determine what each bit represents:

1. The high 16 bits of the message are all the same. I’ve referred to these as the signature for the message, as they help determine that you’re receiving a message from this specific remote.
2. The next four bits correspond directly to the button being pressed, with bit 7 high for ‘Up’, 6 for ‘Down’, 5 for ‘Left’ and 4 for ‘Right’.
3. Bits 0-3 appear random at first, but there’s typically a reason for each bit being transmitted. After some analysis, I found out that XORing bits 4-7 with bits 0-3 in all messages provided the same result (1010). Thus, I determined that bits 0-3 were the checksum bits, which help ensure you did not receive bogus data.

When a user presses, say, Down and Left at the same time, bits 6 and 5 are set high and bits 0-3 are set to 1100 so the checksum is valid. Overall, what this command structure lacks in robustness, it makes up for in simplicity.

Figuring out the messages sent is great, but ensuring that the Arduino can read the commands takes some actual programming. In my next update, I’ll walk through this process and provide some (surprisingly simple) code for parsing the commands.