The standard way of connecting a unipolar stepper motor to the parallel port is to use a Darlington driver such as ULN2003 and there are already many examples out there on how to do this. In this post, I will show you how to build a simple stepper motor driver using discrete MOSFETs.

In the circuit diagram below, you will find that the four power MOSFETs are used as switches for each coil in the stepper motor (the stepper motor I used in this example is a MITSUMI M35SP-9. In theory, any uni-polar stepper motors should work with this circuit). A pull-down resistor is attached to the gate of each MOSFET. This is important as otherwise the interference from the port would prevent the MOSFET from switching reliably.

Controller Circuit

The benefit of using discrete MOSFET is that they can handle extremely high current loads. Using IRFZ22, the coil current can be as high as 10 Amp.

Stepper Motor Controller

The following C code sets the data port (pin 2, 3, 4, 5) to high in order so that the stepper motor would rotate clockwise. If you want the motor to rotate counter-clockwise, simply change the output order to 8,4,2,1.

#include <sys/io.h>

#define PAR_PORT 0x378 

void main()
{
	while (1)
	{
		outb(1, PAR_PORT);
		outb(2, PAR_PORT);
		outb(4, PAR_PORT);
		outb(8, PAR_PORT);
	}
}                                            

#include <stdio.h>
#include <stdlib.h>
#include <limits.h>
#include <math.h>
#include <iostream>

using namespace std;

bool IsPrime(unsigned long long n) {
	bool r = true;

	for (unsigned long long i = 3; i < sqrt((double) n) + 1; i+= 2)
	{
		if (n % i ==0) {
			r = false;
			break;
		}
	}

	return r;
}

bool IsPalindrome(unsigned long long n) {
	bool r = true;
	char s[30];
	int l = sprintf(s, "%llu", n);

	if (l == 1 && n != 1) {
		r = true;
	} else	{
		for (int i = 0; i < l/2; i++) {
			if (s[i] != s[l-i-1]) {
				r = false;
				break;
			}
		}
	}

	return r;
}

/*
 * usage: palprime [lbound] [ubound]
 */
int main(int argc, char** argv) {
	unsigned long long beginNum = 3;
	unsigned long long endNum = 3;

	if (argc == 2) { // lbound default to 3
#ifdef _WIN32
		endNum = _strtoui64(argv[1], NULL, 10);
#else
		endNum = strtoull(argv[1], NULL, 10);
#endif

	} else if (argc == 3) {
#ifdef _WIN32
		beginNum = _strtoui64(argv[1], NULL, 10);
		endNum = _strtoui64(argv[2], NULL, 10);
#else
		beginNum = strtoull(argv[1], NULL, 10);
		endNum = strtoull(argv[2], NULL, 10);
#endif
	}

        unsigned long long i = beginNum;

        while (i < endNum) {
                char s[30];
                int l = sprintf(s, "%llu", i);

		//length cannot be even as even length palindrome numbers
		//can be divided by 11.
                if (l % 2 == 0) {
                    i = ((unsigned long long) (i / 10)) * 100 + 1;
                    continue;
                }

		if (IsPalindrome(i)) {
			if (IsPrime(i)) {
				cout << i << endl;
			}
		}

                i+=2;

                if (s[0] % 2 == 0) {
                    i+=pow(10, l-1);

		    //leading/ending number cannot be 5
                    if (((int) (s[0] - '0')) + 1 == 5) {
                        i += 2 * pow(10, l-1);
                    }
                }
	}
	return (EXIT_SUCCESS);
}

At first, I was trying to find all the palprimes that can be represented by 64 bit integers. But soon I realized that it would take months to do so using the code above with a quad-core PC (using 4 processes with different ranges). Anyway, here’s the last few palindromic primes less than 10,000,000,000,000:

9999899989999
9999901099999
9999907099999
9999913199999
9999919199999
9999938399999
9999961699999
9999970799999
9999980899999
9999987899999

And here are a few interesting ones:

11357975311
1112345432111
1300000000031
1700000000071
1900000000091
7900000000097
9200000000029
1357900097531

Results

When presented with images that are clear, the algorithm correctly identified most of them (see images below):

Building (Microsoft Research Digital Image)

Street (Microsoft Research Digital Image)

Sky

Flower

The following images illustrate how the original image (top right) is divided into sub regions. Canny detection is performed on each of the sub images.

When performing Hough Transform, I chose to detect up to ten lines in each image, with the following stepping parameter (the detection results are very sensitive to these parameters, the following parameters were chosen based on experiment results):
\[\rho=1, \theta=0.01\]

Out of all the detected lines, a few sections are selected based on line continuity and the calculated average gradients around the detected lines. The following images shows the chosen Hough line segments based on the algorithm.

The table below shows the gradient index calculated within each area (the average of the gradients along all the chosen line segments within a sub-image. The result is scaled by 1000, the scale factor is chosen such that for clear images the resulted index is greater than 1 and for blur images the resulted index is less than 1).

And the sub images are indexed as follows:

The algorithm can also detect images with deliberate blur regions (e.g. Bokeh). The results are illustrated below:

Plant (Microsoft Research Digital Image)

Note that the 0’s in the detection results indicate that within those regions, no lines could be reliably detected and thus those regions are considered blurred.
Generally speaking, when an image contains both blurred and clear regions, some of the indexes will be zero and others will be greater than one.

The following images are detected as blurred, with the detected indexes far less than one.

Boat (Microsoft Research Digital Image)

Candle

Limitations

when image contrast is low, or when objects borders are not clearly defined, the algorithm may have difficulty in distinguishing whether an image is blurred. Take the following two cloud images for instance, the image on the left was correctly classified as a clear image due to the relatively high contrast around the center. But the image to the right was classified as blurred due to its lack of contrast.

Cloud

Cloud

Gradient Calculation

If you remember how the Hough parameters were determined (in polar form, see figure below), it is not difficult to obtain the pixel coordinates centered around the points on the detected line.

Region selection for gradient calculation

In fact, we can formulate a line that is perpendicular to the line section detected during Hough transform, and use the line section that is within a predefined region (e.g. the between the dotted lines) to calculate the gradients of luminance. The following code snippet shows how the perpendicular line’s parameters are obtained.

    /**
     * Get the equation parameters for the line that passes through (x,y) and is perpendicular
     * to the line specified by parameters (p0, theta0) in normal form.
     *
     * @param p0 : distance to line from origin.
     * @param theta0 : the slope of p0.
     * @param x : x coordinate of the point where the perpendicular line passes through
     * @param y : y coordinate of the point where the perpendicular line passes through
     * @param &p : the perpendicular line's distance from origin.
     * @param &theta : the slope of p.
     **/
    void LineUtils::GetPerpendicularLineParameters(float p0, float theta0, float x, float y, float &p, float &theta) {
        float x0 = p0 * cos(theta0);
        float y0 = p0 * sin(theta0);

        p = sqrt((x0 - x)*(x0 - x) + (y0 - y)*(y0 - y));

        float a1 = theta0 - PI / 2.0;
        float a2 = theta0 + PI / 2.0;

        //d=|x0 * cos a + y0 * sin a - p|
        float d1 = abs(x * cos(a1) + y * sin(a1) - p);
        float d2 = abs(x * cos(a2) + y * sin(a2) - p);

        if (d1 < d2)
            theta = a1;
        else
            theta = a2;
    }

In an ideal situation, the gradient of the line points obtained via the method above looks like the figures below, where the edges are clearly identified.

Edge (dark to bright)

Edge (dark to bright to dark)

Sometimes, when the lighting condition is poor, the image would appear to be “grainy”, which sometimes led to poor line detection. For instance, the following figure shows the the gradient when the region around the detected edge is grainy:

None-Edge

We use the number of times the luminance increases above or decreases below its mean along the perpendicular line interval as a measure of whether we accept the detection results as gradients or not. Typically when such number of crossings is less than 3 (see the first two images above) the curve is either monotonic or has a single peak, we assume that gradients can be correctly calculated and if the crossings are more than 3 we discard the results. The gradient is calculated as the slope of the curve. In the examples above, we used ten pixels on each side of the Hough line to calculate gradients.

Image Regions

For images with complex contents, it becomes difficult for the Hough transform to reliably identify line structures within the image. Future more, certain photography techniques (i.e. Bokeh) leave portions of images deliberately blurred. Without dividing image into different sub-regions, the classification results would be compromised.

Thus, images are divided into 9 (3×3) sub-images after Canny edge detection, and Hough transform is performed against each sub-images. The figure below illustrates how an image is divided:

Image divided into 3x3 sub-images (Microsoft Research Digital Image)

This technique is especially useful when portions of images are deliberately blurred, like the image shown above. It also helps the line detection accuracy when the image contains complex scenes. By dividing up the image, each sub area’s complexity is greatly reduced. Other methods in scene separation might achieve even better results, but it is out of the scope for our discussion here.

In my next post, I will show some results obtained from using the method mentioned in this and the previous articles and will also discuss its limitations.

Experience tells us that blur images tend to contain less details then their sharper counterparts. And the areas where intensity transitions occur (e.g. the border of an object) are more well defined in clear images. Mathematically speaking, the slope of the intensity transition is statistically deeper in clear images than blur ones. Since whether an image is blurred or not is not affected by color space, it is sufficient to perform detection in the luminance space (i.e. gray scale images):

\[I=0.299R\times0.587G\times0.114B\]

Canny Edge Detection

Thus the very first step in deciding whether an image or an area within an image is blurred is to use some sort of edge detection algorithms to obtain a collection of the edges in the image.

Canny edge detection is a good candidate since it is optimal in terms of good detection and localization. And hysteresis is used to reduce streaking and thus Canny edge detection achieves relatively continuous edge boundaries comparing to other edge detection methods.

In one of my previous posts, I discussed Canny edge detection using auto thresholding utilizing Intel’s Integrated Performance Primitives (IPP). I used the same algorithm here for the image pre-processing process.

Hough Transform

In order to analyze the gradients along detected edges, it is necessary to first parameterize them. Hough transform comes in handy for this task.

While Hough transform is capable of identifying arbitrary shapes, for the purpose of detecting image blurs simple line detection is more robust. Besides, IPP has an implementation for line detection using Hough transform out of the box.

Since our goal is to determine the quality of the image within a region of interest, we do not need to analyze all the edges identified within that region. But rather, we could select a few based on some pre-determined criteria that would maximize our ability to correctly determine the luminance gradients.

The following code snippet shows my implementation of the edge parameterization method using IPP:

    bool CompareIppiPoint(IppiPoint &p1, IppiPoint &p2) {
        if (p1.x == p2.x) {
            return p1.y < p2.y;
        } else {
            return p1.x < p2.x;
        }
    }

    void IPPGrayImage::HoughLine(IppPointPolar delta, int threshold, int maxLineCount, int* pLineCount, IppPointPolar pLines[], list<IppiPoint> pList[]) {
        IppStatus sts;
        IppiSize roiSize = {_width, _height};

        int bufSize;
        sts = ippiHoughLineGetSize_8u_C1R(roiSize, delta, maxLineCount, &bufSize);
        assert(sts == ippStsNoErr);
        Ipp8u * pBuf = ippsMalloc_8u(bufSize);

        Ipp8u *imgBuf;
        int stepSize;
        imgBuf = ippiMalloc_8u_C1(_width, _height, &stepSize);

        sts = ippiConvert_32f8u_C1R(_imgBuffer, _width * PIXEL_SIZE, imgBuf, _width, roiSize, ippRndNear);
        assert(sts == ippStsNoErr);
        assert(imgBuf != NULL);

        sts = ippiHoughLine_8u32f_C1R(imgBuf, _width, roiSize, delta, threshold, pLines, maxLineCount, pLineCount, pBuf);
        assert(sts == ippStsNoErr);

        //d=|x0 * cos a + y0 * sin a - p|
        int val = 0;
        float d;
        for (int x = 0; x < _width; x++) {
            for (int y = 0; y < _height; y++) {
                if (imgBuf[x + y * _width] > 0) {
                    val = imgBuf[x + y * _width];

                    for (int i = 0; i < *pLineCount; i++) {
                        d = abs((float) cos(pLines[i].theta) * (float) x + (float) sin(pLines[i].theta) * (float) y - abs(pLines[i].rho));
                        if (d < threshold) {
                            IppiPoint p;
                            p.x = x;
                            p.y = y;

                            pList[i].push_back(p);
                        }
                    }
                }
            }
        }

        for (int i = 0; i < *pLineCount; i++) {
            pList[i].sort(CompareIppiPoint);
        }

        ippsFree(pBuf);
        ippiFree(imgBuf);
    }

Note that the selection for delta is crucial for the line detection quality. In the code above, up to maxLineCount number of lines are detected and each line’s parameters are stored in pLines[]. The edge image pixel coordinates within a given range that is less than threshold are collected into the list. Because the Hough detection does not indicate the begin and the end point of a line, we need to iterate through the pixels in pList[] and find the sections that the detected lines pass through. The image below illustrates how the edge regions are chosen based on detected line parameters. Image features within a region between the dotted line are recorded based on the distances to the line defined by:
\[d=\vert x_0cos\alpha+y_0sin\alpha-p\vert\]

Region selection for gradient calculation

In practice, for each parameterized line I chose to select the longest section where the line approximates a section of the edge. Other line sections can be used as well, but the longest portion tends to offer better classification results. The code is shown below, and the thickened section in the figure above illustrates the selected section.

    void LineUtils::GetLongestConnectedLinePoints(IppiPoint points[], int len, int threshold, IppiPoint lpoints[], int &maxLen) {
        int maxIndexEnd = 0, curRunLength = 0, maxRunLength = 0;

        lpoints[curRunLength] = points[0];

        for (int i = 1; i < len; i++) {
            if ((abs(points[i].x - lpoints[curRunLength].x) > threshold) || (abs(points[i].y - lpoints[curRunLength].y) > threshold)) {
                if (curRunLength > maxRunLength) {
                    maxRunLength = curRunLength;
                    maxIndexEnd = i;
                }
                curRunLength = 0;
                lpoints[curRunLength] = points[i];
            } else {
                if (curRunLength >= maxLen) {
                    maxRunLength = maxLen;
                    break;
                }

                curRunLength++;
                lpoints[curRunLength] = points[i];
            }
        }

        for (int i = maxIndexEnd - maxRunLength - 1; i < maxIndexEnd; i++) {
            if (i - (maxIndexEnd - maxRunLength) - 1 < maxRunLength) {
                lpoints[i - (maxIndexEnd - maxRunLength - 1)] = points[i];
            }
        }

        maxLen = maxRunLength;
    }

In my next post, I will continue with the gradient calculation and some methods to enhance the detection accuracies.

Background

Image blur is typically caused by

• Motion (e.g. the relative movement between the camera and the object during exposure)
• Out-of-focus
• Low lighting condition

Other types of image blur may also occur in photography (such as Bokeh). And it is desirable to be able to automatically distinguish desired image blur (e.g. Bokeh caused by camera lens with a shallow depth of field) from un-intended blur (e.g. out of focus).

When reference images (e.g. a series of images with different focus settings) are present, detecting image blur is a relatively simple task. For instance, in passive auto-focus cameras, a series of scene images are captured with progressive focus settings. And the intensity differences are calculated within the same region across these different images. Thus, the image with the highest intensity difference corresponds to the correct focus setting. In Active-focus cameras, image blur detection problem is circumvented by measuring the distance between the lens and the object.

The problem becomes more difficult if we are only given a single image. Since many parameters that we used above in camera auto-focusing are not present in a single image setting, we could not reliably infer image sharpness by the method used in passive auto-focusing as we do not have sufficient knowledge to re-construct the series of images necessary for comparison.

We can however, tell whether an image is in-focus by calculating the intensity differences along the edges in an image. If the calculated intensity is higher than a predefined threshold, we deduce that the image is sharp. And if the calculated intensity is lower than a predefined threshold, we conclude that the image is blurred. So now we shifted the problem to identifying such optimal regions where the intensities are calculated.

Algorithm Overview

In this algorithm, we first use Canny edge detection to obtain the edges in the image. The edges are then parameterized using Hough transform. We then calculate the pixel gradients along the parameterized lines detected and finally we use the gradients to decide whether the image is blurred.

I will discuss some technical and implementation details in the up-coming posts. Stay tuned.

For those who are impatient, the function to perform recursive directory search is here:

#include <sys/types.h>
#include <sys/stat.h>
#include <dirent.h>
#include <errno.h>
#include <vector>
#include <string>
#include <iostream>
#include <boost/regex.hpp>

void GetFileListing(vector<string> &files, string dir, string filter, bool ignoreCase) {
    DIR *d;
    if ((d = opendir(dir.c_str())) == NULL) return;
    if (dir.at(dir.length() - 1) != '/') dir += "/";

    struct dirent *dent;
    struct stat st;
    boost::regex exp;

    if (ignoreCase)
        exp.set_expression(filter, boost::regex_constants::icase);
    else
        exp.set_expression(filter);

    while ((dent = readdir(d)) != NULL) {
        string path = dir;

        if (string(dent->d_name) != "." && string(dent->d_name) != "..") {
            path += string(dent->d_name);
            const char *p = path.c_str();
            lstat(p, &st);

            if (S_ISDIR(st.st_mode)) {
                GetFiles(files, (path + string("/")).c_str(), filter, ignoreCase);
            } else {
                if (filter == ".*") {
                    files.push_back(path);
                } else {
                    if (boost::regex_match(string(dent->d_name), exp)) files.push_back(path);
                }
            }
        }
    }

    closedir(d);
}

I used boost library to perform regular expression matches for file names. If you just want to obtain a listing of all the files, you can do without using the boost library.

The following code snippet demonstrates how to use the function. The results are stored in a vector container passed in. Note that the “filter” parameter needs standard regular expressions (so if you are looking for any files, the expression should be .* instead of just *) to work properly.

The code should be pretty self-explanatory. If ignoreCase is set to true, then the match will be case-insensitive.

vector<string> files;

FileUtils::GetFiles(files,"/tmp", ".*", true);
for (int i = 0 ; i < files.size(); i++) {
    cout << files[i] << endl;
}

Here I will examine a few commonly used method in measuring time intervals under Linux. All of the following timing routines are timed against the same OpenMP parallel for loop (on a quad-core CPU, the parallel for will spawn four concurrent threads).

time()

The time() function returns time with the accuracy to a second. So this function is generally useful for measuring long-running processes.

clock()

In single-core systems, clock() is often used for time measurements. The resolution of this timer is determined by CLOCKS_PER_SEC and is usually a microsecond. Since it determines the number of CPU clock cycles elapsed, it is not particularly useful in measuring time on a multi-core processor system when there are concurrent executing threads as the result of clock() function is the accumulation of CPU clocks across all active CPUs. On a quad-core system, if all cores are at full utilization then the result time is roughly four times the wall time.

gettimeofday()

Similar to the clock() function, gettimeofday() has a resolution up to one microsecond. As the function name suggests, gettimeofday() measures the wall time and thus is suitable for time measurement in multi-core, multi-cpu systems.

rdtsc

The time stamp counter is available on most modern CPUs (since Pentium). There are many implementations based on rdtsc (e.g. on Windows systems, the Win32 API call QueryPerformanceCounter). Implementation based on rdtsc is generally very accurate (with resolution up to one nanosecond) but depending on implementation, its accuracy might be susceptible to CPU clock throttling (common in mobile CPUs). In my implementation below, the rdtsc results are divided by the CPU frequency.

grep “cpu MHz” /proc/cpuinfo | cut -d’:’ -f2

This implementation assumes that CPU frequency remains constant during operations, which could lead to poor accuracy should the CPU frequency change during the measurement. For desktop CPU though, this is less of a concern however.

clock_gettime

Like rdtsc, this function has a nanosecond accuracy and is available on all POSIX compliant systems.

Intel Threading Building Block also provides a timer function tick_count::now() and the time can easily measured using the code snippet below:

    t_start = tick_count::now();
    //statements
    t_end = tick_count::now();
    cout << (t_end - t_start).seconds() * 1000 << " ms" << endl;

The following lists the code for time measuring using the methods mentioned above.

#include <iostream>
#include <stdio.h>
#include <stdlib.h>
#include <sys/time.h>
#include <time.h>
#include <ctime>

using namespace std;

void Foo() {
#pragma omp parallel
    {
        for (long i = 0; i < 50000; i++)
            for (long j = 0; j < 50000; j++);
    }

}

unsigned long long rdtsc() {
    unsigned a, d;

    __asm__ volatile("rdtsc" : "=a" (a), "=d" (d));

    return ((unsigned long long) a) | (((unsigned long long) d) << 32);
}

void Time() {
    time_t t1, t2;

    time(&t1);
    Foo();
    time(&t2);

    cout << "time() : " << t2 - t1 << " s" << endl;
}

void Clock() {
    clock_t c1 = clock();
    Foo();
    clock_t c2 = clock();
    cout << "clock() : " << (float) (c2 - c1) / (float) CLOCKS_PER_SEC << " s" << endl;
}

void GetTimeOfDay() {
    timeval t1, t2, t;
    gettimeofday(&t1, NULL);
    Foo();
    gettimeofday(&t2, NULL);
    timersub(&t2, &t1, &t);

    cout << "gettimeofday() : " << t.tv_sec + t.tv_usec / 1000000.0 << " s" << endl;
}

void RDTSC() {
    unsigned long long t1, t2;

    t1 = rdtsc();
    Foo();
    t2 = rdtsc();

    cout << "rdtsc() : " << 1.0 * (t2 - t1) / 3199987.0 / 1000.0 << " s" << endl;
}

void ClockGettime() {
    timespec res, t1, t2;
    clock_getres(CLOCK_REALTIME, &res);

    clock_gettime(CLOCK_REALTIME, &t1);
    Foo();
    clock_gettime(CLOCK_REALTIME, &t2);

    cout << "clock_gettime() : "
         << (t2.tv_sec - t1.tv_sec)  + (float) (t2.tv_nsec - t1.tv_nsec) / 1000000000.0
         << " s" << endl;
}

int main() {
    cout.setf(ios::fixed);
    cout.setf(ios::showpoint);
    cout.precision(5);

    Time();
    Clock();
    GetTimeOfDay();

    RDTSC();
    ClockGettime();

    return (EXIT_SUCCESS);
}

And here is the output from my quad-core computer in debug mode.

time() : 6 s
clock() : 21.85000 s
gettimeofday() : 5.67368 s
rdtsc() : 5.65957 s
clock_gettime() : 5.65894 s

After a quick search on ImageMagick’s site I realized that I was missing the following libraries:

libpng12
libjpeg62

Note, that you might want to use the following commands to find out the exact package names:

sudo apt-cache search libpng
sudo apt-cache search libjpeg

sudo apt-cache search libjpeg
And then use the following commands to install the packages:

sudo apt-get install libpng12-0 libpng12-dev
sudo apt-get install libjpeg62 libjpeg62-dev

Note that after that, you will need to re-configure and re-build the Magick++ library (e.g. ./configure, ./make ./make install) so that Magick++ library can be linked correctly with the above libraries.

The simplest way is to use the mean value of the gray scale image pixel values. As a rule of thumb, we set the low threshold to 0.66*[mean value] and set the high threshold to 1.33*[mean value]. Another way is to use the median color in the gray scale image and uses 0.66*[median value] and 1.33*[median value] accordingly. For typical images, these two methods achieve comparable results.

For example, the following shows the picture of a building along with its histogram (original image from Microsoft Research Digital Image. Please see Microsoft Research Digital Image License Agreement for more information):

Building

Building Histogram

The following shows the edge detection results using Canny algorithm (left image uses mean value auto-thresholding, right image uses median value auto-thresholding) and the results exhibit very little visible differences.

Building (Canny Mean)

Building (Canny Median)

However, for images has non-equalized histogram (see the picture of cloud and its histogram below):

Cloud

Cloud Histogram

Canny Edge detection result based on mean value auto-thresholding is pretty poor (see image on the left below), while edge detection based on median value auto-thresholding achieved much better result (see image on the right below)

Cloud (Canny Mean)

Cloud (Canny Median)

Alternatively, we could have performed histogram equalization on the image first before applying Canny edge detection with mean auto-thresholding:

Cloud Equalized

Cloud Equalized Histogram

And after image equalization, both mean and median value auto-thresholding achieved similar results.

Cloud Equalized (Canny Mean)

Cloud Equalized (Canny Median)

So the Canny edge detection using median value auto-thresholding seems to adapt to different types of images very well (note selecting the median value selection can be thought as equalizing the histogram, except that the pixel values are not changed during such operation).

The following is the code listing for the histogram calculation using IPP (based on the image class I created earlier)

/** @brief Get the min max mean value statistics for the current image
 *   @param min, max, mean: these are output parameters that are passed
 *         back by reference.
 */
void IPPGrayImage::MinMaxMean(float& min, float& max, double& mean)
{
    IppStatus sts;
    IppiSize origImgSize = {_width, _height};

    sts = ippiMinMax_32f_C1R(_imgBuffer, _width * PIXEL_SIZE, origImgSize, &min, &max);
    assert(sts == ippStsNoErr);
    sts = ippiMean_32f_C1R(_imgBuffer, _width * PIXEL_SIZE, origImgSize, &mean, ippAlgHintFast);
    assert(sts == ippStsNoErr);
}

/** @brief Calculate the histogram of the image
 *   @param nLevel: the number of bins in the histogram
 *        levels: this is the optional levels user can pass in (histogram will be then
 *                calculated with these levels instead of the uniform levels by default.
 *   @return an integer array which contains the histogram.
 *   @note the histogram is by default calculated using uniform bins across the color range.
 */
unsigned int * IPPGrayImage::GetHistogram(unsigned int nLevel, float levels[])
{
    IppStatus sts;
    Ipp32f* l = new Ipp32f[nLevel];
    IppiSize origImgSize = {_width, _height};

    unsigned int* bins = new unsigned int[nLevel - 1];

    float minVal = 0, maxVal = 0, stepVal = 0;
    double meanVal = 0;
    if (levels != NULL)
    {
        for (unsigned int i = 0; i < nLevel; i++)
        {
            l[i] = levels[i];
        }
    }
    else
    {
        MinMaxMean(minVal, maxVal, meanVal);
        stepVal = (maxVal - minVal) / (float) nLevel;

        for (unsigned int i = 0; i < nLevel; i++)
        {
            l[i] = minVal + stepVal * i;
        }
    }

    sts = ippiHistogramRange_32f_C1R(_imgBuffer, _width * PIXEL_SIZE, origImgSize, (Ipp32s*) bins, (Ipp32f*) l, nLevel);
    return bins;
}

In my opinion, the above three IDEs all have their strengths and weaknesses depending on what kind of project you are working on. Here are some of my observations.

KDevelop is probably the most comprehensive IDE of the three. This should not come as a surprise as it has been around for more than 10 years and it bas been the de facto development environment for most of the KDE development work. It is very feature rich and you can use it to develop many different types of applications natively out of box (e.g. KDE, GTK+, Qt, wxWidgets, etc). If you are developing GNU style applications, you will benefit from the Automake project type if provides. Besides the vast functionalities KDevelop provides, it is also very fast, efficient and stable.

Nonetheless, KDevelop is not the most intuitive IDE and does not suit small projects (e.g. prof of concept code) well as the initial project setup can be time consuming.

Code::Blocks is a very capable IDE for C++ development as well. It offers many project templates even though it does not offer native KDE project types. This should not be a problem to most people however. Among the features I like the most is that it does not require explicit make file configuration and the build dependencies are inferred by default. This makes it very attractive for rapid prototyping. Debugging under Code::Blocks is also a pleasant experience. It also integrates (at least in the later SVN versions) Valgrind’s MemCheck and Cachegrind, which are very useful for detecting memory leaks and tweaking algorithms for the maximum performance.

The latest stable version of Code::Blocks is 8.02, it is a little dated as a lot of functionalities have been added in the later SVN builds. If you do not require the most stability (I have run into some issues recently), using SVN build should not be a problem. The editor (e.g. syntax highlighting, collapsible regions) is a little bit buggy though and the contextual help does not always work to the level of detail I desired.

NetBeans C++ IDE is probably the most beautiful one among the three. It offers the most detailed syntax highlighting, and can be configured to display class hierarchy and library function information. The refactor tool works pretty well and is certainly a boon to large scale development. Its contextual help is also of top-notch.

All of these come at a cost of course. NetBeans C++ IDE is the most resource intensive among the three. It can easily use 500 MB memory when doing development and can be sluggish at times. It also comes with a very limited project template (e.g. no out-of-box project templates for Qt, wxWidgets). Certain settings are hard to get at as well. Nevertheless, if I am primarily writing back-end code, NetBeans C++ IDE could easily earn my top choice.

Some people like Anjuta and compare it favorably to Code::Blocks. But I haven’t got a chance to use it yet.

In this article, I will show a simple example of creating a wrapper class using IPP and Magick++. The example I am going to show can be used to calculate the 2-dimensional FFT spectrum of a gray-scale image. This framework can be easily extended to include other algorithms that can be applied to an image using IPP.

Before I show the implementation details, let me first show how easy it is to use the class. The code snippet below shows how to read in an image file, apply 2D FFT with and without a Hamming window and save the results into image files.

    IPPGrayImage *img, *img1;

    img = new IPPGrayImage();
    img->LoadFromFile(IMAGE_FILE);

    img1 = img->Clone();
    img1 = img1->FFT(true);
    img1->SaveToFile(IMAGE_HOME + "/test_fftmag.jpg");
    img1 = img->Clone();
    img1 = img->ApplyHammingWindow();
    img1->SaveToFile(IMAGE_HOME + "/test_hamming.jpg");
    img1 = img1->FFT(true);
    img1->SaveToFile(IMAGE_HOME + "/test_fftmag_hamming.jpg");

And for the following IMAGE_FILE,

Test image used for 2D FFT

Here are the results for FFT spectrum with and without hamming window:

Hamming window applied

FFT spectrum (without Hamming window)

FFT spectrum with Hamming window

The header file for the class is as follows:

#ifndef IPPGRAYIMAGE_H
#define IPPGRAYIMAGE_H

#include <Magick++/Image.h>
#include <Magick++.h>
#include <ipp.h>

using namespace std;
using namespace Magick;

namespace KDW
{
    class IPPGrayImage
    {
    public:
        unsigned int PIXEL_SIZE;

        IPPGrayImage();
        IPPGrayImage(const unsigned int width, const unsigned int height);
        IPPGrayImage(Ipp32f *imgBuffer, const unsigned int width, const unsigned int height);
        IPPGrayImage(const IPPGrayImage& other);
        virtual ~IPPGrayImage();
        IPPGrayImage& operator=(const IPPGrayImage& other);

        void LoadFromFile(string fileName);
        void SaveToFile();
        void SaveToFile(string fileName);

        inline float GetPixel(unsigned int col, unsigned int row) {return _imgBuffer[row * _width + col];}
        inline void SetPixel(unsigned int col, unsigned int row, float clr) {_imgBuffer[row * _width + col] = clr;}

        unsigned int GetWidth() { return _width;}
        unsigned int GetHeight() { return _height;}

        Ipp32f* GetImageBuffer() { return _imgBuffer;}
        Image* GetImage() { return _img;}

        IPPGrayImage* Clone();
        IPPGrayImage* ApplyHammingWindow();
        IPPGrayImage* FFT(bool fftShift = false);
    protected:
    private:
        unsigned int _width;
        unsigned int _height;

        Image* _img;
        Pixels* _view;
        PixelPacket* _pixels;
        Ipp32f* _imgBuffer;
        string _fileName;

        void Init();
    };
}
#endif // IPPGRAYIMAGE_H

And here’s the implementation for the class:

#include <assert.h>
#include <math.h>

#include "IPPGrayImage.h"

namespace KDW
{
    /** @brief Constructor()
     *  Initialize image buffer.
     */
    IPPGrayImage::IPPGrayImage()
    {
        Init();
    }

    /** @brief Constructor(width, height)
     *  Initialize an image size of width x height.
     */
    IPPGrayImage::IPPGrayImage(const unsigned int width, const unsigned int height)
    {
        Init();

        _width = width;
        _height = height;

        int stepByte = 0;
        _imgBuffer = ippiMalloc_32f_C1(_width, _height, &stepByte);
    }

    /** @brief Constructor(imgBuffer, width, height)
     *  Initilize from an Ipp32f buffer (width x height)
     */
    IPPGrayImage::IPPGrayImage(Ipp32f *imgBuffer, const unsigned int width, const unsigned int height)
    {
        Init();

        _width = width;
        _height = height;
        _imgBuffer = imgBuffer;
    }

    /** @brief Copy Constructor
     */
    IPPGrayImage::IPPGrayImage(const IPPGrayImage& other)
    {
        _width = other._width;
        _height = other._height;

        int stepByte = 0;

        _imgBuffer = ippiMalloc_32f_C1(_width, _height, &stepByte);

        for (unsigned int row = 0; row < _height ; row++)
        {
            for (unsigned int column = 0; column < _width ; column++)
            {
                _imgBuffer[column + row * _width] =other._imgBuffer[column + row * _width];
            }
        }

        if (other._pixels == NULL) _pixels = NULL;
    }

    /** @brief Overload =
     */
    IPPGrayImage& IPPGrayImage::operator=(const IPPGrayImage& rhs)
    {
        if (this == &rhs) return *this; // handle self assignment

        return *this;
    }

    /**
     * @brief Destructor
     */
    IPPGrayImage::~IPPGrayImage()
    {
        ippFree(_imgBuffer);
        delete _img;
        delete _view;
    }

    /**
     * @brief Common initialization code
     */
    void IPPGrayImage::Init()
    {
        PIXEL_SIZE = sizeof(Ipp32f);
        _pixels = NULL;
        _imgBuffer = NULL;
        _img = NULL;
    }

    /** @brief Load an image from file
      */
    void IPPGrayImage::LoadFromFile(string fileName)
    {
        _img = new Image(fileName);
        Geometry g = _img->size();

        _width = g.width();
        _height= g.height();

        _view = new Pixels(*_img);
        _pixels = _view->get(0,0, _width, _height);

        int stepByte = 0;
        _imgBuffer = ippiMalloc_32f_C1(_width, _height, &stepByte);

        for (unsigned int row = 0; row < _height ; row++)
        {
            for (unsigned int column = 0; column < _width ; column++)
            {
                PixelPacket *p = &_pixels[column + row * _width];
                Color c = Color(p->red, p->green, p->blue);
                _imgBuffer[column + row * _width] = c.intensity();
            }
        }
    }

    /** @brief Save the current image buffer to file
      */
    void IPPGrayImage::SaveToFile()
    {
        SaveToFile("");
    }

    /** @brief SaveToFile(fileName)
     *  Saves the current image buffer to a file (by file name).
     */
    void IPPGrayImage::SaveToFile(string fileName)
    {
        if (_img != NULL && _pixels != NULL)
        {
            for (unsigned int y = 0; y< _height ; y++)
            {
                for (unsigned int x = 0; x < _width; x++)
                {
                    float clr = (float) _imgBuffer[x + y * _width];
                    _pixels[x+ y * _width] = Color(clr, clr, clr);
                }
            }
            _view->sync();
            _img->syncPixels();

            if (fileName == "")
            {
                _img->write(_fileName);
            }
            else
            {
                _img->write(fileName);
            }
        }
        else
        {
            Image img(Geometry(_width, _height),"white");

            for (unsigned int y = 0; y< _height ; y++)
            {
                for (unsigned int x = 0; x < _width; x++)
                {
                    Color c;
                    float clr = (float) _imgBuffer[x + y * _width];
                    img.pixelColor(x,y, Color(clr, clr, clr));
                }
            }

            img.write(fileName);
        }
    }

    /** @brief Clone
    *   Duplicate the current image to another IPPGrayImage object.
    *   @return an IPPGrayImage pointer to the cloned image
    */
    IPPGrayImage* IPPGrayImage::Clone()
    {
        IPPGrayImage *newImg;

        newImg = new IPPGrayImage(_width, _height);

        int stepByte = 0;
        newImg->_imgBuffer = ippiMalloc_32f_C1(_width,_height, &stepByte);

        for (unsigned int y = 0 ; y < _height ; y++)
        {
            for (unsigned int x = 0 ; x < _width ; x++)
            {
                newImg->_imgBuffer[x + y * _width] = _imgBuffer[x + y * _width];
            }
        }

        return newImg;
    }

    /** @brief Apply Hamming window to the image
     *  @return an IPPGrayImage pointer to the processed image
     */
    IPPGrayImage* IPPGrayImage::ApplyHammingWindow()
    {
        IPPGrayImage *newImg;
        newImg = new IPPGrayImage(_width , _height);
        IppiSize srcImgSize = {_width, _height};

        IppStatus sts;
        int stepByte;
        Ipp32f *imgCache = ippiMalloc_32f_C1(_width , _height , &stepByte);

        sts = ippiWinHamming_32f_C1R(_imgBuffer, _width * PIXEL_SIZE, imgCache, _width * PIXEL_SIZE, srcImgSize);
        assert(sts ==ippStsNoErr);

        for (unsigned int y = 0; y< _height; y++)
        {
            for (unsigned int x = 0; x < _width; x++)
            {
                newImg->_imgBuffer[x+ y * _width] = imgCache[x + y * _width];
            }
        }

        return newImg;
    }

    /** @brief Perform FFT on the image and returns the magnitude component
     *  @param fftShift: if it is true, the the result is with
     *         zero-frequency component shifted to center of spectrum
     *  @return the magnitude FFT component
     */
    IPPGrayImage* IPPGrayImage::FFT(bool fftShift)
    {
        IPPGrayImage *newImg;
        IppiFFTSpec_R_32f *spec;
        IppStatus sts;

        unsigned int n = (int) (logf((float) _width) / logf(2.0f));
        unsigned int m = (int) (logf((float) _height) / logf(2.0f));

        unsigned int N = pow(2, n);
        unsigned int M = pow(2, m);

        if (N < _width)
        {
            n = n + 1;
            N = pow(2, n);
        }

        if (M < _height)
        {
            m = m + 1;
            M = pow(2, m);
        }

        int stepByte;
        Ipp32f *src = ippiMalloc_32f_C1(M , N , &stepByte);
        Ipp32f *dst = ippiMalloc_32f_C1(M , N , &stepByte);
        Ipp32f *mag = ippiMalloc_32f_C1(M , N , &stepByte);

        IppiSize srcImgSize = {_width, _height};
        IppiSize dstImgSize = {N, M};

        sts = ippiCopyConstBorder_32f_C1R(
                  _imgBuffer, _width * PIXEL_SIZE, srcImgSize,
                  src,  N * PIXEL_SIZE, dstImgSize,
                  0,0,0);
        assert(sts ==ippStsNoErr);

        sts = ippiFFTInitAlloc_R_32f(&spec, n , m, IPP_FFT_DIV_BY_SQRTN, ippAlgHintAccurate);
        assert(sts ==ippStsNoErr);

        sts = ippiFFTFwd_RToPack_32f_C1R(src, N*PIXEL_SIZE, dst, N*PIXEL_SIZE, spec, 0);
        assert(sts ==ippStsNoErr);

        sts = ippiMagnitudePack_32f_C1R(dst, N*PIXEL_SIZE, mag, N*PIXEL_SIZE, dstImgSize);
        assert (sts ==ippStsNoErr);

        newImg = new IPPGrayImage(N , M);

        if (fftShift)
        {
#pragma omp sections
            {
#pragma omp section
                {
                    for (unsigned int y = 0 ; y < M/2; y++)
                    {
                        for (unsigned int x = 0 ; x < N/2; x++)
                        {
                            newImg->_imgBuffer[x+ y *N] = mag[x + N/2 + (y + M/2) * N];
                        }
                    }
                }
#pragma omp section
                {
                    for (unsigned int y = 0 ; y < M/2; y++)
                    {
                        for (unsigned int x = N/2 ; x < N; x++)
                        {
                            newImg->_imgBuffer[x+ y *N] = mag[x - N/2 + (y + M/2) * N];
                        }
                    }
                }
#pragma omp section
                {
                    for (unsigned int y = M/2 ; y < M; y++)
                    {
                        for (unsigned int x = 0 ; x < N/2; x++)
                        {
                            newImg->_imgBuffer[x+ y *N] = mag[x + N/2 + (y - M/2) * N];
                        }
                    }
                }
#pragma omp section
                {
                    for (unsigned int y = M/2 ; y < M; y++)
                    {
                        for (unsigned int x = N/2 ; x < N; x++)
                        {
                            newImg->_imgBuffer[x+ y *N] = mag[x - N/2 + (y - M/2) * N];
                        }
                    }
                }
            }
        }
        else
        {
            for (unsigned int y = 0; y< M; y++)
            {
                for (unsigned int x = 0; x <N; x++)
                {
                    newImg->_imgBuffer[x+ y * N] = mag[x + y * N];
                }
            }

        }

        return newImg;
    }
}

The above example showed only the FFT function, but virtually all IPP image routines can be accommodated using the wrapper image class above. Intel IPP utilizes many different data types (e.g. Ipp8u, Ipp16s, Ipp32f etc.), but to provide most of the flexibility and compatibility I chose to use only the 32 bit floating data type. For specific implementations, other data types can be used as well.

Since the implementation of many of the functions are threaded, it makes the task of writing high performance applications much easier. Since it is a set of “performance primitives” as the name suggests, it uses its own data structures (e.g. Ipp8u) and does not provide functions to directly inter-operate with data coming from other sources (e.g. image files).

Fortunately, such data conversion is pretty straight forward. In this post, I will illustrate how to convert an image file (e.g. .jpg, .png, .gif) to the data IPP uses, and how to save the result into a standard image file once the processing is done.

Magick++ is a very comprehensive image-processing C++ library and the Image class it provides handles image files quite well. So I chose to use Magic++’s API to convert image files to the data structure IPP uses. In this particular example, I will use a gray level image. But in practice, color images can be easily handled in a similar fashion.

For the code mentioned below, the following headers and namespaces are used:

#include <Magick++/Image.h>
#include <Magick++.h>
#include <ipp.h>

using namespace std;
using namespace Magick;

And the following code shows how to convert a standard image file data into format that is suitable for IPP.

    IppStatus sts;
    Image img("{Image File Name}");
    Geometry g = img.size();

    unsigned int width = g.width();
    unsigned int height= g.height();

    Pixels view(img);
    PixelPacket *pixels = view.get(0,0,width,height);

    int stepByte = 0;
    //allocating a buffer of unsigned char (Ipp8u) for the image.
    Ipp8u *imgCache = ippiMalloc_8u_C1(width,height, &stepByte);

    for (unsigned int row = 0; row < height ; row++)
    {
        for (unsigned int column = 0; column < width ; column++)
        {
            PixelPacket *p = &pixels[column + row * width];
            Color c = Color(p->red, p->green, p->blue);
            double i = c.scaleQuantumToDouble(c.intensity()) * 255;
            imgCache[column + row * width] = (char) i;
        }
    }

The above code snippet first reads the image data into *PixelPacket and the pixel buffer is then converted into a one channel buffer of chars (color value 0-255). Note that the range of the image data Magick++ reads in is between 0 and QuantumRange, which needs to be converted back to the range 0-255 accepted by the Ipp8u buffer. If other types of IPP image buffers are used (e.g. Ipp32f), this scaleQuantumToDouble() conversion may not be necessary. At the end, the image data is converted into the one dimensional array imgCache which can be used by IPP procedures.

Once we are in the IPP data domain, we can proceed with whatever processing we had in mind. Here I will show the code for edge detection using Canny algorithm.

    IppiSize orgImgSize = {width, height};
    IppiSize newImgSize = {width + 2, height + 2};

    Ipp8u *imgCache1 = ippiMalloc_8u_C1(width + 2,height + 2, &stepByte);
    sts = ippiCopyReplicateBorder_8u_C1R(imgCache, width, orgImgSize,  imgCache1,
            width + 2 , newImgSize, 2, 2);
    IppiSize roi = {width, height};

    Ipp32f low=30.0f, high=100.0f;
    int size, size1, srcStep, dxStep, dyStep, dstStep;

    Ipp8u *src, *dst, *buffer;
    Ipp16s *dx, *dy;

    sts = ippiFilterSobelNegVertGetBufferSize_8u16s_C1R(
            roi, ippMskSize3x3, &size);
    sts = ippiFilterSobelHorizGetBufferSize_8u16s_C1R(
            roi, ippMskSize3x3, &size1);

    if (size<size1) size=size1;
    ippiCannyGetSize(roi, &size1);
    if (size<size1) size=size1;

    buffer = ippsMalloc_8u(size);
    dx = ippsMalloc_16s(size);
    dy = ippsMalloc_16s(size);
    dst = ippsMalloc_8u(size);

    sts = ippiFilterSobelNegVertBorder_8u16s_C1R (
            imgCache1, width + 2 , dx, (width + 2) * 2 ,
            roi, ippMskSize3x3, ippBorderRepl, 0, buffer);
    sts = ippiFilterSobelHorizBorder_8u16s_C1R(
            imgCache1, width + 2, dy, (width + 2) *2 ,
            roi, ippMskSize3x3, ippBorderRepl, 0, buffer);
    sts = ippiCanny_16s8u_C1R(dx,
            (width + 2) * 2, dy, (width + 2) * 2,
            dst, width, roi, low, high, buffer);

The code shown above is adopted from Intel’s IPP manual for image and video processing (by default it is located at /opt/intel/ipp/{version number}/em64t/doc/ippiman.pdf).

Please pay special attention to how the original image is extended via ippiCopyReplicateBorder_8u_C1R. The image is extended by 2 pixels in each direction because the 3×3 mask used for the filtering operation. I omitted error checking code for simplicity, but in general you need to check the return status of each ippi function call. When the call is successful, the status is ippStsNoErr. If you receive a value other than ippStsNoErr (e.g. ippStsStepErr) you will need to check your buffer size to make sure that they are adjusted according to the data type. For instance, in the code above dx and dy are both 16bit signed integers and thus they both occupy two bytes each.

To save the result image, we convert the pixel buffer back to PixelPacket type. Again we need to convert the data in the buffer (Ipp8u) to the range accepted by Magick++ API.

    for (unsigned int y = 0; y< height ; y++)
    {
        for (unsigned int x = 0; x < width; x++)
        {
            Color c;
            float clr = (float) dst[x + y * width] /255;
            float q = c.scaleDoubleToQuantum(clr);
            pixels[x+ y * width] = Color(q, q, q);
        }
    }

    view.sync();
    img.syncPixels();

    img.write("{Image File Name}");

We can also save the result data into a new image (instead of syncing the data back to the image object holding the original image data) using the code below:

    Image img1(Geometry(width, height),"white");

    for (unsigned int y = 0; y< height ; y++)
    {
        for (unsigned int x = 0; x < width; x++)
        {
            Color c;
            float clr = (float) dst[x + y * width] /255;
            float q = c.scaleDoubleToQuantum(clr);
            img1.pixelColor(x,y, Color(q, q, q));
        }
    }

    img1.write("{Image File Name}");

And finally the buffers used are freed.

    ippiFree(imgCache);
    ippsFree(buffer);

The following images show Canny edge detector in action using the code in this article:

Original

Canny Edge Detector applied

Even though a lot of optimizations have been done in the .Net runtime to make it more efficient, it is apparent that scientific programming still favors C and C++ because that the performance advantage is huge.

In this article, I will examine some matrix multiplication algorithms that are commonly used and illustrate the efficiencies of the various methods. All the tests are done using C++ only and matrices size ranging from 500×500 to 2000×2000. When the matrix sizes are small (e.g. <50), you can pretty much use any matrix multiplication algorithms without observing any significant performance differences. This is largely due to the fact that the typical stable matrix multiplication algorithms are O(n^3) and sometimes array operation overheads outweigh the benefit of algorithm efficiencies. But for matrices of larger dimensions, the efficiency of the multiplication algorithm becomes extremely important.

Since Dmitri’s article has already captured pretty detailed data using the standard matrix multiplication algorithm, I will not repeat his findings in this article. What I intended to show was the performance data of uBLAS, OpenMP, cBLAS and MATLAB.

The following sample code are compiled under Ubuntu 8.10 64 bit (kernel 2.6.24.23) on Intel Q9450@3.2GHz.

Standard Matrix Multiplication (Single Threaded)

This is our reference code. Later on, I will only show the critical portion of the code and not repeat the common portion of code that initializes/finalizes the arrays. Similarly, the timing method used is also the same across all the tests and will be omitted later on.

float **A, **B, **C;

A = new float*[matrix_size];
B = new float*[matrix_size];
C = new float*[matrix_size];

for (int i = 0 ; i < matrix_size; i++)
{
    A[i] = new float[matrix_size];
    B[i] = new float[matrix_size];
    C[i] = new float[matrix_size];
}

for (int i=0; i<matrix_size; i++)
{
    for (int j = 0 ; j < matrix_size; j++)
    {
        A[i][j]=rand();
        B[i][j]=rand();
    }
}

timeval t1, t2, t;
gettimeofday(&t1, NULL);

for (int i = 0 ; i < matrix_size; i++)
{
    for (int j = 0;  j < matrix_size; j++)
    {
        C[i][j] = 0;
        for (int k = 0; k < matrix_size; k++)
        {
            C[i][j] += A[i][k] * B[k][j];
        }
    }
}

gettimeofday(&t2, NULL);
timersub(&t2, &t1, &t);

cout << t.tv_sec + t.tv_usec/1000000.0 << " Seconds -- Standard" << endl;

for (int i = 0 ; i < matrix_size; i++)
{
    delete A[i];
    delete B[i];
    delete C[i];
}

delete A;
delete B;
delete C;

OpenMP With Two Dimensional Arrays

Using OpenMP, we are able to multiple threads via the #pragma omp directives. For the simple algorithm we used here, the speed increase is almost proportional to the number of available cores within the system.

...
#pragma omp parallel for shared(a,b,c)
for (long i=0; i<matrix_size; i++)
{
    for (long j = 0; j < matrix_size; j++)
    {
        float sum = 0;
        for (long k = 0; k < matrix_size; k++)
        {
            sum +=a[i][k]*b[k][j];
        }
        c[i][j] = sum;
    }
}
...

OpenMP With One Dimensional Arrays

Cache locality is poor using the simple algorithm I showed above. The performance can be easily improved however by improving the locality of the references. One way to achieve better cache locality is to use one dimensional array instead of two dimensional array and as you will see later, the performance of the following implementation has as much as 50% speed gains over the previous OpenMP implementation using two dimensional arrays.

float *a, *b, *c;

a = new float[matrix_size * matrix_size];
b = new float[matrix_size * matrix_size];
c = new float[matrix_size * matrix_size];

for (long i=0; i<matrix_size * matrix_size; i++)
{
    a[i]=rand();
    b[i] = rand();
    c[i] = 0;
}

#pragma omp parallel for shared(a,b,c)
for (long i=0; i<matrix_size; i++)
{
    for (long j = 0; j < matrix_size; j++)
    {
        long idx = i * matrix_size;
        float sum = 0;
        for (long k = 0; k < matrix_size; k++)
        {
            sum +=a[idx + k]*b[k * matrix_size +j];
        }
        c[idx + j] = sum;
    }
}

delete a;
delete b;
delete c;

Boost Library uBLAS (Single Threaded)

Boost library provides a convenient way to perform matrix multiplication. However, the performance is very poor compared to all other approaches mentioned in this article. The performance of the uBLAS implementation is largely on par with that using C# (see benchmarks towards the end of the article). Intel’s Math Kernal Library (MKL) 10.1 does provide functionality to dynamically convert code using uBLAS syntax into highly efficient code using MKL by the inclusion of header file mkl_boost_ublas_matrix_prod.hpp. I have not tried it myself though, but the performance should be comparible to algorithms using the native MKL BLAS interface.

By default (without using MKL’s uBLAS capability) though, uBLAS is single threaded and due to its poor performance and I would strongly suggest avoid using uBLAS in any high performance scientific applications.

matrix<float> A, B, C;

A.resize(matrix_size,matrix_size);
B.resize(matrix_size,matrix_size);

for (int i = 0; i < matrix_size; ++ i)
{
    for (int j = 0; j < matrix_size; ++ j)
    {
        A(i, j) = rand();
        B(i, j) = rand();
    }
}

C =prod(A, B);

Intel Math Kernel Library (MKL) cBLAS

Intel’s Math Kernel Library (MKL) is highly optimized on Intel’s microprocessor platforms. Given that Intel developed this library for its own processor platforms we can expect significant performance gains. I am still surprised at how fast the code runs using cBLAS though. In fact, it was so fast that I doubted the validity of the result at first. But after checking the results against those obtained by other means, those doubts were putting into rest.

The cBLAS matrix multiplication uses blocked matrix multiplication method which further improves cache locality. And it is more than thirty times faster then the fastest OMP 1D algorithm listed above! Another benefit is that by default it automatically detects the number of CPUs/cores available and uses all available threads. This behavior greatly simplifies the code since threading is handled transparently within the library.

float *A, *B, *C;

A = new float[matrix_size * matrix_size];
B = new float[matrix_size * matrix_size];
C = new float[matrix_size * matrix_size];

for (int i = 0; i < matrix_size * matrix_size; i++)
{
    A[i] = rand();
    B[i] = rand();
    C[i] = 0;
}

cblas_sgemm(CblasRowMajor, CblasNoTrans, CblasNoTrans,
    matrix_size,  matrix_size,  matrix_size, 1.0, A,matrix_size,
    B, matrix_size, 0.0, C, matrix_size);

MATLAB (Single Threaded)

MATLAB is known for its efficient algorithms. In fact it uses BLAS libraries for its own matrix calculation routines. The version of MATLAB I have is a little dated (7.0.1), but nevertheless it would be interesting to see how its performance compares with that of latest MKL’s. MATLAB 7 is single threaded, and given the same matrix size, it runs roughly three times slower than the fastest MKL routine listed above (per core).

    a = rand(i,i);
    b = rand(i,i);
    tic;
    c = a*b;
    t = toc

The following table shows the results I obtained by running the code listed above. The results are time in seconds. (note, S.TH means single threaded and M.TH means multi-threaded).

The following figure shows the results. Since uBLAS and single threaded matrix multiplications took significantly longer to compute, I did not include them in the figure below.

The following figure shows the same data but uses log-scale Y axis instead so that all the data can show up nicely. You can get a sense of various algorithms’ efficiencies here:

To get the driver built on Ubuntu 8.04 however, I needed to make some minor changes to epcam.c. The kernel version I was compiling against is 2.6.24-19-generic. To get the 0.7.3 driver build successfully, I needed to comment out #include <linux/config.h> and then ran

KBUILD_NOPEDANTIC=1 make install

If you do not want to setup the environment variables for the build, you could just modify the #include statements to where the kernel header files are located:

After the build the driver is automatically installed and when the webcam is plugged in, it should be recognized by apps such as camorama (note: /dev/video0 is automatically created when the webcam is connected).

The standard algorithm for generating Mandelbrot Set is extremely easy to adapt to using TBB. In fact the loops look almost identical to those in the single-threaded approach, except that the iterations are calculated within a 2D range block (blocked_range2d) instead of the entire two dimensional space.

Because xlib by itself is not thread-safe, special attention must be made when trying to update the display concurrently. One way to address this issue is to employee a shared memory region (X11/extensions/XShm.h and sys/shm.h), the display is first built in memory and then the shared memory is attached to the display. In my examples above I used code (video.h, xvideo.cpp) from the sample code that come with the TBB library, which uses the shared memory method I mentioned earlier to make the X11 calls thread-safe.

Many optimization methods can be used to further enhance the performance of the algorithm. One of the most efficient methods is to utilize SSE instructions found on all modern Intel processors (examples can be found here: Using SSE3 Technology in Algorithms with Complex Arithmetic). This approach however might be difficult to implement and debug since parallel data structures must be used in order to benefit from SSE instructions. Also, explicit assembly level coding makes porting code to other machine architectures a daunting task. Modern compilers can already take full advantage of the underlying machine architecture. For example, the gcc compiler (4.2.3) already generates SSE instructions for the code snippet above. While hand tweaking using SSE instructions might further improve the performance, we would certainly sacrifice code simplicity and portability.

The approach I am going to take to further optimize the code is to use loop unrolling. Since the inner loop of the standard algorithm is pretty short, unrolling the inner loop should lessen the burden of loop overhead and decrease the chances of stalling the pipeline (when branching must be predicted). So a high-level loop unrolling should be able to improve the performance.

Here is the code after the inner loop is unrolled:

This code generates identical results as the code mentioned previously. As you can see, the inner loop is not unrolled to handle two data points at a time.

As it turned out, this algorithm runs almost twice as fast as the code mentioned earlier(280ms versus 510ms on Intel Q9450 @ 3.4GHz).

Source code for this article can be downloaded here.

I personally like simplicity of Code::Blocks when writing small programs. Even though Code::Blocks does come with Qt support, you will have to tell it where Qt is installed before you can compile your code correctly. Like any other C++ environment, the two things you must provide are the include path and the linker path.

Ubuntu 8.04 comes with Qt3 installed, to install Qt4, please follow the instructions here. Here are the build options you need to add with the default Qt4 install:

As a bare minimum, you will need to set the libQtGui and libQtCore libraries paths. Note thatUbuntu 8.04 comes with Qt3 installed, when you specify the libraries you need to make sure you specify the ones with the correct version. In my example, the required Qt4 libraries are /usr/lib/libQtGui.so.4 and /usr/lib/libQtCore.so.4. You also need to specify where the include files can be found. For typical applications, /ur/incude/qt4 and /usr/include/qt4/QtGui are the paths you will need. If you want contextual help to be available, you will have to provide the search paths for the include files under "C/C++ parser options" tab in Project/targets options settings.

For a very large interval (e.g. 3~1,000,000), the TBB version of the prime program achieved a 4x speed up given a reasonably large grain size (e.g. 100). Smaller grain size resulted in slightly more overhead. On a quad-core machine (Q9450 @ 3.2GHz), it took 217.83 ms for the single threaded routine to find all the prime numbers within 1,000,000, whereas it only took 58.32 ms for the TBB version, which runs roughly four times as fast. The TBB framework took care of dividing the task according to the number of processors automatically.

Like any kind of science, the essence of computer science education was not to teach you any particular languages, but to teach you the theories by which all computational tasks abide.

A typical software engineer today is very different from one even just a couple of decades ago. Back then, creating a computer program usually required some very intimate knowledge of both the software and the hardware. Of course we had no choice back then as we lacked the modern high level languages (e.g. Java, C#) and the hardware resources were usually very limited. Back then we valued efficient algorithms in terms of both footprint and efficiency as a 600K versus 60k application usually means one can fit into the main memory and one cannot. And similarly, an inefficient algorithm usually led to an unusable application.

Things have arguably changed quite a bit nowadays. Personal computers today are becoming ever more powerful and the limit imposed by storage is rapidly disappearing for all practical matters. Nobody would even notice the difference of an inefficient application taking one second to run whereas the same application could be optimized to run a thousand time faster. A 10Meg application and a 1Meg application would rarely pose any problem as today’s computers can easily handle multi-gigabytes of working sets.

And with computers being as ubiquitous as they are today, almost every one knows how to operate a computer and does what his or her heart desires to a certain degree. So naturally, the technological savviness of an average developer is far less than that ten or twenty years ago.

This situation was made even worse in a market economy today. Most companies’ job postings require the knowledge and experience of certain computer languages and without such credentials (e.g. for a fresh colledge graduate) a candidate is simply not considered. Even though we all know that a good developer in one language typically can master any given languages within a very short period of time and be good at them as well. Unfortunately, as human nature, we tend to emphasize more on the surface value that we can see.

The society we live in is one that is driven by demand. Given the corporate culture just mentioned, more and more universities, especially the lesser known ones started to put more emphasis on what is needed in the market and the fundamentals (e.g. Assembly, C/C++) became less desirable to teach or learn as such fundamentals hardly ever translate into the market place value.

And it is only to become worse, as companies like Microsoft make the application development environment easier and easier to use, wring programs becoming seemingly more and more trivial. Some people even believe that anyone, with some training, can easily master a computer language and thus becomes a software developer. There might be some truth to that as far as the "surface value" is concerned, but this is rather dangerous as there is a lot more to writing good programs than being able to drag and drop.

To this day, this is still the program I admire the most. I still wonder how Jim Coplien created such an elegant program which, even trying to understand the reverse engineered version remains a daunting task.

In order for every "modern" programmer to grasp the true beauty of the original program, I have modified it slightly so it will run in Visual Studio 2005. Try it yourself!