Interfacing IPP with Magick++

Intel’s Integrated Performance Primitives (IPP) is a low level C++ library. It provides routines that are highly optimized on Intel processors. I recently started using it because its vast speed advantage in signal and image processing applications.

Since the implementation of many of the functions are threaded, it makes the task of writing high performance applications much easier. Since it is a set of “performance primitives” as the name suggests, it uses its own data structures (e.g. Ipp8u) and does not provide functions to directly inter-operate with data coming from other sources (e.g. image files).

Fortunately, such data conversion is pretty straight forward. In this post, I will illustrate how to convert an image file (e.g. .jpg, .png, .gif) to the data IPP uses, and how to save the result into a standard image file once the processing is done.

Magick++ is a very comprehensive image-processing C++ library and the Image class it provides handles image files quite well. So I chose to use Magic++’s API to convert image files to the data structure IPP uses. In this particular example, I will use a gray level image. But in practice, color images can be easily handled in a similar fashion.

For the code mentioned below, the following headers and namespaces are used:

#include <Magick++/Image.h>
#include <Magick++.h>
#include <ipp.h>

using namespace std;
using namespace Magick;

And the following code shows how to convert a standard image file data into format that is suitable for IPP.

    IppStatus sts;
    Image img("{Image File Name}");
    Geometry g = img.size();

    unsigned int width = g.width();
    unsigned int height= g.height();

    Pixels view(img);
    PixelPacket *pixels = view.get(0,0,width,height);

    int stepByte = 0;
    //allocating a buffer of unsigned char (Ipp8u) for the image.
    Ipp8u *imgCache = ippiMalloc_8u_C1(width,height, &stepByte);

    for (unsigned int row = 0; row < height ; row++)
    {
        for (unsigned int column = 0; column < width ; column++)
        {
            PixelPacket *p = &pixels[column + row * width];
            Color c = Color(p->red, p->green, p->blue);
            double i = c.scaleQuantumToDouble(c.intensity()) * 255;
            imgCache[column + row * width] = (char) i;
        }
    }

The above code snippet first reads the image data into *PixelPacket and the pixel buffer is then converted into a one channel buffer of chars (color value 0-255). Note that the range of the image data Magick++ reads in is between 0 and QuantumRange, which needs to be converted back to the range 0-255 accepted by the Ipp8u buffer. If other types of IPP image buffers are used (e.g. Ipp32f), this scaleQuantumToDouble() conversion may not be necessary. At the end, the image data is converted into the one dimensional array imgCache which can be used by IPP procedures.

Once we are in the IPP data domain, we can proceed with whatever processing we had in mind. Here I will show the code for edge detection using Canny algorithm.

    IppiSize orgImgSize = {width, height};
    IppiSize newImgSize = {width + 2, height + 2};

    Ipp8u *imgCache1 = ippiMalloc_8u_C1(width + 2,height + 2, &stepByte);
    sts = ippiCopyReplicateBorder_8u_C1R(imgCache, width, orgImgSize,  imgCache1, 
            width + 2 , newImgSize, 2, 2);
    IppiSize roi = {width, height};

    Ipp32f low=30.0f, high=100.0f;
    int size, size1, srcStep, dxStep, dyStep, dstStep;

    Ipp8u *src, *dst, *buffer;
    Ipp16s *dx, *dy;

    sts = ippiFilterSobelNegVertGetBufferSize_8u16s_C1R(
            roi, ippMskSize3x3, &size);
    sts = ippiFilterSobelHorizGetBufferSize_8u16s_C1R(
            roi, ippMskSize3x3, &size1);

    if (size<size1) size=size1;
    ippiCannyGetSize(roi, &size1);
    if (size<size1) size=size1;

    buffer = ippsMalloc_8u(size);
    dx = ippsMalloc_16s(size);
    dy = ippsMalloc_16s(size);
    dst = ippsMalloc_8u(size);

    sts = ippiFilterSobelNegVertBorder_8u16s_C1R (
            imgCache1, width + 2 , dx, (width + 2) * 2 , 
            roi, ippMskSize3x3, ippBorderRepl, 0, buffer);
    sts = ippiFilterSobelHorizBorder_8u16s_C1R(
            imgCache1, width + 2, dy, (width + 2) *2 , 
            roi, ippMskSize3x3, ippBorderRepl, 0, buffer);
    sts = ippiCanny_16s8u_C1R(dx, 
            (width + 2) * 2, dy, (width + 2) * 2, 
            dst, width, roi, low, high, buffer);

The code shown above is adopted from Intel’s IPP manual for image and video processing (by default it is located at /opt/intel/ipp/{version number}/em64t/doc/ippiman.pdf).

Please pay special attention to how the original image is extended via ippiCopyReplicateBorder_8u_C1R. The image is extended by 2 pixels in each direction because the 3×3 mask used for the filtering operation. I omitted error checking code for simplicity, but in general you need to check the return status of each ippi function call. When the call is successful, the status is ippStsNoErr. If you receive a value other than ippStsNoErr (e.g. ippStsStepErr) you will need to check your buffer size to make sure that they are adjusted according to the data type. For instance, in the code above dx and dy are both 16bit signed integers and thus they both occupy two bytes each.

To save the result image, we convert the pixel buffer back to PixelPacket type. Again we need to convert the data in the buffer (Ipp8u) to the range accepted by Magick++ API.

    for (unsigned int y = 0; y< height ; y++)
    {
        for (unsigned int x = 0; x < width; x++)
        {
            Color c;
            float clr = (float) dst[x + y * width] /255;
            float q = c.scaleDoubleToQuantum(clr);
            pixels[x+ y * width] = Color(q, q, q);
        }
    }

    view.sync();
    img.syncPixels();

    img.write("{Image File Name}");

We can also save the result data into a new image (instead of syncing the data back to the image object holding the original image data) using the code below:

    Image img1(Geometry(width, height),"white");

    for (unsigned int y = 0; y< height ; y++)
    {
        for (unsigned int x = 0; x < width; x++)
        {
            Color c;
            float clr = (float) dst[x + y * width] /255;
            float q = c.scaleDoubleToQuantum(clr);
            img1.pixelColor(x,y, Color(q, q, q));
        }
    }

    img1.write("{Image File Name}");

And finally the buffers used are freed.

    ippiFree(imgCache);
    ippsFree(buffer);

The following images show Canny edge detector in action using the code in this article:

Original

Original

Canny Edge Detector applied

Canny Edge Detector applied

Be Sociable, Share!

3 Comments

  1. Andriy M says:

    Thanks for sharing this,

    I think your reading and writing from IPP has an error.
    While converting from a standard image file, you should use

    imgCache[column + row * stepByte/sizeof(Ipp8u)] = (char) i;
    instead of
    imgCache[column + row * width] = (char) i;

    The stebByte is a number of bytes between the horizontal lines (as allocated by ippiMalloc_8u_C1).
    This number may or may not be equal to the width.(even though you specify height and width).

    For the Ipp8u type, size(Ipp8u)=1, so the stepByte/sizeof(Ipp8u) is slightly redundant.

    In my problem I used Ipp32f type so I had to use stepByte/sizeof(Ipp32f).
    I used ippiMalloc_32f_C1(width,height, &stepByte); to allocate memory.
    It turned out that stepByte/sizeof(Ipp32f) is slightly longer than actually image width.

    I wonder why? Perhaps ippiMalloc allocates aligned memory for faster memory access, which
    can be slightly larger than actual image. If anybody can comment on this, I’d appreciate!

    Then when performing any operation,e.g. Sobel filter, you should use stepByte as a length
    between lines, instead of width.

    Any comments appreciated, thanks

  2. chip says:

    HI,
    nice tutorial.
    I tried canny detector using IPP with highGUI opencv, but the output was unexpected.
    here is the code:
    #include “cv.h”
    #include “highgui.h”
    #include “ipp.h”
    #include
    // file example.bmp in our images directory
    char name[] = “baboon.jpg”;
    // functions defined after main()
    void openWindow_8u( Ipp8u *img, IppiSize *size, int nChannels,char *name, int wait );
    void closeWindow_8u( char *name );
    int main()
    {
    IppStatus sts;
    // define Ipp8u image pointer rgb
    Ipp8u *ipprgb = NULL;
    Ipp8u *dst = NULL; Ipp8u *buffer = NULL;
    Ipp16s *dx, *dy;

    // size of our images
    IppiSize imgsize,roi;

    Ipp32f low=30.0f, high =100.0f;
    int size, size1, srcStep, dxStep, dyStep, dstStep;
    // pointers to our result images
    Ipp8u *ippxyz = NULL, *ipphls = NULL, *ippluv = NULL;
    IplImage* img = NULL; // new IplImage structure img

    int i = 0, j = 0, wait=0;
    img = cvLoadImage( name, -1 ); // load a BMP ipprgb image into img
    imgsize.width = img->width; // size of IPP images is the same
    imgsize.height = img->height; // as read image img
    roi.width = img->width; // *********************
    roi.height = img->height; // *********************
    // memory allocation for rgb
    ipprgb = ippsMalloc_8u( imgsize.width * imgsize.height * 3 );

    // the pointer of our OpenCV data is our IPP image pointer
    ippiCopy_8u_C3R((Ipp8u *) img->imageData, imgsize.width * 3, ipprgb, imgsize.width * 3, imgsize );

    sts = ippiFilterSobelNegVertGetBufferSize_8u16s_C1R(roi, ippMskSize3x3, &size);//*********************
    sts = ippiFilterSobelHorizGetBufferSize_8u16s_C1R(roi, ippMskSize3x3, &size1);//***********************
    if(size<size1)size=size1;//**********
    ippiCannyGetSize(roi,&size1);//**************
    if (size<size1) size=size1;//******************
    buffer = ippsMalloc_8u(size);//*******************
    dx = ippsMalloc_16s(size);//******
    dy = ippsMalloc_16s(size);//********
    dst = ippsMalloc_8u(size);//********
    sts = ippiFilterSobelNegVertBorder_8u16s_C1R (ipprgb, (imgsize.width*3+2), dx, (imgsize.width+2)*2 ,roi, ippMskSize3x3, ippBorderRepl, 0, buffer);
    sts = ippiFilterSobelHorizBorder_8u16s_C1R(ipprgb, (imgsize.width*3+2 ), dy, (imgsize.width+2 )*2 ,roi, ippMskSize3x3, ippBorderRepl, 0, buffer);
    sts = ippiCanny_16s8u_C1R(dx,(imgsize.width+2)*2 , dy, (imgsize.width+2)*2 , dst, (imgsize.width), roi, low, high, buffer);
    ippsFree(buffer);

    openWindow_8u( dst, &roi, 1, "dest", 0 ); // display xyz image
    openWindow_8u( ipprgb, &imgsize, 3, "rgb", 0 ); // display rgb image

    cvWaitKey( wait ); // wait for key (==0) or wait milliseconds

    closeWindow_8u( "dest" ); // close luv image
    closeWindow_8u( "rgb" ); // close rgb image
    ippsFree( ipprgb ); // memory release with IPP
    ippsFree( dst ); // memory release with IPP
    cvReleaseImage( &img ); // memory release with OpenCV
    return( 0 );
    }

    thanks for help!

    regards.

    • chip says:

      the problem was solved. I am forget to include function convert image RGB to GRAY.

      #include “cv.h”
      #include “highgui.h”
      #include “ipp.h”
      #include
      #include

      char name[] = “lena.jpg”;
      void openWindow_8u( Ipp8u *img, IppiSize *size, int nChannels,char *name, int wait );
      void closeWindow_8u( char *name );

      int main()
      {
      IppStatus sts;
      Ipp8u *ipprgb = NULL;
      Ipp8u *ippgray = NULL;
      Ipp8u *ippedge = NULL;
      Ipp8u *buffer = NULL;
      Ipp16s *dx, *dy;

      IppiSize imgsize,roi;

      Ipp32f low=8.0f, high =12.0f;
      int size, size1;
      IplImage *img = NULL;

      int wait=0;
      img = cvLoadImage( name, -1 );
      imgsize.width = img->width;
      imgsize.height = img->height;
      roi.width = img->width;
      roi.height = img->height;

      ipprgb = ippsMalloc_8u( imgsize.width * imgsize.height * 3 );
      ippgray = ippsMalloc_8u( imgsize.width * imgsize.height );
      ippiCopy_8u_C3R((Ipp8u *) img->imageData, imgsize.width * 3, ipprgb, imgsize.width * 3, imgsize );
      ippiRGBToGray_8u_C3C1R ( ipprgb, imgsize.width*3, ippgray, imgsize.width, imgsize );

      double t = (double)cvGetTickCount();

      sts = ippiFilterSobelNegVertGetBufferSize_8u16s_C1R(roi, ippMskSize3x3, &size);
      sts = ippiFilterSobelHorizGetBufferSize_8u16s_C1R(roi, ippMskSize3x3, &size1);
      if(size<size1)size=size1;
      ippiCannyGetSize(roi,&size1);
      if (sizewidth;
      sizeCv.height = size->height;
      cvImg = cvCreateImage( sizeCv, IPL_DEPTH_8U, nChannels );
      tmp = ippsMalloc_8u( size->width * size->height * nChannels );
      ippiCopy_8u_C3R( img, size->width * nChannels,
      tmp, size->width * nChannels, *size );
      cvSetData( cvImg, (void *) tmp, sizeCv.width * nChannels );
      cvNamedWindow( name, 1 );
      cvShowImage( name, cvImg );
      cvWaitKey( wait );
      cvReleaseImage( &cvImg );
      }
      void closeWindow_8u( char *name )
      {
      cvDestroyWindow( name );
      }

Leave a Reply to chip