A step-by-step guide to the use of the Intel OpenCV 1 library

Accessing the pixels of an image

So far, we used an OpenCV function to perform the processing on our image. To apply our own processing algorithm we need to be able to access each pixel of an image, read their values and possibly assign new value to them. This can be easily achieved because all the pixels of an IplImage are simply stored into an array of bytes; this is the image buffer. The size of this buffer is simply the width times the height of the image times the number of channels.

In the following example, the number of colors in an image is reduced by using a divisor that subdivides the RGB color into cubes of equal size. Each color point is assigned the color value that corresponds to the middle point of the cube it is contained in. We implemented this algorithm in-place (i.e. the original is modified). The processing class is therefore very simple since no initialization is required. Here is the processing method:

 
      void ColorProcessor::
process(IplImage *image) {
     
      int nl= image->height; // number of lines
      // total number of element per line
      int nc= image->width * image->nChannels; 
      int step= image->widthStep; // effective width
             
      // get the pointer to the image buffer
      unsigned char *data= reinterpret_cast<unsigned char *>
                                           (image->imageData);
 
      for (int i=1; i<nl; i++) {
            for (int j=0; j<nc; j+= image->nChannels) {
 
            // process each pixel ---------------------
                 
                  data[j]= data[j]/div * div + div/2;
                  data[j+1]= data[j+1]/div * div + div/2;
                  data[j+2]= data[j+2]/div * div + div/2;
 
            // end of pixel processing ----------------
 
            } // end of line
                   
            data+= step;  // next line
      }
}
Using the default divider value of 64 (4x4x4 colors) the following image is obtained:

Note that for an efficient use of the MMX capabilities of the processor, the line length of an image should always be a multiple of 8 bytes. To ensure that this condition is always met, when an image is created under OpenCV, this one is automatically padded with dummy pixels if necessary. This explains the role of the widthStep attribute; the real width of the image is given by width but if the image has been extended to become quad-word aligned, the effective width will be larger as given by widthStep.

Check point #4: source code of the above example.

Although this is the most efficient way to scan an image, this process can be error prone. In order to simplify this frequent task, an image Iterator can be introduced. The role of this iterator template is to take care of the pointer manipulation involve in the processing of an image. The template is as follows:

 
template <class PEL>
class IplImageIterator {
 
  int i, j, i0;
  PEL* data;
  int step;
  int nl, nc;
  int nch;
 
 public:
 
  /* constructor */
  IplImageIterator(IplImage* image,
     int x=0, int y=0, int dx= 0, int dy=0) :
       i(x), j(y), i0(0) {
    
    data= reinterpret_cast<PEL*>(image->imageData);
    step= image->widthStep / sizeof(PEL);
      CvRect rect= cvGetImageROI(image);
 
      nl= rect.height;
      nc= rect.width;
      x+= rect.x;
      y+= rect.y;
 
    if ((y+dy)>0 && (y+dy)<nl) nl= y+dy;
    if (y<0 || y>=nl) j=0;
    data+= step*j;
 
    if ((x+dx)>0 && (x+dx)<nc) nc= x+dx;
    nc*= image->nChannels;
    if (x>0 && x<nc) i0= x*image->nChannels;
    i= i0;
 
    nch= image->nChannels;
  }
 
  /* has next ? */
  bool operator!() const { return j < nl; }
 
  /* next pixel or next color component */
  IplImageIterator& operator++() {
      i++;
    if (i >= nc) {
            i=i0;
            j++;
            data+= step;
      }
    return *this;
  }
 
  const IplImageIterator operator++(int) {
      IplImageIterator<PEL> copy(*this);
      ++(*this);
      return copy;
  }
 
  IplImageIterator& operator+=(int s) {
      i+= s;
    if (i >= nc) {
            i=i0;
            j++;
            data+= step;
      }
    return *this;
  }
 
  /* pixel access */
  PEL& operator*() {
        return data[i];
  }
 
  const PEL operator*() const {
        return data[i];
  }
 
  const PEL neighbor(int dx, int dy) const {
        return *(data+dy*step+i+dx*nch);
  }
 
  PEL* operator&() const {
        return data+i;
  }
 
  /* current pixel coordinates */
  int column() const {
        return i/nch;
  }
 
  int line() const {
        return j;
  }
};
 
template <class PEL>
class IplMultiImageIterator {
 
  int i, j, i0;
  PEL** data;
  int step;
  int nl, nc;
  int nch;
  int nimages;
 
 public:
 
  /* constructor */
  IplMultiImageIterator(IplImage** images, int n,
     int x=0, int y=0, int dx= 0, int dy=0) :
       i(x), j(y), i0(0) {
    
      nimages= n;
 
      data= new PEL*[nimages];
      for (int i=0; i<nimages; i++) {
 
            data[i]= reinterpret_cast<PEL*>(images[i]->imageData);
      }
 
    step= images[0]->widthStep / sizeof(PEL);
      CvRect rect= cvGetImageROI(images[0]);
 
      nl= rect.height;
      nc= rect.width;
      x+= rect.x;
      y+= rect.y;
 
    if ((y+dy)>0 && (y+dy)<nl) nl= y+dy;
    if (y<0 || y>=nl) j=0;
 
      for (int i=0; i<nimages; i++) {
 
            data[i]+= step*j;
      }
 
    if ((x+dx)>0 && (x+dx)<nc) nc= x+dx;
    nc*= images[0]->nChannels;
    if (x>0 && x<nc) i0= x*images[0]->nChannels;
    i= i0;
 
    nch= images[0]->nChannels;
  }
 
  ~IplMultiImageIterator() {
 
        delete[] data;
  }
 
  /* has next ? */
  bool operator!() const { return j < nl; }
 
  /* next pixel or next color component */
  IplMultiImageIterator& operator++() {
      i++;
    if (i >= nc) {
            i=i0;
            j++;
 
            for (int i=0; i< nimages; i++) {
 
                  data[i]+= step;
            }
      }
    return *this;
  }
 
  const IplMultiImageIterator operator++(int) {
      IplImageIterator<PEL> copy(*this);
      ++(*this);
      return copy;
  }
 
  IplMultiImageIterator& operator+=(int s) {
      i+= s;
    if (i >= nc) {
            i=i0;
            j++;
 
            for (int i=0; i<nimages; i++) {
                  data[i]+= step;
            }
      }
    return *this;
  }
 
  /* pixel access */
  PEL& operator[](int n) {
        return data[n][i];
  }
 
  const PEL neighbor(int n, int dx, int dy) const {
        return *(data[n]+dy*step+i+dx*nch);
  }
 
  /* current pixel coordinates */
  int column() const {
        return i/nch;
  }
 
  int line() const {
        return j;
  }
};
An iterator of this type can be declared by specifying the type of the pixels in the image and by giving a pointer to the IplImage as argument to the iterator constructor, e.g.:
  
IplImageIterator<unsigned char> it(image);
Once the iterator constructed, two operators can be used to iterate over an image. First the ! operator allows to determine if we reach the end of the image and the * operator that give access to the current pixel. A typical loop will therefore look like this:
  
while (!it) {      
  if (*it < 50) {
 
    *it= 0xFF;   // 255
  }
  ++it;
}
Note that if the image contains more than one channel, each iteration will give access to one of the channel of a pixel. This means that in the case of a color pixel, you have to iterate three times for each pixel. In order to access all components of a pixel, the operator & can be used. This one returns an array that contains the current pixel channel values. For example, the previous color reduction example will look like this (note how the iterator is incremented this time to make sure that we go from one pixel to another):
  
      void ColorProcessor::
process(IplImage *image) {
 
  IplImageIterator<unsigned char> it(image);
  unsigned char* data;
 
  while (!it) {
 
     data= ⁢ // get pointer to current pixel
 
       data[0]= data[0]/div * div + div/2;
       data[1]= data[1]/div * div + div/2;
       data[2]= data[2]/div * div + div/2;
 
     it+= 3; // next pixel
  }
}
The use of image iterators is as efficient as directly looping with pointers. This is true as long as you set the compiler to optimize for speed (Project|Properties|C++|Optimization). By default, there is no optimization in Debug mode and the code is optimized for speed in Release mode.

When the processing involves more than one image, more than one iterator can be used. This is illustrated in the following example (a Sobel edge detector):

  
      void Sobel::
process(IplImage *image) {
 
  IplImageIterator<unsigned char>
      src(image,1,1,image->width-2,image->height-2);
  IplImageIterator<unsigned char>
      res(output,1,1,image->width-2,image->height-2);
 
  int sobel;
 
  while (!src) {
 
        sobel= abs(src.neighbor(-1,-1) -  src.neighbor(1,-1) +
                2*src.neighbor(-1,0) - 2*src.neighbor(1,0) +
                    src.neighbor(-1,1) -   src.neighbor(1,1)
              );
 
        sobel+=abs(src.neighbor(-1,-1) -  src.neighbor(-1,1) +
                2*src.neighbor(0,-1) - 2*src.neighbor(0,1) +
                    src.neighbor(1,-1) -   src.neighbor(1,1)
              );
 
        *res= sobel > 255 ? 255 : sobel;
 
        ++src;
        ++res;
  }
}
Since the processing here involves not only the current pixel but also its neighbors, the neighbor method defined by the iterator is used. Also, in this case, a window is specified when creating the iterator (here it defines a 1-pixel strip around the image where no processing is undertaken); this is required since otherwise the neighbors of the first pixels would fall outside of the image. The resulting image is:

Check point #5: source code of the above example.

Top of the page

 

(c) Robert Laganiere 2011