Get the undistorted images from leap

[myfirstleapapp]

Hello everyone, I'm developing an app that needs the leap motion undistorted images. As you can see from the image, I cannot get the rectified image which is useful for the image processing I will develop later by using OpenCV. I've tried to use the code found on the https://developer.leapmotion.com/documentation/cpp/devguide/Leap_Images.html page but I still cannot figure out how to get the "planar" image on the screen.
This is the code I've used to get the image (which is basically the same you can find on the Leap Motion developer page).
Anyone could help me to get out this problem? I'm using Visual Studio 2013 and C++ as programming language.

//Define the destination variables for the image processing


    Mat imgCorrected1(destinationHeight, destinationWidth, CV_8UC1);
    Mat imgCorrected2(destinationHeight, destinationWidth, CV_8UC1);


    controller.setPolicy(Leap::Controller::POLICY_IMAGES);

    ImageList images = controller.images();
    for (int img = 0; img < 2; img++){
        Image image = images[img];
        
        const unsigned char* raw = image.data();
        const float* distortion_buffer = image.distortion();

        //Local variables for values needed in loop
        const int distortionWidth = image.distortionWidth();
        const int width = image.width();
        const int height = image.height();

        for (i = 0; i < destinationWidth; i++) {
            for (j = 0; j < destinationHeight; j++) {
                //Calculate the position in the calibration map (still with a fractional part)
                calibrationX = 63 * i / destinationWidth;
                calibrationY = 62 * (1 - j / destinationHeight); // The y origin is at the bottom
                //Save the fractional part to use as the weight for interpolation
                weightX = calibrationX - truncf(calibrationX);
                weightY = calibrationY - truncf(calibrationY);

                //Get the x,y coordinates of the closest calibration map points to the target pixel
                x1 = calibrationX; //Note truncation to int
                y1 = calibrationY;
                x2 = x1 + 1;
                y2 = y1 + 1;

                //Look up the x and y values for the 4 calibration map points around the target
                dX1 = distortion_buffer[x1 * 2 + y1 * distortionWidth];
                dX2 = distortion_buffer[x2 * 2 + y1 * distortionWidth];
                dX3 = distortion_buffer[x1 * 2 + y2 * distortionWidth];
                dX4 = distortion_buffer[x2 * 2 + y2 * distortionWidth];
                dY1 = distortion_buffer[x1 * 2 + y1 * distortionWidth + 1];
                dY2 = distortion_buffer[x2 * 2 + y1 * distortionWidth + 1];
                dY3 = distortion_buffer[x1 * 2 + y2 * distortionWidth + 1];
                dY4 = distortion_buffer[x2 * 2 + y2 * distortionWidth + 1];

                //Bilinear interpolation of the looked-up values:
                // X value
                dX = dX1 * (1 - weightX) * (1 - weightY) +
                    dX2 * weightX * (1 - weightY) +
                    dX3 * (1 - weightX) * weightY +
                    dX4 * weightX * weightY;

                // Y value
                dY = dY1 * (1 - weightX) * (1 - weightY) +
                    dY2 * weightX * (1 - weightY) +
                    dY3 * (1 - weightX) * weightY +
                    dY4 * weightX * weightY;

                // Reject points outside the range [0..1]
                if ((dX >= 0) && (dX <= 1) && (dY >= 0) && (dY <= 1)) {
                    //Denormalize from [0..1] to [0..width] or [0..height]
                    denormalizedX = dX * width;
                    denormalizedY = dY * height;

                    //look up the brightness value for the target pixel
                    destination[i][j] = raw[denormalizedX + denormalizedY * width];
                }
                else {
                    destination[i][j] = -1;
                }
            }
        }
        if (img == 0) {
            for (int i1 = 0; i1 < destinationHeight; i1++)
                for (int j1 = 0; j1 < destinationWidth; j1++)
                    imgCorrected1.at<unsigned char>(i1, j1) = destination[j1][i1];
        }
        else {
            for (int i1 = 0; i1 < destinationHeight; i1++)
                for (int j1 = 0; j1 < destinationWidth; j1++)
                    imgCorrected2.at<unsigned char>(i1, j1) = destination[j1][i1];
        }        
    }

Thank you.

[Joe_Ward]

I still cannot figure out how to get the "planar" image on the screen.

I'm not sure I understand your question. The image shown above looks like the undistorted image. Is that an example of what you want and aren't getting or it IS what you are getting and you want something different?

[myfirstleapapp]

Hi Joe, thank you for your answer. I've maybe used an incorrect term. For undistorted image I mean the one that fill the window without having those "rounded" borders. Something like this one http://oi45.tinypic.com/2w6fdih.jpg

I've tried to pass the (unsigned char*)image.data() to openCV (Mat myOpenCVimg.data) but it doesn't pass anything.
Thank you

[Joe_Ward]

The camera lenses have fairly large distortion, so when you correct that, you no longer have a rectangular image. You could either crop the result or not apply the undistortion procedure in the first place.

image.data() is a simple array. I'm not sure what your OpenCV function is expecting. At worst case, you could copy the elements from the data array to one that OpenCV likes.

[myfirstleapapp]

Ok, Joe. Thank you very much for your help. I will probably use the first solution: crop the image and use the rectangular part of it. I will post the code and the results I will have once completed this task.

[joonthepark]

Where is the location of the code you wrote?
Is it inside this function?:

void SampleListener::onFrame(const Leap::Controller& controller) {
...
}