Hello,
I need to extract a rectangle just with the hand from the camera images.
To achieve that, I'm projecting the tip position from the fingers with the warp function ( https://developer.leapmotion.com/documentation/python/api/Leap.Image.html?proglang=python#Leap.Image.warp ) over the raw image camera, then planned to take the bounding box and done. It worked really fine before I un-distort the image.
But after I corrected the distortion on the image, those coordenates were no longer precise. I'm using the source in https://developer.leapmotion.com/documentation/python/devguide/Leap_Images.html?proglang=python#correction-using-bilinear-interpolation to un-distort the image (using cv2.remap), so my best guess was that I should keep correcting the image this way, and un-distort only the 6 dots that I stored before, corresponding to the fingers and palm.
For this, I used the code that Joe_Ward provided in this other topic https://community.leapmotion.com/t/python-un-distort-raw-images/3065/3 , with some minor adaptations, and it seemed to work very well... but the positions are a bit off:
The red and blue dots are the same as the previous ones, I just drew them before correct the image, and un-distort them together with the rest of it. The white ones are the dots that I un-distorted apart and then drew over.
I am obviously doing something very wrong, and as draw the coords over the image are pretty simple, my guess is that the un-distort is going wrong somehow. I'm pasting the code from the relevant function here:
def undistort(leap_image, lista_pontos):
import math
distortion_buffer = leap_image.distortion
distortion_width = leap_image.distortion_width
width = leap_image.width
height = leap_image.height
corrected_width = 640
corrected_height = 240
lista_corrigida = np.zeros([0, 1, 2], dtype=np.ubyte)
for p in lista_pontos:
px = p[0][0]
py = p[0][1]
# Calculate position in the calibration map
calibrationX = 63.0 * px / corrected_width
calibrationY = 62.0 * (1.0 - (py* 1.0)/corrected_height)
# Fractional part to be used as weighting in bilinear interpolation
weightX = calibrationX - math.trunc(calibrationX)
weightY = calibrationY - math.trunc(calibrationY)
# Get x,y coordinates of the closest calibration map points to the target pixel
x1 = math.trunc(calibrationX)
y1 = math.trunc(calibrationY)
x2 = x1 + 1
y2 = y1 + 1
# Find x,y grid coordinates from distortion buffer
dX1 = distortion_buffer[x1 * 2 + y1 * distortion_width]
dX2 = distortion_buffer[x2 * 2 + y1 * distortion_width]
dX3 = distortion_buffer[x1 * 2 + y2 * distortion_width]
dX4 = distortion_buffer[x2 * 2 + y2 * distortion_width]
dY1 = distortion_buffer[x1 * 2 + y1 * distortion_width + 1]
dY2 = distortion_buffer[x2 * 2 + y1 * distortion_width + 1]
dY3 = distortion_buffer[x1 * 2 + y2 * distortion_width + 1]
dY4 = distortion_buffer[x2 * 2 + y2 * distortion_width + 1]
# Bilinear Interpolation of target pixel
dX = dX1*(1.0 - weightX)*(1.0 - weightY) + dX2*weightX*(1.0 - weightY) + dX3*(1.0 - weightX)*weightY + dX4*weightX*weightY
dY = dY1*(1.0 - weightX)*(1.0 - weightY) + dY2*weightX*(1.0 - weightY) + dY3*(1.0 - weightX)*weightY + dY4*weightX*weightY
# Reject points outside [0..1]
if (0 <= dX <= 1) and (0 <= dY <= 1):
# Denormalise points from [0..1] to [0..width] or [0..height]
denormalisedX = width - math.trunc(dX * width)
denormalisedY = math.trunc(dY * height)
else:
print(dX, dY)
denormalisedX = 0
denormalisedY = 0
lista_corrigida = np.append(lista_corrigida, [[[denormalisedX, denormalisedY]]], axis=0)
return lista_corrigida
I realised that dX and dY get out of bounds (above 1 or bellow 0) in places where they should work fine (a bit bellow half the image and dY are already above 1, for example), so I guess this is another sign that the problem is here.
This is my first project with python, so sorry for the poor code. I can provide the code of the other functions if you guys believe they may be useful. But as the process work before the un-distort, I really think the problem is in this undistort function.
Is there something wrong in the function? Or maybe I should try another approach to this problem?
Thanks in advance.
