Leapbone local orientation

[jeff_dauphin]

Hello !

For a project using Objective-C and OpenGL, I need to modelize and animate a hand.

First step : hand position and orientation

For the position, I put the hand.palmPosition values in a glTranslatef() function :

glTranslatef(hand.palmPosition.x, hand.palmPosition.y, hand.palmPosition.z);

Then, for the orientation, I create a quaternion from the pitch and yaw values of the hand.direction
and the roll value of the hand.palmNormal. And I create the associated rotation matrix.

[quat setWithPitchY: hand.direction.yaw rollX hand.direction.pitch yawZ: hand.palmNormal.roll];
Matrix *mat = [[Matrix alloc] initWithQuaternion: quat];

nota : my quaternion equation are not with the same axis than the Leap Controller axis, that’s why I put the yaw in the pitch, the pitch in the roll, etc…

Then I multiply my matrix to the current matrix with the glMultMatrixf() function :

glMultMatrixf([mat floatMatrix]);

Finally, I draw a cube and two vectors representing the direction and the normal.

It works perfectly !

Second step : proximal bone position and orientation

I choose an arbitrary position for the proximal bone base joint :

glTranslatef(-0.5, 0, -0.5);

From here I don’t know what to do. The SDK don’t give access to the pitch ans yaw of the bone in the local axis (ie relative to the hand axis). The only data is the bone.direction vector represented in the leap motion Controler axis, not in the local axis of the hand.

Does somebody have an idea ?

Thanks in advance.

[PLeoncini]

As far as I understood, your problem is world vs. local - if so, I see two approaches:

1. as long as your task is drawing, you could draw every hand part (arm, palm, finger phalanxes) in world coords, hust push/pop matrices before/after glMultMatrix. The .basis() gives you the absolute orientation matrix of that part, for each of the #17 parts of the hand;
2. if you want to work in local coords (relative to the palm), just premult the .basis() by the inverse of the palm basis so to make it relative to the palm, i.e.
   In form of C++ code, by memory:

Matrix M_palm_world_inv = hand.basis().rigidInverse();
const FingerList fingers = hand.fingers();
for (int f = 0; f < 5; f++)  // fingers
  for (int p = 0; p < 3; p++)  // phalanxes
  {
      Matrix M_plnx_world = fingers[f].bone(p).basis(),
             M_plnx_local = M_palm_world_inv * M_plnx_world;
      // use M_plnx_local
      // ...
  }

Finally,

I choose an arbitrary position for the proximal bone base joint :

You don't need to invent the root of each finger, the LM SDK supplies it:

fingers[f].bone(Bone::TYPE_PROXIMAL).prevJoint(), f=1..5

[jeff_dauphin]

Thank you very much PLeoncini !

Your method to work in local coords works perfectly, except for a little bug : the axis of the phalange matrix are inverted. Then when I draw the phalange, it points in the wrong direction. It's not a problem of your method. It's a know bug of the Leap Motion. See link :

LeapBone direction 180º bigger than LeadFinger direction

V2 Tracking page :
V2.3.1 Release Notes
Known Issues (General)
Bone direction vector points in the wrong direction. It should point toward the distal end of the bone.

Do you know a method to invert the three axis of a matrix : -x--> x, -y --> y, -z --> z

Thanks

[PLeoncini]

Jeff,

thanks for pointing out the problem.

Do you know a method to invert the three axis of a matrix : -x--> x, -y --> y, -z --> z

Given a matrix M, I'd say:

Matrix M_inverted_axes = Matrix( -M.xBasis(), -M.yBasis(), -M.zBasis() );

[jeff_dauphin]

Hello PLeoncini,

I've tried adapting your code in objective-C. I notice that as M.xBasis() is a LeapVector you can't negate it directly. You need to do more step to achieve the result :

LeapVector *xBasis = [[LeapVector alloc] initWithX:proximalLocalMatrix.xBasis.x y:proximalLocalMatrix.xBasis.y z:proximalLocalMatrix.xBasis.z];
LeapVector *yBasis = [[LeapVector alloc] initWithX:proximalLocalMatrix.yBasis.x y:proximalLocalMatrix.yBasis.y z:proximalLocalMatrix.yBasis.z];
LeapVector *zBasis = [[LeapVector alloc] initWithX:-proximalLocalMatrix.zBasis.x y:-proximalLocalMatrix.zBasis.y z:-proximalLocalMatrix.zBasis.z];

LeapMatrix *proximalLocalMatrixInvertedAxis = [[LeapMatrix alloc] initWithXBasis:xBasis yBasis:yBasis zBasis:zBasis origin:[LeapVector zero]];

As you can see, I only had to negate the zBasis vector to obtain the good behavior. Why ? I don't know.

Thanks for your help.