Understanding where to get positions

[rgaldieri]

Hello there!

I am currently trying to create my first demo with the Leap Motion in Unity, where users can grab capsules and press a button on top of them with the same hand. Unfortunately, no combination of grasp and contact worked for my purpose, and I was forced to implement a simil-anchor structure that allows those capusles to be attached to the hand with a simple gesture (thumb up and 4 fingers closed, similar to the ok gesture).

After experimenting for some days I can say that I don't quite understand the way data is make accessible to programmers, especially relative to coordinates and finger positions.

After digging a lot in the documentation, I found two different ways to access finger positions:

1) Using the RiggedHand component, I can access FingerModel references, and from there directly access the Finger's position in world coordinates and many other informations that I might need to make my calculations.

2) By accessing the Finger instances from the Hand classes. This second method seem to require way more work to obtain the same data, and methods such as TipPosition can be converted in world space coordinates only relative to the LeapHandController, not to the unity World space.

I am quite lost here, even though I can easily access the RiggedHand by reference in my component, I believe there must be something I am missing from the Hand class. What am I doing wrong? Also, how do I derive the TipPosition in world coordinates from the LeapHandController system?

I know this post could be a little confused, but I am new to the Leap and I am doing my best to understand it

[nbenson]

If I'm writing a custom script and want quick access to Hand data in world space, I'll just do something like this:

var hand = Hands.Left;
var indexPosition = hand.Fingers[1].TipPosition.ToVector3();

This returns the index finger tip position in Unity world space. Does this method work for your use-case?

For what it's worth, we're aware of the difficulties related to our original Provider -> Pool -> HandModel -> Hand data workflow, and we've got changes coming that will aim to make the pipeline simpler and easier to understand.

[rgaldieri]

I'm sorry it took me this long to answer but I have been out of office. Yes, that type of script works well when I want to have a single finger tip position, but I am trying to implement complex behaviour such as object grabbing without using the thumb, and that is hard without knowing each single bone position and rotation. I am currently using hand.GrabAngle value to have an average value of how the hand is behaving, but that's temporary and not as precise as calculating each single finger's bone angle.

Anyway, the second part of your message is exactly what I was trying to understand, I really look forward to see future improvements on this side

[nbenson]

In the mean time, I will say that Leap.Hand objects should already contain all of the data you're looking for.

When you get a Hand object through e.g. var hand = Hands.Get(Chirality.Left), you will get a Hand object that is in Unity world space. From there, there's a lot of relevant data you can access:

Finger thumb = hand.Fingers[0], index = hand.Fingers[1]; // etc.
var indexMetacarpalBone = index.bones[0];
var indexProximalBone = index.bones[1];
var indexCurlAngle = Vector3.Angle(indexMetacarpalBone.Direction.ToVector3(),
                                   indexProximalBone.Direction.ToVector3());

Each finger has a "bones" array that contains Bone objects describing each bone of the finger. These bones have a Length, PrevJoint (bone base position), NextJoint (bone end position), Direction, etc.

The only caveats are:
(1) the data is in custom datatypes, e.g. Leap.Vector and Leap.Quaternion, so call e.g. ToVector3() or ToQuaternion() on them to convert them to the Unity-native datatype to use them with Unity methods
(2) The Thumb (anatomically) has one fewer Bone than the other fingers; we represent this by having its first-index Bone (at bones[0]) be a zero-length Bone, so be sure to skip that one if you're doing thumb-bone calculations.

If you aren't getting this data back in Unity world space, that sounds like a bug to me and we should repro and ticket it.