Can't figure how to get WORLD position of finger in Unity

[panosdrak]

Hello

I have been trying for many days how to get the world position of , for example, a finger. To make things clear, i know how to get the position of a finger, palm, etc. but it is RELATIVE to the Transform of LeapHandController.

This is also stated clearly in the following link of your documentation: https://developer.leapmotion.com/documentation/orion/unity/api/Leap_Classes.html

("....The LeapProvider converts the coordinates from Leap Motion space to Unity space and transforms them so that the hands appear in Unity relative to the GameObject to which the LeapProvider is attached (typically the LeapHandController)....")

The problem is, my LeapHandController is attached to an Oculus Rift, thus its transform changes, and as a result the position of fingers returned from LeapServiceProvider , depends on where the Oculus is and points to.

I searched many times and i cant quite find what i am looking for... I tried many methods, such as using TransformPoint, but didnt work...

Thank you...

Just in case, i get the pos of a finger using LeapServiceProvider and this line of code among others:
unityFingerPosition = hand.Fingers[1].TipPosition.ToVector3();

[Apidcloud]

Aren't there any methods like 'ToUnityScale' ?

Check this topic: https://community.leapmotion.com/t/alternatives-to-interaction-box-coordinates/2746/5

[Joe_Ward]

That position IS the world position. The coordinates are transformed into Unity world coordinates relative to the LeapHandController object. This setup assumes that the physical Leap device is also mounted on the HMD, so the Leap Motion coordinates of a hand will change when you move your head. The Unity world coordinates should remain stable for a non-moving hand -- but only because the position is transformed relative to the in-game hand controller object.

If your Leap device isn't mounted on an HMD, then you should detach the LeapHandController from the camera and place it at the position and orientation in the world that corresponds to its physical location.

Note, if what you are looking for are the unmodified Leap Motion coordinates, then you can use the Leap.Controller.Frame() method. These coordinates will still be in a scale of 1=1mm and use the right-handed axes convention (vs. Unity's left-handed convention). You can get the Controller using LeapServiceProvider.GetLeapController() or just create your own.

[panosdrak]

Thanks for your reply...

So to sum it up, (yes leap device is mounted on hmd), in order to get stable coords for a non-moving hand (regardless of the hmd movement), if i understand correctly, i need to do something like this??

Leap.Vector position = finger.TipPosition;
Vector3 unityPosition = position.ToUnityScaled(false);
Vector3 worldPosition = handController.transform.TransformPoint(unityPosition);

supposing that handController.transform represents the transform of the Camera because of the hmd movement/rotation and of course that LeapHandController is a child of my Unity First person camera in the hierarchy?

(i think i tried something like this and didnt work - that is why i am asking)

[Joe_Ward]

No, assuming that you are getting the Finger from LeapProvider.CurrentFrame, it should be just:

Leap.Vector position = finger.TipPosition;

[panosdrak]

i have tried that one also, i tried it again, and it doesnt work... (non-moving finger coords change as hmd moves)

 foreach (Hand hand in provider.CurrentFrame.Hands)
 {
                    if (hand.IsRight)
                    {
                        Leap.Vector LeapFingerPosition = hand.Fingers[1].TipPosition;
                        Debug.Log(LeapFingerPosition.toVector3());
                     
                    }
   }

I dont know if i am missing something, but what you mentioned clearly doesnt work @Joe_Ward

In other words, i use a lineRenderer to draw a straight line from a fixed point to the LeapFingerPosition point in 3d space, just to check what is happening, and what i am seeing is that the line sometimes is close to the visible finger tip, and sometimes(depending on hmd movement) is very far away....

[Joe_Ward]

Are you using the stock LMHeadMountedRig?

[panosdrak]

i guess yes since i followed the original instructions.. this is my hierarchy in unity

[Joe_Ward]

We were able to identify a bug that resulted in the temporal warping code not doing anything. The work-around is to go to the LeapServiceProvider script on the LeapHandController prefab and set OverrideDeviceType to true and override with to "Peripheral":

That said, there are other possible problems with adding the LMHeadMountedRig to a scene with a FirstPersonCharacter -- though maybe you have already taken this into consideration. Namely, the LMHeadMountedRig already contains a camera and bases its temporal warping off that camera. I suspect that Unity's VR support uses the first camera in the scene -- which means that the Leap code is getting the wrong camera information.

[panosdrak]

my settings are already like that..

So in my case, taking everything into account, what is he most viable solution?

[Joe_Ward]

I guess the next step would be to determine whether attaching the LMHeadMountedRig to a FPSController contributes to the issue. Do you see the problem in one of our VR example scenes?

[ChrisMa]

Same problem for the coordinates on my side. I recieve data, but only relative to the Leap not to the world coordinatesystem. Did you find a solution?