Add an offset to the Leap hands independently

[rognod]

Hey,

I have a pointcloud from the Kinect in VR that represents the entire body of the user. I need to match the two Leap hands to the pointcloud hands. I already know the offset vector for each hand, the only thing left is actually shifting the Leap hands to the correct position. I've tired different variants.
(When I say interaction hand, I mean the debug representation of the hand that is drawn when the "Draw Controller Runtime" checkbox on the Interaction Manager is checked.)

1. The LMHeadMountedRig can be translated, but it effects both hands, because they are both children of the Rig which is the Tracking Anchor (defined in the Leap VR Temporal Warping Script). I created a second Rig and just assigned the Hand from the new rig to the old LeapHandController. The graphical representation of the hand is shifted correctly, however the interaction hand remains at the old position. I would've loved this version as it would've been much easier.

2. Another variant, that almost works is to update 3 methods from the scripts. First in HandModel.SetLeapHand. Basically like this:

   LeapTransform offset = new LeapTransform(new Vector(offsetVector.x, offsetVector.y, offsetVector.z), LeapQuaternion.Identity);
   Hand offsethand = hand.TransformedCopy(offset);
   hand_ = offsethand;

Also InteractionManager.refreshInteractionControllers

    foreach (var controller in tempControllers) {
                InteractionHand curr = controller as InteractionHand;
                if (curr != null && curr.handDataMode == HandDataMode.PlayerLeft)
                {
                    controller.transform.Translate(offsetVectorLeft);
                }
                else if (curr != null && curr.handDataMode == HandDataMode.PlayerRight)
                {
                    controller.transform.Translate(offsetVectorRight);
                }

                _interactionControllers.Add(controller);
        }

and finally InteractionController.updateContactBone

        InteractionHand hand = this as InteractionHand;
        if (hand.handDataMode == HandDataMode.PlayerLeft)
        {
            targetPosition += offsetVectorLeft;
        }
        else if (hand.handDataMode == HandDataMode.PlayerRight)
        {
            targetPosition += offsetVectorRight;
        }

Now the graphical and the "interaction" hands are shifted correctly, but the interaction itself seems to be broken. I am not sure if it's because of my changes though.
I was wondering if there is a better or easier way of doing this and one that does not break interaction or require me to change the existing code.

Thanks for any input,
Rognod

[rognod]

Alright then, after another few days trying every last possible solution, I found something that works.
Instead of applying my offset after the Leap Scripts have done their work, I apply it in the Frames of the LeapServiceProvider. Thanks to these guys, for giving me that idea.

To anyone looking for a solution, here's how I did it.
In LeapServiceProvider.transformFrame after dest.CopyFrom(source).Transform(leapTransform); insert something like this

List<Hand> newHands = new List<Hand>();
foreach (var hand in dest.Hands)
        {
        
            if (hand.IsLeft)
            {
                Vector3 off = offsetVectorLeft;
                newHands.Add(hand.TransformedCopy(new LeapTransform(new Vector(off.x, off.y, off.z), LeapQuaternion.Identity)));
            }

            if (hand.IsRight)
            {
                Vector3 off = offsetVectorRight;
                newHands.Add(hand.TransformedCopy(new LeapTransform(new Vector(off.x, off.y, off.z), LeapQuaternion.Identity)));
            }
        }

        dest.Hands = newHands;

Hope it helps someone.
Rognod