I am using Leap Motion to track player hands and also Vive Trackers for real objects which player is able to pick up and use. The issue here that Leap Motion hands don't fully match real hands and also they have some tracking issues periodically and sometimes loose tracking. So it looks weird when the player holds real objects in real hands but there are no virtual hands or they don't match e.g. grip a little bit.
I want to attach virtual object grip of the real tracked object to the player virtual hand when the player picks it up. Also when the player releases the tracked object it should be detached from virtual hands.
It seems possible to compare Leap Motion hands position and the tracked object position in the background and attach if they are close enough and the tracked object starts moving. But it's not so easy to detect when they should be detached. Do you have any suggestion or might be the ready solution?