Implementing picking up small objects by enclosing hand around it

[ZipZap]

Hey everyone.

I've been using the interaction engine (1.0.1) with Unity in a project and I'm finding it generally handles pretty well with objects that are at least roughly as big as the hand. However, if the object is much smaller, the interaction engine seems to only register grasping if I pinch it with my fingers rather than grasp my hands around it. I guess with objects that are closer in size to the hand you're doing more of a pinching motion at that point. I'm guessing that this is just the way the interaction engine detects grasps, but it feels kind of natural (to me anyways) to want to enclose your hand around a small object to pick it up. (Images at the bottom)

But maybe I'm just not aware of some functionality that is already comes with LEAP. Can anyone point me in the right direction of how I could implement this? I would rather not have to modify the source, but if that's what's required then I'm interested.

The last attempt I did was detecting when the hand is closed (with ExtendedFinderDetector) and calling BeginGrasp and EndGrasp on the interaction object when needed. However, the held object wasn't moving along with the hand, although it moved to the proper layer (Interaction No Contact Layer). I'm guessing that I just need to do some other initialization along with calling BeginGrasp.

Any suggestions appreciated!

Pinching:

Enclosing Hand:

[Barrett-Fox]

We expect that what you're attempting here is a common use case. So thanks for the question.

Here's a solution:

• And a second Collider to the transform that has your
  InteractionBehavior component.
• Set this new Collider's IsTrigger value to True
• Make the size of this Collider larger than your object

The Interaction Engine will the check for grasping when your fingers are in this larger volume. But collisions for the object will still work with the original Collider. You can experiment easily with the size of this second trigger volume collider to tune your results.

[ZipZap]

Thanks Barret. I didn't realize that IsTrigger still allows interaction with the interaction engine.

Your suggestion does improve the handling, but it introduces a new issue. Since the trigger collider is bigger than the mesh itself, you can grab the object by the trigger collider and have the actual object itself appear floating outside of the hand. This isn't necessarily a major issue, but it would be nice to get around it.

Grasping by wrapping your hand around these smaller objects is still a bit of a hit or miss. A lot of the time the object falls out of the hand fairly quickly when it moves/rotates. For testing I was using two small cubes sizes with different trigger collider to actual collider ratios, where the actual size of each collider was approximately the same as the large cube collider. It seems that the closer the ratio of the trigger collider to the actual collider is to 1:1, the better it handles. Grasping events are also firing multiple times when I wrap my hand around it before it falls out or I drop it.

I also ended up getting my earlier attempt at this functioning, and it seems to handle ok. I think because it's relying on whether fingers are extended or not it doesn't have too many issues when hands are moved around holding an object. It's fairly hacky, but I added another method to InteractionController (I probably should have just made a class that implemented IInternalInteractionController instead) that allows an external caller to grasp any passed in InteractionBehaviour.

As a side note, it's very hard (but doable) for me to get a score > 80 when running the LEAP calibration. I also experience issues with fingers being inaccurate, especially when hands are more than 10 cm away from the sensor. Even though I don't hardly notice any finger inaccuracies when testing with the small blocks with my hand within 10 cm of the sensor, maybe there is still enough inaccuracy for it to think that grasping has ended?

I'll be playing with this a lot over the next week, so if I make any breakthroughs I'll be sure to update it here.

Thanks again for your help!