Adjusting Grasp Detection Distance / Too Sensitive Grasp Detection Issue?

Tags: #<Tag:0x007f86cc360218>

RowanW 2019-07-31 18:57:30 UTC #1

Hey there all,

I've been working on a project and have run into an issue I'm fairly certain wasn't always present - the grasp detection distance seems to be incredibly high/sensitive, as in, I can hold my hand in just about a loose 'C' shape, and that registers as a grasp whereas it should/used-to be activated when my index finger and thumb are little under 2 inches apart (for reference, the loose 'C' shape equates to around 5+ inches between thumb and index finger tips).

EDIT: This is a gif from a more recent iteration of the project, no changes I can think of to anything regarding the Leap Motion integration, but it seems like the gap has decreased to maybe 3 or 4 inches between the thumb and index finger tips - so it SEEMS better, but also don't know why, especially with this onset issue that it might be changing on it's own over time (perhaps more hardware related than the software...?)

My question is in regards to adjusting this detection distance, however, after LOTS of digging through the source, I can't seem to find where this may be set - the closest I've seem to have gotten is somewhere in HeuristicGrabClassifier.cs, but it's still unclear how this is all working. There doesn't seem to be anything too relevant in InteractionBehaviour.cs either...

I'm fairly certain it's not a calibration issue either, I can achieve a score of 90ish often after a fresh restart.

Is there some setting, field, inspector element, or something that I'm missing that may aid in this issue, or could there be something deeper wrong?

If there's anymore information that might help aid in sussing the issue, I can try my best to provide!

Thanks

RobMostyn 2019-08-08 15:15:22 UTC #2

I'm part of this team as well and am linking to a video of the issue: https://drive.google.com/file/d/1ydUZg5P7PRJhULY5TzicsoHo2tQ9fdjF/view?usp=sharing

The video also demonstrates how we can sort of "pre-grasp", move in to the object, and it will grasp the object. Our hope is that objects would only be grasped when the hand goes from not grasped to grasped while colliding with the object. This is sort of a secondary issue, but we believe it may be related.

We have two Leaps and both present the issue. We've cleaned them and tried different cords as well.

Appreciate any help!

AlexColgan 2019-08-13 20:26:34 UTC #3

Essentially there are two places where you can play around with different variables, one of which you've already identified. Your best bet would be to experiment with the different variables in the HeuristicGrabClassifier script (lines 34-44).

github.com

leapmotion/UnityModules/blob/master/Assets/LeapMotion/Modules/InteractionEngine/Scripts/Internal/HeuristicGrabClassifier.cs#L33


public InteractionHand interactionHand;


private Dictionary<IInteractionBehaviour, GrabClassifierHeuristics.GrabClassifier> _classifiers
  = new Dictionary<IInteractionBehaviour, GrabClassifierHeuristics.GrabClassifier>();
private GrabClassifierHeuristics.ClassifierParameters _defaultGrabParams, _scaledGrabParams;
private Collider[][] _collidingCandidates = new Collider[6][];
private int[] _numberOfColliders = new int[6];
private Vector3[] _fingerTipPositions = new Vector3[5];
private Vector3[] _fingerKnucklePositions = new Vector3[5];


public HeuristicGrabClassifier(InteractionHand intHand,
                               float fingerStickiness = 0F,
                               float thumbStickiness = 0.04F,
                               float maxCurl = 0.65F,
                               float minCurl = -0.1F,
                               float fingerRadius = 0.012F,
                               float thumbRadius = 0.017F,
                               float grabCooldown = 0.2F,
                               float maxCurlVel = 0.0F,
                               float grabbedMaxCurlVel = -0.025F,
                               float maxGrabDistance = 0.05F,

github.com

leapmotion/UnityModules/blob/master/Assets/LeapMotion/Core/Scripts/Utils/HandUtils.cs#L292


/// Returns whether this vector faces from a given world position towards another world position within a maximum angle of error.
/// </summary>
public static bool IsFacing(this Vector3 facingVector, Vector3 fromWorldPosition, Vector3 towardsWorldPosition, float maxOffsetAngleAllowed) {
  Vector3 actualVectorTowardsWorldPosition = (towardsWorldPosition - fromWorldPosition).normalized;
  return Vector3.Angle(facingVector, actualVectorTowardsWorldPosition) <= maxOffsetAngleAllowed;
}


/// <summary>
/// Returns a confidence value from 0 to 1 indicating how strongly the Hand is making a fist.
/// </summary>
public static float GetFistStrength(this Hand hand) {
  return (Vector3.Dot(hand.Fingers[1].Direction.ToVector3(), -hand.DistalAxis() )
        + Vector3.Dot(hand.Fingers[2].Direction.ToVector3(), -hand.DistalAxis() )
        + Vector3.Dot(hand.Fingers[3].Direction.ToVector3(), -hand.DistalAxis() )
        + Vector3.Dot(hand.Fingers[4].Direction.ToVector3(), -hand.DistalAxis() )
        + Vector3.Dot(hand.Fingers[0].Direction.ToVector3(), -hand.RadialAxis() )
        ).Map(-5, 5, 0, 1);
}


/// <summary>
/// Transforms a bone by a position and rotation.

RowanW 2019-08-16 14:52:07 UTC #4

Alrighty so, this definitely helped confirm the areas that we'd need to look into!

I had fiddled with those values a few weeks ago to no avail, but after trying again, it inexplicably worked somewhat this time - I only needed to modify maxCurl down from 0.65F to around 0.2F and it instantly felt much closer to what we are hoping for. With that said, I'm not sure what it's exact influence on the process is (I'm sure the other variables have a stake in how it all sort of comes together) and hope no issues stem from this simple change.

As for our other issue, which is when one can make a preemptive pinch gesture before moving into an Interactable Object which causes the object to become grasped - we ideally want the object to only become grasped if the hand was last known to be 'open' or not pinched before checking if the hand should grasp the object. I was hoping to find maybe a simple state property for the hand (something even as simple as 'open'/'closed', 'pinched'/'not pinched') for which we can override some of the default behavior when it checks for a grasp beginning, but I haven't been able to find anything like it.

Is there perhaps a known way to achieve this, or perhaps we can find some guidance on how to sort of roll our own implementation/safely modify the source?

Thanks for the help so far!

RobMostyn 2019-08-16 15:25:47 UTC #5

@AlexColgan Thanks for the help!

I've included a gif of the pre-grasp issue that Rowan mentions above.

AlexColgan 2019-08-16 15:38:52 UTC #6

You might be able to reduce the grasp volume? From the Interaction Engine FAQ:

If you need to be able to grasp a very small object and the object's physical colliders don't produce good grasping behaviors, try adding a new primitive collider for the object, such as a SphereCollider, with a larger radius than the object itself, and with isTrigger set to true. As long as the InteractionBehaviour is not ignoring grasping, you will be able to grasp the object using this trigger volume.

However, you don't want a grasping-only trigger collider to be too much larger than the object itself. In general, the larger the grasping volume is around an object, the more likely the user is to accidentally grasp objects when they don't intend to. Additionally, having overlapping grasping-only trigger colliders from multiple objects will prevent the grasp classifier from correctly picking which object to grasp.

The inverse is also true -- you could make a slightly smaller collider.

RowanW 2019-08-16 15:53:54 UTC #7

It's not so much that the grasp volumes are too big, it's just that one can pinch well away from the object, and moving into the object's grasp volume pre-pinched like this will grasp it when the behavior we're looking for is checking that the hand wasn't pinched upon entering the grasp volume as part of the requirements for successfully grasping something.