Modifying SteamVR Leap Driver for vive buttons triggered by keystroke not leap gestures

[davisam42]

Most of you are probably already aware of this project going on over HERE

However the Leap gesture triggers mapped to the emulated Vive wand buttons are super unreliable. I have done a bit of research and have found several possible solutions and eventually found this:

Comment from discussion Leap motion with job simulator.

I tried asking him for help, however since he hasnt been active in awhile, I thought i would take a stab at it.
I then was completely overwhelmed, I am not a programmer but i have a good intuition for reverse engineering but could still use some guidance..

Basically I started tearing into all the C++ files and seeing if anything referenced mapping gestures to actions like trigger presses on the vive controllers...
I found this in the driver_leap.cpp file:

 {
    vr::VRControllerState_t NewState = { 0 };

    bool handFound = false;
    GestureMatcher::WhichHand which = (m_nId == LEFT_CONTROLLER ) ? GestureMatcher::LeftHand :
                                      (m_nId == RIGHT_CONTROLLER) ? GestureMatcher::RightHand :
                                      GestureMatcher::AnyHand;

    float scores[GestureMatcher::NUM_GESTURES];
    handFound = matcher.MatchGestures(frame, which, scores);

    if (handFound)
    {
        // Changing unPacketNum tells anyone polling state that something might have
        // changed.  We don't try to be precise about that here.
        NewState.unPacketNum = m_ControllerState.unPacketNum + 1;

        // system menu mapping (timeout gesture)
        if (scores[GestureMatcher::Timeout] >= 0.5f)
            NewState.ulButtonTouched |= vr::ButtonMaskFromId(vr::k_EButton_System);
        if (scores[GestureMatcher::Timeout] >= 0.5f)
            NewState.ulButtonPressed |= vr::ButtonMaskFromId(vr::k_EButton_System);

        // application menu mapping (Flat hand towards your face gesture)
        if (scores[GestureMatcher::FlatHandPalmTowards] >= 0.8f)
            NewState.ulButtonTouched |= vr::ButtonMaskFromId(vr::k_EButton_ApplicationMenu);
        if (scores[GestureMatcher::FlatHandPalmTowards] >= 0.8f)
            NewState.ulButtonPressed |= vr::ButtonMaskFromId(vr::k_EButton_ApplicationMenu);

        // digital trigger mapping (fist clenching gesture)
        if (scores[GestureMatcher::TriggerFinger] > 0.5f)
            NewState.ulButtonTouched |= vr::ButtonMaskFromId(vr::k_EButton_SteamVR_Trigger);
        if (scores[GestureMatcher::TriggerFinger] > 0.5f)
            NewState.ulButtonPressed |= vr::ButtonMaskFromId(vr::k_EButton_SteamVR_Trigger);

        // grip mapping (clench fist with middle, index, pinky fingers)
        if (scores[GestureMatcher::LowerFist] >= 0.5f)
            NewState.ulButtonTouched |= vr::ButtonMaskFromId(vr::k_EButton_Grip);
        if (scores[GestureMatcher::LowerFist] >= 0.5f)
            NewState.ulButtonPressed |= vr::ButtonMaskFromId(vr::k_EButton_Grip);

        // touchpad button press mapping (Thumbpress gesture)
        if (scores[GestureMatcher::Thumbpress] >= 0.2f)
            NewState.ulButtonTouched |= vr::ButtonMaskFromId(vr::k_EButton_SteamVR_Touchpad);
        if (scores[GestureMatcher::Thumbpress] >= 1.0f)
            NewState.ulButtonPressed |= vr::ButtonMaskFromId(vr::k_EButton_SteamVR_Touchpad);

#if 0
        // sixense driver seems to have good deadzone, but add a small one here
        if (fabsf(cd.joystick_x) > 0.03f || fabsf(cd.joystick_y) > 0.03f)
            NewState.ulButtonTouched |= vr::ButtonMaskFromId(vr::k_EButton_StreamVR_Touchpad);
#endif

        // All pressed buttons are touched
        NewState.ulButtonTouched |= NewState.ulButtonPressed;

        uint64_t ulChangedTouched = NewState.ulButtonTouched ^ m_ControllerState.ulButtonTouched;
        uint64_t ulChangedPressed = NewState.ulButtonPressed ^ m_ControllerState.ulButtonPressed;

        SendButtonUpdates(&vr::IServerDriverHost::TrackedDeviceButtonTouched, ulChangedTouched & NewState.ulButtonTouched);
        SendButtonUpdates(&vr::IServerDriverHost::TrackedDeviceButtonPressed, ulChangedPressed & NewState.ulButtonPressed);
        SendButtonUpdates(&vr::IServerDriverHost::TrackedDeviceButtonUnpressed, ulChangedPressed & ~NewState.ulButtonPressed);
        SendButtonUpdates(&vr::IServerDriverHost::TrackedDeviceButtonUntouched, ulChangedTouched & ~NewState.ulButtonTouched);

        NewState.rAxis[0].x = scores[GestureMatcher::TouchpadAxisX];
        NewState.rAxis[0].y = scores[GestureMatcher::TouchpadAxisY];

        NewState.rAxis[1].x = scores[GestureMatcher::TriggerFinger];
        NewState.rAxis[1].y = 0.0f;

        // the touchpad maps to Axis 0 X/Y
        if (NewState.rAxis[0].x != m_ControllerState.rAxis[0].x || NewState.rAxis[0].y != m_ControllerState.rAxis[0].y)
            m_pDriverHost->TrackedDeviceAxisUpdated(m_unSteamVRTrackedDeviceId, 0, NewState.rAxis[0]);

        // trigger maps to Axis 1 X
        if (NewState.rAxis[1].x != m_ControllerState.rAxis[1].x)
            m_pDriverHost->TrackedDeviceAxisUpdated(m_unSteamVRTrackedDeviceId, 1, NewState.rAxis[1]);

        m_ControllerState = NewState;
    }

Hoping someone with more knowledge can show me how to edit the driver to map keyboard presses to triggers and buttons without using the leap drivers.
the source is HERE if you need more context.

Any help is appeciated!

THANKS!

[davisam42]

I noticed in the installation files the leap.vrsettings i can just change TriggerGesture to Pinch becasue in the GestureChecker.exe Pinch was way easier to read than TriggerGesture pulling a 90% success rate at many different angles.

However, upon saving the config and restarting and attempting to use that gesture for triggers, it didnt seem like it was working with a pinch. Im assuming this is due to the Driver cpp also having the event mapped to TriggerGesture as well.

So how would i edit the Driver.cpp and recompile source to install with the necessary edits made?

[lukeschaben]

Rip, i was thinking the same thing, the leap community is so small