Leap for Unity OpenXR

Tags: #<Tag:0x0055d5a4a26870>

XenotechSrl 2021-03-11 19:19:46 UTC #1

Hello everyone,
We're working on getting different hardware pieces (HMDs, controllers) to work with the same code base using OpenXR in Unity (with Unity OpenXR plugin for the XR plug-in Management + Unity XR Interaction Toolkit), and for the most part it's working ok.
What's lacking is the support for hand tracking outside Hololens. We've been trying to get the Leap Motion to work with the Microsoft Mixed Reality OpenXR plugin (as suggested here), but only the rendering works. There's no support for the actual "actions" as implemented in the XR Interaction Toolkit. Also, the branch "OpenXRProvider" on Git (this one) is too old for the new OpenXR plugin and does not work.
Is there any plan to work on/release some Leap Motion OpenXR modules compatible with the new Unity ecosystem?
We might write an abstract layer to cope with this, but it would be great to just be able to use the XR Interaction Toolkit out of the box and have the Leap Motion react to actions / interactable objects.
Thanks!

dankirby 2021-03-12 17:03:20 UTC #2

The Microsoft interactions are driven by an OpenXR extension that's specific to MSFT, so we don't support it and don't currently have any plans to.

Unity's XR Interaction Toolkit doesn't currently support hands, but it's on their roadmap as "under consideration". You can vote on this issue to make Unity aware of how important it is here.

To build interactions around hands with OpenXR the easiest way currently is to use the OpenXRProvider and connect that up to MRTK or the Interaction Engine.

We have tested the provider with v0.1.4 of the plugin and were able to get this working so we suspect there is something else going on here to cause this issue. Please can you send me a screenshot of the exact errors that you are seeing to support@ultraleap.com so I can investigate further?

Please note that the branch on GitHub does not contain the OpenXR plugin package. You will need to add this yourself to get this to work. There are instructions in the readme, but it does mean that when you open the project you will get errors.

XenotechSrl 2021-03-15 10:49:59 UTC #3

Hi @dankirby, thanks for the reply.
Yes we already voted the issue, but unfortunately it looks like it's not being considered too much (it was created jul 2020 if i'm not mistaken, but it still sits there).
Using MRTK is not an option for us.

That sounds interesting, i'll send a screenshot as soon as possible.
The setup was Unity 2020.2, OpenXR plugin, WMR plugin 0.1.4.