Unreal Plugin Latency

[mxo]

Hi,
I tested some Demos and the Visualizer. The latency seemed to be a lot better than the results I get with the official Unreal Engine Plugin running in Unreal Engine 4.14 at 90fps.
Is this a know issue of the UE Plugin?
Was the UE Plugin updated to take advantage of Orions lower latency?

[AlexColgan]

The Unreal plugin hasn't yet been updated to take advantage of the new LeapC API, which is where the raw performance improvements in our Unity assets come from. However, C++ is still pretty close to the metal (compared to C# in Unity), so the difference should be minimal.

You may want to take a look at getnamo's first pass at implementing timewarp, which also affects the user experience.

[mxo]

Hi Alex,

Well, I hope the difference is not minimal, as the latency is huge.
I just recorded a video in which I hold a vive controller. The controller can be seen as reference as it has an unnoticable latency running at 90 fps.

So the video compares the latency of the vive controller to the latency of the leap hands as both are simultaneusly visible. At first the camera is held still and the hand is moved. In this case it is not very good, but acceptable. After that I hold my hand still and just move my head. Then you'll notice, it is way off. The heand easily drifts away 10cm and more from the controller. All in all it is a lot easier to see the difference in VR than in the screen recording. But at least it shows my problem.

So I am still wondering if the reason for the huge latency is just that it doesn't take advantage of the new new LeapC API, or if there is anything else, that can be done.
I haven't compared it myself with the same setup in Unity, but all demos I tried so far seemed to have less latency.

[AlexColgan]

The fact that there seems to be such a radical difference between camera-still/hand-moving and camera-moving/hand-still suggests that there is something else going on.

What's the offset between the virtual controller and the cameras within the scene? (Apologies that I don't know the exact terminology in Unreal.) If the offset is set incorrectly, it could explain the drifting behavior. It's important that the virtual controller be in the same position as the Leap Motion Controller is in the real world. For the Rift, this is 8cm. For the Vive, it should be 9.5 cm.

[Joe_Ward]

This can happen if the head-based transform used to convert the Leap coordinates into world space was captured at a different time than the capture time of the Leap frame. In Unity, we maintain a history of recent head transforms and use the one that most closely matches the frame time to avoid this swimming effect. This is the time warp feature, which Alex mentions above.

[mxo]

Hi,
thanks for your answers!
I checked it and the pre-installed plugin version that shipped with unreal engine 4.14 is 2.0.0. So right before timewarp was implemented. I now downloaded the dev version from Getnamos Plugin and tested the new timewarp function.
Unfortunately I was not able to get any good results. I think I got the best result with a Factor of 1 and Tween of -0.3 to -0.1. But all in all the difference is so small, that I am not even sure if it is really better when toggling it on and off. Larger and smaller numbers made it signifficantly worse though. A further problem is the jitter this introduces when turning the head as Getnamo already mentioned in his commit submission. Thats why I stay without Timewarp for now.

So my next question:
Does the leap support USB 3.0?

In this article http://blog.leapmotion.com/understanding-latency-part-1/ you wrote on an uptdate from June 2014:

The current software currently only uses USB 2.0 data speeds, but in the future will also take advantage of USB 3.0 speeds.

If the game runs at 90 FPS this results in a frametime of 11.1ms. According to your measurements the data transfer in Precision Mode alone takes 16ms with USB 2.0!

According to your graphs from part 2 ( http://blog.leapmotion.com/understanding-latency-part-2/ ) the data transfer over USB 2.0 would make half of the latency on a 90Hz monitor with VSync off.

I tried to connect my leap with a USB 3.0 cable (not the official Leap cable as this is USB 2.0) but I could not see any difference.

So again, is USB 3.0 supported yet, and if not, will it be supported soon?

[getnamo]

Sadly I never got timewarp to work properly when I did a first pass. I'm likely to look into this again when I'll do a modern rewrite for the plugin.

It's currently tracked as an issue on the github repo https://github.com/getnamo/leap-ue4/issues/9

[mni]

Any Update on this issue?
I have the same problem that mxo shows in his video.
Im using the built in version of the leap plugin in 5.15.
Its makes it unusable in any unreal application.
In unity the sensor just works fine without any problems.

[AlexColgan]

Timewarp is on the near-term roadmap for the Unreal plugin.