UnrealEngine4 Plugin

[haraldini]

Hi,

Is there an official (or unofficial) LeapMotion plugin for UnrealEngine4?

Thanks

[LM_Michael]

Hello Haraldini,

Unreal Engine 4 is currently not supported, however we do have software provided by JavDevGames that will allow for integration of the Leap Motion Controller with Unreal Engine 3. You can go ahead and check them out here.

https://github.com/JavDevGames/LeapMotionUDK

https://bitbucket.org/HugoLamarche/leapudk

[bPatrik]

Sorry for being a bit off topic, but i want to ask the same about CryEngine. Iam more familiar with that one. Is there anything for that one, or any plans (offical/unoffical)?

+1 game related question: Any plans to simulate joystick imput with LM? Not exactly like when i grab a joy, but for example 2 axises for tilting my palm, 1 for rotating, or anything. I have some nice ideas whit this and CryEngine.

[LM_Michael]

We do not have any plugins available for Crytek's CryEngine. We do however have some documentation available that will allow you to emulate Joysticks when using the Leap Motion Controller. Check it out here.

Leap4Joy

[zalo]

Can't you recompile the C++ source of UE4 with the C++ .dlls from the SDK?

[haraldini]

Perhaps I could do that, but I was hoping Leap would utilize the UE4 plugin system like other manufactures do. This would make implementation a lot easier for people not too familiar with C++.

[Komposten]

@haraldini
You might want to have a chat with @swhittle. In this post he states that he is "responsible for the Leap integration plugin for that engine" (talking about Unreal 4).

[swhittle]

Hello @haraldini,
If you're an Unreal 4 Licensee, you can check out the current state of my plugin on its pull request page here:

https://github.com/EpicGames/UnrealEngine/pull/112

Given that it's still in development, there's no usage information formally documented yet. Here's a brief rundown:

There's a module that lives in \ThirdParty that contains the Leap DLLs and a plugin module in \Plugins that actually provides the support for the engine. You'll need to copy the dlls from the ThirdParty folder to the binaries folder, then enable support for the plugin in the editor. Once that's done you can add a LeapInputComponent to any actor blueprint you want to be able to respond to Leap events like a new frame or gesture, and then use the 'new frame received' blueprint event to write your event handler.

[Komposten]

@swhittle
It seems that the repo you linked to either doesn't exist, or is private. Just thought it might be worth pointing that out (even though I'm personally not into UE4).

[swhittle]

The repo is private as a result of my license agreement with Epic. Anybody who has a valid UE4 subscription and links their Github account to their UE4 one can access it. Thanks for pointing it out - I should have clarified for the benefit of others who might be interested.

[haraldini]

Progress seems to be there. Really exciting

I tried to apply it as a patch to the current 4.1 github source and I'm getting some compile errors....

Error	1	error C2039: 'MovePinLinksToIntermediate' : is not a member of 'FKismetCompilerContext'	C:\Program Files\Unreal Engine\GitHub\Engine\Plugins\LeapBlueprintSupport\Source\LeapBlueprintSupport\Private\K2Node_LeapGestureAction.cpp	106	1	UE4
Error	2	error C2039: 'MovePinLinksToIntermediate' : is not a member of 'FKismetCompilerContext'	C:\Program Files\Unreal Engine\GitHub\Engine\Plugins\LeapBlueprintSupport\Source\LeapBlueprintSupport\Private\K2Node_LeapGestureAction.cpp	109	1	UE4
Error	3	error C2039: 'MovePinLinksToIntermediate' : is not a member of 'FKismetCompilerContext'	C:\Program Files\Unreal Engine\GitHub\Engine\Plugins\LeapBlueprintSupport\Source\LeapBlueprintSupport\Private\K2Node_LeapNewFrameAction.cpp	58	1	UE4
Error	4	error C2039: 'MovePinLinksToIntermediate' : is not a member of 'FKismetCompilerContext'	C:\Program Files\Unreal Engine\GitHub\Engine\Plugins\LeapBlueprintSupport\Source\LeapBlueprintSupport\Private\K2Node_LeapNewFrameAction.cpp	61	1	UE4
Error	5	error C2039: 'BindUFunction' : is not a member of 'TScriptDelegate<FWeakObjectPtr>'	C:\Program Files\Unreal Engine\GitHub\Engine\Plugins\LeapBlueprintSupport\Source\LeapBlueprintSupport\Private\LeapEventDelegateBinding.cpp	30	1	UE4
Error	6	error : Failed to produce item: C:\Program Files\Unreal Engine\GitHub\Engine\Plugins\LeapBlueprintSupport\Binaries\Win64\UE4Editor-LeapBlueprintSupport.dll	C:\Program Files\Unreal Engine\GitHub\Engine\Intermediate\ProjectFiles\ERROR	UE4
Error	7	error MSB3073: The command "..\..\Build\BatchFiles\Build.bat UE4Editor Win64 Development" exited with code -1.	C:\Program Files (x86)\MSBuild\Microsoft.Cpp\v4.0\V120\Microsoft.MakeFile.Targets	38	5	UE4

[swhittle]

You'll need to check out the latest-preview tag and apply it there, sorry, rather than a stable build.

[swhittle]

Figured I'd also link this :
https://dl.dropboxusercontent.com/u/4692574/Leap_UE4_Integration_WIP.wmv

Just a WIP video showing the sensor data (the black lines and coordinate axes) being used to drive the skeleton - the inaccuracies in that video are due to me still looking for the best way to interpret the sensor data for the rigged hand, rather than the raw sensor data itself being wrong.

[AlexPaschall]

Hi,

I'm actually an engine tech for Epic and I wanted to say that this is really cool. I am interested in all of the VR hardware that has been coming out in recent years and how it can be used for game design. If you are on our forums, please send me a PM I enjoyed the video a lot too, please keep us all updated on your progress.

I will be ordering a Leap Motion so that I can try this out.

[jaxzin]

@swhittle I merged your fork into the latest Oculus fork of UE 4.3.

https://github.com/jaxzin/UnrealEngine/tree/4.3-leap

I'm brand-new to the Unreal Engine, so this could be a PEBKAC issue, but I don't seem to be getting frame data as I'm trying to write my first blueprint. Are you able to share the project you demoed in that screencast above? Or walk me through a simple blueprint that displays the frame data to the screen? Thanks.

[swhittle]

I'll have to dig through my files and get a working copy out to share (I've bought a new computer since I last worked on the plugin), but in the meantime the first thing I'll mention as a gotcha is that you need to recompile my plugin using the headers and libs corresponding to the Leap SDK version that you have installed - my repository contains a version linked to the latest development build available at the time I compiled, but if there's a mismatch between the SDK version you have installed and the version I linked to, you won't get any data.

[jaxzin]

I double-checked but yeah I had updated both the lib and includes folders under ThirdParty/LeapMotionLib and then rebuilt the entire UnrealEngine project. I was hung up for a while because I hadn't copied the Leap.dll and Leap.lib to the Binaries/Win64 dir. Once I did that the project stopped crashing. I definitely want to have the build copy those 2 files automatically.

[jaxzin]

@swhittle
It was a newbie mistake. I hadn't added the LeapInputComponent to the list of Components in my Blueprint. Once I did that I started getting data on screen.

Now that I've got data, I'm trying to change the position of bones in a skeletal mesh. But it appears I can only do that inside an AnimationBlueprint, but using New Frame Received there crashes the Editor. Can you point me in the right direction for adding support for Animation Blueprints? Which parts of your code would that affect?

UPDATE: I figured out how to save the Leap data to a variable in the ActorBlueprint and then access it from the AnimationBlueprint.

[swhittle]

You can also get the AnimationBlueprint and 'push' variables into it from your Actor Blueprint.
If you've got any feedback on the system as it is implemented now, feel free to raise issues or comments on the Github repo

[jaxzin]

@swhittle I've made some decent progress. Hands are positioned correctly, but positioning the finger joints is not working yet. I'm using your FLeapBone Position (which exposes Leap's Bone.nextJoint()) for the skeletal joint position. Joints appear to be offset so they start where they should be ending.

Also, I sent you a small pull request to add IsLeft/IsRight to FLeapHand. I tested it and its working in my build.