UnrealEngine4 Plugin

[matthewtytel]

Hey I'm the dev that works on the Unity Assets.
It seems there's a lot of interest in an official UnrealEngine plugin which is contrary to what we thought a couple months ago!

So, I'd like to port all the logic and content (there's a bunch of really cool new models coming out soon) from the Unity assets into an Unreal plugin if you all are interested. I see a lot of you have done a lot of work already and I'd like to use that as well if that's alright

Anyway, I'd probably start on this end of September because I've got a lot on my plate at the moment, so let me know any feature requests you have in the meantime.

[turick]

Thanks Matthew! Official support for a plugin would be amazing!!! I think you'll find with UE4's licensing scheme making it so available to indie developers and the powerful new features that are causing all sorts of devs to move to UE4, you'll only see a growing interest.

I can't speak for everybody else, but my main interest is for my full skeleton character to be able to move his hand(s) around the screen, with the arm bones simply following along. I only require 1 hand, but based on the latest skeletal tracking stuff, it seems that it shouldn't be any harder to support both hands.

Thanks for paying attention to the community and being willing to take this on!

Josh

[DFinsterwalder]

I would be interested in UE 4 support as well.

[Michel Jongbloed]

Great news Matthew. Official Unreal support would be greatly appreciated.

[Michel Jongbloed]

Jaxzin,

Your link doesn't seem to work any more. I would really love to experiment with it. Could you please point me to a working link?

Btw. thanks for all the work you've put into this. Really great if the leap could be fully usable in the Unreal Engine.

[turick]

Michel, if you don't have your UE4 account linked to your github account, it will show up with a 404. You need to log into unrealengine.com, then go to account settings and you'll be able to link the accounts.

[Michel Jongbloed]

Turick, thank you. Didn't know you had to link these together. It works now, so I'm fine.

[swhittle]

Jaxzin has done a fair bit of work on his fork from the looks of things, but you've certainly got my blessing to take my part of the contribution and turn it into an official plugin. Might I suggest you make it available on the UE4 marketplace when you're done? I was originally going to do this with my code, but I had a few concerns re redistributing your libraries, ongoing support, that sort of thing.
Sorry for the delayed response, I didn't get any sort of ping from the forums that people have been making additional comments here.

[Quixotic]

Thanks to everyone who's worked on this, especially @jaxzin.

Having built and run the 4.4-leap branch, can anyone offer pointers on what to do next? I've tried creating a blueprint project, opening an blueprint, turning off context-sensitivity, and looking for some of the nodes in jaxin's screenshot, but I can't find anything to do with "Leap".

Did I build the project wrong? Am I trying to access the components in the wrong way?

Edit: I forgot to enable the plugin under Window->Plugins. Now I'm getting the same error as @num3ric.

[shess]

You need to copy Leap.dll into [YourUnrealEngine]\Engine\Binaries\Win64.

[LagoLuis]

Where can I get the leap.dll? I just threw the zip file content into my built 4.4 build and I didn't see the leapBlueprintSupport in my plug in folder.

Nevermind I was able to download and compile correctly thanks Jaxzin and swhittle! Is there any documentation or some pngs of blueprints? I would love to see the BP for the hand.

[getnamo]

Just wanted to give a heads up: made an event-driven Leap Motion plugin for UE4 (currently compiled for 4.4), see unreal engine thread for a download link and an example of how to use it.

If you want to have a quick and easy way to use the leap with UE4 without needing a separate engine fork, this would be the option you're looking for. Event driven architecture also means you don't have to use loops and poll checks, but rather just subscribe to events you're interested in.

Example use style showing debug hands:

Result:

Can be added to any blueprint where you want to receive leap events.

[LagoLuis]

My friend emailed me your plug in a couple days ago, I haven't had time to use it but most likely will later today. Super happy you were able to update marc's build.

[getnamo]

Updated the plugin to 0.7, which adds proper real-time image support thanks to minimal copying implementation. With image support and new property-based api, the plugin should provide all the functionality defined in 2.1.5, let me know if anything is missing!

If you do the following in a blueprint

where the Texture2D target in this case is a Billboard Sprite, you get the image on the left

The distortion map, shown as an example to the right, will not show if you only implement the above functions.

Also note that many of the nodes have changed into properties, here is a more complete example of debug hands with the new api style

This will give you the following in the blank template with starter content.

[getnamo]

Updated the API to include a cleaner way to get raw images. Plugin supports both methods as it maintains Leap API throughout, e.g. old style example grabbing both raw images as textures and updating two separate sprites

resulting in something like this (note the finger and palm debug visualization is a result of a separate graph)

the same result can now instead be done by using the Raw Image Received event

Note that both the texture and the image pointer are available for the event allowing you to grab any additional relevant contextual information such as image id (shown) or say distortion map. You can also toggle image support to fully reclaim performance, bandwidth and image memory when not in use e.g.

If image support is not enabled or the emit toggle is off, the image event will simply not emit.

[getnamo]

Since v0.8, convenience content is now available. Once you've installed the plugin its as easy as

then hit Play to try it out!

If you want to try this out in rift mode all you have to do is Play Standalone and hit Alt+Enter and the character will automatically adjust to rift mode. Note that your arms are scaled to be 1:1, but due to hand/finger models, fingers may be longer or shorter than your own.

Go to the main thread to grab the latest release!

[AlexColgan]

Hey everyone, there's now an official Unreal Leap Motion plugin! More info here: https://community.leapmotion.com/t/official-ue4-plugin

[getnamo]

This weekend's release of v0.9 of the unofficial plugin now adds support for VR passthrough in UE4!

Simply set your convenience character to

And use a simple gesture to transition between the two states

If you wish to use your own methods for transitions or adding additional functionality to AR such as depth masking, simply sub-class or change any of the blueprints that are included in the plugin. Refer to the diagram below for an overview how the convenience content is structured.

get the update at the usual place

[getnamo]

Due to popular demand the unofficial plugin now has collision assets included in the convenience rigged content.

The plugin now includes a BasicBody_Physics PhysicsAsset

Which allows you to change the shapes and sizes of collisions attached to your mesh. You can then simply change the CollisionPreset of any of the convenience characters to PhysicsActor and you will gain collision capability. If you don't know how to do that then simply use the LeapCollisionCharacter instead which has that enabled by default (subclass of LeapRiggedCharacter)

This gives you

And if you're patient you can even play some simple jenga

Hope you find it useful, latest update is available at the usual place.

[turick]

I've had a bit of a hiatus from my project until recently. Going from threads like this and other threads just last year where I asked how to use a rotational matrix to use the Leap Motion in a position other than on a table facing up and seeing the crazy amount of progress since then... official UE4 support and Oculus Rift mounts specifically to position the unit facing away from you so it can see your hands! It's so amazing how fast things are moving.

That being said, I just tried this out and it seems the leap motion is still expecting to be flat on a table. Is there some other setting in UE that would interpret the input as if the device were mounted to the front of an Oculus Rift?