Networking in Unity with the Leap Motion

[damocles]

I am working on a multiplayer experience for the Leap and I'm having a bit of trouble figuring out the best approach for sending hand data over the network with Photon networking. My current approach is to treat the objects that compose users' hands (ie the capsules representing each bone) as any other gameobject I'd want to sync over the network and send their position/rotation separately. This has lead to some messiness and confusion about how it could work with other hand models that don't have discrete bones.

Is there a better way of sending hand data over the network and displaying it in a scene?

[michaelwolf95]

the objects that compose users' hands (ie the capsules representing each bone) as any other gameobject I'd want to sync over the network and send their position/rotation separately. This has lead to some messiness and confusion about how it could work with other hand models that don't have discrete bones.

Is there a better way of sending hand data over the network and displaying it in a scene?

I'm working on the same problem at the moment. Did you ever figure it out?

[jhamman11]

Hi @damocles and @michaelwolf95. I am just getting started with a multiplayer leap motion experience. Have either of you had any success in figuring this out, or any tutorials to point me towards?
Best Regards
Jacob

[michaelwolf95]

Hey, yeah, I'm just about to finish getting this working, and I talked to some Leap developers at the Vision Summit, and they might use my scripts in the next SDK.

Email me a reminder sometime later this week, and I'll send you my code.

[fgermain]

Hello !

Having the same problem right now ! Would love to see some code if you managed to have a "networking" Leap Motion.

[damocles]

Apologies for taking so long to follow up. I haven't found a good answer to this question yet and moved on to other projects in the mean time.

[user-295940283]

Having same problems as OP, which I tried to approach in a separate thread.

It's quite difficult to assess the workload for implementing the networked hands at the current state of Orion (in my case it seems easy, basic display of remote hands for connected player). Looking around here I can't quite tell if that's something that's manageable for a beginner developer, or is a major effort and we better wait for next versions of Orion.

Can someone from Team Leap Motion weigh in on that please?

[AlexColgan]

For beginner devs, networking the hands would be fairly difficult. We do have something like that further down our roadmap, but it will be some time before we build it.

[ddutchie]

I'd also like to get this working. Lets see how this goes...

[ProtoTypeMan]

Hello, I am an amateur developer. I just turned to Photon to create Leap applications and I am lucky to find that you have finished this problem. If possible, could you please give me some inspiration or some samples? I am very grateful.

[DaGoedie]

Again, I would love to hear if there has been an update on that front. I could implement a separate OSC/UDP solution but I would not like to open another network framework if I can possibly avoid it.

Any pointers would be much appreciated.

Best Wishes
D.

[camdenbock]

At the University of Maine, we are trying to develop a solution for this at the moment, using PUN.

We have a very primitive demo working - named hyperball in our repository. The idea behind our solution should work with other networking solutions. In short, we have made all of the LeapMotion interactions happen locally and change ownership of objects rather than having multiple instances of the LeapMotion service provider in a scene.

For objects (other than the hands) we have the client take ownership of the object (incl transform and rigidbody) when the object is contacted or grasped by the interaction behaviour. This can be extended

For the hands, we plan to network graphical representations of each of the components. To begin with, we have just networked the palm position as a proof of concept.

We'd welcome any comments and are happy to share anything we've done.

maine-imre/handwaver

[rSalway]

Any update on this front? Is there a git Repo or some other way of checking out your solution.
I have some things working in mirror networking but the hands have major issues when the leap losses tracking. A full IK body that doesn't hide the hands but sets it to a default pose when out of the tracking view may help but I would love to see what others have managed. Thanks.

[rSalway]

For your solution you have the hands locally with objects interacted with through a swap between server and client authority. What do you do if one player wants to hand an object to another player rather then say toss it to them. Also how are you representing the body presence of one client to another client or host? Do you have a separate network player that represents the player and if so are you using child network transforms on each joint or just using a static hand pose and networking the wrist or palm?
Thanks.

P.S. I will check out the git project when I can get back to a device with the leap available.

[michaelwolf95]

Hi rSalway,
Sorry, I wish I could help - I haven't touched this SDK in 6 years, and my solution was incredibly rudimentary, and had a ton of limitations. Also, it used UNet, which is now deprecated - so even if I could dig up my old repo, it wouldn't be very useful to you.
Good luck figuring it out!

[michaelwolf95]

To anyone finding this post in current year - don't get your hopes up.
This post was made in 2016, and my solution was incredibly rudimentary, and barely worked on a local network. I was just syncing the bone transforms using UNet, which has been deprecated since ~2018.
Sorry! I wish I could help!

[rSalway]

Thanks michaelmolf95.
Mirror is built on top of UNet so it could work but I already have a rudimentary bone sync on local network so it looks like I have gotten as far as you. Anyway thank you for the reply and best wishes.