Compiler errors on Demo Scene - Window Store / SDK: Universal 10. Orion 3.1.2 - Unity Hololens 5.4.0b14

[Bierro]

New project in Unity started with last version of the Unity core asset package v4.1.0.
I use the last version of Orion and the last version of Unity Hololens.

Loaded the scene "Leap_Hands_Demo_desktop" and tried to build it.

Everything works fine in play mode and when I build to Standalone, but when I switch to Windows Store platform choose Universal 10 SDK, D3D as UWP Build Type and Build and Run on Local Machine, I get compiler errors like:

-Assets\LeapC\Connection.cs(152,33): error CS0246: The type or namespace name 'ThreadStart' could not be found (are you missing a using directive or an assembly reference?)
-Assets\LeapC\Connection.cs(154,16): error CS1061: 'Thread' does not contain a definition for 'Start' and no extension method 'Start' accepting a first argument of type 'Thread' could be found (are you missing a using directive or an assembly reference?)
-Assets\LeapC\Connection.cs(332,64): error CS1061: 'byte[]' does not contain a definition for 'LongLength' and no extension method 'LongLength' accepting a first argument of type 'byte[]' could be found (are you missing a using directive or an assembly reference?)

It seems it can't access some of the .NET files (in System.Threading and System)...

Any guess on what I could do ?

Thanks!
Pierre

[Joe_Ward]

I don't think Windows Store supports apps that need to communicate directly with other processes, which the Leap Motion system does.

[Bierro]

Thanks for the reply.

Yes it seems I'll have to get the data from the websocket and to parse it inside Unity. You mentioned somewhere that it meant replacing the Controller script in the Orion version. I'll try to get inspiration from leap.js in order to do so.
Do you know if someone has already tried to do it in the past few months so that I could rely on that to get started ?

Thanks!
Pierre

[Joe_Ward]

No, I don't.

Here's the WebSocket code I used in V2. It worked at one point, but is broken now, but it might help you get started.

using UnityEngine;
using System.Collections;
using System.Collections.Generic;
using System;
using MiniJSON;

namespace LeapWrapper{
	public class LeapWebSocketController : MonoBehaviour {
		const string WEBSOCKET_URI = "ws://localhost:6437/v6.json";
		const string GET_FOCUS = "{\"focused\" : \"true\"}";
		const string LOST_FOCUS = "{\"focused\" : \"false\"}";
		const string BACKGROUND_ON = "{\"background\" : \"true\"";
		const string BACKGROUND_OFF =  "{\"background\" : \"false\"";
		const string GESTURES_ON = "{\"enableGestures\" : \"true\"";
		const string GESTURES_OFF = "{\"enableGestures\" : \"false\"";
		const string HMD_ON = "{\"optimizeHMD\" : \"true\"";
		const string HMD_OFF = "{\"optimizeHMD\" : \"false\"";

		public Dictionary<System.String, System.Object> currentFrame;

		IEnumerator Start () {
			WebSocket w = new WebSocket(new Uri(WEBSOCKET_URI));
			yield return StartCoroutine(w.Connect());
			int i=0;
			Debug.Log ("Created WebSocket connection");
			w.SendString(GET_FOCUS);
			while (true)
			{
				string reply = w.RecvString();
				Debug.Log ("Rep string " + reply);
				if (reply != null)
				{
					Dictionary<System.String, System.Object> Message = (Dictionary<System.String, System.Object>)Json.Deserialize(reply);
					Debug.Log ("Dict: " + Message);
					if(Message.ContainsKey("id")){
						Debug.Log ("Received: " + Message["id"]);
						currentFrame = Message;
					}
				}
				if (w.Error != null)
				{
					Debug.LogError ("Error: "+w.Error);
					break;
				}
				yield return 0;
			}
			w.Close();
		}
	
	}
}

The next step would be to take the currentFrame variable and create C# Frame, Hand, Finger, etc objects. (All the Dictionary-related code is because of how the MiniJSON library I used works.)

[Bierro]

That's already a good start !

Thanks a lot Joe !

[Joe_Ward]

Todays Unity release of 5.3.5 also includes support for JSON: http://docs.unity3d.com/Manual/JSONSerialization.html

[Bierro]

I actually didn't use Unity support for Json as it is not handling properly arrays yet. I used "LitJson" library instead and it works fine.

So I got the data and now as you mentioned earlier I am creating the Frame, Hands, Fingers and Bones objects out of it. For the Bone objects something's a bit tricky though.

I am given the basis vectors for each bone and I should be providing a LeapQuaternio to construct a Bone object. However your documentation specifies:
"Note that converting the basis vectors directly into a quaternion representation is not mathematically valid. If you use quaternions, create them from the derived rotation matrix not directly from the bases."
Any guess on how I should do that?

Thanks in advance
Pierre

[Joe_Ward]

I think that is mostly a mathematician's quibble -- namely, that using the basis vectors as a rotation implies rotating from the standard world basis to this basis. There is more than one way to do that (you can rotate clockwise or counterclockwise around the rotation axis, for example) -- which means that more than one quaternion could provide the same result.

The easiest, most efficient way might be to use the Unity Quaternion.LookRotation(z-vector, y-vector) function. You will need to make sure that things end up pointing the right way, though, since Unity and the Leap Motion system use different handed coordinate system. Otherwise, you can make a matrix out of the basis vectors and convert that to the Quaternion. http://www.euclideanspace.com/maths/geometry/rotations/conversions/matrixToQuaternion/

[Bierro]

Ok I see what you mean. Thanks a lot!

Yes I guess I'll use LookRotation after having changed the x-Axis to its opposite. But it seems I only have to do it for bones of the right hand because I read that :

The bases provided for the right hand use the right-hand rule; those for the left hand use the left-hand rule. Thus, the positive direction of the x-basis is to the right for the right hand and to the left for the left hand. You can change from right-hand to left-hand rule by multiplying the basis vectors by -1.

And unity uses the left hand rule right?

[Joe_Ward]

Yes, Unity is left-handed.

[Bierro]

I finished parsing the data, everything works fine and in realtime.

I hadn't read the docs before doing all this, but I was disappointed to learn that the LeapMotion websocket Server could only send data over the port on localhost domain. My computer is running as an internet server and I would have liked to send data directly via my ipv4 address as it would have been more convenient and faster when communicating with an unwired Hololens. Do you know if there is a plan for this to be done?

What I will do instead for now is send the Json files through another websocket in Unity

[Joe_Ward]

You can set your configuration to turn off the localhost restriction:

https://developer.leapmotion.com/documentation/csharp/devguide/Leap_Configuration.html#websocket-options