Official 3D Jam Discussion

[RonanDQuigley]

I'm running into issues with the ImageHands.

Basically I've made it that when both hands can be seen by the oculus, the camera in my unity scene changes from the main camera to a secondary camera. However when using the Image Hands, there are rendering issues to what I think is to do with depth, with the skybox appearing in front of portions of my hands. Would this be a shader issue?

[Tastykilljoy]

I'm having the same issue. What version of windows, unity, oculus sdk are you using?

I am Win 8.1, 5.2.1f1, and 6.0.1.0

[RonanDQuigley]

@Tastkilljoy - I'm running Windows 7, unit 5.2.1f1 and oculus sdk 0.7 (I'm not sure what you mean by the 6.0.1.0?)

On top of my previous issues, rendering multiple cameras with the leap motion package in use causes a massive performance hit for me: when two hands are on screen (image or any of the vr mesh hands) my frame rate dips from a solid 80 all the way down to the mid 30s and for oculus this of course looks awful).

In terms of specs I'm running a core i7 cpu, and an amd radeon hd 7850. I can appreciate the card isn't the best it could be for VR development, but there shouldn't be such a performance hit as there is with something like 200k tris being rendered on screen. I'll try it with VR disabled to see if maybe it's the oculus integration itself, as that's the only other thing I can think of. Either way, if anyone has any advice on how to mitigate this when working with leap I'm definitely all ears!

[dplemmons_leap]

@Tastykilljoy @RonanDQuigley,

Hey y'all, I'm one of the developers who works on our Core Assets. I can provide a bit of insight into what you might be seeing. There are a number of shader tricks we're using to produce the image hand effect, and a number of them involve creative use of depth buffers and multiple render passes. I didn't write that particular part of the code, but I know there can be conflicts between sky boxes and a number of the passthrough effects. Historically when I've run into these render order conflicts, I've used a piece of "sky sphere" geometry with a texture to replace my skybox.

As to the multiple hand performance issues, I've observed similar performance issues in our applications, like VR Cockpit and Planetarium, when running on machines with AMD video cards. Hard to say if this is the issue you're seeing though. In the past, when we've QA'd between the GTX 980 I'm developing with and a similar AMD card, we've very occasionally seen performance degradation on the AMD card. (Note: I need to have someone from our QA team fact check me on my details here, the last time I saw the issue was a while back). Notably I haven't seen this issue reliably across applications, so it is hard to say the exact cause of the issue, or if this issue still exists with the latest AMD drivers. Given the beta nature of so much of the tech stack, it is difficult to pin down some of these more intermittent performance issues.

[RonanDQuigley]

Hey thanks for the suggestion there for the hands.I'll give that a shot and see how I get on.

I think it's just a combination of two things for me. Firstly I've found that the integrated VR support in 5.2.1 causes a notable drop in performance when compared to me testing the same scene in 5.1.2. Combined with just the nature of the Image hands having a large tri count and the use of multiple cameras being rendered concurrently this is what I think is causing the performance problems for me. Furthermore I did another quick test in 5.2.0 and with no alt cameras available my frame rate is at 65 which shows how erratic it can be.

So the two solutions I have that I could use advice on are:

1) Is there any chance that there are image hands available with just the hand itself - i.e. no forearms? I noticed you have cut rigged hands but if I apply the mesh to your prefabs they just completely deform - I. assuming that they just aren't compatible in that case? A bigger plus to this would be if it's possible to get image hands with a much simpler tri count, on top of just hacking the forearms off (why do they need to be so high at present?).

2) The method I'm using for cameras is to switch between each camera using a depth value. However I can get the performance back up to 80fps if I just use one at a time. Initially I tried to have each camera turn itself off when not in use, but this caused the tracking of the hands to break off from the oculus tracking - perhaps because of the associated scripts? I hoped that by just applying the same depth buffer and image retriever script to every object that had its own camera. Is there a way around this at all? I also put the hand rendering on its own separate camera using depth only and culling everything else in the scene, but even with just two cameras in this instance - the main camera and this depth only one for the hands - the performance took a hit for some reason.

EDIT: In more tests, if I just placed a rigged hand model in the scene, I could get up to 2-3m tris before oculus would start to slow down. However as soon as I start using the leap, that's when performance drops would hit for me. (In this case I was trying out 5.1.3p3 - I'm aware this was below the requirements listed on the downloads page but oculus performance certainly improves when I move down to this version).

[I've also tried the VR planetarium demo and it ran completely smooth for me.]

[RonanDQuigley]

Back with another round of testing this evening and some results:

So interestingly the awful performance I am getting when two hands are shown on screen using the oculus does not occur at all with game builds and it stays rock solid at 75fps; the sluggishness only happens in the editor.I've found that 5.1.2p2 is what has given the best performance overall using a leap and oculus combo: anything beyond that and performance takes a hit. Is there something I should be aware of when using the leap 3.0 plugins with this version of unity as no problems have shown up so far?

[dplemmons_leap]

@RonanDQuigley The editor versus build performance difference is interesting. I know @mrschube was running into something like this recently but I think his issue was with shadow rendering a bunch of the objects in the scene. I'll ask and see if he discovered anything relevant. Re: your request for lower poly image hands, I know there was a push to make some lower poly image hands. The current ones are definitely absurdly high poly. I'll look into if that ever got finished, though I know nothing has been added to the main Core Assets branch, so I'm guessing it got back-burnered. (EDIT: Yup, the dev who was working on it has another project that took prescience. He tells me it is still something he's looking to work on in the future though).

Do you see anything in particular spike on the CPU during a deep profile when the performance issue occurs? That might give us some better insight into what's happening.

I don't totally follow the end goal on your camera switching issue. If you're looking for a sense of "transporting" the player view from one camera position to another, I might try just placing empty game objects where you want the camera to "cut" to, and then move the root transform of the Player to those points instead of swapping cameras. There's definitely some complexity inherent to cutting between multiple camera rigs. I don't think it's really a case that was designed for in the way the image retrievers and such initialize and update, the individual components (especially the LeapCameraAlignment and the LeapImageRetriever scripts) are pretty tightly bound once execution gets going. We have to do some tricks to guarantee script execution order and the like that aren't super robust to components being turned on and off or camera hierarchies changing.

Have you tried disabling only the camera components instead of the whole camera game object and seeing if that prevents issues caused by disabling scripts? This might cause some issues because we expect to get OnPreRender and OnPreCull calls from the camera and I'm not sure what the behavior of some of our scripts will be if those are disabled but the image retrievers and image based materials aren't.

This may be a red haring, but something we've been working on improving with Core Assets recently has been our memory usage. You might look to see if you're losing a lot of time to garbage collection or to overuse of system memory (profile it using the task manager rather than the Unity profiler, there's memory use that Unity doesn't know about from our API's C# => C++ bindings). The reason this occurs to me is that it is not uncommon to see very different memory and garbage collection profiles between the editor and a build. I don't know if I'll have a great fix for you if this is the issue (it's something we're investigating) but it would be good for us to know about at the very least.

[RonanDQuigley]

On the image hands subject: yeah that would be a huge help if it's possible to get a release for that. Is there a way to perhaps even just remove the forearm components completely from each hand? I'm not sure what the tri count on each of these parts is like, but it could always help potentially if there's a decent number of tris in them (that and I don't actually need the forearms to be rendered for the project I'm making).

As for the camera issue, I'll try and explain it better:

So in my project, I have four different cameras running (basically the VR game I'm developing is about basically accessing alternate realities between three visually different spaces). What I have is the following set up

1) A camera set to depth only, rendering nothing but the hands. This camera is always at the highest depth so that it is always rendered.
2) A camera that renders a alternate space, triggered by the player rotating the wrist of their left hand.
3) A camera that renders a second alternate space, triggered by the player rotating the wrist of their right hand
4) A "main" camera that just renders the main player space. This space would be accessed by just rotating either your left or right wrists again

Basically I tried shutting off each of the game objects to save resources when they weren't in use (except the camera rendering just the hands, that one is always active). All of the cameras are rendered from the same position (so from the point of oculus tracking). Changing camera depth for this worked, but of course that has a huge performance impact. I did initially try shuttting off objects but this was causing the tracking issues I mentioned before with the hands breaking away from tracking. I also tried shutting off camera components but it wasn't working for some reason (I'll give it a go again to see if I can get it working this time).

I'll look into the last option as well, however I think my best trade off at present is a combination of making sure everything is as low poly as possible and just running the game on a lower quality setting in the editor (would still be great to hear any ideas or methods I can use about the hand mesh stuff I mentioned though).

EDIT: Also I'm not having any luck with your proposed method for depth; the background still cuts into the image hands - do you have an example at all I could see?

[antony1316]

Hello there,

I am building a VR experience for the 3djam. I'm using Unity 5.2.1 and Oculus SDK 0.7.0 on Windows.
Everything runs great in the Unity Editor. However, when I attempt to run a built .exe, I get the following error:

OVRPlugin.dll caused an Access Violation (0xc0000005)
in module OVRPlugin.dll at 0023:66004714.

Anyone else have this problem or have an idea on what may be going on?
Any help is greatly appreciated.

-Antony

[antony1316]

I was able to find a solution after much tinkering and researching. It was as simple as upgrading Unity to 5.2.2 and re-building.

[randomfoo]

Dunno how closesly the devs are watching this. So I'm a bit of a Unity noob, working w/ it really for the first time for this project. I'm chugging along on my project, but what's the best way for me to capture screenshots/video of it?

[antony1316]

I recently tried Open Broadcaster Software for video captures, and it's been awesome so far. It's also free.
https://obsproject.com/

On Windows, I take screenshots with Snipping Tool, an official Microsoft tool.

[AlexColgan]

We can also vouch for OBS -- we use it for our Twitch episodes (twitch.tv/leapmotiondeveloper).

[AlexColgan]

Hey all, less than 14 hours until the Jam closes! Submissions to https://itch.io/jam/leapmotion3djam end at 11:59pm PST, so let us know if you need any support or have any questions.

This thread has information on how to submit: https://community.leapmotion.com/t/3d-jam-submissions-thread/4137

You can also access our live support channel here: https://www.hipchat.com/gOcNQjp9k

[yosun]

Hmm after building the exe from Unity 5.2.2 (Win 8.1) the resulting exe does not seem to trigger either Leap or Oculus display? Works fine in the editor...

[mrschube]

Hi yosun. If it works fine in the editor that means the assets in your project are set up right and VR supported is toggled on. The issue must be after that stage.

Are you building out for the correct platform? And do you have the same runtime installed as the SDK you're compiling against?

[yosun]

ovr 0.7.0 and leap unity assets 3... i didn't import the oculus sdk into unity - i assume that's handled directly by unity now?

[mrschube]

You're correct you don't need to import the Oculus Unity integration package into your project, the key components are already in the Leap Motion Core Assets. I was asking if you have both the Oculus 0.7.0 Runtime installed as well compiling your project against the Oculus 0.7.0 SDK.

[yosun]

that's where i'm confused - if you aren't importing oculus unity, how are you compiling against oculus 0.7.0 sdk?

[curling247]

Are updates allowed after the Jam ends? Obviously itch doesn't seem to care and will let you continue to update. Does this affect the 3D Jam contest at all?