Hi everyone,
I’m working on a new tool tracking project using Leap Motion. The idea is to build a 6DOF pen that works on any surface. For example it lets you “write” on a physical table while you have a VR headset on. I think this gives the best of both worlds since you can transition between working in 2D as you would on a phone/tablet, or in 3D as you would in Tilt Brush, all with one controller (pen). I want to spend my entire workday in VR, and I don’t see it happening with room-scale apps or midair UIs. I want to get rid of my screen and work mostly at a desk. I know others on this forum have had similar ideas (1), (2), but I’m not sure if anyone ever built a finished product. Unity has a cool experiment with the idea of a table-top interface (3) but they're still using motion controllers - I want to take it further.
I’ve built a working prototype of both the hardware and software side using Leap Motion’s Image API and Oculus Rift. Since LM uses active IR the first prototype (image below) didn’t need any electronics, I built it using retro-reflective tape for less than $5. Unfortunately to support more features and tracking systems the latest prototype is a lot more complicated, but LM is an awesome prototyping platform 
The latest prototype has perfect tracking (comparable to Vive controller tracking) at 90fps using Leap Motion. Having the power of 3D input PLUS the efficiency/precision of working on a hard surface feels AMAZING!
Personally I think pen+hand input paired with much higher resolution VR headsets (4) will be the first viable all-day VR workstation setup, and potentially a turning point for professional VR adoption.
The project is tentatively called “Drawboard” and I just put up a site with some more info and a video:
https://www.drawboardvr.com/
I have some funding to work on this idea so I'm looking for people that want to work on this with me. If you have hardware, computer vision, or game engine experience, send me a message.
Finally - we need ideas for applications. So far we've just built a VR whiteboard which is an obvious application for a pen, and works extremely well - but what else should we build? Also interested in ideas for other kinds of tools that could be based on the tracking tech we already built.
1 https://community.leapmotion.com/t/idea-for-leap-motion-3d-pen-accessory-concept-art-image/1495
2 https://community.leapmotion.com/t/the-ideal-tool-for-leap-motion-tool-tracking/1955
3 https://labs.unity.com/article/carte-blanche
4 https://www.varjo.com/
