Idea for Leap Motion 3d Pen Accessory + Concept Art Image

[IntoTheRift]

Lately I've been inspired by the leap motion V2's ability to track tools. I find that a number 2 pencil works best if you want to play around with it. Working with 3d software I see how this will really come in handy for imputing absolute XYZ coordinates. Unlike the popular 3d 3Dconnexion mouse which move relatively in 3d. Such devices for absolute control exist as tools for 3d professionals but are quite pricey like the one listed here. http://news.softpedia.com/newsImage/Haptic-Pen-Guides-Hand-of-Visually-Impaired-2.jpg/

While the leap doesn't have the benefit of haptic feedback and may not have the same accuracy it has the big advantage of cost and a large range to work with. Though one problem I see is that not having a button limits the functionality of it. Using up a hand that can be used for rotation and movement. Below is my concept art on an idea to help with this that should be very low cost to implement if it works well. I created a quick concept in the 3d program blender.

I know it looks a little odd. Basically its a pen designed to not only be tracked more accurately but also have a control button like on a wacom tablet pen. When the button is press it turns on a IR led with a diffuser to send light out in 360 degrees so it can be seen by the leap's IR camera at many angles. The leap then interprets the turning on and off as button presses and passes them on to apps.

Some applications I can think of off the top of my head.
Easy 2D and 3D painting
Laser pointer like computer mouse control
3d modeling and sculpting
Interface control
Games and more

I'm interested in some feedback if you can help.

1. Is the IR system I mention possible to do with the leap motion without causing too much interference?
2. Would spherical ends help aid the tracking?
3. Would you be interested in this?
4. Would you be interested in using this with your project?
5. Any other thoughts on the concept?
6. Any ideas to improve it?

Thanks, I'm not a developer at the moment but a 3d hobbyist that would like to see something like this exist.

[asmith]

Hi, IntoTheRift,
I work at LM and have done some work on the tool tracking.

This is definitely a workable idea in most regards, and in fact we had built something similar in the past internally. For points 1 and 2:
1]]. I would not recommend an IR strobe or emitter because if it's too bright or close to the device it will likely interfere with the ability to adjust for the dynamic range of the scene: basically the image can become overexposed. At short ranges the IR illumination from the device is plenty. It would be better to just make the tool highly visible (uniform, light-colored, Lambertian). If you want to do some special tracking of the tool, you could use the (to-be-released-any-minute-now) image API which allows you to see the raw images from the device: you could put some glyph on the object with retroreflective ink and then use image processing techniques to follow this shape in the image. Honestly though, just a tool with a clickable button would have use cases, and that wouldn't require any special image processing.
2]]. For best tracking of tools spherical end caps are best, but they should be the same radius as the tool body.

Wold love to hear about your progress. Good luck.

[IntoTheRift]

Thanks for the feedback. Have you considered releasing a pen with maximum visibility? As that would be something that would be great to bundle with future versions.

Thanks for the update on the IR emitter I was hoping it would be the key to a very cheap controller transmitter. So it's not ideal to send button presses. It would have to either have a usb or bluetooth connection for button presses. In this case it likely won't be worth pursuing as adding it would easily quadruple the price and add much more complexity. Thanks anyway.

Nice to hear the image API is being released soon!