3D Jam: "Virtual Guitar" project

[zachkinstner]

Hi everyone! I'm working on a "Virtual Guitar" project for the upcoming 3D Jam. The general idea is to strum guitar strings and select chords, in VR, using Leap Motion input.

As part of that work, I'm going to make occasional development update (#DevUp) videos. These will show the early/intermediate stages of the project, and will include discussion about new features, challenges and issues, my next steps, etc.

It would be great to hear feedback from the community as this project moves along!

DevUp: Virtual Guitar video playlist

[zachkinstner]

DevUp: 2015-07-29

This video shows three distinct builds from the past few days. The first uses the mouse to strum the strings, and was primarily a test of the MIDI output. The second introduces Leap Motion input in a virtual reality environment. The third adds a visual indicator that shows when your finger is in the "strum area".

[zachkinstner]

DevUp: 2015-07-31

This video shows the initial chord-selection functionality. Chords are selected by bringing your left hand/finger near to the boxes on the left side. This is a very early UI, of course, and this will definitely need improvement.

Other changes include the string colors, the use of audio clips (instead of MIDI) for realtime sound output, and several internal changes to the structure/organization of the project.

[zachkinstner]

DevUp: 2015-08-04

This video demonstrates the new visual indicators that wrap around the guitar strings. A colorful meter shows your hand's position in space, with an emphasis on the hand's depth. Highlighted areas help you find the chord selection areas and the individual guitar strings.

This build also includes some latency improvements, so that the actual motion of your hand matches up more closely to the audio/visual reactions in the application.

[Kip]

This is looking (and sounding) really great Zach - looking forward to playing around with a build.

[zachkinstner]

Thanks @Kip! My schedule has filled up for this week, so unfortunately that's going to delay a demo build. I'm excited to get feedback from users, but I need to fix some issues and make some improvements first...

[baatkarlo]

Looks really good in the second video. You may want to consider open chord tunings and slide as a way to make a VR slide guitar. Also, consider marking frets directly on the string itself, with 3, 5,7,9,12,15,17,19,21,24 as a different color marked fret than the rest on each string. Allow tilting the string plane via settings, in all three axes. Replace string vibration with a color change on finger tip on contact. Allow right four fingers to latch so index finger owns string three, middle owns string two, ring owns string one and thumb owns strings six, five, four.

[Aki_Kanerva]

Good to see air guitar isn't dead yet. Keep rocking on!

--
Aki Kanerva
Virtual Air Guitar Company Ltd. (est. 2006)

[zachkinstner]

@baatkarlo Thanks for the feedback -- lots of ideas to consider!

The repositioning of the guitar is something I plan to do; I have an initial concept in mind for how to accomplish this. For now, I think I'm going to stick with the simple chord+strum interactions, but it's possible that I'll explore individual frets and improved single-string plucking in the future. I would want to give some very careful consideration to usability -- how the user learns about these features, how they can perform them accurately and consistently, etc.

@Aki_Kanerva If this ever becomes a game, maybe the player gets bonus points for jumping around like a rockstar on stage... and then facial recognition can make sure they're really getting into it

[pronto]

Any chance to get piano version of this app?

[zachkinstner]

@pronto, with the MIDI output (currently turned off in the app), you could potentially select any kind of instrument sound you want. I tried it with piano and harp sounds, which was a pretty nice effect.

However, I assume you're thinking more of a traditional piano layout (vs. the guitar layout). The answer to that would be yes, I have a whole variety of possible instruments in mind. These include both "traditional" and "only possible with VR+Leap" types of instruments. If this guitar project continues to get good feedback and interest from the community, I'm definitely interested in continuing this work.

[zachkinstner]

DevUp: 2015-08-20

This video shows off the new "glow" effect on the strings. The strings react to your finger position, and light up when you strum them. The visual indicators are more subtle in this build, with toned-down colors and fewer lines.

This build also includes several bug fixes (notably, with the audio playback) and some internal changes (including new strumming mechanics).

[AlexColgan]

This project has just been featured on our blog! http://blog.leapmotion.com/video-series-taking-vr-guitar-whole-new-depth/

[zachkinstner]

Thanks @AlexColgan! I got that video done just in time...

I made an animation loop from the latest video. Anyone have an Imgur upvote to spare?

Bask in the neon glow of these VR guitar strings

[CadePeterson]

I know it's just cosmetic, but I am loving the neon glow choice for the strings. #ChildOfThe80s

[zachkinstner]

Haha, thanks @CadePeterson. My early 90s school photos:

I'm really happy with the guitar's glow effect so far. Like I mention in the video, it's a very satisfying experience to have the strings glow and react to your hands. It makes the strings feel "warm" or "alive" to me.

[CadePeterson]

omg, that gives me serious flashbacks to my own 80s photo days at school.

[CadePeterson]

Secret (SHhhh! Don't tell), but my elementary school was built and the administration made the mistake of letting the students choose the school mascot and colors—being children of the late 80s, we chose "lasers" and hot pink and purple.
(I can't even make this up if I tried)

BEHOLD! Lopez Lasers!

Yeah, we were rad, even at a young age.

[zachkinstner]

Hah, that's awesome. My middle school let the kids name the groups(?) of students within each grade. I'm pretty sure I was a "Jazzy Jaguar" one year...

[zachkinstner]

DevUp: 2015-09-01

In this video, we look at the new lighting effect that illuminates your hands with the glow of the guitar strings (and the cursors). The chord selectors have a new design, and the strings have new size/vibration features.

We also visit the Unity development environment to review details about the new "thumb gesture" (for major/minor chords) and the strumming mechanics.