cancel
Showing results for 
Search instead for 
Did you mean: 

New Unity UI + OVR Look-Based Input HOWTO

Anonymous
Not applicable
LookInputModule.jpg

Edit6: version 5 of example project/code:
https://www.dropbox.com/s/g8ptl7w9xdewp ... 5.zip?dl=0
Version removed old Oculus integration and uses Native VR support + 0.1.2 utils so it works with latest runtimes (0.7 or 0.8).

Edit5: version 4 of example project/code (fixed for and tested with Unity 4.6.4f1):
https://www.dropbox.com/s/3bpz5rgimbxdk ... 4.zip?dl=0
This version also adds support for a cursor that scales size with distance so it will always appear the same size but be at the right depth.

Edit4: version 3 of example project/code:
[removed - see updated example above]

Edit3: New version of example project and code posted:
[removed - see updated example above]

Edit2: If you don't have time to read all of this thread and just want the code and a sample project, I uploaded one here:
[removed - see updated example above]

Edit: fixed some issues with the code to handle InputField and also was selecting wrong game object in some cases.

I have been messing around with the new Unity GUI stuff in 4.6 release, and I figured I would share a basic solution to get up and running quickly with a look-based input system. Here is how to get a basic look UI working:

1. Add some kind of UI element to your scene. This will also add a Canvas object and an EventSystem object if you don't already have them in your scene. All UI elements must be parented by a Canvas.

2. On the UI Canvas object, set the Render Mode to World Space

3. Arrange your Canvas in world space and add more UI elements as you would like. To get started, I would just add some buttons.

4. Under the OVRCameraRig->CenterEyeAnchor object, add a regular Unity Camera at position and rotation 0,0,0, and set this camera's Culling Mask to "Nothing" so it won't actually bother rendering anything and waste CPU/GPU cycles. I named it "Look Camera". This camera is just for use by the UI event system. By putting it under the CenterEyeAnchor object, it tracks perfectly with the head. The OVR cameras don't seem to work with the UI system, probably because they utilize render textures and not screen space rendering. This dummy look camera solves that problem.

5. For every UI Canvas object you have in the scene, drag this "Look Camera" object into the "Event Camera" field.

6. create a new script called BasicLookInputModule.cs. Copy in this code:

using UnityEngine;
using UnityEngine.EventSystems;
using UnityEngine.UI;
using System.Collections;

public class BasicLookInputModule : BaseInputModule {

public const int kLookId = -3;
public string submitButtonName = "Fire1";
public string controlAxisName = "Horizontal";
private PointerEventData lookData;

// use screen midpoint as locked pointer location, enabling look location to be the "mouse"
private PointerEventData GetLookPointerEventData() {
Vector2 lookPosition;
lookPosition.x = Screen.width/2;
lookPosition.y = Screen.height/2;
if (lookData == null) {
lookData = new PointerEventData(eventSystem);
}
lookData.Reset();
lookData.delta = Vector2.zero;
lookData.position = lookPosition;
lookData.scrollDelta = Vector2.zero;
eventSystem.RaycastAll(lookData, m_RaycastResultCache);
lookData.pointerCurrentRaycast = FindFirstRaycast(m_RaycastResultCache);
m_RaycastResultCache.Clear();
return lookData;
}

private bool SendUpdateEventToSelectedObject() {
if (eventSystem.currentSelectedGameObject == null)
return false;
BaseEventData data = GetBaseEventData ();
ExecuteEvents.Execute (eventSystem.currentSelectedGameObject, data, ExecuteEvents.updateSelectedHandler);
return data.used;
}

public override void Process() {
// send update events if there is a selected object - this is important for InputField to receive keyboard events
SendUpdateEventToSelectedObject();
PointerEventData lookData = GetLookPointerEventData();
// use built-in enter/exit highlight handler
HandlePointerExitAndEnter(lookData,lookData.pointerCurrentRaycast.gameObject);
if (Input.GetButtonDown (submitButtonName)) {
eventSystem.SetSelectedGameObject(null);
if (lookData.pointerCurrentRaycast.gameObject != null) {
GameObject go = lookData.pointerCurrentRaycast.gameObject;
GameObject newPressed = ExecuteEvents.ExecuteHierarchy (go, lookData, ExecuteEvents.submitHandler);
if (newPressed == null) {
// submit handler not found, try select handler instead
newPressed = ExecuteEvents.ExecuteHierarchy (go, lookData, ExecuteEvents.selectHandler);
}
if (newPressed != null) {
eventSystem.SetSelectedGameObject(newPressed);
}
}
}
if (eventSystem.currentSelectedGameObject && controlAxisName != null && controlAxisName != "") {
float newVal = Input.GetAxis (controlAxisName);
if (newVal > 0.01f || newVal < -0.01f) {
AxisEventData axisData = GetAxisEventData(newVal,0.0f,0.0f);
ExecuteEvents.Execute(eventSystem.currentSelectedGameObject, axisData, ExecuteEvents.moveHandler);
}
}
}
}



7. Drag this script onto the EventSystem object in the scene.

8. fix Submit Button Name and Control Axis Name to match what you use in the Input Settings if you don't use Unity defaults.

9. disable or remove Standalone Input Module and Touch Input Module on the Event System object.

That's it. Push play. Look around with headset and you should see it highlight the buttons/sliders/etc. as you are looking around. When you hit the submit button it will trigger that UI element (buttons On Value Changed operations will be called if you set any of those up). Sliders get selected when you click them and can then be manipulated with the axis that was chosen on the BasicLookInputModule component.

I hope someone finds this useful since there isn't much out there on how to do this yet.

ccs
105 REPLIES 105

Anonymous
Not applicable
That is coming from the Oculus API. I'm still using 4.6.4 for my projects, although I have tinkered with 5.0 on the side. I think current OVR API is not 100% compatible with 5.0, but will still work. I know Unity just came out with the beta that integrates OVR support internally, but I haven't tried it so I don't know how well it works.

Basically Oculus and Unity need to get this sorted out to have a cleaner solution for 5.* going forward.

http://unity3d.com/unity/beta

In the beta release notes:
VR: Added integrated support for Oculus Rift in Editor and Standalone Players. Once Virtual Reality is enabled in Player Settings, pressing play in the editor will show your game view on the rift. It is recommended to "Maximize on Play" the game view in order to achieve smooth rendering. Be sure to remove any previous Oculus plugins from your project.

ndaviden
Honored Guest
Hi again css. Your work has been incredibly helpful!

Would you be able to turn your demo into a package that could be imported into any project? That way it would be easy to tinker with all that you made for the demo.

Thanks again!

Anonymous
Not applicable
Yeah, let me see about making a package. I agree that is cleaner. For the demo to work right I needed to add stuff on the OVR center eye anchor, so the package will include an OVR camera rig. You will have to be selective on import if you don't want that.

ccs

Anonymous
Not applicable
Hey, any news on this project ? It's pretty great ! Been playing with it in Unity 5.

A great addition would be the ability to click buttons by looking at them for an X amount of seconds. Combined with some sort of animation to make it clear what is happening.
I will dive in the code myself to see If i can manage it myself but I'm new to programming in Unity, all the tips are welcome 🙂

ZY1991
Honored Guest
Awesome work! But I have a question to ask. I am developing with Gear VR. I have trouble with calling a keyboard in Gear VR. Using your demo, I can click the InputField but it doesn't call a Android's mobile keyboard. Do you know how to call the keyboard so I can input using GearVR's touchPad? Thank you!

Suwappertjes
Honored Guest
Hey ccs! Thx a lot for the program it seems to work pretty damn well... unless you turn the standard InputModule back on.
I made a worldspace canvas that follows the player around and automatically selects the first button when you press p: a pauze menu (HUD style). However, this "automatically selecting the first button" doesnt work with the InputModule off, and when I turn the Module on all kinds of weird things start to happen. Do you know how to fix this? And/or can you tell me what piece of code I need to add to have that "automatically select the first button" feature?

(PS: I understand this inputmodule is for pressing buttons on static world objects (and not in a HUD because you cant specifically look at a button when it follows you around) but i want to have both my pauze menu working AND use your script for selecting other UI elements within the world)

SDAS_Hodgson
Explorer
*Slow Clap*

You gentlemen are the future. Thank you for this wonderful example.

I wonder how much Unity is going to integrate when it comes to the uGUI and VR support. I haven't seen much news on that.

wazmack
Honored Guest
Would this work with gear VR?
we want interactive videos, with hotspots that when you hover your crosshair over, 2d content pops up..

anyone thing they can help my company out on this one?

Anonymous
Not applicable
I have trouble with calling a keyboard in Gear VR. Using your demo, I can click the InputField but it doesn't call a Android's mobile keyboard.


On Android, the soft keyboard is an additional view that pops up on phone in 2d fashion which can't be displayed in a 3d app. You want all GearVR applications to disable the built-in soft keyboard, which is already taken care of for you by the setup that Oculus provides for GearVR. To solve this, you have to create your own world-space keyboard using UI Buttons arranged in a keyboard pattern, and have the buttons pass key events to the input field. It is quite a pain to rig all that up, but it works. I made a GearVR webbrowser (not released) that uses the UI shown in this post and can enter text input this way. Do note that bluetooth keyboards work with GearVR with no additional code.

Would this work with gear VR?


Yes this package works with GearVR. Just change Submit Button Name on the LookInputModule component to "Fire" and the GearVR touchpad will work here to click buttons and sliders. You can enable Use Look Drag to allow dragging sliders/scrollbars.

For hover-over to trigger some action (submit or pop-up another UI element), you will need to add some code right after this part:

// see if there is a UI element that is currently being looked at
PointerEventData lookData = GetLookPointerEventData();
currentLook = lookData.pointerCurrentRaycast.gameObject;


If currentLook is not null, it will be the currently looked-at UI game object. Here you could see if that currentLook object has a specific custom component, and if so, trigger it to do something, like this:

if (currentLook != null) {
FooComponent foo = currentLook.GetComponent<FooComponent>();
if (foo != null) {
foo.callCustomAction();
}
}


And/or can you tell me what piece of code I need to add to have that "automatically select the first button" feature?


Just use the Select(GameObject go) function provided in LookInputModule.cs. That function will deselect current selected UI object, if any, and select the passed UI object. In the code where you handle "p" to bring up your menu, add a call to this to select the button you want selected when the menu comes up.

OculusDev
Honored Guest
Hey,

Hope this thread is still active...I've just stumbled across this as the perfect example of what I'm trying to achieve. I've downloaded the dropbox code from post #1 and the UI works really well in Rift (Unity 5.1.1, Rift DK2 with runtime 6.0.0).

I have an issue that I'm trying to solve with the UI button. If I look at the button and click on it then the button continues to be rendered in a "clicked" state even when I look away from it. I have to click on a non-UI element to deselect the button. I've tried changing the button's "Navigation" option from "Auto" to "None" but it still doesn't work as expected. I'm working through the BasicLookInputModule code to understand the behavior but if anybody else has seen (and preferably solved!) this then please let me know. Thanks in advance.