/ Forums
New To The Forum? Click Here To Read The How To Guide. -- Developers Click Here.

New Unity UI + OVR Look-Based Input HOWTO

ccsccs Posts: 219
Hiro Protagonist
edited February 2016 in Unity


Edit6: version 5 of example project/code:
https://www.dropbox.com/s/g8ptl7w9xdewp ... 5.zip?dl=0
Version removed old Oculus integration and uses Native VR support + 0.1.2 utils so it works with latest runtimes (0.7 or 0.8).

Edit5: version 4 of example project/code (fixed for and tested with Unity 4.6.4f1):
https://www.dropbox.com/s/3bpz5rgimbxdk ... 4.zip?dl=0
This version also adds support for a cursor that scales size with distance so it will always appear the same size but be at the right depth.

Edit4: version 3 of example project/code:
[removed - see updated example above]

Edit3: New version of example project and code posted:
[removed - see updated example above]

Edit2: If you don't have time to read all of this thread and just want the code and a sample project, I uploaded one here:
[removed - see updated example above]

Edit: fixed some issues with the code to handle InputField and also was selecting wrong game object in some cases.

I have been messing around with the new Unity GUI stuff in 4.6 release, and I figured I would share a basic solution to get up and running quickly with a look-based input system. Here is how to get a basic look UI working:

1. Add some kind of UI element to your scene. This will also add a Canvas object and an EventSystem object if you don't already have them in your scene. All UI elements must be parented by a Canvas.

2. On the UI Canvas object, set the Render Mode to World Space

3. Arrange your Canvas in world space and add more UI elements as you would like. To get started, I would just add some buttons.

4. Under the OVRCameraRig->CenterEyeAnchor object, add a regular Unity Camera at position and rotation 0,0,0, and set this camera's Culling Mask to "Nothing" so it won't actually bother rendering anything and waste CPU/GPU cycles. I named it "Look Camera". This camera is just for use by the UI event system. By putting it under the CenterEyeAnchor object, it tracks perfectly with the head. The OVR cameras don't seem to work with the UI system, probably because they utilize render textures and not screen space rendering. This dummy look camera solves that problem.

5. For every UI Canvas object you have in the scene, drag this "Look Camera" object into the "Event Camera" field.

6. create a new script called BasicLookInputModule.cs. Copy in this code:
using UnityEngine;
using UnityEngine.EventSystems;
using UnityEngine.UI;
using System.Collections;

public class BasicLookInputModule : BaseInputModule {

	public const int kLookId = -3;
	public string submitButtonName = "Fire1";
	public string controlAxisName = "Horizontal";
	private PointerEventData lookData;

	// use screen midpoint as locked pointer location, enabling look location to be the "mouse"
	private PointerEventData GetLookPointerEventData() {
		Vector2 lookPosition;
		lookPosition.x = Screen.width/2;
		lookPosition.y = Screen.height/2;
		if (lookData == null) {
			lookData = new PointerEventData(eventSystem);
		}
		lookData.Reset();
		lookData.delta = Vector2.zero;
		lookData.position = lookPosition;
		lookData.scrollDelta = Vector2.zero;
		eventSystem.RaycastAll(lookData, m_RaycastResultCache);
		lookData.pointerCurrentRaycast = FindFirstRaycast(m_RaycastResultCache);
		m_RaycastResultCache.Clear();
		return lookData;
	}

	private bool SendUpdateEventToSelectedObject() {
		if (eventSystem.currentSelectedGameObject == null)
			return false;
		BaseEventData data = GetBaseEventData ();
		ExecuteEvents.Execute (eventSystem.currentSelectedGameObject, data, ExecuteEvents.updateSelectedHandler);
		return data.used;
	}

	public override void Process() {
		// send update events if there is a selected object - this is important for InputField to receive keyboard events
		SendUpdateEventToSelectedObject();
		PointerEventData lookData = GetLookPointerEventData();
		// use built-in enter/exit highlight handler
		HandlePointerExitAndEnter(lookData,lookData.pointerCurrentRaycast.gameObject);
		if (Input.GetButtonDown (submitButtonName)) {
			eventSystem.SetSelectedGameObject(null);
			if (lookData.pointerCurrentRaycast.gameObject != null) {
				GameObject go = lookData.pointerCurrentRaycast.gameObject;
				GameObject newPressed = ExecuteEvents.ExecuteHierarchy (go, lookData, ExecuteEvents.submitHandler);
				if (newPressed == null) {
					// submit handler not found, try select handler instead
					newPressed = ExecuteEvents.ExecuteHierarchy (go, lookData, ExecuteEvents.selectHandler);
				}
				if (newPressed != null) {
					eventSystem.SetSelectedGameObject(newPressed);
				}
			}
		}
		if (eventSystem.currentSelectedGameObject && controlAxisName != null && controlAxisName != "") {
			float newVal = Input.GetAxis (controlAxisName);
			if (newVal > 0.01f || newVal < -0.01f) {
				AxisEventData axisData = GetAxisEventData(newVal,0.0f,0.0f);
				ExecuteEvents.Execute(eventSystem.currentSelectedGameObject, axisData, ExecuteEvents.moveHandler);
			}
		}
	}	
}


7. Drag this script onto the EventSystem object in the scene.

8. fix Submit Button Name and Control Axis Name to match what you use in the Input Settings if you don't use Unity defaults.

9. disable or remove Standalone Input Module and Touch Input Module on the Event System object.

That's it. Push play. Look around with headset and you should see it highlight the buttons/sliders/etc. as you are looking around. When you hit the submit button it will trigger that UI element (buttons On Value Changed operations will be called if you set any of those up). Sliders get selected when you click them and can then be manipulated with the axis that was chosen on the BasicLookInputModule component.

I hope someone finds this useful since there isn't much out there on how to do this yet.

ccs
«134

Comments

  • kingrocketkingrocket Posts: 50
    Lawnmower Man (or Woman)
    Cool, gonna try this out later.

    Thanks for the writeup.
  • knupknup Posts: 34
    Thanks for sharing! Good to see some custom input module code.
  • jonomfjonomf Posts: 14
    Virtual Boy (or Girl)
    Awesome, I was just about to tackle this with 0.4.3... thanks for posting!
  • virrorvirror Posts: 402
    Lawnmower Man (or Woman)
    Great post! Thanx : )
  • Finally. I've been scouring the internet all night looking for something that works. Thanks for putting this all together with such clear instructions. It's such a relief.

    One problem I'm currently facing now is that I can't actually enter any keyboard input into the InputField UI. Do you have any idea how I might go about doing that while still being able to keep what you've made? I've noticed that unity can still read when I press "space" to bring up the VR menu, so I know the keyboard is not disabled completely.
  • ccsccs Posts: 219
    Hiro Protagonist
    Sorry I haven't tried an input field yet. I'll try that in a minute. There may be something special that needs to happen with keyboard input focus that my basic example doesn't do. I'm only using Submit and Select events in the example code. There are many others to experiment with.

    Maybe add the standalone input module back to the Event System object and re-enable? They should be able to work together.

    BTW - if you use "Fire1" as the submit button and you haven't changed the Unity default input settings, both mouse click and gamepad button 0 (A on xbox controller) will click buttons, select sliders, etc.

    ccs
  • ccsccs Posts: 219
    Hiro Protagonist
    I fixed a minor bug in the code I posted. It has been corrected in the original post:

    Was:
               if (newPressed != null) {
                   eventSystem.SetSelectedGameObject(go);
                }
    

    Is now:
               if (newPressed != null) {
                   eventSystem.SetSelectedGameObject(newPressed);
                }
    

    I can't seem to get the InputField to work either. It responds to the submit action and the cursor starts blinking so I know it is activating, but the keyboard input is not getting picked up to change the text.
  • virrorvirror Posts: 402
    Lawnmower Man (or Woman)
    Did you try posting in the Unity 4.6 beta forum about this?
  • ccsccs Posts: 219
    Hiro Protagonist
    I figured it out by looking at the StandaloneInputModule code here:

    https://gist.github.com/stramit/ce455682b7944bdff0e7

    What I was missing was this procedure that handles the update events that need to get passed to the currently selected UI object:
    	private bool SendUpdateEventToSelectedObject() {
    		if (eventSystem.currentSelectedGameObject == null)
    			return false;
    		BaseEventData data = GetBaseEventData ();
    		ExecuteEvents.Execute (eventSystem.currentSelectedGameObject, data, ExecuteEvents.updateSelectedHandler);
    		return data.used;
    	}
    

    I fixed it in the original post. I make a call to this function right at the beginning of the Process() function. This is what passes on the keyboard input events to the selected object.

    Keep in mind, this doesn't disable all the key bindings made elsewhere in code, like in the OVR code. You will have to find a way to disable those other scripts from trying to also use the keystroke. OVR SDK has a lot of keyboard shortcuts.
  • ccsccs Posts: 219
    Hiro Protagonist
    edited November 2014
    I found something I didn't like too much about the built-in slider/scrollbar axis move handling. It is set at a fixed step of 10%. Unfortunately the step size is not user-settable. So I changed around my Input Module to do custom value setting for Sliders and Scrollbars instead of using the built-in move handlers for these components.

    Instead of updating code in original post, I'm just posting a more comprehensive look-based input module in this post. It includes additional features like setting some values to indicate if the input module consumed the button or axis or whether or not the raytracing hit something. This can be useful if you want to multi-purpose a specific axis for other things. I also added a select color. This make it clear which scrollbar/slider is selected when you aren't looking directly at it. You can still manipulate it when you are looking elsewhere with the axis.

    To enable the smooth-scrolling sliders and scrollbars, just set useSmoothAxis to true (the default). You can adjust the rate by setting smoothAxisMultiplier to different values.

    Here is the more comprehensive look-based input module:
    using UnityEngine;
    using UnityEngine.EventSystems;
    using UnityEngine.UI;
    using System.Collections;
    
    public class LookInputModule : BaseInputModule {
    
    	private static LookInputModule _singleton;
    	public static LookInputModule singleton {
    		get {
    			return _singleton;
    		}
    	}
    	public const int kLookId = -3;
    	public string submitButtonName = "Fire1";
    	public string controlAxisName = "Horizontal";
    	public bool useSmoothAxis = true;
    	public float smoothAxisMultiplier = 0.01f;
    	// if smooth axis is off - the input is stepped - need an interval
    	public float steppedAxisStepsPerSecond = 10f;
    	private bool _guiRaycastHit;
    	public bool guiRaycastHit {
    		get {
    			return _guiRaycastHit;
    		}
    	}
    	private bool _controlAxisUsed;
    	public bool controlAxisUsed {
    		get {
    			return _controlAxisUsed;
    		}
    	}
    	private bool _buttonUsed;
    	public bool buttonUsed {
    		get {
    			return _buttonUsed;
    		}
    	}
    	private PointerEventData lookData;
    	private Color currentSelectedNormalColor;
    	private Color currentSelectedHighlightedColor;
    	public bool useSelectColor = true;
    	public Color selectColor = Color.blue;
    	private float nextAxisActionTime;
    
    	// use screen midpoint as locked pointer location, enabling look location to be the "mouse"
    	private PointerEventData GetLookPointerEventData() {
    		Vector2 lookPosition;
    		lookPosition.x = Screen.width/2;
    		lookPosition.y = Screen.height/2;
    		if (lookData == null) {
    			lookData = new PointerEventData(eventSystem);
    		}
    		lookData.Reset();
    		lookData.delta = Vector2.zero;
    		lookData.position = lookPosition;
    		lookData.scrollDelta = Vector2.zero;
    		eventSystem.RaycastAll(lookData, m_RaycastResultCache);
    		lookData.pointerCurrentRaycast = FindFirstRaycast(m_RaycastResultCache);
    		if (lookData.pointerCurrentRaycast.gameObject != null) {
    			_guiRaycastHit = true;
    		} else {
    			_guiRaycastHit = false;
    		}
    		m_RaycastResultCache.Clear();
    		return lookData;
    	}
    
    	private void SetSelectedColor(GameObject go) {
    		if (useSelectColor) {
    			Selectable s = go.GetComponent<Selectable>();
    			if (s != null) {
    				ColorBlock cb = s.colors;
    				currentSelectedNormalColor = cb.normalColor;
    				currentSelectedHighlightedColor = cb.highlightedColor;
    				cb.normalColor = selectColor;
    				cb.highlightedColor = selectColor;
    				s.colors = cb;
    			}
    		}
    	}
    
    	private void RestoreColor(GameObject go) {
    		if (useSelectColor) {
    			Selectable s = go.GetComponent<Selectable>();
    			if (s != null) {
    				ColorBlock cb = s.colors;
    				cb.normalColor = currentSelectedNormalColor;
    				cb.highlightedColor = currentSelectedHighlightedColor;
    				s.colors = cb;
    			}
    		}
    	}
    
    	public void ClearSelection() {
    		if (eventSystem.currentSelectedGameObject) {
    			RestoreColor(eventSystem.currentSelectedGameObject);
    			eventSystem.SetSelectedGameObject(null);
    		}
    	}
    
    	public void Select(GameObject go) {
    		ClearSelection();
    		SetSelectedColor(go);
    		eventSystem.SetSelectedGameObject(go);
    	}
    
    	private bool SendUpdateEventToSelectedObject() {
    		if (eventSystem.currentSelectedGameObject == null)
    			return false;
    		BaseEventData data = GetBaseEventData ();
    		ExecuteEvents.Execute (eventSystem.currentSelectedGameObject, data, ExecuteEvents.updateSelectedHandler);
    		return data.used;
    	}
    
    	public override void Process() {
    		_singleton = this;
    		// send update events if there is a selected object - this is important for InputField to receive keyboard events
    		SendUpdateEventToSelectedObject();
    		PointerEventData lookData = GetLookPointerEventData();
    		HandlePointerExitAndEnter(lookData,lookData.pointerCurrentRaycast.gameObject);
    		_buttonUsed = false;
    		if (Input.GetButtonDown (submitButtonName)) {
    			ClearSelection();
    			if (lookData.pointerCurrentRaycast.gameObject != null) {
    				GameObject go = lookData.pointerCurrentRaycast.gameObject;
    				GameObject newPressed = ExecuteEvents.ExecuteHierarchy (go, lookData, ExecuteEvents.submitHandler);
    				if (newPressed == null) {
    					// try select handler instead
    					newPressed = ExecuteEvents.ExecuteHierarchy (go, lookData, ExecuteEvents.selectHandler);
    					if (newPressed != null) {
    						Select(newPressed);
    					}
    				} else {
    					InputField infield = newPressed.GetComponent<InputField>();
    					if (infield != null) {
    						Select(newPressed);
    					}
    				}
    				if (newPressed != null) {
    					_buttonUsed = true;
    				}
    			}
    		}
    		_controlAxisUsed = false;
    		if (eventSystem.currentSelectedGameObject && controlAxisName != null && controlAxisName != "") {
    			float newVal = Input.GetAxis (controlAxisName);
    			if (newVal > 0.01f || newVal < -0.01f) {
    				if (useSmoothAxis) {
    					Slider sl = eventSystem.currentSelectedGameObject.GetComponent<Slider>();
    					if (sl != null) {
    						sl.value += newVal*smoothAxisMultiplier;
    						_controlAxisUsed = true;
    					} else {
    						Scrollbar sb = eventSystem.currentSelectedGameObject.GetComponent<Scrollbar>();
    						if (sb != null) {
    							sb.value += newVal*smoothAxisMultiplier;
    							_controlAxisUsed = true;
    						}
    					}
    				} else {
    					_controlAxisUsed = true;
    					float time = Time.unscaledTime;
    					if (time > nextAxisActionTime) {
    						nextAxisActionTime = time + 1f/steppedAxisStepsPerSecond;
    						AxisEventData axisData = GetAxisEventData(newVal,0.0f,0.0f);
    						if (!ExecuteEvents.Execute(eventSystem.currentSelectedGameObject, axisData, ExecuteEvents.moveHandler)) {
    							_controlAxisUsed = false;
    						} 
    					}
    				}
    			}
    		}
    	}	
    }
    
    

    If this gets any longer, I'll post over on github. :)

    edit: made some minor fixes/improvements to the code above since original post:
    1. only things that respond to axis input will get the SelectColor
    2. have option to enable/diable SelectColor
    3. removed the color option that wasn't used
  • Any possibility of creating a sample project or .unitypackage for download?
  • ccsccs Posts: 219
    Hiro Protagonist
    Yeah. I'll try to put one together. :)
  • ccsccs Posts: 219
    Hiro Protagonist
    Here ya go. Includes latest code and a sample scene with a button, slider, scrollable text window, toggle, and input field all working. :)

    https://dl.dropboxusercontent.com/u/87937814/LookInputSample.zip
  • Thanks for getting back so quickly! I I'll be sure to test this out when I have time tomorrow. I'll get back to you if I run into anything else with your code. :D
  • AguaAgua Posts: 37
    Thanks for writing this out.
    I'm just getting into the basics of UI stuff.
    Hasn't been easy to find explanations of UI, especially VR related.
    Always appreciated.
  • kingrocketkingrocket Posts: 50
    Lawnmower Man (or Woman)
    ccs wrote:
    Here ya go. Includes latest code and a sample scene with a button, slider, scrollable text window, toggle, and input field all working. :)

    https://dl.dropboxusercontent.com/u/87937814/LookInputSample.zip

    Slider doesn't work for me in the built demo.
    Edit: ...because I don't have a controller :lol:

    I'm gonna make a simple GUI that follows the camera like a HUD, will post it here once it's done and somewhat useable.

    Can't find any tutorials or examples anywhere else so might as well make them ourselves :D

    Also, you can enable a crosshair with "C" but at the moment it follows the mouse making it pretty much unusable as a tracker for where you're looking. Will disable and see if it's accurate enough to use.

    Building a HUD = easy, making it follow the cam = hard.
  • weasel47weasel47 Posts: 296
    Nexus 6
    To the OP, have you tried this with 5.0 at all? In 5.0.0b11, it throws this error while compiling:

    Assets/Scripts/BasicLookInputModule.cs(48,83): error CS1061: Type `UnityEngine.EventSystems.RaycastResult' does not contain a definition for `gameObject' and no extension method `gameObject' of type `UnityEngine.EventSystems.RaycastResult' could be found (are you missing a using directive or an assembly reference?)
  • ccsccs Posts: 219
    Hiro Protagonist
    I'm stuck on 4.6 for now as my current project won't easily port to 5.0.

    I guess they changed this particular class around. The 4.6 docs show gameObject as a valid field:

    http://docs.unity3d.com/460/Documentation/ScriptReference/EventSystems.RaycastResult.html

    I can't find 5.0 documentation. You might be able to figure this out by opening the code in the editor and poke around with the available fields on RaycastResult. What you need is the game object that was hit by the raycast. That is the UI element you are looking at. That needs to be passed the various events.

    At some point I will dive into 5, but I unfortunately I don't have time to do that right now.
  • weasel47weasel47 Posts: 296
    Nexus 6
    ccs wrote:
    I'm stuck on 4.6 for now as my current project won't easily port to 5.0.

    I guess they changed this particular class around. The 4.6 docs show gameObject as a valid field:

    http://docs.unity3d.com/460/Documentation/ScriptReference/EventSystems.RaycastResult.html

    I can't find 5.0 documentation. You might be able to figure this out by opening the code in the editor and poke around with the available fields on RaycastResult. What you need is the game object that was hit by the raycast. That is the UI element you are looking at. That needs to be passed the various events.

    At some point I will dive into 5, but I unfortunately I don't have time to do that right now.

    Thanks! I had thought that autocomplete would show me what I needed, but it only listed "Equals" and "ReferenceEquals."
    I just right-clicked -> Go to Declaration and found that there's a public GameObject property "go." I guess they changed the name.

    Sure enough, I changed "pointerCurrentRaycast.gameObject" to "pointerCurrentRaycast.go" and it works now!

    It looks like Sliders aren't covered. I'll see if I can figure out what they need...
  • kingrocketkingrocket Posts: 50
    Lawnmower Man (or Woman)
    Has anyone managed to get the a UI component to properly follow the OVRcam?
  • ccsccs Posts: 219
    Hiro Protagonist
    What do you mean, follow the cam?

    If you are using the OVRPlayerController, you can put the UI transforms under ForwardDirection transform and they will always be in front of the player, but you can still look around on the HUD, but I'm not sure that is what you mean.

    If you want something that perfectly tracks the camera, just stick UI transforms under CenterEyeAnchor with a Z of about 0.6 and rotation 180. The UI transform will always be looking back at the camera. You won't be able to use look-based input though because the UI will be fixed on the cam.

    ccs
  • kingrocketkingrocket Posts: 50
    Lawnmower Man (or Woman)
    Ergh, that shouldv'e been obvious... :oops:
    Thanks, whatever I was doing caused it to either show in only one eye or move in one axis.

    Is there a way to make a world space canvas always render over everything else? Using a dummy cam and setting the culling mask to the layer of the canvas doesn't work with the rift. Using sorting layers also doesn't help.

    It'd be kinda immersion breaking if your HUD just decided to clip through walls.
  • ccsccs Posts: 219
    Hiro Protagonist
    Sorting layer and order in layer is just for sorting between UI canvases themselves. This allows you to specify a render order for those if you have multiple canvases.

    I haven't tried this yet, but it looks like you can use a custom material on the UI elements. For the material I'm guessing you will need a custom shader with render queue set as overlay and set ZTest Always. Then assign that material to all of your UI elements. Render queue of overlay makes sure material is rendered after normal geometry, and ztest always makes the shader ignore depth buffer and always render (so there will be no clipping). To make the shader, you'll have to find the built-in UI shader code, make a copy, and modify. I haven't looked into the new UI shaders yet, but you can download the source to all the built-in Unity shaders from unity3d:

    http://netstorage.unity3d.com/unity/builtin_shaders-4.6.0f2.zip

    Some docs on shader cull and depth syntax.
    http://docs.unity3d.com/Manual/SL-CullAndDepth.html
  • weasel47weasel47 Posts: 296
    Nexus 6
    I'm trying to update the OP's script to send all of the events that would be used for a Standalone Input Module as described here:http://docs.unity3d.com/460/Documentation/Manual/script-StandaloneInputModule.html

    There are a lot of things I'm not sure about, but this is what I have so far:
    using UnityEngine;
    using UnityEngine.EventSystems;
    using UnityEngine.UI;
    using System.Collections;
    
    public class BasicLookInputModule : BaseInputModule {
    	
    	public const int kLookId = -3;
    	public string submitButtonName = "Fire1";
    	public string cancelButtonName = "Fire2";
    	public string controlAxisName = "Horizontal";
    	private GameObject dragHandler;
    	public GameObject buttonPressed;
    	public PointerEventData lookData;
    	
    	// use screen midpoint as locked pointer location, enabling look location to be the "mouse"
    	private PointerEventData GetLookPointerEventData() {
    		Vector2 lookPosition;
    		lookPosition.x = Screen.width/2;
    		lookPosition.y = Screen.height/2;
    		if (lookData == null) {
    			lookData = new PointerEventData(eventSystem);
    		}
    		//lookData.Reset();
    		//lookData.delta = Vector2.zero;
    		//need to figure out how to detect movement for these deltas...
    		lookData.delta = new Vector2(0.005f,0.0f);//lookData.position - lookPosition;
    		lookData.scrollDelta = new Vector2(0.005f,0.0f);//lookData.position - lookPosition;
    		lookData.position = lookPosition;
    		//lookData.scrollDelta = Vector2.zero;
    		eventSystem.RaycastAll(lookData, m_RaycastResultCache);
    		lookData.pointerCurrentRaycast = FindFirstRaycast(m_RaycastResultCache);
    		m_RaycastResultCache.Clear();
    		return lookData;
    	}
    	
    	private bool SendUpdateEventToSelectedObject() {
    		if (eventSystem.currentSelectedGameObject == null)
    			return false;
    		BaseEventData data = GetBaseEventData ();
    		ExecuteEvents.Execute (eventSystem.currentSelectedGameObject, data, ExecuteEvents.updateSelectedHandler);
    		return data.used;
    	}
    	
    	public override void Process() {
    		// send update events if there is a selected object - this is important for InputField to receive keyboard events
    		SendUpdateEventToSelectedObject();
    		PointerEventData lookData = GetLookPointerEventData();
    		// use built-in enter/exit highlight handler
    		HandlePointerExitAndEnter(lookData,lookData.pointerCurrentRaycast.go);
    		if (eventSystem.currentSelectedGameObject && controlAxisName != null && controlAxisName != "") {
    			float newVal = Input.GetAxis (controlAxisName);
    			if (newVal > 0.01f || newVal < -0.01f) {
    				AxisEventData axisData = GetAxisEventData(newVal,0.0f,0.0f);
    				ExecuteEvents.Execute(eventSystem.currentSelectedGameObject, axisData, ExecuteEvents.moveHandler);
    			}
    		}
    		if (Input.GetButtonDown (submitButtonName)) {
    						eventSystem.SetSelectedGameObject (null);
    						if (lookData.pointerCurrentRaycast.go != null) {
    								GameObject go = lookData.pointerCurrentRaycast.go;
    								GameObject newPressed = ExecuteEvents.ExecuteHierarchy (go, lookData, ExecuteEvents.submitHandler);
    								if (newPressed == null) {
    										// submit handler not found, try select handler instead
    										newPressed = ExecuteEvents.ExecuteHierarchy (go, lookData, ExecuteEvents.selectHandler);
    								}
    								if (newPressed != null) {
    										eventSystem.SetSelectedGameObject (newPressed);
    								}
    								ExecuteEvents.ExecuteHierarchy (go, lookData, ExecuteEvents.pointerEnterHandler);
    								buttonPressed = ExecuteEvents.ExecuteHierarchy (go, lookData, ExecuteEvents.pointerDownHandler);
    								GameObject tempDragHandler = ExecuteEvents.ExecuteHierarchy (go, lookData, ExecuteEvents.dragHandler);
    								if (tempDragHandler != null)
    										dragHandler = tempDragHandler;
    								ExecuteEvents.Execute (dragHandler, lookData, ExecuteEvents.beginDragHandler);
    
    						}
    				} 
    		else if (Input.GetButton (submitButtonName)) {
    			if (dragHandler != null) {
    					ExecuteEvents.Execute (dragHandler, lookData, ExecuteEvents.dragHandler);
    			}
    			if (eventSystem.currentSelectedGameObject != lookData.pointerCurrentRaycast.go) {
    					ExecuteEvents.ExecuteHierarchy (eventSystem.currentSelectedGameObject, lookData, ExecuteEvents.pointerExitHandler);
    					ExecuteEvents.ExecuteHierarchy (lookData.pointerCurrentRaycast.go, lookData, ExecuteEvents.pointerExitHandler);
    			}
    		} 
    		else if (Input.GetButtonUp (submitButtonName)) {
    			if (buttonPressed !=null)
    			{
    				ExecuteEvents.ExecuteHierarchy (buttonPressed, lookData, ExecuteEvents.pointerUpHandler);
    				if (lookData.pointerCurrentRaycast.go == buttonPressed)
    				{
    					ExecuteEvents.ExecuteHierarchy (buttonPressed, lookData, ExecuteEvents.pointerClickHandler);
    				}
    			}
    			if (dragHandler != null) {
    				ExecuteEvents.Execute (dragHandler, lookData, ExecuteEvents.endDragHandler);
    				ExecuteEvents.Execute (dragHandler, lookData, ExecuteEvents.dropHandler);
    			}
    			dragHandler = null;
    		}
    
    
    		if (Input.GetButtonDown (cancelButtonName)) {
    			eventSystem.SetSelectedGameObject(null);
    			if (lookData.pointerCurrentRaycast.go != null) {
    				GameObject go = lookData.pointerCurrentRaycast.go;
    				GameObject newPressed = ExecuteEvents.ExecuteHierarchy (go, lookData, ExecuteEvents.cancelHandler);
    				if (newPressed == null) {
    					// submit handler not found, try select handler instead
    					newPressed = ExecuteEvents.ExecuteHierarchy (go, lookData, ExecuteEvents.selectHandler);
    				}
    				if (newPressed != null) {
    					eventSystem.SetSelectedGameObject(newPressed);
    				}
    			}
    		}
    	}   
    }
    

    I'm not sure what the difference between "Submit button pressed" and "pointer button pressed" should be if they're both the same button. The script doesn't wait for a full down-up click for the Submit button, but that doesn't seem ideal.

    Sliders are sort of working, but they jump to their max value as soon as they're touched. They do not respond to dragging, but I think that's because the method of using the center of the screen on a dedicated camera will never result in a position delta. That would have to be calculated with some other method (I tried to set the deltas to low values for testing purposes, but that didn't seem to change anything). I would think that a slider would jump to whatever position is clicked even without the delta, but that doesn't seem to be working.

    Anyone have any thoughts or suggestions on this?
  • ccsccs Posts: 219
    Hiro Protagonist
    I have drag and drop working. It required some more changes not captured in my earlier posts and package I posted. I used Unity's own drag and drop sample to test it. I'll have to see if head drag of scrollbars works as well, although I'm not sure you would want to use that in reality so I will make it an option. I'll clean up the code a bit, and package up a new demo with a drag and drop sample in addition to the existing samples, and I'll post the code as well. Shouldn't be too long...
  • weasel47weasel47 Posts: 296
    Nexus 6
    ccs wrote:
    I have drag and drop working. It required some more changes not captured in my earlier posts and package I posted. I used Unity's own drag and drop sample to test it. I'll have to see if head drag of scrollbars works as well, although I'm not sure you would want to use that in reality so I will make it an option. I'll clean up the code a bit, and package up a new demo with a drag and drop sample in addition to the existing samples, and I'll post the code as well. Shouldn't be too long...

    Awesome! I look forward to seeing it!
    I do agree that using one's head to drag scrollbars would probably be weird, but I thought it might be nice to have the option anyway. Options are never bad.
  • ccsccs Posts: 219
    Hiro Protagonist
    Here is the updated example project and code that supports head drag and several other options, including a world space cursor when you are looking at UI elements. Comments are in the code to explain each option. I added Unity's own drag and drop UI sample that works with head drag and drop here.

    Full project including source code (used 4.6.0b21 - still need to try with 4.6.0RC2):
    https://www.dropbox.com/s/3oafblrsk4amuvg/LookInputSampleV2.zip?dl=0

    Code (although I recommend just trying the project above to see how it all works together):
    using UnityEngine;
    using UnityEngine.EventSystems;
    using UnityEngine.UI;
    using System.Collections;
    
    public class LookInputModule : BaseInputModule {
    
    	// singleton makes it easy to access the instanced fields from other code without needing a pointer
    	// e.g.  if (LookInputModule.singleton != null && LookInputModule.singleton.controlAxisUsed) ...
    	private static LookInputModule _singleton;
    	public static LookInputModule singleton {
    		get {
    			return _singleton;
    		}
    	}
    
    	// name of button to use for click/submit
    	public string submitButtonName = "Fire1";
    
    	// name of axis to use for scrolling/sliders
    	public string controlAxisName = "Horizontal";
    
    	// smooth axis - default UI move handlers do things in steps, meaning you can smooth scroll a slider or scrollbar
    	// with axis control. This option allows setting value of scrollbar/slider directly as opposed to using move handler
    	// to avoid this
    	public bool useSmoothAxis = true;
    	// multiplier controls how fast slider/scrollbar moves with respect to input axis value
    	public float smoothAxisMultiplier = 0.01f;
    	// if useSmoothAxis is off, this next field controls how many steps per second are done when axis is on
    	public float steppedAxisStepsPerSecond = 10f;
    
    	// guiRaycastHit is helpful if you have other places you want to use look input outside of UI system
    	// you can use this to tell if the UI raycaster hit a UI element
    	private bool _guiRaycastHit;
    	public bool guiRaycastHit {
    		get {
    			return _guiRaycastHit;
    		}
    	}
    
    	// controlAxisUsed is helpful if you use same axis elsewhere
    	// you can use this boolean to see if the UI used the axis control or not
    	// if something is selected and takes move event, then this will be set
    	private bool _controlAxisUsed;
    	public bool controlAxisUsed {
    		get {
    			return _controlAxisUsed;
    		}
    	}
    
    	// buttonUsed is helpful if you use same button elsewhere
    	// you can use this boolean to see if the UI used the button press or not
    	private bool _buttonUsed;
    	public bool buttonUsed {
    		get {
    			return _buttonUsed;
    		}
    	}
    
    	public enum Mode {Pointer,Submit};
    	// LookInputModule supports 2 modes:
    	// 1 - Pointer
    	//     Module acts a lot like a mouse with pointer locked where you look. Where you look is where 
    	//     pointerDown/pointerUp/pointerClick events are used
    	//     useCursor is recommended for correct precision
    	//     axis control of sliders/scrollbars/etc. is optional
    	// 2 - Submit
    	//     controls are selected and manipulated with axis control only
    	//     submit/select events are used
    	//     in this mode you can't click along a slider/scrollbar to set the slider/scroll value
    	//     useLookDrag option is ignored
    	public Mode mode = Mode.Pointer;
    
    	// useLookDrag allows you to use look-based drag and drop (see example)
    	// and also drag sliders/scrollbars based on where you are looking
    	// only works if usePointerMethod is true
    	public bool useLookDrag = true;
    	public bool useLookDragSlider = true;
    	public bool useLookDragScrollbar = false;
    
    	// useCursor only applies when usePointerMethod is true
    	// the cursor works like a mouse pointer so you can see exactly where you are clicking
    	// not recommended to turn off
    	public bool useCursor = true;
    
    	// the UI element to use for the cursor
    	// the cursor will appear on the plane of the current UI element being looked at - so it adjusts to depth correctly
    	// recommended to use a simple Image component (typical mouse cursor works pretty well) and you MUST add the 
    	// Unity created IgnoreRaycast component (script included in example) so that the cursor will not be see by the UI
    	// event system
    	public RectTransform cursor;
    
    	// when UI element is selected this is the color it gets
    	// useful for when want to use axis input to control sliders/scrollbars so you can see what is being
    	// manipulated
    	public bool useSelectColor = true;
    	public bool useSelectColorOnButton = false;
    	public bool useSelectColorOnToggle = false;
    	public Color selectColor = Color.blue;
    
    	// interal vars
    	private PointerEventData lookData;
    	private Color currentSelectedNormalColor;
    	private bool currentSelectedNormalColorValid;
    	private Color currentSelectedHighlightedColor;
    	private GameObject currentLook;
    	private GameObject currentPressed;
    	private GameObject currentDragging;
    	private float nextAxisActionTime;
    
    	// use screen midpoint as locked pointer location, enabling look location to be the "mouse"
    	private PointerEventData GetLookPointerEventData() {
    		Vector2 lookPosition;
    		lookPosition.x = Screen.width/2;
    		lookPosition.y = Screen.height/2;
    		if (lookData == null) {
    			lookData = new PointerEventData(eventSystem);
    		}
    		lookData.Reset();
    		lookData.delta = Vector2.zero;
    		lookData.position = lookPosition;
    		lookData.scrollDelta = Vector2.zero;
    		eventSystem.RaycastAll(lookData, m_RaycastResultCache);
    		lookData.pointerCurrentRaycast = FindFirstRaycast(m_RaycastResultCache);
    		if (lookData.pointerCurrentRaycast.gameObject != null) {
    			_guiRaycastHit = true;
    		} else {
    			_guiRaycastHit = false;
    		}
    		m_RaycastResultCache.Clear();
    		return lookData;
    	}
    
    	// update the cursor location and whether it is enabled
    	// this code is based on Unity's DragMe.cs code provided in the UI drag and drop example
    	private void UpdateCursor(PointerEventData lookData) {
    		if (cursor != null) {
    			if (useCursor) {
    				if (lookData.pointerEnter != null) {
    					RectTransform draggingPlane = lookData.pointerEnter.GetComponent<RectTransform>();
    					Vector3 globalLookPos;
    					if (RectTransformUtility.ScreenPointToWorldPointInRectangle(draggingPlane, lookData.position, lookData.enterEventCamera, out globalLookPos)) {
    						cursor.gameObject.SetActive(true);
    						cursor.position = globalLookPos;
    						cursor.rotation = draggingPlane.rotation;
    					} else {
    						cursor.gameObject.SetActive(false);
    					}
    				} else {
    					cursor.gameObject.SetActive(false);
    				}
    			} else {
    				cursor.gameObject.SetActive(false);
    			}
    		}
    	}
    		
    	// sets color of selected UI element and saves current color so it can be restored on deselect
    	private void SetSelectedColor(GameObject go) {
    		if (useSelectColor) {
    			if (!useSelectColorOnButton && go.GetComponent<Button>()) {
    				currentSelectedNormalColorValid = false;
    				return;
    			}
    			if (!useSelectColorOnToggle && go.GetComponent<Toggle>()) {
    				currentSelectedNormalColorValid = false;
    				return;
    			}
    			Selectable s = go.GetComponent<Selectable>();
    			if (s != null) {
    				ColorBlock cb = s.colors;
    				currentSelectedNormalColor = cb.normalColor;
    				currentSelectedNormalColorValid = true;
    				currentSelectedHighlightedColor = cb.highlightedColor;
    				cb.normalColor = selectColor;
    				cb.highlightedColor = selectColor;
    				s.colors = cb;
    			}
    		}
    	}
    
    	// restore color of previously selected UI element
    	private void RestoreColor(GameObject go) {
    		if (useSelectColor && currentSelectedNormalColorValid) {
    			Selectable s = go.GetComponent<Selectable>();
    			if (s != null) {
    				ColorBlock cb = s.colors;
    				cb.normalColor = currentSelectedNormalColor;
    				cb.highlightedColor = currentSelectedHighlightedColor;
    				s.colors = cb;
    			}
    		}
    	}
    
    	// clear the current selection
    	private void ClearSelection() {
    		if (eventSystem.currentSelectedGameObject) {
    			RestoreColor(eventSystem.currentSelectedGameObject);
    			eventSystem.SetSelectedGameObject(null);
    		}
    	}
    
    	// select a game object
    	private void Select(GameObject go) {
    		ClearSelection();
    		if (ExecuteEvents.GetEventHandler<ISelectHandler> (go)) {
    			SetSelectedColor(go);
    			eventSystem.SetSelectedGameObject(go);
    		}
    	}
    
    	// send update event to selected object
    	// needed for InputField to receive keyboard input
    	private bool SendUpdateEventToSelectedObject() {
    		if (eventSystem.currentSelectedGameObject == null)
    			return false;
    		BaseEventData data = GetBaseEventData ();
    		ExecuteEvents.Execute (eventSystem.currentSelectedGameObject, data, ExecuteEvents.updateSelectedHandler);
    		return data.used;
    	}
    
    	// Process is called by UI system to process events
    	public override void Process() {
    		_singleton = this;
    
    		// send update events if there is a selected object - this is important for InputField to receive keyboard events
    		SendUpdateEventToSelectedObject();
    
    		// see if there is a UI element that is currently being looked at
    		PointerEventData lookData = GetLookPointerEventData();
    		if (lookData.pointerCurrentRaycast.gameObject) {
    			currentLook = lookData.pointerCurrentRaycast.gameObject;
    		}
    
    		// handle enter and exit events (highlight)
    		// using the function that is already defined in BaseInputModule
    		HandlePointerExitAndEnter(lookData,currentLook);
    
    		// update cursor
    		UpdateCursor(lookData);
    
    		// button handling
    		_buttonUsed = false;
    		if (Input.GetButtonDown (submitButtonName)) {
    			ClearSelection();
    			lookData.pressPosition = lookData.position;
    			lookData.pointerPressRaycast = lookData.pointerCurrentRaycast;
    			lookData.pointerPress = null;
    			if (currentLook != null) {
    				currentPressed = currentLook;
    				GameObject newPressed = null;
    				if (mode == Mode.Pointer) {
    					newPressed = ExecuteEvents.ExecuteHierarchy (currentPressed, lookData, ExecuteEvents.pointerDownHandler);
    					if (newPressed == null) {
    						// some UI elements might only have click handler and not pointer down handler
    						newPressed = ExecuteEvents.ExecuteHierarchy (currentPressed, lookData, ExecuteEvents.pointerClickHandler);
    						if (newPressed != null) {
    							currentPressed = newPressed;
    						}
    					} else {
    						currentPressed = newPressed;
    						// we want to do click on button down at same time, unlike regular mouse processing
    						// which does click when mouse goes up over same object it went down on
    						// reason to do this is head tracking might be jittery and this makes it easier to click buttons
    						ExecuteEvents.Execute (newPressed, lookData, ExecuteEvents.pointerClickHandler);
    					}
    				} else if (mode == Mode.Submit) {
    					newPressed = ExecuteEvents.ExecuteHierarchy (currentPressed, lookData, ExecuteEvents.submitHandler);
    					if (newPressed == null) {
    						// try select handler instead
    						newPressed = ExecuteEvents.ExecuteHierarchy (currentPressed, lookData, ExecuteEvents.selectHandler);
    					}
    				}
    				if (newPressed != null) {
    					lookData.pointerPress = newPressed;
    					currentPressed = newPressed;
    					Select(currentPressed);
    					_buttonUsed = true;
    				}
    				if (mode == Mode.Pointer) {
    					if (useLookDrag) {
    						bool useLookTest = true;
    						if (!useLookDragSlider && currentPressed.GetComponent<Slider>()) {
    							useLookTest = false;
    						} else if (!useLookDragScrollbar && currentPressed.GetComponent<Scrollbar>()) {
    							useLookTest = false;
    							// the following is for scrollbars to work right
    							// apparently they go into an odd drag mode when pointerDownHandler is called
    							// a begin/end drag fixes that
    							if (ExecuteEvents.Execute(currentPressed,lookData, ExecuteEvents.beginDragHandler)) {
    								ExecuteEvents.Execute(currentPressed,lookData,ExecuteEvents.endDragHandler);
    							}
    						} 
    						if (useLookTest) {
    							if (ExecuteEvents.Execute(currentPressed,lookData, ExecuteEvents.beginDragHandler)) {
    								lookData.pointerDrag = currentPressed;
    								currentDragging = currentPressed;
    							}
    						}
    					} else {
    						// the following is for scrollbars to work right
    						// apparently they go into an odd drag mode when pointerDownHandler is called
    						// a begin/end drag fixes that
    						if (ExecuteEvents.Execute(currentPressed,lookData, ExecuteEvents.beginDragHandler)) {
    							ExecuteEvents.Execute(currentPressed,lookData,ExecuteEvents.endDragHandler);
    						}
    					}
    				}
    			}
    		} else if (Input.GetButtonUp(submitButtonName)) {
    			if (currentDragging) {
    				ExecuteEvents.Execute(currentDragging,lookData,ExecuteEvents.endDragHandler);
    				if (currentLook != null) {
    					ExecuteEvents.ExecuteHierarchy(currentLook,lookData,ExecuteEvents.dropHandler);
    				}
    				lookData.pointerDrag = null;
    				currentDragging = null;
    			}
    			if (currentPressed) {
    				ExecuteEvents.Execute(currentPressed,lookData,ExecuteEvents.pointerUpHandler);
    				lookData.rawPointerPress = null;
    				lookData.pointerPress = null;
    				currentPressed = null;
    			}
    		}
    
    		// drag handling
    		if (currentDragging != null) {
    			ExecuteEvents.Execute (currentDragging,lookData,ExecuteEvents.dragHandler);
    		}
    
    		// control axis handling
    		_controlAxisUsed = false;
    		if (eventSystem.currentSelectedGameObject && controlAxisName != null && controlAxisName != "") {
    			float newVal = Input.GetAxis (controlAxisName);
    			if (newVal > 0.01f || newVal < -0.01f) {
    				if (useSmoothAxis) {
    					Slider sl = eventSystem.currentSelectedGameObject.GetComponent<Slider>();
    					if (sl != null) {
    						float mult = sl.maxValue - sl.minValue;
    						sl.value += newVal*smoothAxisMultiplier*mult;
    						_controlAxisUsed = true;
    					} else {
    						Scrollbar sb = eventSystem.currentSelectedGameObject.GetComponent<Scrollbar>();
    						if (sb != null) {
    							sb.value += newVal*smoothAxisMultiplier;
    							_controlAxisUsed = true;
    						}
    					}
    				} else {
    					_controlAxisUsed = true;
    					float time = Time.unscaledTime;
    					if (time > nextAxisActionTime) {
    						nextAxisActionTime = time + 1f/steppedAxisStepsPerSecond;
    						AxisEventData axisData = GetAxisEventData(newVal,0.0f,0.0f);
    						if (!ExecuteEvents.Execute(eventSystem.currentSelectedGameObject, axisData, ExecuteEvents.moveHandler)) {
    							_controlAxisUsed = false;
    						} 
    					}
    				}
    			}
    		}
    	}	
    }
    
  • kingrocketkingrocket Posts: 50
    Lawnmower Man (or Woman)
    ccs wrote:


    Awesome, didn't know that with the materials. Just changed one of the default shaders, kinda hacky but works.

    I noticed that there's an "OVRGUIShader" that comes with the Oculus package, but it overwrites the (GUI) text's properties. There's also a handy "GUIHUD" material.

    Although, having said that, the GUI/Text Shader works perfectly! No messing around needed.

    For now, I just made a copy of the GUIHUD material and put the GUI/Text Shader on that.
    That saved me some serious time.
  • weasel47weasel47 Posts: 296
    Nexus 6
    ccs wrote:
    Here is the updated example project and code that supports head drag and several other options, including a world space cursor when you are looking at UI elements. Comments are in the code to explain each option. I added Unity's own drag and drop UI sample that works with head drag and drop here.

    Full project including source code (used 4.6.0b21 - still need to try with 4.6.0RC2):
    https://www.dropbox.com/s/3oafblrsk4amuvg/LookInputSampleV2.zip?dl=0

    Thanks for doing this. I tested it out and it works pretty well. I'm using head movement for sliders. I encountered weird bugs twice, but was unable to reproduce them:
    Once, I got "stuck" on a slider and couldn't interact with anything else afterwards. The slider kept responding to my head movement and wouldn't respond to the fire1 button.
    The next time, while trying to reproduce that bug, I ended up with two different sliders recolored with the "select color." Everything still functioned fine, and I could still only interact with one slider at a time.

    Another issue which I think is inherited from the way GUI elements are supposed to work by default, and I haven't looked into how to deal with it yet: selected elements can still be interacted with when the cursor is outside of the canvas. This is problematic for me because I have a 3D cursor that will be used to interact with other objects besides the gui. I appreciate the extra functions that you put in to deal with the opposite (detecting when input is going to the gui)!
  • ccsccs Posts: 219
    Hiro Protagonist
    I'll have to keep playing with it to see if I can work out the kinks. I haven't encountered the double selection or frozen selection issue. I'm not perfectly replicating how the pointer stuff works in StandaloneInputModule, so I am probably missing some minor detail. Unity has some pretty complex handling in there and I was trying to simplify here for clarity and for my own sanity as well. :)

    I also have world interaction I am doing outside of the UI stuff where I want to use the same button for selecting these other game objects and I have a separate raycast routine for them. I also want to reuse the controller axis for other purposes when not interacting with the UI. For that I made LookInputModule function ClearSelection public, and I call that in my other input stuff when button is clicked while raycasting over one of those other objects outside of the UI. In the code I posted, ClearSelction was private, but it is a useful public function. In your other code you could call:

    if (!LookInputModule.singleton.guiRaycastHit) {
    LookInputModule.singleton.ClearSelection();
    }

    You could actually put this into LookInputModule Process() if you want the selection to clear when you are looking away from UI stuff, or another option is to not accept the input button and axis control when guiRaycastHit is true. That way UI only has controller focus if you are looking at some UI element. This has given me a good idea to add another option to my example to control that: focusOnlyWhenUILook: when not looking at any UI element, ignore the controller/keyboard/etc.

    Another option is to simply remove the Select() calls. The Select() is only there so UI elements like scrollbars/sliders get the controller axis focus and can be used while not looking at the slider/scrollbar, but if you don't need that you can remove it. InputField won't work without selection though, so if you need that you might keep the Select() in there and use the ClearSelection() method.
«134
Sign In or Register to comment.