Unity - Interactable 2D UI Labels in 3D

Unity3D
, ,
#1

Hi there. I’m working on a project that requires a 2D UI label with buttons to exist in 3D, while also being able to use the mouse or zSpace stylus to properly interact with it, and I’m having some trouble.

For context, here are my current requirements for this label:

  • This label needs to exist in 3D space and move with the body. (I’ve got this working)
  • This label needs to face the camera. (I’ve got this working)
  • This label needs to always render on top of other 3D models.
  • This label needs to be interactable with the mouse and stylus controls.

Right now, the last two bullets are causing some issues. I seem to be able to get one bullet working at the expense of the other. I’ve tried using a number of different shaders and messed with ZWrite and ZTest settings to little success.

Here’s what seems to be happening for different shaders:

  • With the Unity default UI shader or Unity Standard shader, the label renders behind 3D models (which is not ideal) and correctly has the mouse and stylus rendering on top of the label.
  • With a modified shader using the default UI shader as a base and changing ZWrite/ZTest to Off and making some minor modifications to the vertex function, I’m able to get the label to appear over 3D models, but this also has the side effect of causing the mouse and stylus to render behind the label, making it a bad user experience.

Does anyone have advice for things I could try? I’m nervous about editing the stylus and mouse shaders, since they’re working so well with zSpace.

#2

It sounds like you’re overthinking/overcomplicating things for yourself. Unity’s canvas based UI is already fully capable of doing what you want, and zCore is written to integrate with and extend the UI event system.

Here is a sample project/scene demonstrating so.
https://cdn.zspace.com/pdt-build-review-images/3d_ui_sample.zip

Take note of the ZGraphicRaycaster component attached to the canvas and the ZInputModule component attached to the ZEventSystem object which replaces Unity’s standard InputModule component. With those two items, the ZStylus and ZMouse are fully capable of interacting with unity’s UI.

With regards to your intention to force UI to always draw on top of 3D elements, DON’T. Figure out another way. Scenes are very hard to look at when they’re littered with depth occlusion violations, especially for users that are already sensitive to stereo VR experiences to begin with. Our eyes don’t naturally know how to handle focusing on one object drawing on top of another but at a deeper depth. Think of the UI as a normal physical object occupying 3D space just like any other. It may take a little extra effort to ensure that the UI elements are accessible, but it’s well worth the effort for the pain you’re going to spare your users.

1 Like