What is the suggested method for integrating UI Toolkit with the zSpace camera system on Unity? I have built a basic overlay UI for use on the Inspire platform using UI Toolkit. It shows up in the editor and in Play mode but when building the executable to use stereo on the Inspire, the UI disappears visually. The mouse interactions still work with the UI, it just does not show up.
Is there a method to integrate UI Toolkit visuals with the zSpace camera system for stereo viewing?
Hey cyjoe,
When you imported ZCore into your project, it should have come with some sample scenes. I advise using those as reference for how to set up UI elements on zSpace. The first key concept to understand will be that you cannot use Unity UI canvases in Screen space modes on zSpace. They must be configured to exist in World space.
Try looking at Assets/zSpace/Core/Samples/Scenes/06_ZPointer_MaxHitRadius.unity for reference, among others in that directory.
I advise keeping most UI attached to the ZCameraRig->ZCanvas GameObject so that it aligns to the viewport. You can also detach UI from the viewport and let it float freely within the scene, but be wary of letting UI get too far from the viewport plane. Parallax ghosting tends to become problematic for text when this is done.
Alex S
Hey Alex,
As far as I can tell, all of those Samples use the Unity UI (uGUI) framework. I’m attempting to use the UI Toolkit framework. Do you know if there is a sample that uses the UI Toolkit Framework for building a UI?
I have experimented with moving it around the scenegraph including under the ZCameraRig->ZCanvas object. Detaching it does not seem to allow it to show up visually either. The buttons work as input, they are just not being rendered by the zCameraRig in stereo. I’m hoping you can direct me to a sample that uses UI Toolkit or confirm if UI Toolkit is supported at all.
Hey cyjoe,
My mistake, we do not support UI Toolkit at this time. This is due to UI Toolkit’s own lack of world-space support. We can’t use it for pretty much the same reason that HMD app developers can’t use it. If it’s not world-space, then it can’t be integrated into the scene’s depth and obey occlusion.
UI Toolkit looks pretty great, but world-space support It has been a bullet point on the roadmap almost as long as UI Toolkit has been around, so I’m not sure if/when we’ll be able to take advantage of it.
https://portal.productboard.com/rcczqdfvurr8zuws3eth2ift/c/290-display-ui-in-world-space?&utm_medium=social&utm_source=starter_share
Alex S
I’m trying to do a post-render overlay interface. So depth sorting and occlusion have zero impact on what I’m trying to do.
Either way. Good to know you don’t officially support it. I will find a work around. Thanks.
Hey cyjoe,
In that case, if you have the skills to manage the render pipeline yourself, then I would recommend examining ZCamera.cs. There you’ll find the per-eye render textures and a special presentation endpoint that you’ll need to integrate with.
Be aware that without UI depth, you can’t draw any 3d content in negative parallax without depth occlusion violations when that content passes over the UI elements. This is particularly important if you intend to implement the classic zSpace stylus interaction scheme where you extend a beam out into the scene from the tip of the stylus. It will always draw behind the UI because its a part of the 3d scene. I don’t know if you intend to use the stylus, but if you do, you’ll probably have to compromise by coming up with some alternative like a viewport-locked cursor instead. That, or maybe a special 3rd pass just for the stylus beam that draws after the UI to make up the negative parallax portion of the beam.
Alex S
I am aware of those issues Alex. Thank you for the suggestions.