Support for LookingGlass autostereoscopic display

[shole]

I've added support for the LookingGlass autostereoscopic 3d display.
It's basically a display that renders 45 horizontal views and then you can look around freely within that lightfield space.

Display website;
https://lookingglassfactory.com/

Here's a short video of doom on the display, showing the 3d effect;
https://imgur.com/a/CnPRi5o
(Yes, you can literally peek around corners now)

I used the existing stereo rendering system as base, added 45 eyes and implemented my own eye merge shader.
Rather than saving each eye to a texture, they are rendered directly additively in sequence, where undesirable subpixels are just black.

Relevant code is in the main branch here;
https://github.com/shole/gzdoom

I've shared and discussed this on discord and biospud recommended I post this here.
You can build and run it without the display but it will look petty messy.
If you set vr_ipd to 0 it's rendered flat and viewable on normal displays.
Correct way to run this would be
gzdoom +vr_mode 15 -width 2560 -height 1600

There's a few caveats I have for current implementation;

• Each display is unique, so they need to have calibration data entered for it to display correctly.
  These are currently implemented as cvars you have to type in manually.
  They could be read from the USB HID device and I would welcome some help with that if that were to be implemented.
• The display depth is shallow, and distant focus with default parameters is unplayably blurry.
  I have currently fixed this by adjusting frustumShift to fit the scene depth within the display's optimal range.
  I don't know if I can justify this kind of magic number screen adjustment.
  But if not corrected for, this should at least be communicated to the user somehow.
• Another effect of adjusting frustumShift is that gun is still rendered at it's original depth and is now underground.
  This is an issue with regular stereo modes too so I think it would be useful to make a separate feature to match gun depth to vr_screendist/vr_idp or to make it adjustable.
• It seems that this is triggering a bug where ui and menu turn invisible when I open a door.
  Some side effect of having 45 eyes?

[biospud]

That is a very interesting display! I wish I had one to test on. I compiled and ran your code. The business with the menus disappearing definitely needs to be fixed.

The code here overlaps some of the changes in the vulkan branch, currently under development. It might be worth waiting until the vulkan branch gets incorporated into master, assuming that might happen somewhat soon.

We will also need to add entries to wadsrc/static/menudef.txt and language.csv, or wherever those strings go now.

[shole]

I did not yet bother adding them to the menu as manually setting the calibration in cvars is required anyway.
If the USB HID device read was done then it would be fully automatic and make sense to be in the menus.

[biospud]

The closest thing to a HID device in the gzdoom code might be the joystick detection, but I'm unsure exactly how the joystick stuff works, nor whether it would be a good example of what you need.

Do I understand correctly that all 45 views are offset only horizontally, and there is no relative vertical offset? If so, the use of the term "light field" for this display might be a bit of an exaggeration. When there is only one dimension of variation, this would be equivalent to putting a calibrated lenticular screen over a monitor, yes?

I understand that you did not exactly follow the flow of the two-eye stereo modes, because one shader cannot access 45 different textures at once. It looks like you are still using 45 little eye framebuffers plus textures for the eye buffers. That seems inefficient.

I don't understand why items at distance would be blurry. Is it because you place your face very close to the display? In that case, it seems reading glasses would help, but I guess that sorta defeats the purpose.

I still think it would be useful to have this mode in the menus. Most folks cannot use most of the other modes as they are; yet the presence of the different modes teach something about what is possible. And in my opinion this one looks rather interesting on an ordinary display. More importantly, I like to use the menu to quickly scan through and sanity check the various 3D modes for testing and maintenance, even when I'm not using the exact eyewear or display.

[shole]

The closest thing to a HID device in the gzdoom code might be the joystick detection, but I'm unsure exactly how the joystick stuff works, nor whether it would be a good example of what you need.

Existing APIs might be enough and I played around with it a bit but couldn't figure out how to read a page from the device.
The procedure itself is really simple. The C SDK was just released yesterday so I'll need to review it.

Do I understand correctly that all 45 views are offset only horizontally, and there is no relative vertical offset? If so, the use of the term "light field" for this display might be a bit of an exaggeration. When there is only one dimension of variation, this would be equivalent to putting a calibrated lenticular screen over a monitor, yes?

Yes, horizontal only. It's still technically a lightfield with all the properties that come with it - just a very thin one.
Practically it is just a tilted lenticular lens, but also has a huge glass brick in front of it that also works to augment the optics.
Autostereoscopic is possibly more descriptive of the horizontal-only nature, but sells short that you can still actually move forward/back, left/right within that volume and peek around corners etc.
If there was a cracked door, you could lean forward and peek through the crack.
But if there was a keyhole, you could lean forward and peek through the keyhole, but it would be vertically squished, unlike a full lightfield.

I understand that you did not exactly follow the flow of the two-eye stereo modes, because one shader cannot access 45 different textures at once. It looks like you are still using 45 little eye framebuffers plus textures for the eye buffers. That seems inefficient.

Hmm, looks like i need to rereview the eye render code path.

I don't understand why items at distance would be blurry. Is it because you place your face very close to the display? In that case, it seems reading glasses would help, but I guess that sorta defeats the purpose.

Blurry is maybe the wrong word.
The pixel offset at distance becomes larger than a single lenticular cell visible angle and it jumps onto the next pixel over, which is a visual artifact and effectively looks blurry.
Doom depth is also centered around the gun/menu and most of the scene is in negative depth only, wasting half of the screen's lenticular depth range.

I still think it would be useful to have this mode in the menus. Most folks cannot use most of the other modes as they are; yet the presence of the different modes teach something about what is possible. And in my opinion this one looks rather interesting on an ordinary display. More importantly, I like to use the menu to quickly scan through and sanity check the various 3D modes for testing and maintenance, even when I'm not using the exact eyewear or display.

Yea fair enough.