New To The Forum? Click Here To Read The How To Guide. -- Developers Click Here.

Capsule - CUDA-based omni-stereo image viewer

joanjoan Posts: 124
Hiro Protagonist
edited March 2016 in Showcase
This is a small viewer for omni stereo images. Your feedback is appreciated :D

Capsule-0.3.0-x64.zip

The application does basically one thing: display omnidirectional stereoscopic images to the Rift.

CBbH907.png

Supported formats
  • Equirectangular projections in Left/Right and Over/Under configurations.
  • Stereo-cubemaps in the "long strip" format used by GearVR "Oculus 360 Photos" and OTOY's ORBX app.
  • *.vr.jpeg files produced by Google Cardboard Camera app.

Usage
  • Open files by using the menu or by dragging and dropping from Windows Explorer.
  • Navigate through files in the current directory with the Left and Right arrow keys.
  • Use the UI options to control the quality/performance tradeoff.

System requirements
  • Nvidia GPU with CUDA compute capability 3.0+. (GTX 600 series or greater).
  • Windows 7, 8 or 10 in x64 and with .NET 4.5.
  • Oculus runtime 0.8.0.0.
  • Face with 2 or more eyes.

Design goals
  • Being able to quickly view OTOY's Render the Metaverse stereo cubemaps on the DK2.
  • Being able to compare and study omni-stereo images from different sources and formats.
  • Improving my CUDA programming and exploring the CUDA/Oculus interop.

Performances
  • I maintain the 75fps on my GTX 660 at 2.0 pixel density for a 18K stereocubemap and 8K equirect.
  • Note that the equirectangular kernel is slightly slower than the cubemap one. Adjust pixel density accordingly if you get judder.
  • If you get flickering on one eye, make sure there is no other GPU heavy application running.

Please keep in mind that this is baked-in stereo, you have 2 DOF. Avoid translating your head too much and tilting. A slight distortion wobble is to be expected when rotating your head. If there is a lot of wobble it might be coming from a higher than normal latency, verify this with the performance hud (F11).

Thanks for reading. Your feedback is appreciated :idea:

Comments

  • cyberealitycybereality Posts: 22,414 Oculus Staff
    Very cool.
    AMD Ryzen 7 1800X | MSI X370 Titanium | G.Skill 16GB DDR4 3200 | EVGA SuperNOVA 1000 | Corsair Hydro H110i
    Gigabyte RX Vega 64 x2 | Samsung 960 Evo M.2 500GB | Seagate FireCuda SSHD 2TB | Phanteks ENTHOO EVOLV
  • dithernetdithernet Posts: 25
    Lawnmower Man (or Woman)
    Awesome. Very goog job. :shock:
  • joanjoan Posts: 124
    Hiro Protagonist
    Updated to 0.3.0 (edited links in OP).
    • Added support for Google Cardboard "VR photos".
    ----
    These stereo images are starting to crop up, and we couldn't experience them on the high-end headsets? Totally unacceptable :D

    Some random thoughts about these files,
    - The stereo effect is indeed good.
    - Some of the files posted are more than 9000 pixels wide per eye, enough to give a quality result on the DK2.
    - Unfortunately the nadir and zenith holes are massive, leaving about only 70° vertical fov. You always see the holes in front of you so that's quite an immersion breaker.
    - We'll get a lot of casually taken images with moving objects, breaking stereo.
    - Overall I think it's a pretty damn fine engineering achievement to go from someone spinning around holding a phone to a panoramic stereoscopic image. Kudos to the team at Google for this app.
  • stupendousmanstupendousman Posts: 1
    Virtual Boy (or Girl)
    What to do, when it says "failed head initialization" ?

    Works with other apps/games.
  • joanjoan Posts: 124
    Hiro Protagonist
    What to do, when it says "failed head initialization" ?
    Can you confirm that you are using runtime 0.8.0.0?
    I will push a new version with proper logging to diagnose such issues.
  • fs_at12fs_at12 Posts: 1
    Virtual Boy (or Girl)
    hi joan

    I wanted to try your viewer, but for me i get the same error
    "Failed headset initialization"

    i can not run SDK 0.8.0.0, my system don´t work (Win 8.1), so i had to go back to the last working SDK 0.5.0.1.
    do you have a solution?
  • joanjoan Posts: 124
    Hiro Protagonist
    fs_at12 wrote:
    i can not run SDK 0.8.0.0, my system don´t work (Win 8.1), so i had to go back to the last working SDK 0.5.0.1.
    do you have a solution?
    The application was built with SDK 0.8 and thus should only support runtimes 0.7 and 0.8 (I only ever tested on 0.8 though).
    Windows 8.1 works well with rutime 0.8 (it's what I run), you do need to have the latest Nvidia drivers. Maybe try the runtime switcher if you have applications requiring older runtimes.
  • Thanks Joan! Worked fine with both spherical and cubemaps, using DK2
  • ClodoClodo Posts: 61
    Hiro Protagonist
    Any image i tested (resolution doesn't matter), it seem locked to 60 FPS (Confirmed by Render Timing, Compositor Frame Rate: 60 hz). Any other player works perfectly (Virtual Desktop, MaxVR).
    GeForce GTX 980 Ti - Intel Core i7-6700 CPU @3.40Ghz (8 CPU)
    Any idea? Thanks.

    p.s. if you need some kind of stress test, i successfully render ODS stereo images with POV-Ray, so i can render in any kind of resolution.
  • joanjoan Posts: 124
    Hiro Protagonist
    Clodo wrote:
    It seem locked to 60 FPS (Confirmed by Render Timing, Compositor Frame Rate: 60 hz). Any other player works perfectly (Virtual Desktop, MaxVR).
    GeForce GTX 980 Ti - Intel Core i7-6700 CPU @3.40Ghz (8 CPU)

    That is strange. Would you happen to have a dual GPU system? I also have a GTX 980Ti now, and I don't reproduce this issue.
  • ClodoClodo Posts: 61
    Hiro Protagonist
    I have an Intel Core i7-6700 CPU @3.40Ghz (8 CPU)
    with only one GeForce GTX 980 Ti.

    It really seem locked to 60 fps...
    289965a1852d8e906a5e351b4f59beaece9aa6f0.png
    Changing pixel-density or queue-ahead don't change anything.
    I don't have any issue with other software for DK2, only this.
    I don't have any idea about what i can try, sorry.


    Apart this issue, i'm very very very interested in your player.
    I made a patch for POV-Ray to render ODS images. Here you can find high-resolution examples that maybe can be useful for stress-test of your image viewer. I can render in more highest resolution if you need that.

    .... WOW i notice just now you are the same guy that reply to me in this other topic!

    I still think it's incredible that is not currently possible to build a pre-rendered video with a correct resolution (at least 6000 x 6000 as you explain to me) at correct 90 framerate (Rift CV1). Apart the codec problem, there isn't any player that playback this kind of resolution & fps.
    I'm asking what low are the average resolution choosed by major cinema studios for pre-rendered films.

    Do you plan to improve this image-viewer to support also videos or jpeg image sequence?

    Thanks for all your great work!
  • ClodoClodo Posts: 61
    Hiro Protagonist
    Note: Virtual Desktop, the same image, the Render HUD OSD display show a "Compositor Frame-rate" always around 75 Hz, and never drop any "Compositor Missed V-Sync Count".
    With your image-viewer, 60 Hz almost constant and a lot of "Missed" counter.
  • joanjoan Posts: 124
    Hiro Protagonist
    Render CPU time: 13.43 ms
    This is awkward. It's at 1.8 ms tops on my machine. This is certainly where the framerate limit is coming from, but as to the underlying cause, I have no idea. Is POV-Ray running simultaneously?

    I don't have plans to add video or image sequences to it at the moment. I'm using the codebase as a sandbox for lightfield rendering experiments for the time being.

    I have trouble with the stereo in your image, does it look ok in all directions in Virtual Desktop for you?
  • ClodoClodo Posts: 61
    Hiro Protagonist
    "Render CPU time" it's always high (>10) with your Capsule.
    Close Capsule, launch Virtual Desktop, 75 fps with a "Render CPU time" always under 1.
    Close VD, relaunch Capsule, >10.
    There isn't any other app open or running, test done after a reboot.
    I have trouble with the stereo in your image, does it look ok in all directions in Virtual Desktop for you?
    For me, yes. I build the formulas for POV-Ray based on the 'raytracing' section of this doc:
    https://developers.google.com/cardboard ... ontent.pdf
  • husakmhusakm Posts: 54
    Hiro Protagonist
    The 60 FPS sounds to me like the limit of the 2D preview on the monitor. Are you sure you had disabled V-sync for the 2D preview and the FPS in limited only by the Rift V-sync ?

    Related question: Why do you need CUDA for the viewer ? What is wrong with normal methods of mapping on cube, sphere cylinder e.t.c. ... ?
  • joanjoan Posts: 124
    Hiro Protagonist
    husakm wrote:
    Related question: Why do you need CUDA for the viewer ? What is wrong with normal methods of mapping on cube, sphere cylinder e.t.c. ... ?
    Nothing wrong, the reason is I did this project almost specifically to learn/improve on CUDA. Which is also why it's focused on a single task. I'm not worried that there will be plenty of more general purpose applications to do the same and more.

    I do call wglSwapIntervalEXT(0); to disable VSync and let the Oculus runtime handle it.
  • husakmhusakm Posts: 54
    Hiro Protagonist
    wglSwapIntervalEXT(0); works only if the user have set in nVida control panel Vertical Sync "Off" or "Use 3D application setting". If the user set "On" he will get the 60 Hz refresh rate because the wglSwapIntervalEXT(0); command will be overwritten and ignored ... I believe this is the reason of the 60 Hz mysterious issue ;) ...
  • joanjoan Posts: 124
    Hiro Protagonist
    husakm wrote:
    wglSwapIntervalEXT(0); works only if the user have set in nVida control panel Vertical Sync "Off" or "Use 3D application setting". If the user set "On" he will get the 60 Hz refresh rate because the wglSwapIntervalEXT(0); command will be overwritten and ignored ... I believe this is the reason of the 60 Hz mysterious issue ;) ...
    Aha! Yes, I get exactly the reported behavior when I switch this option. Thanks.
    I followed the OculusRoomTinyGL code sample on this so it will probably crop up in other applications as well. Good to know.
  • ClodoClodo Posts: 61
    Hiro Protagonist
    Confirmed, setting Vertical Sync "Off" in nVida control panel resolve the issue, 75 fps now. Re-set to On reproduce the issue, 60 fps max.
    Thanks.
  • husakmhusakm Posts: 54
    Hiro Protagonist
    I am happy the problem is solved. I was playing a lot with your application. I work on a commercial version of similar code - playing movies instead of still images.
    Suggestions for improvement:
    1) Add joypad control for the image changes + as additional rotation tool. Without controlling the rotation
    by something else than HMD sensors it is hard to look backward in the scene without standing up.
    2) Thing about speeding up the image reading. What about silently pre-read more images from HD to make faster visualization possible ?
  • ClodoClodo Posts: 61
    Hiro Protagonist
    husakm said:
    I am happy the problem is solved. I was playing a lot with your application. I work on a commercial version of similar code - playing movies instead of still images.
    I'm very happy to see someone that are working on a video player.
    I recently done some rendering experiment with POV-Ray, some day ago i rendered a testing/stress video: 120 frames rendered at 7680 x 7680 ODS Stereo Top/Bottom (around 32 MB per frame, PNG loseless). Perfectly loopable (1 sec at 120fps, 2 sec at 60fps etc.).

    But i'm stuck about different tests of encoding choices: resizing to "4k uhd" vs "4k dci" vs "8k", framerate (120/90/75), codec (h264/h265).
    Currently it's a pain. No VR video-player support smooth loop (i mean restart from frame 0 instantly when reach the last frame, any standard player like WMP, MPC-HQ, VLC do that). Virtual Desktop don't have loop at all.
    And, in general, any video player i tested can't playback this kind of fps/resolution smoothly.
    Anyway, i soon post some .mp4 files if anyone is interested.

  • diogofigueiredodiogofigueiredo Posts: 156
    Art3mis
    Any news on upgrading this to SDK1.3? That would be really great.
    Thanks in advance!
    Owner of a DK2 since October 2014.
    Anxiously waiting for the delivery of the CV1.
  • joanjoan Posts: 124
    Hiro Protagonist
    edited April 2016
    I'll probably do that after I receive my CV1 in April June.

    The goal however was to have a modest and lightweight application that goes fast from startup to seeing the image. Something for content authors to check their work rapidly for example. I'll see how this can be done in 1.3. This is a small side project, I don't really want to implement a full blown UI in VR just to tweak pixel density or select an image, and not sure I want to bother with application signing and submission to Home.
  • JeffreyKJeffreyK Posts: 7
    Virtual Boy (or Girl)
    Really hope to see this in 1.3 too :) I am also waiting for my CV1, but DK2 works great in 1.3 so i had to upgrade. At the moment to view my cardboard 3d images I have to extract the images and view them side by side in an app and use virtual desktop to do the stereo split for the rift. :(

    You could totally make a super simple "Cardboard Photo Viewer" and sell it for a couple bucks on the rift store. I would buy it instantly. Wouldn't need a gui, just designate a folder and then allow the user to go forward or back through the images using the game pad. 
  • auxbrainauxbrain Posts: 1
    Super bummed to see this not working with my just arrived CV1 :(

    I'm using this on a project and Oculus 360 photos will have to do in the pinch, but there a very noticeable increase in distortion in the Oculus viewer.

    If there is any way to get a "blind" build for 1.3 out before you get your CV1 it would be greatly appreciated. Happy to make a donation.

    Thanks!
  • kibacheetahkibacheetah Posts: 4
    Virtual Boy (or Girl)
    Im looking forward to this working with cv1 ^^
  • kartskarts Posts: 4
    Would be awesome to see this on the CV1, any news on that?
Sign In or Register to comment.