VR Player

[stephanelx]

Hello,

I have been working on VR Player in my spare time since the end of last year. It's my first open source contribution and I hope that it will help the adoption of virtual reality technologies.

VR Player is an experimental Virtual Reality Media Player for Head-Mounted Display devices like the Oculus Rift.

It's still an early alpha but I decided to distribute it anyways to get as much feedback as possible to make it a better product.

I upload the latest version here: http://vrplayer.codeplex.com/

Short term roadmap

*Oculus rift support
*Leap motion support
*Dynamic depth map
*Vrpn integration
*XMP metadata support
*Position trackers for using with the rift: IrTracker, ARToolkit
*Plugin system
*UI improvements


If you have other suggestions, ideas, bugs etc, please do not hesitate to contact me!

-Stephane

[MrGeddings]

so it basically will allow us to view those 3D side by side pictures and videos using the Rift in 3D? that would be awesome!
hopefully youll have a wide range of codecs support like flash/mp4 divx etc would be nice.

That would be awesome if you get it working right might tempt me to get that 3d camera on amazon is saw for not a bad price while i wait for the Rift .

Will it correctly warp the video/images to be viewed correctly on the rift? This is really great. while due to the low res of the dev kit it wont be perfect it would be a nice little plus for those of us (like me) who do not have 3d moniters yet .

Great stuff!

Edit: a little confusing at first perhaps in time with the Oculus Rift you can have a spacific setting so it sets up everything to work with the Rift and ...perhaps options for it with the 3d side by side video and photos first start but im sure it will be great thanks!

[stephanelx]

Hello MrGeddings,

hopefully youll have a wide range of codecs support like flash/mp4 divx etc would be nice.

If you install a good codec pack, you should be able to play/stream about any kind of video. Usually, if you can play a video in windows media player, it should be playable in VR Player.

Will it correctly warp the video/images to be viewed correctly on the rift? This is really great.

You have control over the field of view, the distance between each eyes, and the amount of lenses distortion. I don't know if it will be enough to get a good result but as soon as I receive my dev kit, I will do my best to tweak the settings and add missing features.

That would be awesome if you get it working right might tempt me to get that 3d camera on amazon is saw for not a bad price while i wait for the Rift

Personally, I would wait before bying a 3d camera because of the low resolution, I don't know if this will ruin the 3d effect.

a little confusing at first perhaps in time with the Oculus Rift you can have a spacific setting so it sets up everything to work with the Rift

This is definitely on my todo list! The xmp meta data will also help to avoid confusion by automatically changing the settings related to the media (type of projection, mono/stereo format, etc). As an example, when loading a strereoscopic spherical video with over/under disposition, the player will select the right options.

Thank you for the feedback

[XavierMidnight]

Thanks for this. I've been following along with your project. It took me a minute to discover the view settings were hiding under the question mark.
I ripped an avatar 3d sidebyside trailer from youtube.
Set my options to plane, side , and pincushion.
And set the viewport to -100, which seems just about perfect.

The closer your move the video to your face, or lower the fov, the bigger the 3d effect.
The 3d was just slightly less than what you get from a good 3D tv. That may be due to the fact that you want to move the video as close to your face as possible, but then you start cutting off what is intended to be your main viewing area.
So, if you film something 3d, make sure you have a lot of extra space around your scene.
Also, since movies are filmed with not everything in focus. it's not going to be as good as a game. Since in a game, everything is in focus all the time, so you can look at anything you want, not just what they want you to.
Tracking will be cool, similar to the theater example someone posted on youtube.
Good work, it shouldn't take you much to polish it off once you have a rift to view it with.

Off to find some 360 videos to try out the other modes. Do you have any good video examples I should try?

[stephanelx]

Thank you very much for the description. Very useful and reassuring.

For video content, Fredz from MTBS forum has made a very nice listing of 360 and 3D videos:
http://vr.wikinet.org/wiki/Main_Page#Videos

These ones look particularly good :
http://makingview.no/makingview.com/?page_id=41

It took me a minute to discover the view settings were hiding under the question mark.

Yeah sorry.. I will create a Settings menu or something more intuitive to edit the configurations.
You can also edit more parameters in the .config file but a lot of things will change because I would like to have individual configs for each plugins. This will allow you to install VR Player with only the trackers projections and effects that you intend to use.

[XavierMidnight]

Thanks,
I pulled down a couple of the 360 vids, and tried them in cylinder mode, and they seemed to work pretty well. The 3D 360 vid I tried was incredibly low res, which kindof killed the 3D effect.
When you add the rift tracking in there, it should be pretty cool.

[cybereality]

Looks interesting, but I was having trouble getting the image looking good on the Rift. Seems there should be some options for adjusting the stereo overlap, as I just wasn't able to merge the images together and get the picture in focus. But still, it looks promising.

[Sebbi]

Hi there,

I tried out your VR Player and I am impressed. Would it be possible to playback live video from a webcam or other connected cameras (like a videograbber)?

When my Rift kit arrives I want to try out if the latency is good enough for FPV flying. Being able to move around in a projected video (i tested with some FPV GoPro videos from my flights) could be an awesome FPV experience, even if the camera itself isn't moving on the airplane/copter (I have a gimbal on my copter so this could be used for headtracking supported camera movement).

[stephanelx]

For cybereality:

Seems there should be some options for adjusting the stereo overlap, as I just wasn't able to merge the images together and get the picture in focus.

Thank you, this is some thing I am looking forward to try when I receive the rift.
What feature would would you like to see to improve the focus? Do you think that being able to move images would be enough?

For Sebbi:

Would it be possible to playback live video from a webcam or other connected cameras (like a videograbber)?

It would be possible but this is not something I was planning to do short term.
In the mean while, maybe it can be achieved via a windows media live stream but I did not try it yet.
Medium term, I will probably add custom directshow graph support. This will cover a lot of media inputs.
Long term, the best will be to detect available devices.

[wylde]

love this idea.

is there any way to zoom out? every vid I try is crazy zoomed in.

[stephanelx]

The only way to zoom out without moving the camera is to increase the field of view.
It's in "?" menu under "debug...->config". (I will relocate the menu)
You can also change this value in the .config file.

Otherwise, you can use positional trackers like the hydra to step backward. Arrows on the keyboard are also activated as a debug tool but don't expect movement relative to camera like in a fps (first person shooter). At least not yet

[XavierMidnight]

Hey,
Have you made any progress getting the rift tracker added? I've seen a few projects which just hook into the mouse look.

[stephanelx]

Hi!

Unfortunately, the rift integration is on hold because I did not receive my dev kit yet.
All I need is a c# wrapper of the OVR library and the integration should be very quick.
I am following these two projects:
-https://github.com/Entroper/VRapper
-https://github.com/SiS-Shadowman/RiftDotNet

Currently, I am working on the plugin system and Dynamic depth map but rift integration is my priority and I will do it as soon as I can.

So far, the following features have been implemented for the next version:
-Leap motion support (Navigation with fingers, no playback control yet)
-UI for plugins configuration (Projections, trackers, effects and distortions)
-Vrpn integration (only tested with razer hydra)
-XMP metadata support + Adobe custom panel for xmp editing
-Keyboard navigation FPS style

Also, MTBS user Cymatic Bruce was kind enough to test the application with his Rift and send me recommendations about the default settings:
A field of view of 112 deg and an horizontal offset of -130px seems to gives very good results.
Note: It is possible to set these settings in version 0.1.5 in the .config file

Hope this helps.

-Stephane

[XavierMidnight]

how much work would it be to add multiple videos at once. I'd like to place several videos at varying distance/sizes.

[stephanelx]

Hi XavierMidnight,

I think it would take a lot of work and it might introduce lag and sync problems.
It is currently possible to use one big video divided in multiple parts but it would require to create a custom projection plugin for supporting multiple layers. I don't recommend coding a plugin yet because I am still working on the signatures/contracts and it is not documented.

If you really need to do it and you know what you are doing, these are the steps you need to follow for creating a multiple layer video with a foreground and a background.
-Merge both video in a single file. (Foreground can be on top and background under, it doesn't matter)
-Create a new projection plugin with 2 planes, sphere or cube depending if its a flat or 360 video.
-Adjust mapping to apply each video parts to corresponding geometries
-When playing the video, color key alpha effect will need to be applied to see trough the foreground layer.

In the long term, I would like to include a projection loader plugin so that users can use maya and 3dmax for importing geometries and uv mapping instead of having to code all manually.

[XavierMidnight]

Thank you for the response!
What you said makes a lot of sense.
They added rift support to stereoscopic player, but it doesn't have the ability to manipulate the screen like vr player does. And I feel that's an important feature when dealing with videos not originally intended for the rift.
I'm going to try out a few things first, but I may end up mapping video layers inside UDK. But, as you said performance and syncing could be a difficult task.

[XavierMidnight]

How about the ability to connect the tracker to the video source, not the whole view.
Such that, if a video is recorded at 1080p, the view of the window is only a portion of that like 720 or rift resolution of 640x800.
Then allow the tracker to move the view within that window.

This would give non-360 videos a similar feel to those recorded in 360.
It would be very similar to using some of the other projections like cylinder, but could be implemented a bit differently.

[mediavr]

Hi
here is a stereoscopic 360 panorama twin image of mine you can use with the VRPlayer
http://www.mediavr.com/lrintersection.jpg
To use it you need to set the following settings:
Media/Format to Side by Side
/Projection to Sphere
/Effect to None

Display/Layout to Side by Side
/Distortion to Barrel Distortion

I havent got a Rift but I think these are the correct settings

I have a blog about Stereo panoramas
http://www.stereopanoramas.com/blog/

I put together a rig today to acquire some stereo video source files too.
I have 2 Gopro Hero2 cameras, with the 3d kit (a synchronizing system).
When these are vertical the 48fps 720p setting gives you angular coverage of 88 degrees horizontal
and 118 degrees vertical (I have calibrated them)
So slight vertical cropping will give you stereo fisheye videos which will work straight off as side by side videos
with the Rift I think. I will shoot some test videos tomorrow.

PeterM

[stephanelx]

Hello XavierMidnight,

How about the ability to connect the tracker to the video source, not the whole view ... This would give non-360 videos a similar feel to those recorded in 360.

Interesting idea, this would avoid distorting medias so it would be better for anti-aliasing.
This is some thing I will be trying with the rift, especially after reading this blog post from mediavr:
http://www.stereopanoramas.com/blog/flat-scrolling-3d-panorama-viewers/.

If you want to achieve a alternative zoom effect in the current version, you can:
-Use the up arrow key to zoom in. (Rotating a tracker will not result in a translation)
-Use a dome projection + unwrap fish eye effect. (You will get a lot of distortion on the sides if the media is not wide-angle)

[stephanelx]

Hi PeterM,

Thank you for your kind offer. I will include your picture in the samples directory of the application.
I will contact you in PM about copyright notice you would like me to include.

Tuesday of this week, I'll receive my rift SDK so I will finally be able test stereoscopic content!!
I just hope that the distortion and the anti aliasing will not destroy the 3d effects.

I am really looking forward to know how your tests goes with the gopros and I will keep and eye on your blog!

-Stephane