Tutorial: Low cost FPV setup for the rift (130ms latency)

A forum to discuss development for the Oculus Rift.

Tutorial: Low cost FPV setup for the rift (130ms latency)

Postby Grix » July 7th, 2013, 11:54 am

EDIT/UPDATE: Read post 5. Basically, latency has been reduced to only 130ms!


Yesterday night, it finally happened: I got my rift! And one of the first things I tried was flying FPV with my RC quadcopter. I've been wanting to do this since I heard about the rift last year, and I've been researching how to get FPV working since. And now I finally have the confirmation: Although it's not perfect, it definitely works! Basically, what I do is convert the analog signal from the video receiver to an USB webcam video, and feed that to the rift with proper duplication and warping.

What you need:
- An Oculus Rift
- An RC vehicle
- Standard FPV gear (Camera, video transmitter and receiver)
- An RCA to USB video capture device (I use this: http://bit.ly/12daZIR)
- A laptop
- A 12V battery
- 5V voltage regulator (or 5V battery)
- A power cable you don't need that fits the rift
- RCA cable
- Optional, but recommended: A customizable OSD (I uses minimOSD)

First, setup your FPV system if you are new to FPV. I won't cover that in this tutorial, google it. There are however a few things to note: For the rift, a high FOV camera is obviously preferable. I currently use a GoPro which has 110 degrees which matches the rift pretty perfectly. I have however ordered a different camera for reasons I will explain later, and I went with this: http://bit.ly/11sR7Tk . It has roughly the same FOV.
If you are using an OSD, which is really recommend that you do, you'll probably want one that lets you customize the layout, like minimOSD. Otherwise the text may be too hard to read due to placement or size. This is my layout in minimOSD, everything is perfectly readable: http://pastebin.com/qvrGLxya (Save as x.osd and import)

Another note about the capture device: Ebay is riddled with dirt cheap "easycaps". I recommend that you avoid them, because they are most likely fake and will lead only to problems. I first bought one for $5, but it was useless for my purposes, it used some obscure codec that only the program included in the disk would recognize. Fake devices also have worse image quality and latency. As said, I bought this one: http://bit.ly/12daZIR . It uses the same chip as many higher end capture cards, and that site provides proper and updated drivers for download.

After you have installed the capture card drivers and verified that it works, download this program: http://lab.neosolve.de/RemoteEyes/RemoteEyes.zip
(Credit goes to this guy: https://www.spacefish.biz/blog/ ).
It should automatically detect the camera and start outputting in rift format. Press f to go fullscreen and esc to exit. If like me you have a webcam integrated in the laptop, you need to disable it in device manager, because the program won't let you change the camera it uses, it just picks the first one it sees. I've tried to get the guy to put the source on github or something but he hasn't responded. Out of the box the program has wrong warp parameters. Open the file fshader.txt and replace it with this: http://pastebin.com/57YkYYfa . Feel free to experiment with params, I sacrificed a bit of FOV for better overview.

If you turn everything on you should now get the image from the FPV camera to the rift! But the setup is not yet portable. You need to make the rift and video receiver battery powered. For this I soldered together a custom cable: On one end it has a connector for a 12V liPo battery, I had a few of those because I use them on my quadcopter. The cable splits, where one end goes directly to the video receiver with the proper plug, and the other end goes to a 5V regulator with a small heatsink, and then to the rift (I had a spare cable with a plug that fit the rift control box, so I soldered it on).

You might be tempted to run the rift off the laptop's USB power. In fact I tried this first, but after five minutes the computer bluescreened due to too high USB power draw. Luckily the quad was on the ground, if that happens when you are in the air, you're screwed. Also, make sure to disable any power saver settings on the laptop, the program does not stop the screen from dimming after x minutes, and that would also be terrible if it happened while in the air.

So how well does this work? There are a few problems, the biggest is latency. Imeasured my system to around 0.25 seconds. It's flyable, especially in open areas, but it can get a bit annoying. However, my gopro camera is partially at fault, the live feed has a slight delay in itself. That's why I ordered the other camera I mentioned above. Latency is 130ms with the new camera. Much better. Here's a demonstration video: http://youtu.be/6rTkuO_44S8 .
That, and the fact that the image quality is much better, especially in low or high light conditions. The controller radio can also be a factor, I use a very cheap radio (Gotta upgrade that some day), and there might be a delay between me moving the sticks and the motors responding, which would also affect the total latency. Another factor is the software, if someone makes a program that really focuses on low latency, it should shave a few additional milliseconds off. I bet that with proper equipment the latency can definitely become acceptable. Though in fact, I would say it already is acceptable. I tried flying for a few minutes but crashed, but I think this was because it was in a small, bulky area, and this is the first time I've ever flied FPV, so I just need some practice in better conditions.

Another problem is image quality. This would depend on the camera, warp parameter and capture device, but in my case it is not that great, but of course, this is nothing new to FPV. PAL is the highest resolution one can transmit currently, and that's obviously not great. The rift has a limited resolution as well, of course. But again, it's definitely flyable, and I think the image quality will improve a bit when I get the new camera as well.

Two missing features are 3D, and headtracking. Both of these are possible to implement, using a gimbal and two cameras, however, it would be much more expensive. Perhaps someone else has the dough to test it out? :)



Here are some pictures:
http://i.imgur.com/o3Gg366.jpg
http://i.imgur.com/K6KBA9c.jpg
http://i.imgur.com/bsWR9Zq.jpg
I can take a video of a flight once I get to try it properly in an open area.

One a sidenote, I'd like to link to this article, another take on the rift and FPV: http://intuitiveaerial.com/home/2013/6/8/oculus-rift-fpv
These guys are doing it a bit differently. They have two cameras for 3D, and better latency, perhaps better resolution as well. However, I believe there is a few downsides compared to this tutorial:
- It's much more expensive, of course.
- It looks like they are using a digital wireless link. This allows for things like better bandwidth, but it also vastly reduces range, and you'd have to buy a whole new, expensive setup even if you already have traditional FPV gear.
- I'm not sure they would even give the community the details on their hardware and software unless you buy it, because they are a company their solution will most likely be proprietary.
Last edited by Grix on July 9th, 2013, 10:13 am, edited 3 times in total.
Grix
 
Posts: 36
Joined: April 28th, 2013, 2:58 pm

Re: Tutorial: Low cost FPV setup for the rift

Postby Sebbi » July 7th, 2013, 3:30 pm

I'm interested in FPV with the Rift, but I hope I can get it to work with a Raspberry Pi or an Android tablet instead of a laptop. And i want the OSD to be drawn in the ground station ...

Do you plan on developing your own software to improve the state of FPV with the Rift?
User avatar
Sebbi
 
Posts: 88
Joined: March 29th, 2013, 2:12 pm
Location: Germany

Re: Tutorial: Low cost FPV setup for the rift

Postby Grix » July 8th, 2013, 2:29 am

Sebbi wrote:Do you plan on developing your own software to improve the state of FPV with the Rift?


Maybe, but I'm not that good at graphical programs, so I don't know if I have the skills.
Grix
 
Posts: 36
Joined: April 28th, 2013, 2:58 pm

Re: Tutorial: Low cost FPV setup for the rift

Postby Medium » July 8th, 2013, 5:16 am

This is the best idea i have heard since the major games got announced for the rift.

Very cool man! Great that you wrote a tutorial on the whole thing as well.

I am definitely going to give this a shot when my rift arrives. Do you have any tips for quad rotor shopping?

Cheers

Medium
Medium
 
Posts: 11
Joined: June 19th, 2013, 1:38 am

Re: Tutorial: Low cost FPV setup for the rift

Postby OculusChris » July 8th, 2013, 7:21 am

Grix wrote:Two missing features are 3D, and headtracking. Both of these are possible to implement, using a gimbal and two cameras, however, it would be much more expensive. Perhaps someone else has the dough to test it out? :)


Hi there,

i've planned it as part of my diploma thesis ;)!
3-Axis Gimbal (Alexmos http://www.armbgc.com/armbgc-v3-0-alexmos-controller-3-axis/) attached to a cablecam, controlled by the Oculus Rift Headtracker.

Whether or not 3D, I'm not quite sure yet...
Lifeview from a Canon, or this:
http://www.nghobbies.com/cart/index.php?main_page=product_info&products_id=393

For ideas and input i'm also grateful :lol:

Chris
OculusChris
 
Posts: 3
Joined: June 3rd, 2013, 11:32 pm

Re: Tutorial: Low cost FPV setup for the rift

Postby Grix » July 9th, 2013, 10:04 am

Very good news, guys! The new camera arrived today, and boy does it make a difference! The latency actually HALVED! Measured now to 4 frames at 30fps, or ~130ms. Moral of the story: Do not use a gopro for live video.

Here's a video demonstrating the latency: http://youtu.be/6rTkuO_44S8

Only problem is, now my OSD is not working for some reason. Maybe I fried it somehow, that would be the second minimosd to fail on me.
Grix
 
Posts: 36
Joined: April 28th, 2013, 2:58 pm

Re: Tutorial: Low cost FPV setup for the rift

Postby OculusChris » July 10th, 2013, 8:00 am

Grix wrote:Here's a video demonstrating the latency: http://youtu.be/6rTkuO_44S8


Awesome! My Rift arrived today, so i can also start experimenting :P .

Cheers
Chris
OculusChris
 
Posts: 3
Joined: June 3rd, 2013, 11:32 pm

Re: Tutorial: Low cost FPV setup for the rift (130ms latency

Postby vande361 » July 11th, 2013, 5:46 pm

Has anyone recorded 360 video with multiple GoPros for playback through the rift? Does this work???
vande361
 
Posts: 11
Joined: July 3rd, 2013, 1:35 pm


Re: Tutorial: Low cost FPV setup for the rift (130ms latency

Postby Felix12g » July 16th, 2013, 11:12 am

Wow, that one drags a whole laptop up.
Code Monkey
User avatar
Felix12g
 
Posts: 142
Joined: April 7th, 2013, 8:35 am
Location: Minneapolis, Minnesota

Re: Tutorial: Low cost FPV setup for the rift (130ms latency

Postby captain3d » July 16th, 2013, 11:29 am

I have been flying 3D FPV for 18 months using the '3D Cam FPV' at nghobbies and love it. I think it has to be 3D if you are going to do it but I know many people will be happy with 2D on the large FOV. I see quite a lot of Rift youtube videos of games where the person does not realize it is 2D.

The Rift is a natural target for FPV flight and I have teamed up with Trevor the 3D Cam FPV designer for 'Transporter3D' that will enable the Rift to be used for 2D or 3D FPV. He made the first 3D rift FPV flight a month ago...

http://youtu.be/u74rf4ztJTI
http://youtu.be/M7WZXzhsFgY
http://youtu.be/xiqIPjAh0JQ

Still in progress but no laptop needed :-)

phil
User avatar
captain3d
 
Posts: 19
Joined: April 24th, 2013, 7:39 pm

Re: Tutorial: Low cost FPV setup for the rift (130ms latency

Postby SkyDrone » August 18th, 2013, 8:07 pm

We have just launched our Indiegogo campaign for Sky Drone FPV. A digital low latency HD FPV solution with Oculus Rift support coming: http://skydrone.aero/fpv-indiegogo
User avatar
SkyDrone
 
Posts: 15
Joined: August 18th, 2013, 8:06 pm

Re: Tutorial: Low cost FPV setup for the rift (130ms latency

Postby Grix » August 29th, 2013, 6:55 am

The guy that made the program has now made a new version, and best of all it's now open source!

https://code.google.com/p/remote-eyes/source/browse/
Grix
 
Posts: 36
Joined: April 28th, 2013, 2:58 pm

Re: Tutorial: Low cost FPV setup for the rift (130ms latency

Postby feuerhahn » October 13th, 2013, 1:34 pm

hey man, great stuff. I fly FPV since 2 years and was one of the first that got the oculus. At the beginning there was not much about rift fpv. I'll definetly will give it a try the next days. Any updates from your side? the last status is past a while.

cheers and thanks for your effort!
feuerhahn
 
Posts: 28
Joined: March 31st, 2013, 4:40 pm

Re: Tutorial: Low cost FPV setup for the rift (130ms latency

Postby Grix » October 14th, 2013, 10:00 am

feuerhahn wrote:hey man, great stuff. I fly FPV since 2 years and was one of the first that got the oculus. At the beginning there was not much about rift fpv. I'll definetly will give it a try the next days. Any updates from your side? the last status is past a while.

cheers and thanks for your effort!


I recently moved to a more densely populated area and so I haven't been able to fly the copter much unfortunately. However, many other people have been making progress here, with dual 3D cameras and headtracking setups. You can probably find a lot of info by googling
Grix
 
Posts: 36
Joined: April 28th, 2013, 2:58 pm

Re: Tutorial: Low cost FPV setup for the rift (130ms latency

Postby Regulator » November 2nd, 2013, 9:30 am

What's the bandwidth usage on the sky drone? LTE I assume. I'm pulling about 3mb / min. Using my Verizon so that would give a suitable 28 hrs
Regulator
 
Posts: 1
Joined: November 2nd, 2013, 9:29 am

Re: Tutorial: Low cost FPV setup for the rift (130ms latency

Postby killerm1 » December 11th, 2013, 11:17 pm

Hi,

I have been trying this out and have found that fpv program for converting the video to not be so great. It just comes up all blury for me. I am using one of those fake easy caps u mentioned though. I played around a little and got it to work with vr player in conjunction with vlc. All you have to do is stream your capture device with vlc and then open the vlc process with vr player. Or you can open the stream with vr player aswell using "dshow://" as the url. Having vlc gives a lot more functionality aswell.
killerm1
 
Posts: 1
Joined: November 15th, 2013, 8:35 pm

Re: Tutorial: Low cost FPV setup for the rift (130ms latency

Postby fatslugg » March 10th, 2014, 7:43 am

I don't know if that might be of interest to anyone, but I've been doing some work to use the Oculus Rift as a FPV system. My first attempt at flying FPV with it was pretty successful (it was my first ever FPV flight so I can't compare much to other systems). Here are some links:

The flight
http://bitoniau.blogspot.fr/2013/10/first-oculus-based-fpv.html

And some more recent work on adding pan-tilt:
http://bitoniau.blogspot.fr/2014/02/pan-tilt-fpv-using-oculus-rift.html
fatslugg
 
Posts: 4
Joined: July 10th, 2013, 2:58 am

Re: Tutorial: Low cost FPV setup for the rift (130ms latency

Postby fatslugg » March 10th, 2014, 7:47 am

Oh and I forgot: I've also run some latency tests on various FPV setups including the Oculus Rift. It's about 130 ms as someone mentionned previously, so not great but I guess still acceptable for FPV

http://bitoniau.blogspot.fr/2013/10/video-latency-investigation.html
http://bitoniau.blogspot.fr/2014/02/video-latency-investigation-part-2.html
fatslugg
 
Posts: 4
Joined: July 10th, 2013, 2:58 am

Re: Tutorial: Low cost FPV setup for the rift (130ms latency

Postby vizual54 » March 20th, 2014, 10:41 am

This is really interesting as I have just started a project with the aim of mounting a stereo rig on my multicopter and using a Oculus for viewing.
I've been wanting to do this for a long time but have not had the time. I guess I won't have too much time for this now either with small kids and a toddler : )
Anyway my plan is to use two web cameras, probably Logitech c310, connected to a Intel nuc or a odroid to make the stereoscopic stream barrel correction etc. To start with I will probably use wireless lan or 4g though I have a 1.2 GHz analog video link an a happhauge USB stick to test with as well. The cameras will be mounted on a 3 axis brushless gimbal that will take ptz pwm signals from a APM flight controller.
I just ordered the new oculus today ; )

The system will not be used as FPV primarily since the latency will be to high. Though it will be fun to test the oculus for that purpose as well.
vizual54
 
Posts: 4
Joined: February 26th, 2014, 4:04 am


Return to “Oculus General Development”


Who is online

Users browsing this forum: Google [Bot], Smarticus and 7 guests