Regards community
I just want to build a similar app like this,
with my own content of course.
How to capture 360 degree video (cameras, format, ingest, audio)?
Implementation:
2.1 Which one Cardboard SDK works best for my interests (Android or Unity)
2.2 Do you know any blogs, websites, tutorials, samples in which I can support.
Thank you
MovieTextures are a great way to do this in Unity, but unfortunately MovieTextures are not implemented on Android (maybe this will change in Unity 5). See the docs here:
For a simple wrap-a-texture-on-a-sphere app, the Cardboard Java SDK should work. But if you would rather use Unity due to other aspects of the app, the next best way to do this would be to allocate a RenderTexture and then grab the GL id and pass it to a native plugin that you would write.
This native code would be decoding the video stream, and for each frame it would fill the texture. Then Unity can handle the rest of the rendering, as detailed by the previous answer.
First of all, you need content, and to record stereo 360 video, you'll need a rig of at least 12 cameras. Such rigs can be purchased for GoPro cams. That's gonna be expensive.
The recently released Unity 5 is a great option and I strongly suggest using it. The most basic way of doing 360 stereo videos in Unity is to create two spheres with MovieTextures showing your 360 video. Then you turn them "inside out", so that they display their back faces instead of the front faces. This can be done with a simple shader, turning on front face culling and removing the mirror effect. Then you place your cameras inside the spheres. If you are using the google cardboard sdk camera rig, put the spheres on different culling layers and make the cameras only see the appropriate spheres. Remember to put the spheres in proper positions regarding the cameras.
There may be other ways to do this, leading to better results, but they won't be as simple. You can also look for some paid scripts/plugins/assets to do 360 video in VR.
Related
I'm working on a robot that is controller via the VR headset and sends a real-time video feed to the headset.
I've chosen to go the native way on Android and now have everything I need to receive the video stream and encode it (using GStreamer) and also to send the control data to the robot via UDP.
The last thing to do (and the one I most struggle with as I nave no prior experience with computer graphics) is to draw the image (encoded camera feed) to the screen. In the last few days, I've been reading stuff about how Vulkan and OpenGL works, I've also went through the examples provided in Oculus Mobile SDK (mainly VRCubeWorld_SurfaceView) but that's way to complex for what I need, I've tried to simplify it so I could just draw two images, but then I thought.
Do I even need any of that? And this question might sound stupid, but I really don't have any prior experience doing this.
I mean, the example is using OpenGL to basically compute all the layers of the 3D scene, apply colors and then fuse them together to get a final frame that is passed to VR_API via the function:
vrapi_SubmitFrame2(appState.Ovr, &frameDesc);
Can I just take those images, and somehow force them into the frameDesc structure to skip the whole OpenGL pipeline? If so, can anyone knowledgeable enough point me to a working solution?
I don't need any kind of panning over the images, just to render them. Later I'll be using head sensor data, but it won't actually do anything with the "scene".
I am fairly new to Android and especially to the various camera systems in this platform. I'm building an app where I need to integrate ARCore only to track the camera pose (among other things like objects in the scene, planes etc). I don't want to augment anything in the "real world" , so I am not looking to preview the frames being fed to the camera. I've looked through all of the examples in the arcore-sdk and sample code in google's documentation. None of them cover my use case where I want to be able to fetch camera's pose without previewing the camera images onto a surface view or something. I also don't want to 'fake' it by creating a view and hiding it. I would like to know if anyone has experience with such a thing or any ideas how we can achieve it or if we can achieve this at all? Does ARCore even support this?
UPDATE: I found this https://github.com/google-ar/arcore-android-sdk/issues/259 where they mention that it's possible with just an OpenGL context. But I have no clue how to get started. Any samples or pointers would be appreciated!
You can run an ArSession for tracking. ArSession doesn't depend on View.
I need to play a video on a OpenGL surface. I think I will need to render each frame of the video to a texture in a loop and then render it via OpenGL. is this possible under ios and/or android ?
It is possible on iOS, but it's pretty tricky business to get it to run fast enough to keep up with a video stream.
There is an old demo app from Apple called ChromaKey that takes a CVPixelBuffer from Core Video and maps it directly into an OpenGL texture without having to copy the data. That makes performance MUCH better, and is the approach I would suggest.
I don't know if there is more current sample code available that shows how it's done. That code is back from the days of iOS 6, and was written in Objective-C. (I would suggest doing new iOS development in Swift, since that's where Apple is putting its emphasis.)
My VR-researcher friend asked me to develop a simple android (and eventually iOS) app for google cardboard that displays her 180 degree video. She stitched it together from her gopro 3 hero rig, and it's in a quite unusual projection I believe. I've been researching this a lot but I haven't seen anything like it, although I must admit I'm new to VR video.
Google's documentation for the cardboard specifies the equirectangular-panoramic projection as the only thing supported at the time and it doesn't seem like anyone uses this bizarre projection besides her. She normally loads it into koloreyes - where it looks perfectly fine - on her macbook and displays it on an Oculus and claims this projection based on the eyes curvature improves immersion, and I must agree it was deeply immersive when I tried it myself for her research project. (Full length video available here, but youtube compresses it to a degree that ruins immersion completely which is why the app is even justified to begin with).
She would like a more portable version for demonstration with a modified cardboard with straps or something added, but when I load it into the simplevideowidget in the google VR SDK it looks terrible, presumably because of the unexpected projection.
Unity appears to be a no-go as it does not support high resolution videos without using expensive plugins like Easy Movie Texture. It also seems quite overkill to use unity just to display a video.
TL;DR: How do I display this oddly projected 180 degree stereoscopic video in an android app with the google VR SDK (or something entirely different?) Why does the video file work fine in the Oculus but not at all with the cardboard? I'm thinking it must have to do with the field of view, somehow.
Try to use from this example.May be this one will help you. In this change sphere to 180, 230 whatever you want. May be it helps you.
https://github.com/ashqal/MD360Player4Android
I'm making android game.(using andengine)
I need to record game play screen .
This is not for making promotion video, It is for game players to review their game play.
My app should record video by itself.
So I can't solve this problem using available recording app in market.
I already checked below code.
http://code.google.com/p/andengineexamples/source/browse/src/org/anddev/andengine/examples/ScreenCaptureExample.java?spec=svn66d3057f672175a8a21f9d06e7a045942331f65c&r=66d3057f672175a8a21f9d06e7a045942331f65c
It works very well..
But I want to record game play video, not a one screenshot.
At least I need 24fps for smooth replay, But If I use glreadpixels , I can get 5 fps at my xoom device.
I searched various websites to solve this optimization problem.
most people saying glreadplxels is too slow to record video.
http://www.gamedev.net/topic/473794-glreadpixel-takes-tooooo-much-time/
they recommend glcopyteximage2d instead of glreadpixels.
because glcopyteximage2d is much more faster than glreadpixels.
but I can't find how to use glcopyteximage2d in andengine.
even someone say that android opengl ES do not support glcopyteximage2d.
Maybe Another method exists to record smooth video.
It is read framebuffer of android device.
most of recording app in market using this method. but these app needs root permission to grab framebuffer.
I've read some news that android will be support capture screen from suface_flinger after gingerbread.
But I can't find out how to use framebuffer without root permission. T_T
These are my guessing solution.
use another opengl API which has better speed than glreadpixels.
find some android API can get framebuffer without root permission.
(Maybe I can access to android SURFACE_FLINGER ??)
draw another offscreen texture to record video.
But I don't know how to implement these methods.
Which approach is correct?
Do you have a example code to record video for android?
please help me to solve this problem.
If you know any other method, That will be helpful.
any help will be appreciated
Does the GPU vendor of your device support es3.0, if it does you can try to use PBO.
Here is a topic I you can refer to :Low readback performance with PBO , help !!!!!