I have a standard GLSurfaceView class:
public class TestSurfaceView extends GLSurfaceView {
public MainRenderer mRenderer;
public GStreamerSurfaceView(Context context) {
super(context);
setEGLContextClientVersion(2);
mRenderer = new MainRenderer(context);
setRenderer(mRenderer);
setRenderMode(GLSurfaceView.RENDERMODE_CONTINUOUSLY);
}
}
I have a Renderer class that implements GLSurfaceView.Renderer:
public class MainRenderer implements GLSurfaceView.Renderer {
private int[] hTex;
private SurfaceTexture mSTexture;
private Context context;
MainRenderer(Context c) {
context = c;
}
public void onSurfaceCreated(GL10 unused, EGLConfig config) {
}
public void onDrawFrame(GL10 unused) {
}
public void onSurfaceChanged(GL10 unused, int width, int height) {
}
public void onSurfaceCreated(GL10 arg0, javax.microedition.khronos.egl.EGLConfig arg1) {
}
}
In a separate JNI thread, I have some video data (YUV format) uploaded to an OpenGLES texture. I receive in Java the notification that a new texture is available and I have the corresponding texture id.
How can I display the content of this texture in the Renderer class with minimum performance impact?
For cases where the frames are coming from Camera or MediaCodec there are some very efficient solutions. It sounds like you're generating or decoding the video in software, though, which puts a mild spin on things (though you do have a SurfaceTexture declared in your code, which is odd). The trick is the "OpenGL ES texture" part, because the texture is associated with the EGL context, and the EGL context can only be active in one thread at a time.
Because you're using GLSurfaceView, rather than plain SurfaceView, you don't have control over the EGL context in the GLSurfaceView's renderer thread. The easiest way to work around this is to jump through some hoops to create a second EGL context that is shared with the first. Once you've done that, the texture created in the separate JNI thread will be available to the GLSurfaceView renderer thread.
You can find an example of this in Grafika's "show + capture camera" activity. If you look at the onDrawFrame() method in CameraCaptureActivity.java you can see it calling updateSharedContext(), which passes a message to the thread running TextureMovieEncoder to cause it to run handleUpdateSharedContext(), which (re-)creates the surface used to feed a video encoder.
If you use plain SurfaceView instead, and do your own EGL and thread management, the code will be less twisty. You can create both contexts at once, then just pass one to the thread that's producing images. You could also create a single context and use eglMakeCurrent() to shift it between threads, but that could be expensive on some platforms.
Update: the Grafika "show + capture camera" implementation has a race condition; see this bug report for details on the problem and solution. You have to perform some extra steps when creating a texture in one thread and using it in another. It's usually better to do everything in one context on one thread. The other activities in Grafika use a plain SurfaceView and do their own EGL context and thread management.
Related
I have 2 separate Android app (apk).
App 1 creates SurfaceView inside it and should provide AIDL methods for other apps to obtain an instance of SurfaceHolder for such SurfaceView. So the other apps will be able to draw on that view, displayed inside app number 1.
I was able to transfer Surface itself via aidl easily, since it implements Parcelable interface.
// IMainService.aidl
package com.commonsware.cwac.preso.demo.service;
import android.view.Surface;
interface IMainService {
Surface getSurf();
}
But 3-rd party sdk need SurfaceHolder to draw on. So the question is - how can I create SurfaceHolder for a given Surface instance, or how can I transfer SurfaceHolder via AIDL. Is there any examples of how I can implement Parcelable for SurfaceHolder?
My use-case (if its matter): App 1 starts as a Service and draw UI on Presentation Screen. And I need a way to get another apps display Navigation Data via Nokia Here Mobile SDK inside app 1. I need SurfaceHolder in order to use this api.
Any help will be appreciated.
After a while, finally create a solution that works well with Here OffscreenRender API.
At one app I create SurfaceHolder and this app provide AIDL (see at the post above). With this AIDL I was able to correctly send Surface object and then, on app 2 side, make SurfaceHolder from it.
public class MySurfaceHolder implements SurfaceHolder {
private Surface surface;
private Rect rect;
MySurfaceHolder(Surface surface, Rect rect) {
this.surface = surface;
this.rect = rect;
}
///....
#Override
public Canvas lockCanvas() {
return surface.lockCanvas(this.rect);
}
#Override
public Canvas lockCanvas(Rect rect) {
return surface.lockCanvas(rect);
}
#Override
public void unlockCanvasAndPost(Canvas canvas) {
surface.unlockCanvasAndPost(canvas);
}
#Override
public Rect getSurfaceFrame() {
return rect;
}
#Override
public Surface getSurface() {
return surface;
}
}
Basically, its my own SurfaceHolder, which get initialized by my own, instead of system. This approach need extend AIDL to be able to request Rect sizes, as well.
Hope that will be helpful for someone.
I'm building a simple application which acts similar to built-in camera application - takes a photo and saves it locally, with some additional aŃtions. First I took Firemonkeys' TCameraComponent, placed it to the form and added transfered image from it to TImage. It works OK, I can change resolution, control flash power and so on. The problem is that preview framerate is very low while running at good quality. If I switch camera to 320x240 resolution, it works fast enough, but quality is poor and I have to switch to some higher resolution to take a good photo - loosing time on re-focusing and light adaptation. If I set high resolution (such as 1280x720) for preview, it slows down to about 3-5 fps.
So I need to use some another technique to access camera. App interface must still unchanged during usage, so I can't use TakePhotoFromCameraAction. I'm looking to JNI interfaces. So I need, using JNI calls, access camera and get a preview at some image object at my form. I use the such code to get preview (I'm a bit lack of understanding native android so it may look crazy):
J.Cam := TJCamera.JavaClass.open(0); // J.Cam is JCamera object
J.View := TJView.JavaClass.init( TAndroidHelper.Context ); // JView
J.ViewParams := TJViewGroup_LayoutParams.JavaClass.init(Width, Height); // i got Width and Height from J.Cam.getParameters()
TAndroidHelper.Activity.addContentView( J.View, J.ViewParams );
J.SurfaceView := TJSurfaceView.JavaClass.init( J.View.getContext );
J.LayoutParameters := TJViewGroup_LayoutParams.JavaClass.init(Width, Height);
J.SurfaceView.setLayoutParams( J.LayoutParameters );
J.Cam.setPreviewDisplay( J.SurfaceView.getHolder );
J.Cam.startPreview;
and it doesn't show anything except black screen. Google developer reference says that "If you are using SurfaceView, you will need to register a SurfaceHolder.Callback with addCallback(SurfaceHolder.Callback) and wait for surfaceCreated(SurfaceHolder) before calling setPreviewDisplay() or starting preview."
So the next step I should create the callback class, extending SurfaceView and implementing SurfaceHolder.Callback. There are many samples can be found like this:
public class TestActivity extends Activity {
#Override
public void onCreate(Bundle savedInstanceState) {
super.onCreate(savedInstanceState);
requestWindowFeature(Window.FEATURE_NO_TITLE);
setContentView(new MySurfaceView(this));
}
public class MySurfaceView extends SurfaceView implements SurfaceHolder.Callback {
public MySurfaceView(Context context) {
super(context);
getHolder().addCallback(this);
}
#Override
public void surfaceChanged(SurfaceHolder holder, int format, int width, int height) { }
#Override
public void surfaceCreated(SurfaceHolder holder) { }
#Override
public void surfaceDestroyed(SurfaceHolder holder) { }
}
}
Now a have to translate this code to Delphi. Java code is classes only, but Delphi's JNI is interfaces AND classes so I can't figure out how to represent this:
class MySurfaceView extends SurfaceView implements SurfaceHolder.Callback
since both JNI's JSurfaceView and JSurfaceHolder_Callback are interfaces.
So, does anybody knows how do do this? (or completely another way to access camera)
I have the following (simplified) rig so far:
MyActivity.java:
public class MyActivity extends Activity {
public GLSurfaceView myGLView;
#Override
protected void onCreate(Bundle savedInstanceState) {
super.onCreate(savedInstanceState);
myGLView = new MyGLSurfaceView(this);
setContentView(myGLView);
}
}
MyGLSurfaceView.java:
public class MyGLSurfaceView extends GLSurfaceView {
private MyRenderer mMyRenderer = new MyRenderer();
private MyThread mMyThread = new MyThread();
public MyView(Context context) {
super(context);
setRenderer(mGameRenderer);
setRenderMode(GLSurfaceView.RENDERMODE_CONTINUOUSLY);
mGameThread.setRunning(true);
}
}
MyRenderer.java:
public class GameRenderer implements GLSurfaceView.Renderer {
#Override
public void onDrawFrame(GL10 gl) {
gl.glClear(GL10.GL_COLOR_BUFFER_BIT);
// ...
}
}
MyThread.java:
Here I'm doing all initializations, creating objects and so on.
public class MyThread extends Thread {
private MyObject mMyObject = new MyObject();
public MyThread {
// ...
mMyObject.setRot();
this.start();
}
public void run() {
// Fixed Timestep Loop goes here
mMyObject.getRot();
}
}
MyObject.java:
This is a sample object which holds different fileds and methods.
public class MyObject {
private double mRot;
// Getters & Setters
protected double getRot() { return mRot; }
protected void setRot() {
// ... Do calculations
}
public void draw() {
// OGL Instructions go here
gl.glRotatef(1.2f, 0, 0, setRot());
}
}
Now the problem I was running into is the following: (I guess, I missed something very basic and simple :) )
As stated above, I'm creating my object instances in the MyThread class. The Thread is created in the MyGLSurface class, same goes for the Renderer. Now, that I have that two threads I can't figure out, how to use that one instance and their methods in that two separate threads.
I tried different approaches, but nothing did work. So in my opinion I made a mistake in the class design. I mean, I don't just want to get it running (that'd be quite easy), but I want to know how to do it correctly.
The main problem is actually that I can't access the MyObject's instance and simply use the draw() method in the renderer - because I don't get it.
I thought, it would be possible to call the draw() method of MyObject within the rendering thread without the need of using a singleton and so on. So simply referencing the instance to it. But somehow that seemed weird and dirty (besides that it doesn't work for me).
I tried dozens of different approaches, but I really need a bump into the right direction. I'm quite familar with OOP, but here I might really miss something.
In that many samples I found on the web (stackoverflow, Replica Island, different tutorial sites, Google I/O, DevCentral, etc.) they either didn't use a multithreaded system or they split it directly (GL objects from regular objects).
Any hint into the right direction would be much appreciated!
Another example to peruse:
https://code.google.com/p/android-breakout/
The wiki and code comments discuss the threading issues inherent in using GLSurfaceView. In particular, the game does as much setup as it can before the Renderer thread starts; once it's running, as much work as possible is done on that thread. The game state objects are effectively "owned" by the renderer thread, and the Activity is owned by the UI thread, so any "off-thread" interactions are handled by sending messages rather than making direct method calls. For example, see the handling of touch events.
It's good that you're thinking about this -- you either need synchronization or careful object discipline to avoid nasty race conditions.
See also: android game loop vs updating in the rendering thread
The nice thing about GLSurfaceView is that it creates the OpenGL rendering thread for you, so you don't need to create one yourself. The main UI thread will call the OnDraw() method in your view class and that's all the threads you need. If you really want to create your own thread for OpenGL rendering, use TextureView instead of GLSurfaceView. Here is an article that I think will help:
http://software.intel.com/en-us/articles/porting-opengl-games-to-android-on-intel-atom-processors-part-1
In my openGL game, I draw my scene normally using a GLSurfaceView.Renderer class in the onDrawFrame(). However, when I am displaying a loading screen I would like to force the screen to draw after each item of data is loaded so the loading bar can be displayed.
Is it possible to force a bufferswap during this draw call somehow? My only alternative is to stagger my loading across multiple frames which means a lot of rework..
I guess what I am trying to call is eglSwapBuffers() but I cannot find a way to access the egl context from the GLSurfaceView or GLSurfaceView.Renderer.
Thank you for your time.
You can add also this method to your GLSurfaceView.Renderer class:
import javax.microedition.khronos.egl.EGL10;
import javax.microedition.khronos.egl.EGLDisplay;
import javax.microedition.khronos.egl.EGLSurface;
public void swapBuffers()
{
EGL10 curEgl = (EGL10)EGLContext.getEGL();
EGLDisplay curDisplay = curEgl.eglGetCurrentDisplay();
if (curDisplay == EGL10.EGL_NO_DISPLAY) { Log.e("myApp","No default display"); return; }
EGLSurface curSurface = curEgl.eglGetCurrentSurface(EGL10.EGL_DRAW);
if (curSurface == EGL10.EGL_NO_SURFACE) { Log.e("myApp","No current surface"); return; }
curEgl.eglSwapBuffers(curDisplay, curSurface);
}
Much the same as OpenUserX03's answer, just in Java.
No you can't (or shouldn't) force swapping buffers in the onDraw method of your Renderer.
What you should do is to make the loading of your data in a separate Thread. Your onDraw method will still be called regularly, which will let you ask to the loading thread how many items were loadede to display a progress bar / message accordingly.
It's been awhile since the answer has been accepted but you can (and there is no reason you shouldn't) force swapping the buffers in the onDrawFrame() method of your Renderer.
I had the exact same problem in my OpenGL app - I needed to render a loading screen while data was being loaded. Here is my pseudo-code example of calling eglSwapBuffers() during a load (I use JNI):
public void onDrawFrame(GL10 gl)
{
MyJNILib.render();
}
MyJNILib native pseudo-code:
#include <EGL\egl.h>
...
void render()
{
...
while (loading)
{
// Do loading stuff
...
eglSwapBuffers( eglGetCurrentDisplay(), eglGetCurrentSurface( EGL_DRAW ) );
}
...
}
A solution by force is to make your custom version GLSurfaceView class based on the source code of Android.
In the source, you can find a method called swap:
/**
* Display the current render surface.
* #return the EGL error code from eglSwapBuffers.
*/
public int swap() {
if (! mEgl.eglSwapBuffers(mEglDisplay, mEglSurface)) {
return mEgl.eglGetError();
}
return EGL10.EGL_SUCCESS;
}
This should be what you want. Unfortunately, however, it is a method of private inner class called EglHelper.
/**
* An EGL helper class.
*/
private static class EglHelper {
So in your custom GLSurfaceView class (copied from Google's source), make this EglHelper class public and you can use EglHelper.swap method.
public static class EglHelper {
Is it possible to share the GLES20 context between different GLSurfaceViews (within one Activity)? Alternatively, how would one share a set of texture between different GLSurfaceViews?
On iOS, if you want to conserve memory and reuse (large) textures in different CAEAGLLayer-backed UIViews, you can pass around a EAGLContext object between them or use different EAGLContexts which share a common EAGLSharegroup object.
I wonder how to accomplish this on Android. Is there any equivalent technique?
Edit1
The initial suggestion, to implement your own EGLContextFactory, which will return the same EGLContext, doesn't work since every GLSurfaceViews dispatches the rendering to it's own private gl render thread and sharing the same EGLContext between different threads is not possible.
To rephrase my initial question:
You have several GLSurfaceViews in one screen (one Activity) and you need to access a set of common but large texture in the individual EGLContext of every surfaces, but loading your textures multiple times exceeds the memory of your device. How would you share your textures between GLSurfaceViews then?
The following code works on some of the devices, but not all of them:
public EGLContext createContext(EGL10 egl, EGLDisplay display, EGLConfig eglConfig) {
EGLContext shared = .....;
int[] attrib_list = { EGL_CONTEXT_CLIENT_VERSION, 2, EGL10.EGL_NONE };
EGLContext context = egl.eglCreateContext(display, eglConfig, shared == null ? EGL10.EGL_NO_CONTEXT : shared,
attrib_list);
return context;
}
}
GLSurfaceView setEGLContextFactory
GLSurfaceView.java
It seems that setEGLContextFactory enables to use the same GLES20 context between different GLSurfaceViews.
pseudo code:
private class MyEGLContextFactory implements EGLContextFactory {
private static EGLContext mEGLContext;
public EGLContext createContext(EGL10 egl, EGLDisplay display, EGLConfig config) {
/* create EGLContext for GLES20 in first time */
return mEGLContext;
}
public void destroyContext(EGL10 egl, EGLDisplay display,
EGLContext context) {
}
}