Develop Reference

How to use OpenGL to draw a text over a Android MediaPlayer?

I have a cpp code implementing a media player behavior on Android.
I'm using the media player for playing a mp4 file however, I need to draw text above this.
For testing purposes, I've already tried to do as drawText() function from BootAnimation.cpp however without success.
I'm guessing there is some OpenGL calls I'm missing. Is there some call to be added inside drawText() for it to draw above the mp4?
void BootAnimation::drawText(const char* str, const Font& font, bool bold, int* x, int* y) {
glEnable(GL_BLEND); // Allow us to draw on top of the animation
glBindTexture(GL_TEXTURE_2D, font.texture.name);
const int len = strlen(str);
const int strWidth = font.char_width * len;
if (*x == TEXT_CENTER_VALUE) {
*x = (mWidth - strWidth) / 2;
} else if (*x < 0) {
*x = mWidth + *x - strWidth;
}
if (*y == TEXT_CENTER_VALUE) {
*y = (mHeight - font.char_height) / 2;
} else if (*y < 0) {
*y = mHeight + *y - font.char_height;
}
int cropRect[4] = { 0, 0, font.char_width, -font.char_height };
for (int i = 0; i < len; i++) {
char c = str[i];
if (c < FONT_BEGIN_CHAR || c > FONT_END_CHAR) {
c = '?';
}
// Crop the texture to only the pixels in the current glyph
const int charPos = (c - FONT_BEGIN_CHAR); // Position in the list of valid characters
const int row = charPos / FONT_NUM_COLS;
const int col = charPos % FONT_NUM_COLS;
cropRect[0] = col * font.char_width; // Left of column
cropRect[1] = row * font.char_height * 2; // Top of row
// Move down to bottom of regular (one char_heigh) or bold (two char_heigh) line
cropRect[1] += bold ? 2 * font.char_height : font.char_height;
glTexParameteriv(GL_TEXTURE_2D, GL_TEXTURE_CROP_RECT_OES, cropRect);
glDrawTexiOES(*x, *y, 0, font.char_width, font.char_height);
*x += font.char_width;
}
glDisable(GL_BLEND); // Return to the animation's default behaviour
glBindTexture(GL_TEXTURE_2D, 0);
}
PS: this is no android app, so it won't be done in app layer.

The Bootanimation.cpp use of OpenGL ES changed a bit and now it's using a more modern way to deal with graphics.
That being said, I found that my case would need a some abstraction as done here. Basic OpenGL manipulation, as use of common vertex and fragment shaders (position and color, really nothing different from fundamentals) and VBO/VAO for data buffering and glDrawArrays is enough for my usage.
I still need to understand and apply some texture and understand the best way (in my scenario) for manipulate text, however I think that is the all.

Read from GL_TEXTURE_EXTERNAL_OES to GL_TEXTURE_2D have perfomance issues and glitches

I'm need to send data from GL_TEXTURE_EXTERNAL_OES to simple GL_TEXTURE_2D (Render image from Android player to Unity texture) and currently do it through read pixels from buffer with attached source texture. This process work correctly on my OnePlus 5 phone, but have some glitches with image on phones like xiaomi note 4, mi a2 and etc (like image is very green), and also there is perfomance issues becouse of this process works every frame and than more pixels to read, than worser perfomance (even my phone has low fps at 4k resolution). Any idea how to optimize this process or do it in some other way?
Thanks and best regards!
GLuint FramebufferName;
glGenFramebuffers(1, &FramebufferName);
glBindFramebuffer(GL_FRAMEBUFFER, FramebufferName);
glFramebufferTexture2D(GL_FRAMEBUFFER, GL_COLOR_ATTACHMENT0, GL_TEXTURE_EXTERNAL_OES, g_ExtTexturePointer, 0);
if (glCheckFramebufferStatus(GL_FRAMEBUFFER) != GL_FRAMEBUFFER_COMPLETE)
{
LOGD("%s", "Error: Could not setup frame buffer.");
}
unsigned char* data = new unsigned char[g_SourceWidth * g_SourceHeight * 4];
glReadPixels(0, 0, g_SourceWidth, g_SourceHeight, GL_RGBA, GL_UNSIGNED_BYTE, data);
glBindTexture(GL_TEXTURE_2D, g_TexturePointer);
glTexImage2D(GL_TEXTURE_2D, 0, GL_RGBA, g_SourceWidth, g_SourceHeight, 0, GL_RGBA, GL_UNSIGNED_BYTE, data);
glDeleteFramebuffers(1, &FramebufferName);
glBindFramebuffer(GL_FRAMEBUFFER, 0);
glBindTexture(GL_TEXTURE_2D, 0);
delete[] data;
UPDATE.
Function which contain this code and function which calls it from Unity side
static void UNITY_INTERFACE_API OnRenderEvent(int eventID) { ... }
extern "C" UnityRenderingEvent UNITY_INTERFACE_EXPORT UNITY_INTERFACE_API UMDGetRenderEventFunc()
{
return OnRenderEvent;
}
Which called from Unity Update function like this:
[DllImport("RenderingPlugin")]
static extern IntPtr UMDGetRenderEventFunc();
IEnumerator UpdateVideoTexture()
{
while (true)
{
...
androidPlugin.UpdateSurfaceTexture();
GL.IssuePluginEvent(UMDGetRenderEventFunc, 1);
}
}
And Android plugin do this on its side (surfaceTexture its texture which contain this external texture on which ExoPlayer render video)
public void exportUpdateSurfaceTexture() {
synchronized (this) {
if (this.mIsStopped) {
return;
}
surfaceTexture.updateTexImage();
}
}

On the C++ side:
You're creating and destroying pixel data every frame when you do new unsigned char[g_SourceWidth * g_SourceHeight * 4]; and delete[] data and that's expensive depending on the Texture size. Create the texture data once then re-use it.
One way to do this is to have static variables on the C++ side hold the texture information then a function to initialize those variables::
static void* pixelData = nullptr;
static int _x;
static int _y;
static int _width;
static int _height;
void initPixelData(void* buffer, int x, int y, int width, int height) {
pixelData = buffer;
_x = x;
_y = y;
_width = width;
_height = height;
}
Then your capture function should be re-written to remove new unsigned char[g_SourceWidth * g_SourceHeight * 4]; and delete[] data but use the static variables.
static void UNITY_INTERFACE_API OnRenderEvent(int eventID)
{
if (pixelData == nullptr) {
//Debug::Log("Pointer is null", Color::Red);
return;
}
GLuint FramebufferName;
glGenFramebuffers(1, &FramebufferName);
glBindFramebuffer(GL_FRAMEBUFFER, FramebufferName);
glFramebufferTexture2D(GL_FRAMEBUFFER, GL_COLOR_ATTACHMENT0, GL_TEXTURE_EXTERNAL_OES, g_ExtTexturePointer, 0);
if (glCheckFramebufferStatus(GL_FRAMEBUFFER) != GL_FRAMEBUFFER_COMPLETE)
{
LOGD("%s", "Error: Could not setup frame buffer.");
}
glReadPixels(_x, _y, _width, _height, GL_RGBA, GL_UNSIGNED_BYTE, pixelData);
glBindTexture(GL_TEXTURE_2D, g_TexturePointer);
glTexImage2D(GL_TEXTURE_2D, 0, GL_RGBA, _width, _height, 0, GL_RGBA, GL_UNSIGNED_BYTE, pixelData);
glDeleteFramebuffers(1, &FramebufferName);
glBindFramebuffer(GL_FRAMEBUFFER, 0);
glBindTexture(GL_TEXTURE_2D, 0);
}
extern "C" UnityRenderingEvent UNITY_INTERFACE_EXPORT UNITY_INTERFACE_API
UMDGetRenderEventFunc()
{
return OnRenderEvent;
}
On the C# side:
[DllImport("RenderingPlugin", CallingConvention = CallingConvention.Cdecl)]
public static extern void initPixelData(IntPtr buffer, int x, int y, int width, int height);
[DllImport("RenderingPlugin", CallingConvention = CallingConvention.StdCall)]
private static extern IntPtr UMDGetRenderEventFunc();
Create the Texture information, pin it and send the pointer to C++:
int width = 500;
int height = 500;
//Where Pixel data will be saved
byte[] screenData;
//Where handle that pins the Pixel data will stay
GCHandle pinHandler;
//Used to test the color
public RawImage rawImageColor;
private Texture2D texture;
// Use this for initialization
void Awake()
{
Resolution res = Screen.currentResolution;
width = res.width;
height = res.height;
//Allocate array to be used
screenData = new byte[width * height * 4];
texture = new Texture2D(width, height, TextureFormat.RGBA32, false, false);
//Pin the Array so that it doesn't move around
pinHandler = GCHandle.Alloc(screenData, GCHandleType.Pinned);
//Register the screenshot and pass the array that will receive the pixels
IntPtr arrayPtr = pinHandler.AddrOfPinnedObject();
initPixelData(arrayPtr, 0, 0, width, height);
StartCoroutine(UpdateVideoTexture());
}
Then to update the texture, see the sample below. Note that there are two methods to update the texture as shown on the code below. If you run into issues with Method1, comment out the two lines which uses texture.LoadRawTextureData and texture.Apply and un-comment the Method2 code which uses the ByteArrayToColor, texture.SetPixels and texture.Apply function:
IEnumerator UpdateVideoTexture()
{
while (true)
{
//Take screenshot of the screen
GL.IssuePluginEvent(UMDGetRenderEventFunc(), 1);
//Update Texture Method1
texture.LoadRawTextureData(screenData);
texture.Apply();
//Update Texture Method2. Use this if the Method1 above crashes
/*
ByteArrayToColor();
texture.SetPixels(colors);
texture.Apply();
*/
//Test it by assigning the texture to a raw image
rawImageColor.texture = texture;
//Wait for a frame
yield return null;
}
}
Color[] colors = null;
void ByteArrayToColor()
{
if (colors == null)
{
colors = new Color[screenData.Length / 4];
}
for (int i = 0; i < screenData.Length; i += 4)
{
colors[i / 4] = new Color(screenData[i],
screenData[i + 1],
screenData[i + 2],
screenData[i + 3]);
}
}
Unpin the array when done or when the script is about to be destroyed:
void OnDisable()
{
//Unpin the array when disabled
pinHandler.Free();
}

Calling glReadPixels is always going to be slow; CPUs are not good at bulk data transfer.
Ideally you'd managed to convince Unity to accept an external image handle, and do the whole process zero copy, but failing that I would use a GPU render-to-texture and use a shader to transfer from the external image to the RGB surface.

Vuforia 6.0.117 0x501 error when rendering texture

I'm trying to get Vuforia 6.0.117 working in my Android app. I'm using this specific version since its the last version supporting FrameMarkers. The detection of FrameMarkers is working fine, but when i'm trying to render a texture over the FrameMarker on my phone I get an error stating:
After operation FrameMarkers render frame got glError 0x501
My renderFrame method:
// Clear color and depth buffer
GLES20.glClear(GLES20.GL_COLOR_BUFFER_BIT | GLES20.GL_DEPTH_BUFFER_BIT);
// Get the state from Vuforia and mark the beginning of a rendering
// section
State state = Renderer.getInstance().begin();
// Explicitly render the Video Background
Renderer.getInstance().drawVideoBackground();
GLES20.glEnable(GLES20.GL_DEPTH_TEST);
GLES20.glEnable(GLES20.GL_BLEND);
GLES20.glBlendEquation(GLES20.GL_FUNC_ADD);
// GLES20.glBlendFunc(GLES20.GL_SRC_ALPHA, GLES20.GL_ONE_MINUS_SRC_ALPHA);
GLES20.glBlendFunc(GLES20.GL_ONE, GLES20.GL_ONE_MINUS_SRC_ALPHA);
// We must detect if background reflection is active and adjust the
// culling direction.
// If the reflection is active, this means the post matrix has been
// reflected as well,
// therefore standard counter clockwise face culling will result in
// "inside out" models.
GLES20.glEnable(GLES20.GL_CULL_FACE);
GLES20.glCullFace(GLES20.GL_BACK);
if (Renderer.getInstance().getVideoBackgroundConfig().getReflection() == VIDEO_BACKGROUND_REFLECTION.VIDEO_BACKGROUND_REFLECTION_ON) {
GLES20.glFrontFace(GLES20.GL_CW); // Front camera
} else {
GLES20.glFrontFace(GLES20.GL_CCW); // Back camera
}
// Did we find any trackables this frame?
if (mActivity.isHelpVisible() || state.getNumTrackableResults() == 0) {
// no marker scanned
mActivity.hideInfoButton();
} else {
// Get the trackable:
TrackableResult trackableResult = state.getTrackableResult(0);
float[] modelViewMatrix = Tool.convertPose2GLMatrix(trackableResult.getPose()).getData();
// Check the type of the trackable:
MarkerResult markerResult = (MarkerResult) trackableResult;
Marker marker = (Marker) markerResult.getTrackable();
if (markerId != marker.getMarkerId()) {
markerId = marker.getMarkerId();
tag = DataManager.getInstance().getTagByMarkerId(markerId);
if (tag != null) {
texture = Texture.loadTexture(tag.getTexture());
setupTexture(texture);
tag.addToDB();
}
}
if (tag != null) {
String poiReference = tag.getPoiReference();
if (!poiReference.isEmpty()) {
mActivity.showInfoButton(poiReference);
}
// Select which model to draw:
Buffer vertices = planeObject.getVertices();
Buffer normals = planeObject.getNormals();
Buffer indices = planeObject.getIndices();
Buffer texCoords = planeObject.getTexCoords();
int numIndices = planeObject.getNumObjectIndex();
float[] modelViewProjection = new float[16];
float scale = (float) tag.getScale();
Matrix.scaleM(modelViewMatrix, 0, scale, scale, scale);
Matrix.multiplyMM(modelViewProjection, 0, vuforiaAppSession.getProjectionMatrix().getData(), 0, modelViewMatrix, 0);
GLES20.glUseProgram(shaderProgramID);
GLES20.glVertexAttribPointer(vertexHandle, 3, GLES20.GL_FLOAT, false, 0, vertices);
GLES20.glVertexAttribPointer(normalHandle, 3, GLES20.GL_FLOAT, false, 0, normals);
GLES20.glVertexAttribPointer(textureCoordHandle, 2, GLES20.GL_FLOAT, false, 0, texCoords);
GLES20.glEnableVertexAttribArray(vertexHandle);
GLES20.glEnableVertexAttribArray(normalHandle);
GLES20.glEnableVertexAttribArray(textureCoordHandle);
GLES20.glActiveTexture(GLES20.GL_TEXTURE0);
GLES20.glBindTexture(GLES20.GL_TEXTURE_2D, texture.mTextureID[0]);
GLES20.glUniformMatrix4fv(mvpMatrixHandle, 1, false, modelViewProjection, 0);
GLES20.glUniform1i(texSampler2DHandle, 0);
GLES20.glDrawElements(GLES20.GL_TRIANGLES, numIndices, GLES20.GL_UNSIGNED_SHORT, indices);
GLES20.glDisableVertexAttribArray(vertexHandle);
GLES20.glDisableVertexAttribArray(normalHandle);
GLES20.glDisableVertexAttribArray(textureCoordHandle);
SampleUtils.checkGLError("FrameMarkers render frame");
}
}
GLES20.glDisable(GLES20.GL_DEPTH_TEST);
Renderer.getInstance().end();
}
I'm loading a texture of the size 640x482 and is loading as follows:
public class Texture {
public int mWidth; // The width of the texture.
public int mHeight; // The height of the texture.
public int mChannels; // The number of channels.
public ByteBuffer mData; // The pixel data.
public int[] mTextureID = new int[1];
public boolean mSuccess = false;
public static Texture loadTexture(String fileName) {
try {
InputStream inputStream = new FileInputStream(fileName);
BufferedInputStream bufferedStream = new BufferedInputStream(inputStream);
Bitmap bitMap = BitmapFactory.decodeStream(bufferedStream);
bufferedStream.close();
inputStream.close();
int[] data = new int[bitMap.getWidth() * bitMap.getHeight()];
bitMap.getPixels(data, 0, bitMap.getWidth(), 0, 0, bitMap.getWidth(), bitMap.getHeight());
return loadTextureFromIntBuffer(data, bitMap.getWidth(), bitMap.getHeight());
} catch (IOException e) {
Log.e(Constants.DEBUG, "Failed to load texture '" + fileName + "' from APK");
Log.i(Constants.DEBUG, e.getMessage());
return null;
}
}
public static Texture loadTextureFromIntBuffer(int[] data, int width, int height) {
// Convert:
int numPixels = width * height;
byte[] dataBytes = new byte[numPixels * 4];
for (int p = 0; p < numPixels; ++p) {
int colour = data[p];
dataBytes[p * 4] = (byte) (colour >>> 16); // R
dataBytes[p * 4 + 1] = (byte) (colour >>> 8); // G
dataBytes[p * 4 + 2] = (byte) colour; // B
dataBytes[p * 4 + 3] = (byte) (colour >>> 24); // A
}
Texture texture = new Texture();
texture.mWidth = width;
texture.mHeight = height;
texture.mChannels = 4;
texture.mData = ByteBuffer.allocateDirect(dataBytes.length).order(ByteOrder.nativeOrder());
int rowSize = texture.mWidth * texture.mChannels;
for (int r = 0; r < texture.mHeight; r++) {
texture.mData.put(dataBytes, rowSize * (texture.mHeight - 1 - r), rowSize);
}
texture.mData.rewind();
texture.mSuccess = true;
return texture;
}
}
Anybody got an idea why i'm getting this error and how to fix it?

I cannot go over your entire code right now, and even if I could I'm not sure it would help. You first need to narrow down the problem, so I will first give you the method to do that, and I hope it will serve you in other cases as well.
You managed to find out that there was an error - but you are checking it only at the end of the rendering function. What you need to do is to place the checkGLError call in several places inside the rendering code (print a different text message), until you can pin-point the exact line after which the error first appears. Then, if you cannot understand the problem, comment here what is the problematic line and I will try to help.
UPDATE:
After looking at the shader code, following your report that normalHandle is -1, I got to the following conclusions:
The error, which indicates the variable vertexNormal cannot be found in the shader, may be due to the fact that this variable is probably optimized out during shader compilation, since it is not really required.
Explanation: in the vertex shader (CUBE_MESH_VERTEX_SHADER), vertexNormal is assigned to a varying called normal (variable that is passed to the fragment shader). In the fragment shader, this varying is declared but not used.
Therefore, you can actually delete the variables vertexNormal and normal from the shader, and you can delete all usages of 'normalHandle' in your code.
This should eliminate the error.

How to get textures applied on mobile devices (android 6.0.1)

I've got this webgl application (360 panorama viewer), that wraps an equirectangular image over a sphere, positioning the camera at the center of that sphere to simulate a real world illusion.
It works on desktop browsers (chrome, ff) but does not on android 6.0.1 google chrome (the result is a black screen) and does work on device's built in browser.
However, without applying a texture things work correctly everywhere and I can see a simple primitives (spheres, cubes, pyramids etc).
the part of the code that creates such view:
function load360View(r, center, lats, longs, textureImage) {
let vertexBuffer = gl.createBuffer(),
uvBuffer = gl.createBuffer(),
indexBuffer = gl.createBuffer(),
texture = gl.createTexture(),
vertices = [],
texCoords = [],
indices = [];
let theta, phi, x, y, z;
for (let i = 0; i <= lats; i++) {
theta = i * Math.PI/lats;
for (let j = 0; j <= longs; j++) {
phi = j * 2 * Math.PI/longs;
x = Math.sin(theta) * Math.cos(phi);
y = Math.cos(theta);
z = Math.sin(theta) * Math.sin(phi);
vertices.push(r * x + center.x);
vertices.push(r * y + center.y);
vertices.push(r * z + center.z);
texCoords.push( (j / longs) );
texCoords.push( (i / lats) );
}
}
for (let i = 0; i < lats; i++) {
for (let j = 0; j < longs; j++) {
let first = (i * (longs + 1)) + j;
let second = first + longs + 1;
indices.push(first);
indices.push(second);
indices.push(first + 1);
indices.push(second);
indices.push(second + 1);
indices.push(first + 1);
}
}
gl.bindBuffer(gl.ELEMENT_ARRAY_BUFFER, indexBuffer);
gl.bufferData(gl.ELEMENT_ARRAY_BUFFER, new Uint16Array(indices), gl.STATIC_DRAW);
gl.bindBuffer(gl.ARRAY_BUFFER, vertexBuffer);
gl.enableVertexAttribArray(vPositionLoc);
gl.vertexAttribPointer(vPositionLoc, 3, gl.FLOAT, false, 0, 0);
gl.bufferData(gl.ARRAY_BUFFER, new Float32Array(vertices), gl.STATIC_DRAW);
gl.bindBuffer(gl.ARRAY_BUFFER, uvBuffer);
gl.enableVertexAttribArray(vtexCoordLoc);
gl.vertexAttribPointer(vtexCoordLoc, 2, gl.FLOAT, false, 0, 0);
gl.bufferData(gl.ARRAY_BUFFER, new Float32Array(texCoords), gl.STATIC_DRAW);
gl.bindTexture(gl.TEXTURE_2D, texture);
gl.texImage2D(gl.TEXTURE_2D, 0, gl.RGBA, gl.RGBA, gl.UNSIGNED_BYTE, textureImage);
if (isPowerOf2(textureImage.width) && isPowerOf2(textureImage.height)) {
gl.generateMipmap(gl.TEXTURE_2D);
} else {
gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_MAG_FILTER, gl.LINEAR);
gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_MIN_FILTER, gl.LINEAR);
gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_WRAP_S, gl.CLAMP_TO_EDGE);
gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_WRAP_T, gl.CLAMP_TO_EDGE);
}
return {
vertexBuffer: vertexBuffer,
indexBuffer: indexBuffer,
uvBuffer: uvBuffer,
indices: indices,
texture : texture
}
}
Is this a bug of android's chrome browser or its related to the actually application which would mean that something should be changed ?
p.s.
I Also get this error on firefox desktop (not nightly build):
Error: WebGL: texImage2D: Chosen format/type incured an expensive reformat: 0x1908/0x1401

android ffmpeg opengl es render movie

I am trying to render video via the NDK, to add some features that just aren't supported in the sdk. I am using FFmpeg to decode the video and can compile that via the ndk, and used this as a starting point. I have modified that example and instead of using glDrawTexiOES to draw the texture I have setup some vertices and am rendering the texture on top of that (opengl es way of rendering quad).
Below is what I am doing to render, but creating the glTexImage2D is slow. I want to know if there is any way to speed this up, or give the appearance of speeding this up, such as trying to setup some textures in the background and render pre-setup textures. Or if there is any other way to more quickly draw the video frames to screen in android? Currently I can only get about 12fps.
glClear(GL_COLOR_BUFFER_BIT);
glEnableClientState(GL_VERTEX_ARRAY);
glEnableClientState(GL_TEXTURE_COORD_ARRAY);
glBindTexture(GL_TEXTURE_2D, textureConverted);
//this is slow
glTexImage2D(GL_TEXTURE_2D, /* target */
0, /* level */
GL_RGBA, /* internal format */
textureWidth, /* width */
textureHeight, /* height */
0, /* border */
GL_RGBA, /* format */
GL_UNSIGNED_BYTE,/* type */
pFrameConverted->data[0]);
glEnableClientState(GL_TEXTURE_COORD_ARRAY);
glTexCoordPointer(2, GL_FLOAT, 0, texCoords);
glVertexPointer(3, GL_FLOAT, 0, vertices);
glDrawElements(GL_TRIANGLES, 6, GL_UNSIGNED_BYTE, indices);
glDisableClientState(GL_VERTEX_ARRAY);
glDisableClientState(GL_TEXTURE_COORD_ARRAY);
EDIT
I changed my code to initialize a gltextImage2D only once, and modify it with glSubTexImage2D, it didn't make much of an improvement to the framerate.
I then modified the code to modify a native Bitmap object on the NDK. With this approach I have a background thread that runs that process the next frames and populates the bitmap object on the native side. I think this has potential, but I need to get the speed increased of converting the AVFrame object from FFmpeg into a native bitmap. Below is currently what I am using to convert, a brute force approach. Is there any way to increase the speed of this or optimize this conversion?
static void fill_bitmap(AndroidBitmapInfo* info, void *pixels, AVFrame *pFrame)
{
uint8_t *frameLine;
int yy;
for (yy = 0; yy < info->height; yy++) {
uint8_t* line = (uint8_t*)pixels;
frameLine = (uint8_t *)pFrame->data[0] + (yy * pFrame->linesize[0]);
int xx;
for (xx = 0; xx < info->width; xx++) {
int out_offset = xx * 4;
int in_offset = xx * 3;
line[out_offset] = frameLine[in_offset];
line[out_offset+1] = frameLine[in_offset+1];
line[out_offset+2] = frameLine[in_offset+2];
line[out_offset+3] = 0;
}
pixels = (char*)pixels + info->stride;
}
}

Yes, texture (and buffer, and shader, and framebuffer) creation is slow.
That's why you should create texture only once. After it is created, you can modify its data by calling glSubTexImage2D.
And to make uploading texture data more faster - create two textures. While you use one to display, upload texture data from ffmpeg to second one. When you display second one, upload data to first one. And repeat from beginning.
I think it will still be not very fast. You could try to use jnigraphics library that allows to access Bitmap object pixels from NDK. After that - you just diplay this Bitmap on screen on java side.

Yes, you can optimized this code:
static void fill_bitmap(AndroidBitmapInfo* info, void *pixels, AVFrame *pFrame)
{
uint8_t *frameLine;
int yy;
for (yy = 0; yy < info->height; yy++)
{
uint8_t* line = (uint8_t*)pixels;
frameLine = (uint8_t *)pFrame->data[0] + (yy * pFrame->linesize[0]);
int xx;
for (xx = 0; xx < info->width; xx++) {
int out_offset = xx * 4;
int in_offset = xx * 3;
line[out_offset] = frameLine[in_offset];
line[out_offset+1] = frameLine[in_offset+1];
line[out_offset+2] = frameLine[in_offset+2];
line[out_offset+3] = 0;
}
pixels = (char*)pixels + info->stride;
}
}
to be something like:
static void fill_bitmap(AndroidBitmapInfo* info, void *pixels, AVFrame *pFrame)
{
uint8_t *frameLine = (uint8_t *)pFrame->data[0];
int yy;
for (yy = 0; yy < info->height; yy++)
{
uint8_t* line = (uint8_t*)pixels;
int xx;
int out_offset = 0;
int in_offset = 0;
for (xx = 0; xx < info->width; xx++) {
int out_offset += 4;
int in_offset += 3;
line[out_offset] = frameLine[in_offset];
line[out_offset+1] = frameLine[in_offset+1];
line[out_offset+2] = frameLine[in_offset+2];
line[out_offset+3] = 0;
}
pixels = (char*)pixels + info->stride;
frameLine += pFrame->linesize[0];
}
}
That will save you some cycles.

A couple of minor additions will solve your problem, first convert your AVFrame to RGB with swscale, then apply it directly to your texture i.e.:
AVPicture *pFrameConverted;
struct SwsContext img_convert_ctx;
void init(){
pFrameConverted=(AVPicture *)avcodec_alloc_frame();
avpicture_alloc(pFrameConverted, AV_PIX_FMT_RGB565, videoWidth, videoHeight);
img_convert_ctx = sws_getCachedContext(&img_convert_ctx,
videoWidth,
videoHeight,
pCodecCtx->pix_fmt,
videoWidth,
videoHeight,
AV_PIX_FMT_RGB565,
SWS_FAST_BILINEAR,
NULL, NULL, NULL );
ff_get_unscaled_swscale(img_convert_ctx);
}
void render(AVFrame* pFrame){
sws_scale(img_convert_ctx, (uint8_t const * const *)pFrame->data, pFrame->linesize, 0, pFrame->height, pFrameConverted->data, pFrameConverted->lineSize);
glClear(GL_COLOR_BUFFER_BIT);
glTexSubImage2D(GL_TEXTURE_2D, 0, 0, 0, videoWidth, videoHeight, GL_RGB, GL_UNSIGNED_BYTE, pFrameConverted->data[0]);
glDrawTexiOES(0, 0, 0, videoWidth, videoHeight);
}

Oh,maybe you can use jnigraphics as https://github.com/havlenapetr/FFMpeg/commits/debug.
but if when you get yuv data after decode frame,you should convert it to RGB555,it is too slowly.Use android's mediaplayer is a good idea