I have a stream of H264 video that I need to show in my Android app. If I consigure MediaCodec with a surface, then the video gets decoded to my app no problem, and I can see it in this surface.
But I also need to obtain a Bitmap of the video in certain moments (i.e. to store certain frames on the SD card). Is it possible to configure the MediaCodec library to return an array of bytes instead of working directly towards a surface??
Another option would be to obtain the bitmap directly from the surface, but I couldn't find this option on the SDK either.
You can use MediaMetadataRetriever:
MediaMetadataRetriever mediaMetadataRetriever=new MediaMetadataRetriever(); //should be stored as an instance variable
mediaMetadataRetriever.setDataSource(mVideo.getVideoUrl(Video.SD));
int time = videoView.getCurrentPosition()* 1000; //micro-to-milliseconds
Bitmap bmFrame = mediaMetadataRetriever.getFrameAtTime(time);
Edit:
To retrieve raw byte data without the involvement of UI, you can advance time frame by frame, get the Bitmap and convert it to byte array.
To be more efficient, consider forking FFmpegMediaMetadataRetriever, whose FFmpegMediaMetadataRetriever.java has method private native byte [] _getFrameAtTime(long timeUs, int option); skipping the whole Bitmap conversion process.
Try this:
public class GameView extends SurfaceView {
private Bitmap bmp;
private SurfaceHolder holder;
public GameView(Context context) {
super(context);
holder = getHolder();
holder.addCallback(new SurfaceHolder.Callback() {
#Override
public void surfaceDestroyed(SurfaceHolder holder) {
}
#Override
public void surfaceCreated(SurfaceHolder holder) {
Canvas c = holder.lockCanvas(null);
onDraw(c);
holder.unlockCanvasAndPost(c);
}
#Override
public void surfaceChanged(SurfaceHolder holder, int format,
int width, int height) {
}
});
bmp = BitmapFactory.decodeResource(getResources(), R.drawable.icon);
}
#Override
protected void onDraw(Canvas canvas) {
canvas.drawColor(Color.BLACK);
canvas.drawBitmap(bmp, 10, 10, null);
}
}
Related
I am trying for weeks to render a vlc streaming to OpenGL on Android.
I guess I am missing something. Here is what I have so far.
This is my custom class:
public class VLCVideoView extends SurfaceView implements SurfaceHolder.Callback, IVideoPlayer, GLSurfaceView.Renderer
My initialization on VLC:
try {
// Create a new media player
libvlc = LibVLC.getInstance();
libvlc.setHardwareAcceleration(LibVLC.HW_ACCELERATION_FULL);
libvlc.eventVideoPlayerActivityCreated(true);
libvlc.setSubtitlesEncoding("");
libvlc.setAout(LibVLC.AOUT_OPENSLES);
libvlc.setVout(LibVLC.VOUT_OPEGLES2);
//libvlc.setVout(LibVLC.VOUT_ANDROID_SURFACE);
libvlc.setTimeStretching(true);
libvlc.setChroma("RV32");
libvlc.setVerboseMode(true);
LibVLC.restart(context);
holder.setFormat(PixelFormat.RGBA_8888);
mSurface = holder.getSurface();
libvlc.attachSurface(mSurface, this);
Implemented the entire GLSurfaceView.Renderer methods:
#Override
public void onSurfaceCreated(GL10 gl, EGLConfig config)
#Override
public void onSurfaceChanged(GL10 gl, int width, int height)
#Override
public void onDrawFrame(GL10 gl)
As well, as:
#Override
protected void onDraw(Canvas canvas) for the VLCVideoView
{
if (mSurface!= null)
{
try {
Canvas surfaceCanvas = mSurface.lockCanvas(null);
if(surfaceCanvas != null)
{
super.onDraw(surfaceCanvas);
mSurface.unlockCanvasAndPost(surfaceCanvas);
}
} catch (Surface.OutOfResourcesException excp) {
excp.printStackTrace();
}
}
}
I have a rotating cube being draw, but instead of having the streaming frames as textures, it simple appears streaming on a flat surface.
Any clue?
I have this android application.
It use a SurfaceView, from where I get the Surface through the SurfaceHolder.
It also use ExoPlayer to stream videos. However I have instantiated an ImageReader, getting its Surface and passing to the ExoPlayer.
Now, I am in the ImageReader.OnImageAvailableListener#onImageAvailable and I access the latest Image.
I want to manipulate the Image and send the new data to the "SurfaceView" Surface.
How can I "draw" an android.media.Image to an android.view.Surface ?
The question is not clear to me.
The way to get android.media.Image is by the Camera2 API, and there you can provide a surface and the "camera" will draw over it. Please refer to Camera2Video example
Another way to get the Image object is from ImageReader (while decoding video for example). In this case you want to draw the image, but you can not provide a surface to the ImageReader(there is an internal surface that is not displayed). In this case you can draw the Image on a SurfaceView.
Assuming this is the case, you need to convert an Image to a Bitmap objects.
You have discussion about how perform this here
Possible duplicate of: how to draw image on surfaceview android
First get your canvas by using lockCanvas() (see here), second get your image and make it a drawable using:
my_bitmap = Bitmap.createBitmap(
MediaStore.Images.Media.getBitmap(getContentResolver(), uri),
0,0,90, 90);
drawable=new BitmapDrawable(my_bitmap);
After that you can draw the drawable to the locked canvas and use unlockCanvasAndPost (Canvas canvas) to post the updated canvas back to the surfaceview.
here is the answer for your question.
MainActivity.java
public class MainActivity extends Activity {
#Override
public void onCreate(Bundle savedInstanceState) {
super.onCreate(savedInstanceState);
mySurfaceView mySurfaceView = new mySurfaceView(getApplicationContext());
setContentView(mySurfaceView);
}
}
mySurfaceView.java
public class mySurfaceView extends SurfaceView implements
SurfaceHolder.Callback {
private TutorialThread _thread;
public mySurfaceView(Context context) {
super(context);
getHolder().addCallback(this);
_thread = new TutorialThread(getHolder(), this);
}
#Override
protected void onDraw(Canvas canvas) {
super.onDraw(canvas);
Bitmap _scratch = BitmapFactory.decodeResource(getResources(),
R.drawable.icon);
canvas.drawColor(Color.BLACK);
canvas.drawBitmap(_scratch, 10, 10, null);
}
public void surfaceChanged(SurfaceHolder arg0, int arg1, int arg2, int arg3) {
}
public void surfaceCreated(SurfaceHolder arg0) {
_thread.setRunning(true);
_thread.start();
}
public void surfaceDestroyed(SurfaceHolder arg0) {
boolean retry = true;
_thread.setRunning(false);
while (retry) {
try {
_thread.join();
retry = false;
} catch (InterruptedException e) {
}
}
}
class TutorialThread extends Thread {
private SurfaceHolder _surfaceHolder;
private mySurfaceView _panel;
private boolean _run = false;
public TutorialThread(SurfaceHolder surfaceHolder, mySurfaceView panel) {
_surfaceHolder = surfaceHolder;
_panel = panel;
}
public void setRunning(boolean run) {
_run = run;
}
#Override
public void run() {
Canvas c;
while (_run) {
c = null;
try {
c = _surfaceHolder.lockCanvas(null);
synchronized (_surfaceHolder) {
_panel.onDraw(c);
}
} finally {
if (c != null) {
_surfaceHolder.unlockCanvasAndPost(c);
}
}
}
}
}
}
I have TextureView that I set to MetoaPlayer to play video:
TextureView textureView = new TextureView(context);
addView(textureView, new LayoutParams(LayoutParams.MATCH_PARENT, LayoutParams.MATCH_PARENT));
textureView.setSurfaceTextureListener(this);
#Override
public void onSurfaceTextureUpdated(SurfaceTexture surface) {}
#Override
public void onSurfaceTextureSizeChanged(SurfaceTexture surface, int width, int height) {}
#Override
public boolean onSurfaceTextureDestroyed(SurfaceTexture surface)
{
return false;
}
#Override
public void onSurfaceTextureAvailable(SurfaceTexture surface, int width, int height)
{
this.surface = new Surface(surface);
mediaPlayer.setSurface(this.surface);
prepareAndPlay();
}
Video plays normally.
But when I try to draw manyally on surface, video playing not started!
#Override
public void onSurfaceTextureAvailable(SurfaceTexture surface, int width, int height)
{
this.surface = new Surface(surface);
mediaPlayer.setSurface(this.surface);
// DRAW FIRST FRAME ON SURFACE
Bitmap bitmap = BitmapFactory.decodeFile(firstFramePath);
Canvas canvas = surface.lockCanvas(null);
canvas.drawBitmap(bitmap, new Rect(0, 0, bitmap.getWidth(), bitmap.getHeight()), new Rect(0, 0, canvas.getWidth(), canvas.getHeight()), null);
bitmap.recycle();
surface.unlockCanvasAndPost(canvas);
prepareAndPlay();
}
When I call play() on mediaplayer playing not started.
I suppose, that Surface move to invalid state and MediaPlayer cannot play video. But logcat is empty.
There is any way to draw on same surface both with media player.
You can't do this.
A "surface" is the producer side of a producer-consumer pair. You can't have two producers. (If you want the gory details, see this doc.)
With a TextureView you can change the underlying SurfaceTexture using setSurfaceTexture(), which should allow you to switch from one to the other. This feature is used (for different reasons) in Grafika's "double decode" activity.
I am working on streaming a video from web. Where I decode the video/audio stuff in native code and get the raw pixels for video
I create a bitmap in java code, with surfaceholder and canvas and update pixels for each bitmap from native code and then stream the bitmaps as video. My problem here is, the video crashes after a few seconds because of low memory.
I want to know whether there is anything that i need to make sure to not to crash app and use low memory.
Here is my code.
public CanvasThread(SurfaceHolder surfaceHolder, Panel panel) {
_surfaceHolder = surfaceHolder;
_panel = panel; }
public void setRunning(boolean run) {
_run = run; }
#Override
public void run() {
Canvas c;
while (_run) {
c = null;
try {
c = _surfaceHolder.lockCanvas(null);
synchronized (_surfaceHolder) {
_panel.onDraw(c);
}
} finally {
if (c != null) {
_surfaceHolder.unlockCanvasAndPost(c);
}
}
public class Panel extends SurfaceView implements SurfaceHolder.Callback{
private CanvasThread canvasthread;
private static Bitmap mBitmap;
private static boolean ii=false;
public Panel(Context context, AttributeSet attrs) {
super(context, attrs);
getHolder().addCallback(this);
canvasthread = new CanvasThread(getHolder(), this);
setFocusable(true);
mBitmap=Bitmap.createBitmap(480, 320, Bitmap.Config.RGB_565);//bitmap created in constructor
}
public Panel(Context context) {
super(context);
getHolder().addCallback(this);
canvasthread = new CanvasThread(getHolder(), this);
setFocusable(true);
}
private static native void renderbitmap(Bitmap bitmap); //native function
#Override
public void onDraw(Canvas canvas) {
renderbitmap(mBitmap); //Update pixels from native code
canvas.drawBitmap(mBitmap, 0,0,null);//draw on canvas
}
#Override
public void surfaceChanged(SurfaceHolder holder, int format, int width,int height) { }
#Override
public void surfaceCreated(SurfaceHolder holder) {
canvasthread.setRunning(true);
canvasthread.start(); }
#Override
public void surfaceDestroyed(SurfaceHolder holder) {
boolean retry = true;
canvasthread.setRunning(false);
while (retry) {
try {
canvasthread.join();
retry = false;
} catch (InterruptedException e) { }
}
You might check out Richard Quirk's glbuffer. I'm using it in a video player app.
In general you want to hold on to the video packets you're receiving and only decode them right before they are needed for display. It should be easy to integrate your code with glbuffer and bypass any Bitmap allocation in Java code.
There would be one decoded frame at any given time present in the native code and in GL texture memory and several encoded packets that you're keeping around for buffering.
That will probably not because you have limited amount of memory. Probably around 16MB for the emulator. You should not store all the images in memory. You have to options
Fix your algorithm so it doesn't
need everything in memory
Or only keep one image in memory and
the rest on disk
From a quick scan of your code, it looks like your basic problem is that you're creating a new bitmap each time your OnDraw method is called. Instead, comment that line out, and create the bitmap for mBitmap just once at startup.
I started implementation of android live wallpaper, following the examples and tutorials found on the internet, and I can't include png background as wallpaper. Also checked with similar problems here, and still can't make it work.
This is the code:
public class LiveWallpaper extends WallpaperService {
/* IDs of recurces needed for animations*/
private SurfaceHolder holder;
private static final String TAG = "MyActivity";
#Override
public void onCreate() {
super.onCreate();
}
#Override
public void onDestroy() {
super.onDestroy();
}
#Override
public Engine onCreateEngine() {
return new WallpaperEngine();
}
class WallpaperEngine extends Engine {
public final Runnable mDrawWallpaper = new Runnable(){
public void run(){
drawWallpaper();
}
};
#Override
public void onCreate(SurfaceHolder surfaceHolder){
super.onCreate(surfaceHolder);
setTouchEventsEnabled(false);
loadImagesIntoMemory(R.drawable.wallpaper);
holder = getSurfaceHolder();
}
void drawWallpaperContent(Canvas c, int resourceId){
Bitmap decodeResoure = BitmapFactory.decodeResource (getResources(), resourceId);
c.drawBitmap(decodeResoure, 0, 0, null);
}
void drawWallpaper(){
final SurfaceHolder holder = getSurfaceHolder();
Canvas c = null;
c = holder.lockCanvas();
if(c!=null){
c.save();
drawWallpaperContent(c, R.drawable.wallpaper);
c.restore();
}
}
private void loadImagesIntoMemory(int resourceId){
Resources res = getResources();
BitmapFactory.decodeResource(res, resourceId);
}
#Override
public void onDestroy(){
super.onDestroy();
mHandler.removeCallbacks(mDrawWallpaper);
}
}
}
Bitmap is stored in drawable folder, and the version of android sdk is 2.2.
After launching the live wallpaper, I only get 'Loading Wallpaper' without showing the wallpaper image.
Does anyone knows what could be the problem?
Thank you.
Dj.
use this in your draw
' Bitmap image = BitmapFactory.decodeResource(getResources(),R.drawable.image);'
canvas.drawBitmap(image, 0, 0, paint);
you can pass null in paint parameter. m using this and its working
I struggled with a similar problem, c.drawColor(0xff000000); before drawing the bitmap was the solution for me.