I am working on streaming a video from web. Where I decode the video/audio stuff in native code and get the raw pixels for video
I create a bitmap in java code, with surfaceholder and canvas and update pixels for each bitmap from native code and then stream the bitmaps as video. My problem here is, the video crashes after a few seconds because of low memory.
I want to know whether there is anything that i need to make sure to not to crash app and use low memory.
Here is my code.
public CanvasThread(SurfaceHolder surfaceHolder, Panel panel) {
_surfaceHolder = surfaceHolder;
_panel = panel; }
public void setRunning(boolean run) {
_run = run; }
#Override
public void run() {
Canvas c;
while (_run) {
c = null;
try {
c = _surfaceHolder.lockCanvas(null);
synchronized (_surfaceHolder) {
_panel.onDraw(c);
}
} finally {
if (c != null) {
_surfaceHolder.unlockCanvasAndPost(c);
}
}
public class Panel extends SurfaceView implements SurfaceHolder.Callback{
private CanvasThread canvasthread;
private static Bitmap mBitmap;
private static boolean ii=false;
public Panel(Context context, AttributeSet attrs) {
super(context, attrs);
getHolder().addCallback(this);
canvasthread = new CanvasThread(getHolder(), this);
setFocusable(true);
mBitmap=Bitmap.createBitmap(480, 320, Bitmap.Config.RGB_565);//bitmap created in constructor
}
public Panel(Context context) {
super(context);
getHolder().addCallback(this);
canvasthread = new CanvasThread(getHolder(), this);
setFocusable(true);
}
private static native void renderbitmap(Bitmap bitmap); //native function
#Override
public void onDraw(Canvas canvas) {
renderbitmap(mBitmap); //Update pixels from native code
canvas.drawBitmap(mBitmap, 0,0,null);//draw on canvas
}
#Override
public void surfaceChanged(SurfaceHolder holder, int format, int width,int height) { }
#Override
public void surfaceCreated(SurfaceHolder holder) {
canvasthread.setRunning(true);
canvasthread.start(); }
#Override
public void surfaceDestroyed(SurfaceHolder holder) {
boolean retry = true;
canvasthread.setRunning(false);
while (retry) {
try {
canvasthread.join();
retry = false;
} catch (InterruptedException e) { }
}
You might check out Richard Quirk's glbuffer. I'm using it in a video player app.
In general you want to hold on to the video packets you're receiving and only decode them right before they are needed for display. It should be easy to integrate your code with glbuffer and bypass any Bitmap allocation in Java code.
There would be one decoded frame at any given time present in the native code and in GL texture memory and several encoded packets that you're keeping around for buffering.
That will probably not because you have limited amount of memory. Probably around 16MB for the emulator. You should not store all the images in memory. You have to options
Fix your algorithm so it doesn't
need everything in memory
Or only keep one image in memory and
the rest on disk
From a quick scan of your code, it looks like your basic problem is that you're creating a new bitmap each time your OnDraw method is called. Instead, comment that line out, and create the bitmap for mBitmap just once at startup.
Related
I have a stream of H264 video that I need to show in my Android app. If I consigure MediaCodec with a surface, then the video gets decoded to my app no problem, and I can see it in this surface.
But I also need to obtain a Bitmap of the video in certain moments (i.e. to store certain frames on the SD card). Is it possible to configure the MediaCodec library to return an array of bytes instead of working directly towards a surface??
Another option would be to obtain the bitmap directly from the surface, but I couldn't find this option on the SDK either.
You can use MediaMetadataRetriever:
MediaMetadataRetriever mediaMetadataRetriever=new MediaMetadataRetriever(); //should be stored as an instance variable
mediaMetadataRetriever.setDataSource(mVideo.getVideoUrl(Video.SD));
int time = videoView.getCurrentPosition()* 1000; //micro-to-milliseconds
Bitmap bmFrame = mediaMetadataRetriever.getFrameAtTime(time);
Edit:
To retrieve raw byte data without the involvement of UI, you can advance time frame by frame, get the Bitmap and convert it to byte array.
To be more efficient, consider forking FFmpegMediaMetadataRetriever, whose FFmpegMediaMetadataRetriever.java has method private native byte [] _getFrameAtTime(long timeUs, int option); skipping the whole Bitmap conversion process.
Try this:
public class GameView extends SurfaceView {
private Bitmap bmp;
private SurfaceHolder holder;
public GameView(Context context) {
super(context);
holder = getHolder();
holder.addCallback(new SurfaceHolder.Callback() {
#Override
public void surfaceDestroyed(SurfaceHolder holder) {
}
#Override
public void surfaceCreated(SurfaceHolder holder) {
Canvas c = holder.lockCanvas(null);
onDraw(c);
holder.unlockCanvasAndPost(c);
}
#Override
public void surfaceChanged(SurfaceHolder holder, int format,
int width, int height) {
}
});
bmp = BitmapFactory.decodeResource(getResources(), R.drawable.icon);
}
#Override
protected void onDraw(Canvas canvas) {
canvas.drawColor(Color.BLACK);
canvas.drawBitmap(bmp, 10, 10, null);
}
}
I have this android application.
It use a SurfaceView, from where I get the Surface through the SurfaceHolder.
It also use ExoPlayer to stream videos. However I have instantiated an ImageReader, getting its Surface and passing to the ExoPlayer.
Now, I am in the ImageReader.OnImageAvailableListener#onImageAvailable and I access the latest Image.
I want to manipulate the Image and send the new data to the "SurfaceView" Surface.
How can I "draw" an android.media.Image to an android.view.Surface ?
The question is not clear to me.
The way to get android.media.Image is by the Camera2 API, and there you can provide a surface and the "camera" will draw over it. Please refer to Camera2Video example
Another way to get the Image object is from ImageReader (while decoding video for example). In this case you want to draw the image, but you can not provide a surface to the ImageReader(there is an internal surface that is not displayed). In this case you can draw the Image on a SurfaceView.
Assuming this is the case, you need to convert an Image to a Bitmap objects.
You have discussion about how perform this here
Possible duplicate of: how to draw image on surfaceview android
First get your canvas by using lockCanvas() (see here), second get your image and make it a drawable using:
my_bitmap = Bitmap.createBitmap(
MediaStore.Images.Media.getBitmap(getContentResolver(), uri),
0,0,90, 90);
drawable=new BitmapDrawable(my_bitmap);
After that you can draw the drawable to the locked canvas and use unlockCanvasAndPost (Canvas canvas) to post the updated canvas back to the surfaceview.
here is the answer for your question.
MainActivity.java
public class MainActivity extends Activity {
#Override
public void onCreate(Bundle savedInstanceState) {
super.onCreate(savedInstanceState);
mySurfaceView mySurfaceView = new mySurfaceView(getApplicationContext());
setContentView(mySurfaceView);
}
}
mySurfaceView.java
public class mySurfaceView extends SurfaceView implements
SurfaceHolder.Callback {
private TutorialThread _thread;
public mySurfaceView(Context context) {
super(context);
getHolder().addCallback(this);
_thread = new TutorialThread(getHolder(), this);
}
#Override
protected void onDraw(Canvas canvas) {
super.onDraw(canvas);
Bitmap _scratch = BitmapFactory.decodeResource(getResources(),
R.drawable.icon);
canvas.drawColor(Color.BLACK);
canvas.drawBitmap(_scratch, 10, 10, null);
}
public void surfaceChanged(SurfaceHolder arg0, int arg1, int arg2, int arg3) {
}
public void surfaceCreated(SurfaceHolder arg0) {
_thread.setRunning(true);
_thread.start();
}
public void surfaceDestroyed(SurfaceHolder arg0) {
boolean retry = true;
_thread.setRunning(false);
while (retry) {
try {
_thread.join();
retry = false;
} catch (InterruptedException e) {
}
}
}
class TutorialThread extends Thread {
private SurfaceHolder _surfaceHolder;
private mySurfaceView _panel;
private boolean _run = false;
public TutorialThread(SurfaceHolder surfaceHolder, mySurfaceView panel) {
_surfaceHolder = surfaceHolder;
_panel = panel;
}
public void setRunning(boolean run) {
_run = run;
}
#Override
public void run() {
Canvas c;
while (_run) {
c = null;
try {
c = _surfaceHolder.lockCanvas(null);
synchronized (_surfaceHolder) {
_panel.onDraw(c);
}
} finally {
if (c != null) {
_surfaceHolder.unlockCanvasAndPost(c);
}
}
}
}
}
}
I'm trying to work with 3 SurfaceViews on one screen, one on top half (BoardView), one on bottom half (StatusView), and the last one as an extra layer above the top half (TileView) (see main.xml).
I created a class MySurfaceView, which is extended by BoardView, StatusView and TileView.
I've got multiple problems with this.
Let me first give the code.
main.xml:
<?xml version="1.0" encoding="utf-8"?>
<RelativeLayout
xmlns:android="http://schemas.android.com/apk/res/android"
android:layout_width="fill_parent"
android:layout_height="fill_parent"
android:background="#color/main_background">
<com.niek.test.BoardView
android:id="#+id/boardview"
android:layout_width="fill_parent"
android:layout_height="fill_parent" />
<FrameLayout
android:layout_width="fill_parent"
android:layout_height="fill_parent"
android:layout_below="#+id/boardview">
<com.niek.test.StatusView
android:id="#+id/statusview"
android:layout_width="fill_parent"
android:layout_height="fill_parent"
android:background="#F0931E"
android:layout_below="#+id/boardview" />
<com.niek.test.TileView
android:id="#+id/tileview"
android:layout_width="180dip"
android:layout_height="60dip"
android:layout_gravity="bottom"/>
</FrameLayout>
</RelativeLayout>
MainActivity.java:
package com.niek.test;
public class MainActivity extends Activity {
private Board board;
#Override
public void onCreate(Bundle savedInstanceState) {
super.onCreate(savedInstanceState);
setContentView(R.layout.main);
board = new Board();
BoardView boardView = (BoardView) findViewById(R.id.boardview);
boardView.setBoard(board);
StatusView statusView = (StatusView) findViewById(R.id.statusview);
statusView.setBoard(board);
}
}
MySurfaceView.java
package com.niek.test;
public class MySurfaceView extends SurfaceView implements SurfaceHolder.Callback {
protected DrawThread drawThread;
public MyView(Context context, AttributeSet attrs) {
super(context, attrs);
getHolder().addCallback(this);
setFocusable(true);
drawThread = new DrawThread(getHolder());
}
#Override
public void surfaceChanged(SurfaceHolder holder, int format, int width, int height) {
// TODO Auto-generated method stub
}
#Override
public void surfaceCreated(SurfaceHolder holder) {
// TODO Auto-generated method stub
drawThread.setRunning(true);
drawThread.start();
}
#Override
public void surfaceDestroyed(SurfaceHolder holder) {
// we have to tell thread to shut down & wait for it to finish, or else
// it might touch the Surface after we return and explode
boolean retry = true;
drawThread.setRunning(false);
while (retry) {
try {
drawThread.join();
retry = false;
} catch (InterruptedException e) {
// we will try it again and again...
}
}
}
protected class DrawThread extends Thread {
private SurfaceHolder surfaceHolder;
private boolean isRunning;
public DrawThread(SurfaceHolder surfaceHolder) {
this.surfaceHolder = surfaceHolder;
isRunning = false;
}
public void setRunning(boolean run) {
isRunning = run;
}
public void run() {
Canvas c;
while (isRunning) {
try {
Thread.sleep(100);
} catch (Exception e) {
// TODO: handle exception
}
c = null;
try {
c = surfaceHolder.lockCanvas(null);
synchronized (surfaceHolder) {
onDraw(c);
postInvalidate();
}
} finally {
// do this in a finally so that if an exception is thrown
// during the above, we don't leave the Surface in an
// inconsistent state
if (c != null) {
surfaceHolder.unlockCanvasAndPost(c);
}
}
}
}
}
}
These three classes extend MySurfaceView:
BoardView.java
package com.niek.test;
public class BoardView extends MySurfaceView {
private int squareSize, marginX, marginY;
private Board board;
Paint boardBorder;
public BoardView(Context context, AttributeSet attrs) {
super(context, attrs);
board = null;
}
public void setBoard(Board board) {
this.board = board;
}
private void init(SurfaceHolder holder) {
Canvas canvas = null;
try {
canvas = holder.lockCanvas();
/* Initialize the board */
squareSize = canvas.getWidth() / Board.GRIDSIZE;
/* Size the view */
LayoutParams lp = getLayoutParams();
lp.height = (squareSize * Board.GRIDSIZE) + 4;
setLayoutParams(lp);
/* Place the board neatly in the center */
marginX = (canvas.getWidth() - (squareSize * Board.GRIDSIZE)) / 2;
marginY = 1;
} finally {
holder.unlockCanvasAndPost(canvas);
}
boardBorder = new Paint();
boardBorder.setColor(Color.RED);
boardBorder.setStyle(Style.STROKE);
}
#Override
public void onDraw(Canvas canvas) {
drawBoard(board, canvas);
}
#Override
public void surfaceCreated(SurfaceHolder holder) {
init(holder);
super.surfaceCreated(holder);
}
private void drawBoard(Board board, Canvas canvas) {
synchronized (board) {
if (board != null) {
for (Square[] ys : board.getSquares()) {
for (Square xs : ys) {
xs.onDraw(canvas, squareSize, squareSize, marginX, marginY);
}
}
}
canvas.drawRect(marginX - 1, marginY - 1, marginX + squareSize * Board.GRIDSIZE + 1, marginY + squareSize * Board.GRIDSIZE + 1, boardBorder);
}
}
}
StatusView.java
package com.niek.test;
public class StatusView extends MySurfaceView {
private Board board;
private Paint textPaint;
public StatusView(Context context, AttributeSet attrs) {
super(context, attrs);
board = null;
textPaint = new Paint();
textPaint.setColor(Color.BLACK);
textPaint.setTextSize(20);
textPaint.setTypeface(Typeface.DEFAULT_BOLD);
}
public void setBoard(Board board) {
this.board = board;
}
int tmp=0;
#Override
public void onDraw(Canvas c) {
if (board != null) {
c.drawText(tmp+"", 10, 20, textPaint);
tmp++;
System.out.println(tmp);
}
}
}
TileView.java
package com.niek.test;
public class TileView extends MySurfaceView {
public TileView(Context context, AttributeSet attrs) {
super(context, attrs);
System.out.println(0);
}
int tmp =0;
#Override
public void onDraw(Canvas c) {
System.out.println(2);
Paint p= new Paint();
p.setColor(Color.RED);
c.drawColor(Color.RED);
c.drawText(tmp+"",10,10,p);
tmp++;
}
}
Now what are my problems?
First off, as you can see in MySurfaceView I've got this:
try {
c = surfaceHolder.lockCanvas(null);
synchronized (surfaceHolder) {
onDraw(c);
postInvalidate();
}
}
When I only use onDraw(c), only the BoardView gets drawn, the StatusView doesn't get drawn, but the tmp increments in the onDraw of StatusView are being executed.
When I only use postInvalidate(), same story, but only StatusView gets drawn, BoardView doesn't.
So that's why I use both methods, and both Views get drawn.
Then there's TileView, the System.out(2) is being shown in logcat, but the view doesn't get drawn. It is a black square instead of the red square I ask it to be in the onDraw method.
When I turn the screen off and then on again, the TileView does get drawn, and the tmp increments are shown.
Who can help me?
For clarity, I've created this based on this tutorial.
You can have multiple SurfaceViewsin one layout. The "Multi-surface test" activity in Grafika has three.
The first post cited in #nonsleepr's answer was followed up 9 months later with this post by the same author, which mentioned the existence of SurfaceView#setZOrderMediaOverlay().
The key thing to understand is that SurfaceView is not a regular view. When your app comes to the foreground it gets a surface to draw on. Everything in your app's UI is rendered onto the app's surface by the app, and then that surface is composited with other surfaces (like the status bar and navigation bar) by the system compositor. When you create a SurfaceView, it's actually creating an entirely new surface that is composited by the system, not by your app.
You can control the Z-ordering (i.e. "depth") of the SurfaceView surface very loosely. There are four positions, from top to bottom:
SurfaceView + ZOrderOnTop
(app UI goes here)
SurfaceView + ZOrderMediaOverlay
SurfaceView (default)
If you have two SurfaceViews at the same depth, and they overlap, the results are undefined -- one will "win", but you can't control which.
The system compositor on modern devices is very efficient when you have N surfaces. At N+1 surfaces you hit a performance cliff. So while you can have three SurfaceViews, you're generally better off keeping the number down. The value of N varies from device to device.
Update: if you really want to understand how SurfaceView works, see the Android System-Level Graphics doc.
It looks like you are not supposed to create multiple SurfaceViews on one Layout.
According to this two posts written by Android framework engineer:
The way surface view is implemented is that a separate surface is created and Z-ordered behind its containing window, and transparent pixels drawn into the rectangle where the SurfaceView is so you can see the surface behind. We never intended to allow for multiple surface view.
and
you should effectively think of SurfaceView as an overlay you embed inside your window,
giving you an area in which you can directly draw independently of the normal view update system.
So, what you can do, is use one SurfaceView to draw all the graphics you want.
It sounds like the SurfaceViews are being drawn, but transparency is not enabled for whichever one is on top. In your MySurfaceView class in the surfaceCreated() method, make sure you are calling holder.setFormat(PixelFormat.TRANSPARENT);
I'm currently trying to build an android application to take pictures and I need to freeze the camera preview on a given event (i.e. picture taken) and restart it only after another event.
What I want, in other words, is for the view to display whatever the camera sees until the freeze event occurs and then to freeze the image (i.e. display whatever was on screen at the time of this event -- as if a picture was taken) until the unfreeze event occurs.
Now, I'm currently using a SurfaceView with a SurfaceHolder.Callback to do this and I tried to use a PreviewCallback to freeze the screen, but unfortunately, I can't find an example or a tutorial and I'm really stuck at this point.
If anyone has a guide or some pointers on how to get this done, I would really appreciate the help...
I'm pasting the relevant parts of my code below:
public class CustomCameraView extends SurfaceView {
Camera camera;
SurfaceHolder previewHolder;
//Callback for the surfaceholder
SurfaceHolder.Callback surfaceHolderListener = new SurfaceHolder.Callback() {
public void surfaceCreated(SurfaceHolder holder) {
camera=Camera.open();
try
{
camera.setPreviewDisplay(previewHolder);
}
catch (Throwable t) {
}
}
public void surfaceChanged(SurfaceHolder surfaceHolder, int format, int w, int h)
{
Parameters params = camera.getParameters();
params.setPictureFormat(PixelFormat.JPEG);
camera.setParameters(params);
camera.startPreview();
}
public void surfaceDestroyed(SurfaceHolder arg0)
{
camera.stopPreview();
camera.release();
}
};
public CustomCameraView(Context ctx)
{
super(ctx);
previewHolder = this.getHolder();
previewHolder.setType(SurfaceHolder.SURFACE_TYPE_PUSH_BUFFERS);
previewHolder.addCallback(surfaceHolderListener);
setBackgroundColor(Color.TRANSPARENT);
}
public CustomCameraView(Context context, AttributeSet attrs)
{
super(context, attrs);
}
protected void onDraw (Canvas canvas)
{
}
public void closeCamera()
{
if(camera != null)
camera.release();
}
public void dispatchDraw(Canvas c)
{
super.dispatchDraw(c);
}
}
Thank you very much for your help!
-Billy
Old question, I know, but answering for posterity. You should be able to simply call
camera.stopPreview();
The preview will freeze on whatever you're looking at until you call startPreview() again.
I started implementation of android live wallpaper, following the examples and tutorials found on the internet, and I can't include png background as wallpaper. Also checked with similar problems here, and still can't make it work.
This is the code:
public class LiveWallpaper extends WallpaperService {
/* IDs of recurces needed for animations*/
private SurfaceHolder holder;
private static final String TAG = "MyActivity";
#Override
public void onCreate() {
super.onCreate();
}
#Override
public void onDestroy() {
super.onDestroy();
}
#Override
public Engine onCreateEngine() {
return new WallpaperEngine();
}
class WallpaperEngine extends Engine {
public final Runnable mDrawWallpaper = new Runnable(){
public void run(){
drawWallpaper();
}
};
#Override
public void onCreate(SurfaceHolder surfaceHolder){
super.onCreate(surfaceHolder);
setTouchEventsEnabled(false);
loadImagesIntoMemory(R.drawable.wallpaper);
holder = getSurfaceHolder();
}
void drawWallpaperContent(Canvas c, int resourceId){
Bitmap decodeResoure = BitmapFactory.decodeResource (getResources(), resourceId);
c.drawBitmap(decodeResoure, 0, 0, null);
}
void drawWallpaper(){
final SurfaceHolder holder = getSurfaceHolder();
Canvas c = null;
c = holder.lockCanvas();
if(c!=null){
c.save();
drawWallpaperContent(c, R.drawable.wallpaper);
c.restore();
}
}
private void loadImagesIntoMemory(int resourceId){
Resources res = getResources();
BitmapFactory.decodeResource(res, resourceId);
}
#Override
public void onDestroy(){
super.onDestroy();
mHandler.removeCallbacks(mDrawWallpaper);
}
}
}
Bitmap is stored in drawable folder, and the version of android sdk is 2.2.
After launching the live wallpaper, I only get 'Loading Wallpaper' without showing the wallpaper image.
Does anyone knows what could be the problem?
Thank you.
Dj.
use this in your draw
' Bitmap image = BitmapFactory.decodeResource(getResources(),R.drawable.image);'
canvas.drawBitmap(image, 0, 0, paint);
you can pass null in paint parameter. m using this and its working
I struggled with a similar problem, c.drawColor(0xff000000); before drawing the bitmap was the solution for me.