My current android applications main functionality is to pull the frames (10fps) from a recorded video using the built in camera. Whenever the user selects the video I call my class "FrameCollector" which loops through the video and pulls the frames out and stores them into an ArrayList of Bitmaps. Having this work well for a number of days had me thinking that I was on the right track, but now I'm getting the dreaded "java.lang.OutofMemoryError"
My code is as follows:
Here it is setting the MediaMetaDataRetriever to select 10 frames per second from the video path which was passed from the main class
public class FrameCollector {
MediaMetadataRetriever _mmr;
double _fps;
double _duration;
long _counter = 0;
long _incrementer;
public FrameCollector(String path, Context context)
{
try
{
_mmr = new MediaMetadataRetriever();
_mmr.setDataSource(path);
SharedPreferences pref = PreferenceManager.getDefaultSharedPreferences(context);
String fpsString = pref.getString("prefFPS", "10");
_fps = Double.parseDouble(fpsString);
_incrementer = (long) (1000000 / _fps);
String stringDuration = _mmr.extractMetadata(MediaMetadataRetriever.METADATA_KEY_DURATION);
_duration = Double.parseDouble(stringDuration) * 1000;
}
catch(Exception ex)
{
ex.printStackTrace();
}
}
The below method is then adding the frame at that time to a bitmap and adding the bitmap to an arraylist of bitmaps "bitFrames"
public ArrayList<Bitmap> getBitmaps()
{
try
{
ArrayList<Bitmap> bitFrames = new ArrayList<Bitmap>();
Bitmap b = _mmr.getFrameAtTime(_counter);
while (_counter < _duration && b != null)
{
bitFrames.add(b);
_counter += _incrementer;
b = _mmr.getFrameAtTime(_counter);
}
return bitFrames;
}
catch (Exception ex)
{
ex.printStackTrace();
return null;
}
}
I'm thinking my issue lies with in this method. I believe I need to decode the bitmaps before I store them into the ArrayList but I'm unsure how as
Bitmap b = BitmapFactory.decode____(_mmr.getFrameAtTime(_counter));
-- Whether it be decodeStream, decodeResource, decodeFile all bring errors.
Any help would be much appreciated
Many Thanks,
Never store Bitmaps in an ArrayList unless the number of bitmaps are very small. Bitmaps take up large amounts of memory.
I've not worked with video recording before, but I would suggest you try to get a native codec library like ffmpeg to do it for you.
If you have to grab the frames yourself, however, I suggest storing them in a fixed size buffer in memory from the camera and have a background thread which pulls bitmaps from the buffer and stores it to the disk in parallel.
It's not good to store ArrayList of Bitmaps in RAM.
Anyway if you need more than default heap size to store bitmaps in ArrayList, you can try this hack.
http://habrahabr.ru/post/139717/
It's in Russian, but I guess code is quite clear to understand.
Related
I am working on an app that uses a Recyclerview to display mp3 files, providing its cover art image along with other info. It works but is slow once it starts dealing with a dozen or more cover arts to retrieve, as I am currently doing this from the id3 on the main thread, which I know is not a good idea.
Ideally, I would work with placeholders so that the images can be added as they become available. I've been looking into moving the retrieval to a background thread and have looked at different options: AsyncTask, Service, WorkManager. AsyncTask seems not to be the way to go as I face memory leaks (I need context to retrieve the cover art through MetadataRetriever). So I am leaning away from that. Yet I am struggling to figure out which approach is best in my case.
From what I understand I need to find an approach that allows multithreading and also a means to cancel the retrieval in case the user has already moved on (scrolling or navigating away). I am already using Glide, which I understand should help with the caching.
I know I could rework the whole approach and provide the cover art as images separately, but that seems a last resort to me, as I would rather not weigh down the app with even more data.
The current version of the app is here (please note it will not run as I cannot openly divulge certain aspects). I am retrieving the cover art as follows (on the main thread):
static public Bitmap getCoverArt(Uri medUri, Context ctxt) {
MediaMetadataRetriever mmr = new MediaMetadataRetriever();
mmr.setDataSource(ctxt, medUri);
byte[] data = mmr.getEmbeddedPicture();
if (data != null) {
return BitmapFactory.decodeByteArray(data, 0, data.length);
} else {
return null;
}
}
I've found many examples with AsyncTask or just keeping the MetaDataRetriever on the main thread, but have yet to find an example that enables a dozen or more cover arts to be retrieved without slowing down the main thread. I would appreciate any help and pointers.
It turns out it does work with AsyncTask, as long as it is not a class onto itself but setup and called from a class with context. Here is a whittled down version of my approach (I am calling this from within my Adapter.):
//set up titles and placeholder image so we needn't wait on the image to load
titleTv.setText(selectedMed.getTitle());
subtitleTv.setText(selectedMed.getSubtitle());
imageIv.setImageResource(R.drawable.ic_launcher_foreground);
imageIv.setAlpha((float) 0.2);
final long[] duration = new long[1];
//a Caching system that helps reduce the amount of loading needed. See: https://github.com/cbonan/BitmapFun?files=1
if (lruCacheManager.getBitmapFromMemCache(selectedMed.getId() + position) != null) {
//is there an earlier cached image to reuse? imageIv.setImageBitmap(lruCacheManager.getBitmapFromMemCache(selectedMed.getId() + position));
imageIv.setAlpha((float) 1.0);
titleTv.setVisibility(View.GONE);
subtitleTv.setVisibility(View.GONE);
} else {
//time to load and show the image. For good measure, the duration is also queried, as this also needs the setDataSource which causes slow down
new AsyncTask<Uri, Void, Bitmap>() {
#Override
protected Bitmap doInBackground(Uri... uris) {
MediaMetadataRetriever mmr = new MediaMetadataRetriever();
mmr.setDataSource(ctxt, medUri);
byte[] data = mmr.getEmbeddedPicture();
Log.v(TAG, "async data: " + Arrays.toString(data));
String durationStr = mmr.extractMetadata(MediaMetadataRetriever.METADATA_KEY_DURATION);
duration[0] = Long.parseLong(durationStr);
if (data != null) {
InputStream is = new ByteArrayInputStream(mmr.getEmbeddedPicture());
return BitmapFactory.decodeStream(is);
} else {
return null;
}
}
#Override
protected void onPostExecute(Bitmap bitmap) {
super.onPostExecute(bitmap);
durationTv.setVisibility(View.VISIBLE);
durationTv.setText(getDisplayTime(duration[0], false));
if (bitmap != null) {
imageIv.setImageBitmap(bitmap);
imageIv.setAlpha((float) 1.0);
titleTv.setVisibility(View.GONE);
subtitleTv.setVisibility(View.GONE);
} else {
titleTv.setVisibility(View.VISIBLE);
subtitleTv.setVisibility(View.VISIBLE);
}
lruCacheManager.addBitmapToMemCache(bitmap, selectedMed.getId() + position);
}
}.execute(medUri);
}
I have tried working with Glide for the caching, but I haven't been able to link the showing/hiding of the TextViews to whether there is a bitmap. In a way though, this is sleeker as I don't need to load the bulk of the Glide-library. So I am happy with this for now.
Here's another question for you :)
Basically i made a realtime streaming service, sending multiple jpegs to my android app, that decodes them as soon as he receives them.
// dIn is DataInputStream
// videoFeed is an ImageView
// bitmap is Bitmap
// hand is an Handler of the main thread
//CODE EXECUTED IN ANOTHER THERAD
byte[] inBuff = new byte[8];
byte[] imgBuff;
String inMsg;
while(socket.isConnected()) {
dIn.readFully(inBuff);
inMsg = new String(inBuff, "ASCII").trim();
int size = Integer.parseInt(inMsg);
imgBuff = new byte[size];
dIn.readFully(imgBuff);
out.write("SEND-NEXT-JPEG".getBytes("ASCII"));
bitmap = BitmapFactory.decodeByteArray(imgBuff, 0, size);
hand.post(setImage);
}
}
private Runnable setImage = new Runnable() {
#Override
public void run() {
videoFeed.setImageBitmap(bitmap);
}
};
The problem is that after about 10 or 20 jpegs are perfectly decoded in realtime, the app freezes for 400ms or so and then it continues to decode other 10/20 jpegs before another freeze...
I know that sending multiple jpegs it's not a good way for streaming video but i can only change the client (android app), not the server.
Do you have any idea for get a fluid video and avoid freezes? thanks!
Right now, you are using the three-parameter version of decodeByteArray(). Instead, switch to the four-parameter version, passing in a BitmapFactory.Options as the last value. On there, set inBitmap to be a Bitmap object that can be reused.
This requires you to maintain a small Bitmap object pool. It could be as simple as two Bitmap instances: the one that is presently being displayed and the one that you are preparing for the next "frame" of the video.
The catch is that, for API Level 18 and below, the Bitmap needs to be the same resolution (height and width in pixels). In your case, that's probably not a problem, as I would imagine that each of your bitmaps have the same resolution.
the main problem now i want to ad effects on video
i got that idea to convert the video into bitmaps add effects then return it again
but i cound`t return it to video again
any help
or any other way to do add effects on video without using ffmpeg
public ArrayList<Bitmap> getFrames(Uri path) {
try {
bArray = new ArrayList<Bitmap>();
bArray.clear();
MediaMetadataRetriever mRetriever = new MediaMetadataRetriever();
// mRetriever. setDataSource(path);
mRetriever.setDataSource(this, path);
for (int i = 0; i < 50; i++) {
// time in msec 1000*i
bArray.add(mRetriever.getFrameAtTime(10,
MediaMetadataRetriever.OPTION_CLOSEST));
}
return bArray;
} catch (Exception e) {
return null;
}
}
thanks.
Please pay attention to your getFrameatTime's first parameter that is in microsends. Otherwise you will still get the first frame's bitmap as return. Maybe your can decode the frames and show them in sequence like a movie.
The problem
Hi there,
I'm developing an application where the user specifies some pictures and how long they are going to be on the screen.So sometimes he wants to create something like a small animation or viewing the images for a small amount of time.The problem is that after some time the images are not previewed when they should and we have a few ms of error.In the application that i'm developing time matters so I would like some help on what the problem might be.
The code
So let me explain how it works.I take the pictures from my web app and then I save them in a HashMap
Bitmap image = ImageOperations(url,String.valueOf(frameNum) + ".jpg");
ImageMap.put(String.valueOf(frameNum), image);
where the mathod ImageOperations is like that:
private Bitmap ImageOperations(String url, String saveFilename) {
try {
Display display = getWindowManager().getDefaultDisplay();
InputStream is = (InputStream) this.fetch(url);
Bitmap theImage = BitmapFactory.decodeStream(is);
if (theImage.getHeight() >= 700 || theImage.getWidth() >= 700) {
theImage = Bitmap.createScaledBitmap(theImage,
display.getWidth(), display.getHeight() - 140, true);
}
return theImage;
} catch (MalformedURLException e) {
e.printStackTrace();
return null;
} catch (IOException e) {
e.printStackTrace();
return null;
}
}
So later I run a thread that updates the UI when the user specified.The method that updates it is this one.
public void setPictures(int NumOfFrame) {
if (frameArray.get(NumOfFrame - 1).frame_pic.contains("n/a") != true) {
ImagePlace.setImageBitmap(ImageMap.get(String.valueOf(NumOfFrame)));
} else {
ImagePlace.setImageDrawable(null);
}
}
After we update the image we put the thread for sleep and when runs again it updates the thread.Is there something that creates the problem?Does it have to do with Garbage collection?
Thank you in advance
Probably the issue is in increasing heap size when it loads additional images. I would suggest You to do some profiling so things will be much clearer and You'll get full picture of timings for the app.
First you are missing a null check at here:
ImageMap.get(String.valueOf(NumOfFrame))
And you do not recycle the old bitmap at here:
theImage.recycle(); // missing line
theImage = Bitmap.createScaledBitmap(theImage,
display.getWidth(), display.getHeight() - 140, true);
It may lead to outofmemory exceptions, with is most likely from your description of the problem.
Also I am not sure if BitmapFactory.decodeStream throw exception when he fails. You need to add a null point check there too.
I have a very strange bug. It only happens in the Emulator. I tested it on multiple Android phones and the Acer Tablet, it works fine there.
My program has a loop that loads in Bitmaps into a Bitmap[] bitCards. The Array is set up for 14 elements by bitCards = new Bitmap[14].
Now it loops 12 times to put a Bitmap into the Array as follows:
bitCards[i] = BitmapFactory.decodeStream(inputStream);
When i = 8 it crashes at this statement.
If i replace it with
bitCards[0] = BitmapFactory.decodeStream(inputStream);
it does not crash, I thought maybe somehow the Array was not big enough so I did the following
bitCards[8]=BitmapFactory.decodeStream(inputStream); // Still did not crash.
The only thing that makes sense is that when I have
bitCards[i] = BitmapFactory.decodeStream(inputStream);
It is relasing the old memory and putting in a new object, thus only memory for one object is created, but.... the exception does not go off, shouldn't I get some kind of an error?
Here is my full code:
void Shuffle()
{
Random generator;
generator = new Random();
int[] iCardsUsed;
iCardsUsed = new int[55];
for(int i=0;i<55;i++)
iCardsUsed[i]=0;
try {
bitCards = new Bitmap[14];
iCards = new int[14];
iTurnOver = new int[14];
for (int i = 0; i < 12; i++)
{
iTurnOver[i]=0;
int cardId;
do {
cardId = generator.nextInt(50);
} while( iCardsUsed[cardId] ==1);
iCardsUsed[cardId] =1;
iCards[i]=cardId;
iCards[i]=i;
String fName=new String("card");
fName+=Integer.toString(iCards[i]+1);
fName+=".jpg";
AssetManager assetManager= ctx.getAssets();
InputStream inputStream;
inputStream = assetManager.open(fName);
// this is where it crashes
bitCards[i]=BitmapFactory.decodeStream(inputStream);
inputStream.close();
}
} catch( IOException e)
{
gi++;
}
// update screen
invalidate();
}
Since you have provided no error message, I am taking a shot in the dark and assuming it is going OOM.
You say that it stops after running for a few times ( when i = 8) , I believe that you are not freeing the resources. Bitmaps can sometime take up a lot of space and if you are persisting them in the memory, I would not be surprised if the device goes OutOfMemory. Different devices have different specs for memory and after a few runs it is filling up the memory.
So, my suggestion would be to clear the Bitmaps, using mBitmap.recycle(), and the other storage that you are using for temporary purposes.
Also, have a look at this question!