I am trying to develop an app that takes a 15 seconds of video, allows the user to apply different filters, shows the preview of the effect, then allows to save the processed video to sdcard. I use ffmpeg to split the video into JPEG frames, apply the desired filter using GPUImage to all the frames, then use ffmpeg to encode the frames back to a video. Everything works fine except the part where user selects the filter. When user selects a filter, the app is supposed to display the preview of the video with the filter applied. Though 450 frames get the filter applied fairly quick, displaying the images sequentially at 30 fps (to make the user feel the video is being played) is performing poorly. I tried different approaches but the maximum frame rate I could attain even on the fastest devices is 10 to 12 fps.
The AnimationDrawable technique doesn't work in this case because it requires the entire images to be buffered into memory which in this case is huge. App crashes.
The below code is the best performing one so far (10 to 12 fps).
package com.example.animseqvideo;
import ......
public class MainActivity extends Activity {
Handler handler;
Runnable runnable;
final int interval = 33; // 30.30 FPS
ImageView myImage;
int i=0;
#Override
protected void onCreate(Bundle savedInstanceState) {
super.onCreate(savedInstanceState);
setContentView(R.layout.activity_main);
myImage = (ImageView) findViewById(R.id.imageView1);
handler = new Handler();
runnable = new Runnable(){
public void run() {
i++; if(i>450)i=1;
File imgFile = new File(Environment.getExternalStorageDirectory().getPath() + "/com.example.animseqvideo/image"+ String.format("%03d", i) +".jpg");
if(imgFile.exists()){
Bitmap myBitmap = BitmapFactory.decodeFile(imgFile.getAbsolutePath());
myImage.setImageBitmap(myBitmap);
}
//SOLUTION EDIT - MOVE THE BELOW LINE OF CODE AS THE FIRST LINE OF run() AND FPS=30 !!!
handler.postDelayed(runnable, interval);
}
};
handler.postAtTime(runnable, System.currentTimeMillis()+interval);
handler.postDelayed(runnable, interval);
}
}
I understand that the process of getting an image from SDCard, decoding it, then displaying it onto the screen involves the performance of the SDCard reading, the CPUs performance and graphics performance of the device. But I am wondering if there is a way I could save a few milliseconds in each iteration. Any suggestion would be of great help at this point.
I also did a similar thing. I used a ValueAnimator. The images is an array of the image ids. eg: R.drawable.park1
ValueAnimator animation = ValueAnimator.ofInt(0,44);
animation.setInterpolator(new LinearInterpolator());
animation.setDuration(1000);
animation.setRepeatCount(200);
animation.addUpdateListener(new AnimatorUpdateListener() {
int i;
public void onAnimationUpdate(ValueAnimator animation) {
Log.i("TAG", "::onAnimationUpdate: "+(++i)+" value = "+animation.getAnimatedValue());
mImageViewPreview.setImageResource(images[(Integer) animation.getAnimatedValue()]);
mImageViewPreview.invalidate();
if(mImageViewPreview.getVisibility()!=View.VISIBLE){
animation.cancel();
}
}
});
animation.start();
You are correct in saying that the performance of SDCard reading is slow. In fact look at how many things you want done - creating a new file from an external location, decoding it and creating a new bitmap. And all this done on the UI thread.
I don't know how big your images are but have you tried possibly preloading some of them, putting them in a list and then just picking them from that list and displaying them in your handler code?
I realize storing 450 images in memory is quite a lot but the task could probably be split up. Say you have preload 100 bitmaps and start displaying them, and then have an Async Task preloading the other 100 into the next list(or possibly the same list) at the same time.
My Bad ! Found the culprit. Being used to NSTimer on iOS, I overlooked the importance of the line of code :
handler.postDelayed(runnable, interval);
That code responsible for scheduling the next frame process was being executed AFTER the current frame getting read from the SDCard, decoded and displayed on to the imageview.
ie. If the entire process of displaying the current frame took 25 milliseconds, the next frame processing would be scheduled after 25+33 = 58ms after previous frame.
I moved that line of code to the first line of the run{} method of the runnable thread , and BINGO !! FPS raised to 24-30 FPS even on a $100 android device.
Anyways thanks to #Valentin, #Pedro Bernardo and #Shashika for pointing out some valid possible solutions.
Related
I am writing an Android app where I want to display a consistent stream using the newest Glide 4.11.0. My idea was that I display every 100 ms a new image using a for loop where I call the Glide function. When I start my function loadImageFromURL() it runs through and afterwards display all the images as fast as possible after each other, but I want to get it displayed evenly with a time gap of around 100 ms.
What am I doing wrong? Is there a problem with an internal thread of Glide?
My Code:
public void loadImageFromURL() {
Glide.get(this).setMemoryCategory(MemoryCategory.HIGH); //use more RAM than normal
final long Start = System.currentTimeMillis();
for (int i = 0; i < 5; imageNumber++) { //for loop for displaying continuously stream
if (System.currentTimeMillis() > Start + i * 100) {
Glide.with(this).asBitmap().load(ImageURL).diskCacheStrategy(DiskCacheStrategy.NONE).override(ScreenSizeX, ScreenSizeY).into(new CustomTarget<Bitmap>(){
#Override
public void onResourceReady(#NonNull Bitmap resource, #Nullable Transition<? super Bitmap> transition) {
_imageView.setImageBitmap(resource);
imageNumber++;
}
#Override
public void onLoadCleared(#Nullable Drawable placeholder) {
}
});
i++;
}
}
Glide.get(this).setMemoryCategory(MemoryCategory.NORMAL); //RAM usage back to normal
}
There are multiple flaws with your code. Glide downloads the images in a separate thread, so it's running the download code independently from your loop. Some issues:
You are manipulating imageNumber from multiple threads. This will cause random time-sensitive issues. You should not increment imageNumber in onResourceReady
The loop will just infinitely loop until the desired time is reached. This will cause heavy CPU load and can be solved by using locks, waits or similar.
My suggestion: Try loading each image after another synchronously with the solution from this answer. When an image has loaded, calculate how much time remains for your "image load break" to pass and use Thread.sleep(milliSeconds) to wait out the remaining time. Then you can update the imageView.
Make sure you run this code in a separate thread and use runOnUIThread when updating the imageView.
You can probably further optimize this by using a ConcurrentQueue where the download of the images all happen at once and a separate consumer thread takes from the queue and waits in between. This will probably have higher memory usage though because the images will be also stored in the queue.
I have an activity that show some full screen images in crossfade. There are a total of 6 images. To do this I used 2 ImageViews and 2 animation playing at the same time, one that fades out the first image and one that fades in the second. I used this video as a reference https://www.youtube.com/watch?v=9XbKMUtVnJA
Because I need this to run continuously, I used a timer to schedule the animation every 4 seconds. Everything works, but it uses a huge quantity of memory. To load the images I tried:
the basic not recommended way used in the video, i.e loading all images in a drawable array
setting the images using imageview.setImageResource
loading the images from assets using Picasso
All this method are memory intensive a cause an out of memory exception on older devices (like a Galaxy S2). The Picasso approach isn't working properly.
I'm sure there's a better way of doing this, but I don't know it, any suggestions?
Here's the relevant code:
private void animateImageview(){
prevImageView.animate().alpha(0);
nextImageView.animate().alpha(1).setListener(new AnimatorListenerAdapter() {
#Override
public void onAnimationEnd(Animator animation) {
mCurrentDrawable =
(mCurrentDrawable + 1) % imagesToShow.length;
int nextDrawableIndex =
(mCurrentDrawable + 1) % imagesToShow.length;
prevImageView.setImageResource(imagesToShow[mCurrentDrawable]);
nextImageView.setImageResource(imagesToShow[nextDrawableIndex]);
nextImageView.setAlpha(0f);
prevImageView.setAlpha(1f);
}
});
protected void onCreate(Bundle savedInstanceState) {
...
prevImageView.setImageResource(imagesToShow[0]);
nextImageView.setImageResource(imagesToShow[1]);
new Timer().scheduleAtFixedRate(new TimerTask() {
#Override
public void run() {
animateImageview();
}
}, 5000, 4000);
}
Edit:
the largeHeap option in the manifest seems to solve the problem, but I'm not convinced that I'm doing this thing right anyway.
Edit2:
A gist with the solution I used https://gist.github.com/alexmazza/003e3449c02fe58848a9
Please remove the largeHeap option from the manifest. It's not fixing the problem, it's sweeping the problem under the rug.
This is the problem
The images are 1536x2048 loaded in an imageview
You shouldn't just try to load the image at maximum resolution, because, as you said, in a S2 for instance, which has a display of 800 x 480, what's the point?
Generally speaking, what you should do instead is load a bitmap into memory as big as you need it to be (in your case it should be as big as the ImageView). You can follow a very good tutorial for this in the official docs.
To avoid java.lang.OutOfMemory exceptions, check the dimensions of a bitmap before decoding it, unless you absolutely trust the source to provide you with predictably sized image data that comfortably fits within the available memory.
[....]
For example, it’s not worth loading a 1024x768 pixel image into memory if it will eventually be displayed in a 128x96 pixel thumbnail in an ImageView.
Hope this helps.
I'm trying to hook up a basic audio player. The audio side is done, but I'm using a SeekBar as a track scrubber and I'm having trouble with performance.
The SeekBar is from an XML layout, and I'm using fairly basic code to update it:
updateTask = new Runnable() {
public void run() {
int elapsed = 0;
if (player != null) {
elapsed = player.getElapsedTime(); // Seconds
}
trackSeekbar.setProgress(elapsed);
updateHandler.postDelayed(this, 1000);
}
};
updateHandler.postDelayed(updateTask, 1000);
This code is taking between 20%–50% CPU on a Nexus 10! It makes other parts of my app are very choppy so it has to be faster.
I have already taken out some things in the layout to try and reduce layout redraws (I was displaying the elapsed time in a text box but gave up because it was just too slow), but I do need a basic scrubber. When I take out the call to setProgress() CPU drops to 1%. Is there a better way of doing this?
The cause of the poor performance was a high resolution transparent PNG in the background. When I removed it, all the various performance issues disappeared.
On the Nexus 10, the image was 2560x1440 which is its native resolution. Seemingly Android is not able to cope with full resolution background images.
I'm trying to create an Android app that will include many (sometimes up to about 200) small images that will be animated (relocate and change size) about 20 at a time. After that they'll remain still.
Should I use the simple solution of the View class animation or should I use Drawable animation?
EDIT: I should be more specific..
There are a lot of tutorial out there and a lot of different ways to do the same thing. I'm looking for the best way to implement the next scenario:
Say I have 50 different small (30x30) images currently drawn on the screen.
I need to animate them so they will translate to a different DYNAMIC position. And while they are moving I need the image to be resized up and down (so I get kind of a jump effect if looking from top).
They need to move within a specific timeframe. For example: After the first image starts to move, the second will begin moving 50ms after the last and so on (wave effect)...
After one group of images is translated, another group will be formed, but the last group will still be on screen.
So what I'm asking is a little specifics about the best way to do this. For example: Should I create a XML file for each Image or should I just load them in code? Should I load all the images (there could be up to 200 small images, maybe more) at application start or will it be ok to load them on demand? What would be the best animation technique? Stuff like that.
The easiest solution I found: (API 16+)
Runnable endAction = new Runnable() {
public void run() {
tv.animate().x(400).y(1400).scaleX(1).scaleY(1);
}
};
tv.animate().x(600).y(100).scaleX(3).scaleY(3).withEndAction(endAction);
I would use Drawable animation but it doesn´t matter so much. The important thing you should do if the app runs very slow, is to use diferents threads using this code for example:
Handler mHandler = new Handler();
mHandler.post(new Runnable() {
public void run() {
//YOUR ANIMATION HERE
}
});
In this way, you will be able to process the animation of a lot of images at the same time because the phone will execute the code in different computing threads.
You can use too AsyncTask like that (adding the class into your activity class):
private class doAnimation extends AsyncTask<ImageView, Void, Void>{
#Override
protected Void doInBackground(ImageView... image) {
image.startAnimation(animation);
return null;
}
}
And calling it using:
new doAnimation().execute(image1);
new doAnimation().execute(image2);
new doAnimation().execute(image3);
...
I've created an application that show around 250 images in ImageView. Images are loaded one after another, 15-30 images per second. Basically the whole thing gives an illusion of a rotating 3D object, at least it should.
The problem is next, app hangs when loading certain images(i.e. I see a few seconds of fluid animation and then animation hangs, jump 10-15 frames(images) ahead and continues. It always happens at the same places in animation cycle.
I though that Android might not have enough resources to handle something like this, so I've resized images to half their size, but it did't help. I've tried buffering images but that did't help either(actually, maybe a little, I think that animation looks a little bit smoother).
And now the weirdest thing. I use the touch screen to allow users to "rotate" the 3D object on those images, and while rotating I again experience those hangs at exactly the same places as with the animation.
All images are in .png format and their size vary from 15kB to 40kB.
I use the following code for the animation:
new Thread(new Runnable() {
#Override
public void run() {
while (!stopStartupAnimation && li < images_360.length) {
final int fli = li;
handler.post(new Runnable() {
#Override
public void run() {
//Bitmap b = BitmapFactory.decodeResource(getResources(), R.drawable.icon);
//imageCanvas.setImageResource(images_360[fli]);
imageCanvas.setImageBitmap(imageStackNext.pop());
System.out.println("rawX = " + fli);
}
});
int ti = fli +25;
if(ti > images_360.length-1){
ti = ti - images_360.length;
}
imageStackNext.push(BitmapFactory.decodeResource(getResources(), images_360[ti]));
synchronized (this) {
try {
wait(1000 / 25);
} catch (InterruptedException e) {
e.printStackTrace();
}
}
li++;
li++;
if (li >= images_360.length) {
li = 0;
}
}
}
}).start();
First, 15-40KB is their compressed form. Uncompressed, as Bitmaps, they are probably substantially larger. 250 of them may be using many MB of RAM, which is not a good idea.
Second, given a choice between using OpenGL for 3D (which is its purpose), or the 2D drawing primitives on the Canvas, or using ImageView, you chose the worst-performing option.
Third, postRunnable() does not take effect immediately, but rather puts things on a message queue for the main application thread to process when it gets a chance. If it gets tied up -- say, handling touch events -- it may well skip over some seemingly redundant ImageView redraws, or have them go by so fast they appear to not happen. All your 40ms wait() does is ensure that you are only raising events every 40ms, not that they will paint every 40ms. Besides, you could have more easily just used postDelayed() for your 40ms timing.
Bitmaps should be loaded efficiently.
Refer example on official page: https://developer.android.com/training/displaying-bitmaps/index.html