My app should show a bunch of animated images (within a list or grid, ~10-20 images on a screen depending on the screen size). Each of these images can contain a lot of frames (up to 150). Actually these images are gifs initially. I get them as encoded mp4 files (to reduce the data size transferred through the network) and decode them on a device. For decoding I use android.media.MediaMetadataRetriever (it doesn't work well on Samsung devices) and I do it the following way: when I show the current frame I put a new decoding task - actually a task to get the next frame - to a priority queue. Priority depends on the time to which I need to get the next frame. If task "expires" (isn't processed by workers before expected time) I just put to a queue next task and etc. But my algorithm doesn't work well - animations are very slow and buggy (I don't also exclude that my implementation can have some bugs)..
So, here are the questions:
1) Are there any other possibly better ways to decode mp4?
2) Could someone please give me and advice which algorithm I should use to effectively decode mp4 files, so there weren't any animation lags?
3) I can't use simple AnimationDrawable because of out of memory errors, so how can I effectively manage frames cache?
Thanks!
Related
I trying to create an Android App that using Camera2 API, as part of the functionality I want to develop a module that saving multiple images produced by ImageReader as followed:
Image image = reader.acquireLatestImage();
I'm getting the followed Exception:
IllegalStateException too many images are currently acquired
as mentioned in the documention:
https://developer.android.com/reference/android/media/ImageReader#acquireLatestImage()
This is because the image returned from 'acquireLatestImage' is still belongs to the ImageReader Queue.
Is there any way to detach images returning from 'ImageReader' ?
Is there a way to copy an image, preferably without to store it on disk, that is a resource consuming operation ?
Thank's
If this is a YUV_420_888 image, you can copy each of the plane ByteBuffers to keep the data around indefinitely. That is somewhat expensive, of course.
Unfortunately, there's no way to easily detach the Image from the ImageReader. The Reader is a circular buffer queue internally, so removing an Image would require the Reader to allocate a new image to replace the one removed, which is somewhat expensive as well.
It can be done by using an ImageWriter.queueInputImage, connected to another ImageReader with the same configuration as the original ImageReader. When you receive image A from your first ImageReader, and you want to keep it, you can queue it into the ImageWriter, and then get the Image handle again from the second ImageReader. Still need to set maxImages on the second reader high enough to account for all the Images you want to keep around at once, of course.
That's fairly cumbersome, of course, and if you're doing this continually you'll cause a lot of memory reallocation and copies may be just as expensive (and simpler in many ways).
I am working with an android application, where i have number of urls of gif image. I need to show these gifs in a GridView. I have used Glide, Fresco etc. third party libs to load gifs, but still it talking lot of time to load gifs into the gridview. My requirement is to show thumbnail/preview of image first, while talking time to download and then start playing animation after the download complete. Is there any efficient solution to this problem?
If you own the backend from where the GIF are downloaded. I suggest you to create low res images of this GIFs (tooking the first frame of the GIF for example).
When the user enter the GridView, you fast download the low resolution thumbnails, then in parallel you start downloading the real GIF and substitute them one to one when the single download is completed.
If you don't own the backend and the content owner doesn't provide a thumbnail download system, you can't do anything :(
I wrote an application that normally shows 4 or 5 simple GIFs (size limited to 500KB) I use Glide in a separated thread. The thread first extract bitmaps and later it create an array of resized to view dimensions Drawables, it also creates an array of delays between frames and at the end the thread calls postInvalidate(), nothing are drawn until thread terminates but later due I have the frames saved all is redrawn very fast. I use a Runnable with postDelayed to change frames.
I'm trying to copy a part of a video, and save it as a GIF into the disk. The video can be local or remote, and the copy should be 2s max. I don't need to save every single frame, but every other frame (12-15 fps). I have the "frames to gif" part working, but the "get the frames" part is not great.
Here is what I tried so far:
- MediaMetadataRetriever: too slow (~1s per frame on a Nexus4), and only works with local files
- FFmpegMediaMetadataRetriever: same latency, but works with remote video
- TextureView.getBitmap(): I'm using a ScheduledExecutorService and every 60ms, grab the Bitmap (while playing...) It works well with small size getBitmap(100, 100), but for bigger ones (> 400), the whole process becomes really slow. And as the doc says Do not invoke this method from a drawing method anyway.
It seems that the best solution would be to access every frame while decoding, and save them. I tried OpenCV for Android but couldn't find an API to grab a frame at a specific time.
Now, I'm looking into those samples to understand how to use MediaCodec, but while running ExtractMpegFramesTest.java, I can't seem to extract any frame ("no output from decoder available").
Am I on the right track? Any other suggestion?
edit: went further with ExtractMpegFramesTest.java, thanks for this post.
edit 2: just to clarify, what I'm trying to achieve here is to play a video, and press a button to start capturing the frames.
My Android app does live video processing using OpenGL. I'm trying to save it to video using MediaMuxer and MediaCodex.
The performance it not good enough. Each cycle the screen is updated, and it is saved to file. The screen is smooth, the video file is horrible. By this I mean major motion blur when it changes quickly and the frame-rate appears to be 1/2 or 1/3rd of what it should be.
It seems to be a limitation due to clamping of settings internally. I can't get it to spit out a video with a bit rate greater than 288KBPS. I think it is not clamping the requested parameters because there is no difference in frame rate for 1024x1024, 480x480, and 240x240. If it was having trouble keeping up, it should at least improve when the number of pixels drops by a factor > 10.
The app is here : https://play.google.com/store/apps/details?id=com.matthewjmouellette.snapdat.
I would love to post a code sample, but my program is 10K lines of code, with a lot of relevant code just for this problem.
Any help would be greatly appreciated.
EDIT:
I've tried like 10+ different things. I'm out of ideas right now. I wish I could just save the video uncompressed, the hard-drive should be able to keep up with a small enough image and medium fps.
It seems to be that the encoding method just doesn't work for my video. The frames differ to much, to try to "move" one part of the frame, as a sort of encoding. Instead I need full frames throughout. I am thinking something along the lines of M-JPEG would work really well. JPEGs tend to take 1/10th the size of a bitmap. It should allow a reasonable size, with almost no processing power required by the CPU, since it is image compression not video compression which we are doing. I wish I had a good library for this.
I have been posting quite some questions recently without getting a response. Hope this one gets.
Iv decided to go with mp4 as its compression is much better than gif sequences. Trying movie.decodestream(InputStream) helps to get the stream. But when movie.duration() is aquired, a null pointer exception is thrown. Going around the web, I found that duration works if the movies are frame by frame(with durations for each).
So, is the mp4 sequence a bad way to read the stream and get the duration?
Is there any way to convert the mp4 into a "frame by frame" sequence?
What is required is an easy way to play some 30 frame animations on screen.
The problems of each solution Iv found are: Jpeg sequences are pretty much memory consuming, Gif sequences when used have jagged edges and bigger file sizes and mp4 sequences cannot be acquired for frame by frame playback.
- Since the animations have a dimension of 800*600, I guess Jpeg sequences are out of the question, or please do tell if there are any good options for this.
- Presently Im using gif sequences,with half the dimension. The edges are jagged and are displayed very bad in different devices.
-Mp4 Sequences are just too good. But I need the control over the frames so that I can play it forward and reverse, and display the first and last frames at the resting stage of the animation(the animations are triggered on touch).