Develop Reference

Displaying Coverart (from id3) in Recyclerview in background

I am working on an app that uses a Recyclerview to display mp3 files, providing its cover art image along with other info. It works but is slow once it starts dealing with a dozen or more cover arts to retrieve, as I am currently doing this from the id3 on the main thread, which I know is not a good idea.
Ideally, I would work with placeholders so that the images can be added as they become available. I've been looking into moving the retrieval to a background thread and have looked at different options: AsyncTask, Service, WorkManager. AsyncTask seems not to be the way to go as I face memory leaks (I need context to retrieve the cover art through MetadataRetriever). So I am leaning away from that. Yet I am struggling to figure out which approach is best in my case.
From what I understand I need to find an approach that allows multithreading and also a means to cancel the retrieval in case the user has already moved on (scrolling or navigating away). I am already using Glide, which I understand should help with the caching.
I know I could rework the whole approach and provide the cover art as images separately, but that seems a last resort to me, as I would rather not weigh down the app with even more data.
The current version of the app is here (please note it will not run as I cannot openly divulge certain aspects). I am retrieving the cover art as follows (on the main thread):
static public Bitmap getCoverArt(Uri medUri, Context ctxt) {
MediaMetadataRetriever mmr = new MediaMetadataRetriever();
mmr.setDataSource(ctxt, medUri);
byte[] data = mmr.getEmbeddedPicture();
if (data != null) {
return BitmapFactory.decodeByteArray(data, 0, data.length);
} else {
return null;
}
}
I've found many examples with AsyncTask or just keeping the MetaDataRetriever on the main thread, but have yet to find an example that enables a dozen or more cover arts to be retrieved without slowing down the main thread. I would appreciate any help and pointers.

It turns out it does work with AsyncTask, as long as it is not a class onto itself but setup and called from a class with context. Here is a whittled down version of my approach (I am calling this from within my Adapter.):
//set up titles and placeholder image so we needn't wait on the image to load
titleTv.setText(selectedMed.getTitle());
subtitleTv.setText(selectedMed.getSubtitle());
imageIv.setImageResource(R.drawable.ic_launcher_foreground);
imageIv.setAlpha((float) 0.2);
final long[] duration = new long[1];
//a Caching system that helps reduce the amount of loading needed. See: https://github.com/cbonan/BitmapFun?files=1
if (lruCacheManager.getBitmapFromMemCache(selectedMed.getId() + position) != null) {
//is there an earlier cached image to reuse? imageIv.setImageBitmap(lruCacheManager.getBitmapFromMemCache(selectedMed.getId() + position));
imageIv.setAlpha((float) 1.0);
titleTv.setVisibility(View.GONE);
subtitleTv.setVisibility(View.GONE);
} else {
//time to load and show the image. For good measure, the duration is also queried, as this also needs the setDataSource which causes slow down
new AsyncTask<Uri, Void, Bitmap>() {
#Override
protected Bitmap doInBackground(Uri... uris) {
MediaMetadataRetriever mmr = new MediaMetadataRetriever();
mmr.setDataSource(ctxt, medUri);
byte[] data = mmr.getEmbeddedPicture();
Log.v(TAG, "async data: " + Arrays.toString(data));
String durationStr = mmr.extractMetadata(MediaMetadataRetriever.METADATA_KEY_DURATION);
duration[0] = Long.parseLong(durationStr);
if (data != null) {
InputStream is = new ByteArrayInputStream(mmr.getEmbeddedPicture());
return BitmapFactory.decodeStream(is);
} else {
return null;
}
}
#Override
protected void onPostExecute(Bitmap bitmap) {
super.onPostExecute(bitmap);
durationTv.setVisibility(View.VISIBLE);
durationTv.setText(getDisplayTime(duration[0], false));
if (bitmap != null) {
imageIv.setImageBitmap(bitmap);
imageIv.setAlpha((float) 1.0);
titleTv.setVisibility(View.GONE);
subtitleTv.setVisibility(View.GONE);
} else {
titleTv.setVisibility(View.VISIBLE);
subtitleTv.setVisibility(View.VISIBLE);
}
lruCacheManager.addBitmapToMemCache(bitmap, selectedMed.getId() + position);
}
}.execute(medUri);
}
I have tried working with Glide for the caching, but I haven't been able to link the showing/hiding of the TextViews to whether there is a bitmap. In a way though, this is sleeker as I don't need to load the bulk of the Glide-library. So I am happy with this for now.

Using mp4parser , how can I handle videos that are taken from Uri and ContentResolver?

Background
We want to let the user choose a video from any app, and then trim a video to be of max of 5 seconds.
The problem
For getting a Uri to be selected, we got it working fine (solution available here) .
As for the trimming itself, we couldn't find any good library that has permissive license, except for one called "k4l-video-trimmer" . The library "FFmpeg", for example, is considered not permission as it uses GPLv3, which requires the app that uses it to also be open sourced. Besides, as I've read, it takes quite a lot (about 9MB).
Sadly, this library (k4l-video-trimmer) is very old and wasn't updated in years, so I had to fork it (here) in order to handle it nicely. It uses a open sourced library called "mp4parser" to do the trimming.
Problem is, this library seems to be able to handle files only, and not a Uri or InputStream, so even the sample can crash when selecting items that aren't reachable like a normal file, or even have paths that it can't handle. I know that in many cases it is possible to get a path of a file, but in many other cases, it's not, and I also know it's possible to just copy the file (here), but this isn't a good solution, as the file could be large and take a lot of space even though it's already accessible.
What I've tried
There are 2 places that the library uses a file:
In "K4LVideoTrimmer" file, in the "setVideoURI" function, which just gets the file size to be shown. Here the solution is quite easy, based on Google's documentation:
public void setVideoURI(final Uri videoURI) {
mSrc = videoURI;
if (mOriginSizeFile == 0) {
final Cursor cursor = getContext().getContentResolver().query(videoURI, null, null, null, null);
if (cursor != null) {
int sizeIndex = cursor.getColumnIndex(OpenableColumns.SIZE);
cursor.moveToFirst();
mOriginSizeFile = cursor.getLong(sizeIndex);
cursor.close();
mTextSize.setText(Formatter.formatShortFileSize(getContext(), mOriginSizeFile));
}
}
...
In "TrimVideoUtils" file, in "startTrim" which calls "genVideoUsingMp4Parser" function. There, it calls the "mp4parser" library using :
Movie movie = MovieCreator.build(new FileDataSourceViaHeapImpl(src.getAbsolutePath()));
It says that they use FileDataSourceViaHeapImpl (from "mp4parser" library) to avoid OOM on Android, so I decided to stay with it.
Thing is, there are 4 CTORS for it, all expect some variation of a file: File, filePath, FileChannel , FileChannel+fileName .
The questions
Is there a way to overcome this?
Maybe implement FileChannel and simulate a real file, by using ContentResolver and Uri ? I guess it might be possible, even if it means re-opening the InputStream when needed...
In order to see what I got working, you can clone the project here. Just know that it doesn't do any trimming, as the code for it in "K4LVideoTrimmer" file is commented:
//TODO handle trimming using Uri
//TrimVideoUtils.startTrim(file, getDestinationPath(), mStartPosition, mEndPosition, mOnTrimVideoListener);
Is there perhaps a better alternative to this trimming library, which is also permissive (meaning of Apache2/MIT licences , for example) ? One that don't have this issue? Or maybe even something of Android framework itself? I think MediaMuxer class could help (as written here), but I think it might need API 26, while we need to handle API 21 and above...
EDIT:
I thought I've found a solution by using a different solution for trimming itself, and wrote about it here, but sadly it can't handle some input videos, while mp4parser library can handle them.
Please let me know if it's possible to modify mp4parser to handle such input videos even if it's from Uri and not a File (without a workaround of just copying to a video file).

First of all a caveat: I am not familiar with the mp4parser library but your question looked interesting so I took a look.
I think its worth you looking at one of the classes the code comments say is "mainly for testing". InMemRandomAccessSourceImpl. To create a Movie from any URI, the code would be as follows:
try {
InputStream inputStream = getContentResolver().openInputStream(uri);
Log.e("InputStream Size","Size " + inputStream);
int bytesAvailable = inputStream.available();
int bufferSize = Math.min(bytesAvailable, MAX_BUFFER_SIZE);
final byte[] buffer = new byte[bufferSize];
int read = 0;
int total = 0;
while ((read = inputStream.read(buffer)) !=-1 ) {
total += read;
}
if( total < bytesAvailable ){
Log.e(TAG, "Read from input stream failed")
return;
}
//or try inputStream.readAllBytes() if using Java 9
inputStream.close();
ByteBuffer bb = ByteBuffer.wrap(buffer);
Movie m2 = MovieCreator.build(new ByteBufferByteChannel(bb),
new InMemRandomAccessSourceImpl(bb), "inmem");
} catch (FileNotFoundException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
}
But I would say, there looks to be somewhat of a conflict between what you want to achieve and the approach the parser takes. It is depending on local files to avoid large memory overheads, and random access to bytes can only be done if the entire set of data is available, which differs from a streaming approach.
It will require buffering at least the amount of data required for your clip in one go before the parser is given the buffer. That might be workable for you if you are looking to grab short sections and the buffering is not too cumbersome. You may be subject to IO exceptions and the like if the read from the InputStream has issues, especially if it is remote content, whereas you really aren't expecting that with a file on a modern system.
There is also MemoryFile to consider which provides an ashmem backed file-like object. I think somehow that could be worked in.

Next a snipped shows how to open a MediaStore Uri with IsoFile from Mp4Parser. So, you can see how to get a FileChannel from a Uri.
public void test(#NonNull final Context context, #NonNull final Uri uri) throws IOException
{
ParcelFileDescriptor fileDescriptor = null;
try
{
final ContentResolver resolver = context.getContentResolver();
fileDescriptor = resolver.openFileDescriptor(uri, "rw");
if (fileDescriptor == null)
{
throw new IOException("Failed to open Uri.");
}
final FileDescriptor fd = fileDescriptor.getFileDescriptor();
final FileInputStream inputStream = new FileInputStream(fd);
final FileChannel fileChannel = inputStream.getChannel();
final DataSource channel = new FileDataSourceImpl(fileChannel);
final IsoFile isoFile = new IsoFile(channel);
... do what you need ....
}
finally
{
if (fileDescriptor != null)
{
fileDescriptor.close();
}
}
}

How to get the size of the data received via EXTRA_STREAM in an application handling the action send intent?

When another application is sending a file to my app, I get a Uri via the intent.getExtras().get(EXTRA_STREAM) property. I can then get the bytes of the file using an inputstream : new BufferedInputStream(activity.getContentResolver().openInputStream(uri));
Everything's OK and working so far. Now I'd like to show some kind of progress to my user, but I'm not sure of how to get the total number of bytes of the file without reading the stream completely beforehand (which would defeat the whole purpose of the progress bar) ...
I tried ParcelFileDescriptor fileDesc = activity.getContentResolver().openFileDescriptor(uri, "r"); but this only works with uris of type file://....
For example If I receive a file from Skydrive I get a content://.. Uri, as in : content://com.microsoft.skydrive.content.external/external_property/10C32CC94ECB90C4!155/Sunset.480p.mp4
On such Uri I get (unsurprisingly) a "FileNotFoundException : Not a whole file" exception.
Any sure fire way to get the total size of the stream of data I will get ?

Even though InputStream.available() is (almost) never a recommended way of getting file size, it might be a viable solution in your case.
The content is already available locally. A server is not involved. So, the following should return the exact file size:
try {
InputStream inputStream = getContentResolver().openInputStream(uri);
Log.i("TEST", "File Size: " + inputStream.available());
} catch (FileNotFoundException fnfe) {
fnfe.printStackTrace();
} catch (IOException ioe) {
ioe.printStackTrace();
}
I tested this with SkyDrive and Dropbox. The file sizes returned were correct.

There is no general solution for getting the size of a stream, other than reading the entire stream. This is easily proven: One could create a web server that, for some URL, generates a random stream of text that is terminated at a random time. (In fact, I'm sure such URLs exist, whether by design or not :-) In such a case, the size of the stream isn't known until the last byte has been generated, never mind received.
So, the stream size, if it is sent by the server at all, has to be sent in an application-specific manner.
I've never worked with SkyDrive, but a google search for its API turned up this link, which has the following example for Android Java apps:
public void readFile() {
String fileId = "file.a6b2a7e8f2515e5e.A6B2A7E8F2515E5E!141";
client.getAsync(fileId, new LiveOperationListener() {
public void onError(LiveOperationException exception, LiveOperation operation) {
resultTextView.setText("Error reading file: " + exception.getMessage());
}
public void onComplete(LiveOperation operation) {
JSONObject result = operation.getResult();
String text = "File info:" +
"\nID = " + result.optString("id") +
"\nName = " + result.optString("name");
resultTextView.setText(text);
}
});
}
Based on other examples on that page, I would guess that something like result.optString("size") (or maybe result.optInt("size") ?) would give you the size of the file.

Custom image viewing slows down

The problem
Hi there,
I'm developing an application where the user specifies some pictures and how long they are going to be on the screen.So sometimes he wants to create something like a small animation or viewing the images for a small amount of time.The problem is that after some time the images are not previewed when they should and we have a few ms of error.In the application that i'm developing time matters so I would like some help on what the problem might be.
The code
So let me explain how it works.I take the pictures from my web app and then I save them in a HashMap
Bitmap image = ImageOperations(url,String.valueOf(frameNum) + ".jpg");
ImageMap.put(String.valueOf(frameNum), image);
where the mathod ImageOperations is like that:
private Bitmap ImageOperations(String url, String saveFilename) {
try {
Display display = getWindowManager().getDefaultDisplay();
InputStream is = (InputStream) this.fetch(url);
Bitmap theImage = BitmapFactory.decodeStream(is);
if (theImage.getHeight() >= 700 || theImage.getWidth() >= 700) {
theImage = Bitmap.createScaledBitmap(theImage,
display.getWidth(), display.getHeight() - 140, true);
}
return theImage;
} catch (MalformedURLException e) {
e.printStackTrace();
return null;
} catch (IOException e) {
e.printStackTrace();
return null;
}
}
So later I run a thread that updates the UI when the user specified.The method that updates it is this one.
public void setPictures(int NumOfFrame) {
if (frameArray.get(NumOfFrame - 1).frame_pic.contains("n/a") != true) {
ImagePlace.setImageBitmap(ImageMap.get(String.valueOf(NumOfFrame)));
} else {
ImagePlace.setImageDrawable(null);
}
}
After we update the image we put the thread for sleep and when runs again it updates the thread.Is there something that creates the problem?Does it have to do with Garbage collection?
Thank you in advance

Probably the issue is in increasing heap size when it loads additional images. I would suggest You to do some profiling so things will be much clearer and You'll get full picture of timings for the app.

First you are missing a null check at here:
ImageMap.get(String.valueOf(NumOfFrame))
And you do not recycle the old bitmap at here:
theImage.recycle(); // missing line
theImage = Bitmap.createScaledBitmap(theImage,
display.getWidth(), display.getHeight() - 140, true);
It may lead to outofmemory exceptions, with is most likely from your description of the problem.
Also I am not sure if BitmapFactory.decodeStream throw exception when he fails. You need to add a null point check there too.

Obfuscate JPG by bit-toggle - reading performance on Android

Abstract:
reading images from file
with toggled bits to make unusable for preview tools
cant use encryption, to much power needed
can I either optimize the code below, or is there a better approach
Longer description:
I am trying to improve my code, maybe you got some ideas or improvements for the following situation. Please be aware that I neither try to beat the CIA, nor care much if somebody "brakes" the encryption.
The background is simple: My app loads a bunch of images from a server into a folder on the SD card. I do NOT want the images to be simple JPG files, because in this case the media indexer would list them in the library, and a user could simply copy the whole folder to his harddrive.
The obvious way to go is encryption. But a full blown AES or other encryption does not make sense, for two reasons: I would have to store the passkey in the app, so anyone could get the key with some effort anyway. And the price for decrypting images on the fly is way too high (we are talking about e.g. a gallery with 30 200kB pictures).
So I decided to toggle some bits in the image. This makes the format unreadable for image tools (or previews), but is pretty easy undone when reading the images. For "encrypting" I use some C# tool, the "decrypt" lines are the following ones:
public class CustomInputStream extends InputStream {
private String _fileName;
private BufferedInputStream _stream;
public CustomInputStream(String fileName) {
_fileName = fileName;
}
public void Open() throws IOException {
int len = (int) new File(_fileName).length();
_stream = new BufferedInputStream(new FileInputStream(_fileName), len);
}
#Override
public int read() throws IOException {
int value = _stream.read() ^ (1 << 7);
return value;
}
#Override
public void close() throws IOException {
_stream.close();
}
}
I tried overwriting the other methods (read with more then one byte) too, but this kills the BitmapFactory - not sure why, maybe I did something wrong. Here is the code for the image bitmap creation:
Bitmap bitmap = null;
try {
InputStream i = CryptoProvider.GetInstance().GetDecoderStream(path);
bitmap = BitmapFactory.decodeStream(i);
i.close();
} catch (Exception e1) {
_logger.Error("Cant load image " + path + " ERROR " + e1);
}
if (bitmap == null) {
_logger.Error("Image is NULL for path " + path);
}
return bitmap;
Do you have any feedback on the chosen approach? Any way to optimize it, or a completely different approach for Android devices?

You could try XORing the bytestream with the output of a fast PRNG. Just use a different seed for each file and you're done.
note: As already noted in the question, such methods are trivial to bypass.