Develop Reference

Android 11 + Kotlin: Reading a .zip File

I've got an Android app written in Kotlin targeting framework 30+, so I'm working within the new Android 11 file access restrictions. The app needs to be able to open an arbitrary .zip file in the shared storage (chosen interactively by the user) then do stuff with the contents of that .zip file.
I'm getting a URI for the .zip file in what I'm led to understand is the canonical way:
val activity = this
val getContent = registerForActivityResult(ActivityResultContracts.GetContent()) {
CoroutineScope(Dispatchers.Main).launch {
if(it != null) doStuffWithZip(activity, it)
...
}
}
getContent.launch("application/zip")
My problem is that the Java.util.zip.ZipFile class I'm using only knows how to open a .zip file specified by a String or a File, and I don't have any easy way to get to either of those from a Uri. (I'm guessing that the ZipFile object needs the actual file rather than some kind of stream because it needs to be able to seek...)
The workaround I'm using at present is to turn the Uri into an InputStream, copy the contents to a temp file in private storage, and make a ZipFile instance from that:
private suspend fun <T> withZipFromUri(
context: Context,
uri: Uri, block: suspend (ZipFile) -> T
) : T {
val file = File(context.filesDir, "tempzip.zip")
try {
return withContext(Dispatchers.IO) {
kotlin.runCatching {
context.contentResolver.openInputStream(uri).use { input ->
if (input == null) throw FileNotFoundException("openInputStream failed")
file.outputStream().use { input.copyTo(it) }
}
ZipFile(file, ZipFile.OPEN_READ).use { block.invoke(it) }
}.getOrThrow()
}
} finally {
file.delete()
}
}
Then, I can use it like this:
suspend fun doStuffWithZip(context: Context, uri: Uri) {
withZipFromUri(context, uri) { // it: ZipFile
for (entry in it.entries()) {
dbg("entry: ${entry.name}") // or whatever
}
}
}
This works, and (in my particular case, where the .zip file in question is never more than a couple MB) is reasonably performant.
But, I tend to regard programming by temporary file as the last refuge of the terminally incompetent, thus I can't escape the feeling that I'm missing a trick here. (Admittedly, I am terminally incompetent in the context of Android + Kotlin, but I'd like to learn to not be...)
Any better ideas? Is there a cleaner way to implement this that doesn't involve making an extra copy of the file?

Copying from external source (and risking downvoting to oblivion) and this isn't quite an answer, but too long for a comment
public class ZipFileUnZipExample {
public static void main(String[] args) {
Path source = Paths.get("/home/mkyong/zip/test.zip");
Path target = Paths.get("/home/mkyong/zip/");
try {
unzipFolder(source, target);
System.out.println("Done");
} catch (IOException e) {
e.printStackTrace();
}
}
public static void unzipFolder(Path source, Path target) throws IOException {
// Put the InputStream obtained from Uri here instead of the FileInputStream perhaps?
try (ZipInputStream zis = new ZipInputStream(new FileInputStream(source.toFile()))) {
// list files in zip
ZipEntry zipEntry = zis.getNextEntry();
while (zipEntry != null) {
boolean isDirectory = false;
// example 1.1
// some zip stored files and folders separately
// e.g data/
// data/folder/
// data/folder/file.txt
if (zipEntry.getName().endsWith(File.separator)) {
isDirectory = true;
}
Path newPath = zipSlipProtect(zipEntry, target);
if (isDirectory) {
Files.createDirectories(newPath);
} else {
// example 1.2
// some zip stored file path only, need create parent directories
// e.g data/folder/file.txt
if (newPath.getParent() != null) {
if (Files.notExists(newPath.getParent())) {
Files.createDirectories(newPath.getParent());
}
}
// copy files, nio
Files.copy(zis, newPath, StandardCopyOption.REPLACE_EXISTING);
// copy files, classic
/*try (FileOutputStream fos = new FileOutputStream(newPath.toFile())) {
byte[] buffer = new byte[1024];
int len;
while ((len = zis.read(buffer)) > 0) {
fos.write(buffer, 0, len);
}
}*/
}
zipEntry = zis.getNextEntry();
}
zis.closeEntry();
}
}
// protect zip slip attack
public static Path zipSlipProtect(ZipEntry zipEntry, Path targetDir)
throws IOException {
// test zip slip vulnerability
// Path targetDirResolved = targetDir.resolve("../../" + zipEntry.getName());
Path targetDirResolved = targetDir.resolve(zipEntry.getName());
// make sure normalized file still has targetDir as its prefix
// else throws exception
Path normalizePath = targetDirResolved.normalize();
if (!normalizePath.startsWith(targetDir)) {
throw new IOException("Bad zip entry: " + zipEntry.getName());
}
return normalizePath;
}
}
This apparently works with pre-existing files; however since you already have an InputStream read from the Uri - you can adapt this and give it a shot.
EDIT:
It seems like it's extracting to Files as well - you could store the individual ByteArrays somewhere then decide what to do with them later on. But I hope you get the general idea - you can do all of this in-memory, without having to use the disk (temp files or files) in between.
Your requirement is a bit vague and unclear however, so I don't know what you're trying to do, merely suggesting a venue/approach to try out

What about a simple ZipInputStream ? –
Shark
Good idea #Shark.
InputSteam is = getContentResolver().openInputStream(uri);
ZipInputStream zis = new ZipInputStream(is);

#Shark has it with ZipInputStream. I'm not sure how I missed that to begin with, but I sure did.
My withZipFromUri() method is much simpler and nicer now:
suspend fun <T> withZipFromUri(
context: Context,
uri: Uri, block: suspend (ZipInputStream) -> T
) : T =
withContext(Dispatchers.IO) {
kotlin.runCatching {
context.contentResolver.openInputStream(uri).use { input ->
if (input == null) throw FileNotFoundException("openInputStream failed")
ZipInputStream(input).use {
block.invoke(it)
}
}
}.getOrThrow()
}
This isn't call-compatible with the old one (since the block function now takes a ZipInputStream as a parameter rather than a ZipFile). In my particular case -- and really, in any case where the consumer doesn't mind dealing with entries in the order they appear -- that's OK.

Okio (3-Alpha) has a ZipFileSystem https://github.com/square/okio/blob/master/okio/src/jvmMain/kotlin/okio/ZipFileSystem.kt
You could probably combine it with a custom FileSystem that reads the content of that file. It will require a fair bit of code but will be efficient.
This is an example of a custom filesystem https://github.com/square/okio/blob/88fa50645946bc42725d2f33e143628e7892be1b/okio/src/jvmMain/kotlin/okio/internal/ResourceFileSystem.kt
But I suspect it's simpler to convert the URI to a file and avoid any copying or additional code.

It's easy to check the .zip and .rar files in the Android-Kotlin FileAdapter(work with file manager), add the bellow function to your code:
private fun isZip(name: String): Boolean {
return name.contains(".zip") || name.contains(".rar")
}

How to play cached byte array in android

I have a Xamarin application which plays remote(internet) audio files using the MediaPlayer with the following setup:
_mediaPlayer.SetDataSource(mediaUri);
_mediaPlayer.PrepareAsync();
Now I would like to change the implementation to also cache files. For the caching part I found a really nice library called MonkeyCache which saves the files in this JSON format:
{"$type":"System.Byte[], mscorlib","$value":"UklGRhAoAgBXQVZFZm10IBAAAAABAAIARKwAABCxAgAEABAAZGF0YdAnAgAAAAAAAAAAAAAAAAAA+P/4/+z/7P8KAAoA7//v//L/8v8JAAkA6f/p//j/+P8FAAUA6P/o/wEAAQD+//7/5//n ................."}
So my MediaPlayer setup has now changed to:
if (Barrel.Current.Exists(mediaUri)){
var audio = Barrel.Current.Get<byte[]>(mediaUri);
_mediaPlayer.SetDataSource(???);
}else{
using (var webClient = new WebClient()){
var downloadDataBytes = webClient.DownloadData(mediaUri);
if (downloadDataBytes != null && downloadDataBytes.Length > 0)
{
Barrel.Current.Add(mediaUri, downloadDataBytes, TimeSpan.FromDays(1));
_mediaPlayer.SetDataSource(???);
}
}
}
I would like to play the audio from a byte[] instead of the mediaUri.
Is there any way to actually play an in memory byte[]?
The only solutions that I could find were to create a FileInputStream out of a File by using a filepath, but the implementation of the MonkeyCache actually hashes the file name, before adding it:
static string Hash(string input){
var md5Hasher = MD5.Create();
var data = md5Hasher.ComputeHash(Encoding.Default.GetBytes(input));
return BitConverter.ToString(data);
}
Therefore the downloaded bytes, will be saved under:
/data/data/com.package.name.example/cache/com.package.name.example/MonkeyCacheFS/81-D6-E8-62-F3-4D-F1-64-A6-A1-53-46-34-1E-FE-D1
Even if I were to use the same hashing logic to actually compute the file path myself and use the FileInputStream which might be working by what I've read, it would defeat the purpose of using the var audio = Barrel.Current.Get<byte[]>(mediaUri); functionality of the MonkeyCache.
However, if that is the only way, I will do it.
Edit: Even with my described approach, it would probably not work right away as even if I compute the right file name, it is still in JSON format.
Edit2: A working solution is:
var audio = Barrel.Current.Get<byte[]>(mediaUri);
var url = "data:audio/mp3;base64," + Convert.ToBase64String(audio);
_mediaPlayer.SetDataSource(url);

Finally figured it out:
The MediaPlayer.SetDataSource method has an overload which expects a MediaDataSource object.
MediaDataSource
For supplying media data to the framework. Implement
this if your app has special requirements for the way media data is
obtained
Therefore, you can use your implementation of the MediaDataSource and feed it your byte array.
public class StreamMediaDataSource : MediaDataSource
{
private byte[] _data;
public StreamMediaDataSource(byte[] data)
{
_data = data;
}
public override long Size
=> _data.Length;
public override void Close()
{
_data = null;
}
public override int ReadAt(long position, byte[] buffer, int offset, int size)
{
if (position >= _data.Length)
{
return -1;
}
if (position + size > _data.Length)
{
size -= (Convert.ToInt32(position) + size) - _data.Length;
}
Array.Copy(_data, position, buffer, offset, size);
return size;
}
}
which will lead to the MediaPlayer data source to be
_mediaPlayer.SetDataSource(new StreamMediaDataSource(downloadedDataBytes));
Edit: This only works for API >= 23. I still have to find a solution for API<23(which I still haven't).
Edit2: For API < 23 support I used the
var url = $"data:audio;base64,{Convert.ToBase64String(downloadedDataBytes)}";
_mediaPlayer.SetDataSource(url);
approach.

How to cache a video in background in android?

I am building an android application where a user can view some listed video. Those videos are categories into some channel. Once a channel is selected by user I want to cache all the video related to that channel in my cache memory so can play the video when there is no internet also.
Can anyone have more understanding about video cache without playing please help me in understanding how I can achieve this task.
Right now I am able to cache video If it's played using some library.

I have find the following working solution for caching video in background (single/multiple) using below lib, no need of player/video_view.use AsyncTaskRunner
Videocaching Lib
Add following in line in your gradle file
compile 'com.danikula:videocache:2.7.0'
Since we just need to kick start the prefetching, no need to do anything in while loop.
Or we can use ByteArrayOutputStream to write down the data to disk.
URL url = null;
try {
url = new URL(cachingUrl(cachingUrl));
InputStream inputStream = url.openStream();
int bufferSize = 1024;
byte[] buffer = new byte[bufferSize];
int length = 0;
while ((length = inputStream.read(buffer)) != -1) {
//nothing to do
}
} catch (IOException e) {
e.printStackTrace();
}
Important code from lib. to do
Create static instance in application class using following code
private HttpProxyCacheServer proxy;
public static HttpProxyCacheServer getProxy(Context context) {
Applications app = (Applications) context.getApplicationContext();
return app.proxy == null ? (app.proxy = app.newProxy()) : app.proxy;
}
private HttpProxyCacheServer newProxy() {
//return new HttpProxyCacheServer(this);
return new HttpProxyCacheServer.Builder(this)
.cacheDirectory(CacheUtils.getVideoCacheDir(this))
.maxCacheFilesCount(40)
.maxCacheSize(1024 * 1024 * 1024)
.build();
}
Write following code in your activity to pass url
public String cachingUrl(String urlPath) {
return Applications.getProxy(this).getProxyUrl(urlPath, true);
}

Load image from binary base64

EDIT: This is a bug in Android version <4.3 Kitkat. It relates to the libjpeg library in Android, which can't handle JPEGs with missing EOF/EOI bits, or apparently with metadata/EXIF data that it doesn't like.
https://code.google.com/p/android/issues/detail?id=9064
ORIGINAL QUESTION:
I have an issue when loading an image in my app.
My endpoint sends JSON which contains a BASE64 encoded image. Depending on the REST call, these images can be PNG or JPG. Some of the JPG files suffer from an issue where they are missing an EOF bit at the end. The PNG files work, and some JPG files work, but unfortunately a lot of these JPG files with the issue are present in the Oracle DB (stored as BLOB). I don't have control of the DB.
I have been looking through Google bugs here:
https://code.google.com/p/android/issues/detail?id=9064
and here:
https://code.google.com/p/android/issues/detail?id=57502
The issue is also seen where the encoding is CYMK using a custom ICC profile.
Decoding the image the standard way returns false:
byte[] imageAsBytes = Base64.decode(base64ImageString, Base64.DEFAULT);
return BitmapFactory.decodeByteArray(imageAsBytes, 0, imageAsBytes.length);
According to the bug reports above, the built in JPG parser in Android is to blame.
I'm trying to figure out a workaround for my device, which is stuck on 4.2.2. I have no other option on this OS version.
I thought it might be a good idea to try and use an image loader library like Universal Image Loader, but it requires I either have the image stored locally, or stored on a URL. As I get the data in BASE64 from the REST server, I can't use this. An option is to support decodeByteArray in a custom class that extends BaseImageDecoder, as stated by the dev at the bottom here: https://github.com/nostra13/Android-Universal-Image-Loader/issues/209
Here's where I get stuck. I already have a custom image decoder to try handle the issue of the missing EOF marker in the JPG file, but I don't know how to edit it to add support for decodeByteArray.
Here is my CustomImageDecoder:
public class CustomImageDecoder extends BaseImageDecoder {
public CustomImageDecoder(boolean loggingEnabled) {
super(loggingEnabled);
}
#Override
protected InputStream getImageStream(ImageDecodingInfo decodingInfo) throws IOException {
InputStream stream = decodingInfo.getDownloader()
.getStream(decodingInfo.getImageUri(), decodingInfo.getExtraForDownloader());
return stream == null ? null : new JpegClosedInputStream(stream);
}
private class JpegClosedInputStream extends InputStream {
private static final int JPEG_EOI_1 = 0xFF;
private static final int JPEG_EOI_2 = 0xD9;
private final InputStream inputStream;
private int bytesPastEnd;
private JpegClosedInputStream(final InputStream iInputStream) {
inputStream = iInputStream;
bytesPastEnd = 0;
}
#Override
public int read() throws IOException {
int buffer = inputStream.read();
if (buffer == -1) {
if (bytesPastEnd > 0) {
buffer = JPEG_EOI_2;
} else {
++bytesPastEnd;
buffer = JPEG_EOI_1;
}
}
return buffer;
}
}
}
By the way, using the above custom class, I am trying to load my byte array like this:
byte[] bytes = Base64.decode(formattedB64String, Base64.NO_WRAP);
ByteArrayInputStream is = new ByteArrayInputStream(bytes);
String imageId = "stream://" + is.hashCode();
...
ImageLoader imageLoader = ImageLoader.getInstance();
imageLoader.displayImage(imageId, userImage, options);
and I get this error:
ImageLoader: Image can't be decoded [stream://1097215584_656x383]
Universal Image loader does not allow the stream:// schema, so I created a custom BaseImageDownloader class that allows it:
public class StreamImageDownloader extends BaseImageDownloader {
private static final String SCHEME_STREAM = "stream";
private static final String STREAM_URI_PREFIX = SCHEME_STREAM + "://";
public StreamImageDownloader(Context context) {
super(context);
}
#Override
protected InputStream getStreamFromOtherSource(String imageUri, Object extra) throws IOException {
if (imageUri.startsWith(STREAM_URI_PREFIX)) {
return (InputStream) extra;
} else {
return super.getStreamFromOtherSource(imageUri, extra);
}
}
}
So if anyone can help me create a better CustomImageDecoder that handles a BASE64 encoded string, or a byte[] containing an image so I can use decodeByteArray, I would be grateful!
Thank you.

UnversalImageLoader uses the following schemes to decode the files
"h t t p ://site.com/image.png" // from Web
"file:///mnt/sdcard/image.png" // from SD card
"file:///mnt/sdcard/video.mp4" // from SD card (video thumbnail)
"content://media/external/images/media/13" // from content provider
"content://media/external/video/media/13" // from content provider (video thumbnail)
"assets://image.png" // from assets
"drawable://" + R.drawable.img // from drawables (non-9patch images)
your scheme is stream://
Hope that helps.

Just to close this off:
The issue here is actually a bug in Android <4.3 where Android can't display images that either aren't closed properly (missing end bytes) or contain certain metadata that, for some reason, it doesn't like. I'm not sure what metadata this is, however. My issue was with JPEGs not being terminated properly.
The bug is fixed in Android 4.3 anyway.

Using muPDF with curl/flip effect

I'm using muPDF for reading PDFs in my application. I don't like its default animation (Switching horizontally). In other side i found this brilliant library for curl effect on images, and this project for flip-flap effect on layouts.
In curl sample project, in CurlActivity, all of data are images and set in PageProvider like this:
private class PageProvider implements CurlView.PageProvider {
// Bitmap resources.
private int[] mBitmapIds = { R.drawable.image1, R.drawable.image2,
R.drawable.image3, R.drawable.image4};
And use it like this:
private CurlView mCurlView;
mCurlView = (CurlView) findViewById(R.id.curl);
mCurlView.setPageProvider(new PageProvider());
And CurlView extends from GLSurfaceView and implements View.OnTouchListener, CurlRenderer.Observer
But in muPDF if i'm not mistaken, data are in core object. core is instance of MuPDFCore. And using it like this:
MuPDFReaderView mDocView;
MuPDFView pageView = (MuPDFView) mDocView.getDisplayedView();
mDocView.setAdapter(new MuPDFPageAdapter(this, this, core));
MuPDFReaderView extends ReaderView and ReaderView extends AdapterView<Adapter> and implements GestureDetector.OnGestureListener, ScaleGestureDetector.OnScaleGestureListener, Runnable.
My question is where how can I using curl effect in muPDF? Where should I get pages one by one and converting them to bitmaps? and then changing aspects of the Adapter in muPDF to CurlView.
In flip-flap sample project, in FlipHorizontalLayoutActivity (I like this effect too), we have these:
private FlipViewController flipView;
flipView = new FlipViewController(this, FlipViewController.HORIZONTAL);
flipView.setAdapter(new TravelAdapter(this));
setContentView(flipView);
And FlipViewController extends AdapterView<Adapter>, and data set in TravelAdapter that extends BaseAdapter.
No one has done this before? Or can help me to do that?!
EDIT:
I found another good open source PDF reader with curl effect called fbreaderJ. its developer says "An additional module that allows to open PDF files in FBReader. Based on radaee pdf library."
I got confused! cause radaeepdf is closed source and downloadable project is just for demo and inserted username and password is for this package.
People want to change whole fbreader project such as package name.
Another issue for make me confused is where is this additional module source code?!
Anyway, if someone wants to help me, fbreader has done it very well.
EDIT:
I talked to Robin Watts, who developed muPDF (or one of developers), and he said:
Have you read platform/android/ClassStructure.txt ? MuPDF is
primarily a C library. The standard api is therefore a C one. Rather
than exposing that api exactly as is to Java (which would be the
nicest solution, and something that I've done some work on, but have
not completed due to lack of time), we've implemented MuPDFCore to
wrap up just the bits we needed. MuPDFCore handles opening a PDF file,
and getting bitmaps from it to be used in views. or rather, MuPDFCore
returns 'views', not 'bitmaps'. If you need bitmaps, then you're going
to need to make changes in MuPDFCore.
There are too many errors when changing a little part of MuPDFReaderView class. I get confused! These are related to each other.
Please answer more precisely.
EDIT:
And bounty has expired.

If the muPDF does not support rendering to a bitmap, you have no other choice than rendering to a regular view and take a screen dump to a bitmap like this:
View content = findViewById(R.id.yourPdfView);
Bitmap bitmap = content.getDrawingCache();
Then use this bitmap as input to your other library.

Where should i get pages one by one and converting them to bitmaps?
In our application (newspaper app) we use MuPDF to render PDFs.
The workflow goes like this:
Download PDF file (we have one PDF per newspaper page)
Render it with MuPDF
Save the bitmap to the filesystem
Load the Bitmap from filesystem as background image to a view
So, finally, what we use is MuPDFCore.java and its methods drawPage(...) and onDestroy()
Is this what you want to know or do i miss the point?
EDIT
1.) I think it is not necessary to post code how to download a file. But after downloading i add a RenderTask (extends from Runnable) to a Renderqueue and trigger that queue. The RenderTask needs some information for rendering:
/**
* constructs a new RenderTask instance
* #param context: you need Context for MuPdfCore instance
* #param pageNumber
* #param pathToPdf
* #param renderCallback: callback to set bitmap to the view after
* rendering
* #param heightOfRenderedBitmap: this is the target height
* #param widthOfRenderedBitmap: this is the target width
*/
public RenderTask (Context context, Integer pageNumber, String pathToPdf, IRenderCallback,
renderCallback, int heightOfRenderedBitmap,
int widthOfRenderedBitmap) {
//store things in fields
}
2.) + 3.) The Renderqueue wraps the RenderTask in a new Thread and starts it. So the run-method of the RenderTask will be invoked:
#Override
public void run () {
//do not render it if file exists
if (exists () == true) {
finish();
return;
}
Bitmap bitmap = render();
//if something went wrong, we can't store the bitmap
if (bitmap == null) {
finish();
return;
}
//now save the bitmap
// in my case i save the destination path in a String field
imagePath = save(bitmap, new File("path/to/your/destination/folder/" + pageNumber + ".jpg"));
bitmap.recycle();
finish();
}
/**
* let's trigger the callback
*/
private void finish () {
if (renderCallback != null) {
// i send the whole Rendertask to callback
// maybe in your case it is enough to send the pageNumber or path to
// renderend bitmap
renderCallback.finished(this);
}
}
/**
* renders a bitmap
* #return
*/
private Bitmap render() {
MuPDFCore core = null;
try {
core = new MuPDFCore(context, pathToPdf);
} catch (Exception e) {
return null;
}
Bitmap bm = Bitmap.createBitmap(widthOfRenderedBitmap, heightOfRenderedBitmap, Config.ARGB_8888);
// here you render the WHOLE pdf cause patch-x/-y == 0
core.drawPage(bm, 0, widthOfRenderedBitmap, heightOfRenderedBitmap, 0, 0, widthOfRenderedBitmap, heightOfRenderedBitmap, core.new Cookie());
core.onDestroy();
core = null;
return bm;
}
/**
* saves bitmap to filesystem
* #param bitmap
* #param image
* #return
*/
private String save(Bitmap bitmap, File image) {
FileOutputStream out = null;
try {
out = new FileOutputStream(image.getAbsolutePath());
bitmap.compress(Bitmap.CompressFormat.JPEG, 80, out);
return image.getAbsolutePath();
} catch (Exception e) {
return null;
}
finally {
try {
if (out != null) {
out.close();
}
} catch(Throwable ignore) {}
}
}
}
4.) I think it is not necessary to post code how to set a bitmap as background of a view