Develop Reference

Using mp4parser , how can I handle videos that are taken from Uri and ContentResolver?

Background
We want to let the user choose a video from any app, and then trim a video to be of max of 5 seconds.
The problem
For getting a Uri to be selected, we got it working fine (solution available here) .
As for the trimming itself, we couldn't find any good library that has permissive license, except for one called "k4l-video-trimmer" . The library "FFmpeg", for example, is considered not permission as it uses GPLv3, which requires the app that uses it to also be open sourced. Besides, as I've read, it takes quite a lot (about 9MB).
Sadly, this library (k4l-video-trimmer) is very old and wasn't updated in years, so I had to fork it (here) in order to handle it nicely. It uses a open sourced library called "mp4parser" to do the trimming.
Problem is, this library seems to be able to handle files only, and not a Uri or InputStream, so even the sample can crash when selecting items that aren't reachable like a normal file, or even have paths that it can't handle. I know that in many cases it is possible to get a path of a file, but in many other cases, it's not, and I also know it's possible to just copy the file (here), but this isn't a good solution, as the file could be large and take a lot of space even though it's already accessible.
What I've tried
There are 2 places that the library uses a file:
In "K4LVideoTrimmer" file, in the "setVideoURI" function, which just gets the file size to be shown. Here the solution is quite easy, based on Google's documentation:
public void setVideoURI(final Uri videoURI) {
mSrc = videoURI;
if (mOriginSizeFile == 0) {
final Cursor cursor = getContext().getContentResolver().query(videoURI, null, null, null, null);
if (cursor != null) {
int sizeIndex = cursor.getColumnIndex(OpenableColumns.SIZE);
cursor.moveToFirst();
mOriginSizeFile = cursor.getLong(sizeIndex);
cursor.close();
mTextSize.setText(Formatter.formatShortFileSize(getContext(), mOriginSizeFile));
}
}
...
In "TrimVideoUtils" file, in "startTrim" which calls "genVideoUsingMp4Parser" function. There, it calls the "mp4parser" library using :
Movie movie = MovieCreator.build(new FileDataSourceViaHeapImpl(src.getAbsolutePath()));
It says that they use FileDataSourceViaHeapImpl (from "mp4parser" library) to avoid OOM on Android, so I decided to stay with it.
Thing is, there are 4 CTORS for it, all expect some variation of a file: File, filePath, FileChannel , FileChannel+fileName .
The questions
Is there a way to overcome this?
Maybe implement FileChannel and simulate a real file, by using ContentResolver and Uri ? I guess it might be possible, even if it means re-opening the InputStream when needed...
In order to see what I got working, you can clone the project here. Just know that it doesn't do any trimming, as the code for it in "K4LVideoTrimmer" file is commented:
//TODO handle trimming using Uri
//TrimVideoUtils.startTrim(file, getDestinationPath(), mStartPosition, mEndPosition, mOnTrimVideoListener);
Is there perhaps a better alternative to this trimming library, which is also permissive (meaning of Apache2/MIT licences , for example) ? One that don't have this issue? Or maybe even something of Android framework itself? I think MediaMuxer class could help (as written here), but I think it might need API 26, while we need to handle API 21 and above...
EDIT:
I thought I've found a solution by using a different solution for trimming itself, and wrote about it here, but sadly it can't handle some input videos, while mp4parser library can handle them.
Please let me know if it's possible to modify mp4parser to handle such input videos even if it's from Uri and not a File (without a workaround of just copying to a video file).

First of all a caveat: I am not familiar with the mp4parser library but your question looked interesting so I took a look.
I think its worth you looking at one of the classes the code comments say is "mainly for testing". InMemRandomAccessSourceImpl. To create a Movie from any URI, the code would be as follows:
try {
InputStream inputStream = getContentResolver().openInputStream(uri);
Log.e("InputStream Size","Size " + inputStream);
int bytesAvailable = inputStream.available();
int bufferSize = Math.min(bytesAvailable, MAX_BUFFER_SIZE);
final byte[] buffer = new byte[bufferSize];
int read = 0;
int total = 0;
while ((read = inputStream.read(buffer)) !=-1 ) {
total += read;
}
if( total < bytesAvailable ){
Log.e(TAG, "Read from input stream failed")
return;
}
//or try inputStream.readAllBytes() if using Java 9
inputStream.close();
ByteBuffer bb = ByteBuffer.wrap(buffer);
Movie m2 = MovieCreator.build(new ByteBufferByteChannel(bb),
new InMemRandomAccessSourceImpl(bb), "inmem");
} catch (FileNotFoundException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
}
But I would say, there looks to be somewhat of a conflict between what you want to achieve and the approach the parser takes. It is depending on local files to avoid large memory overheads, and random access to bytes can only be done if the entire set of data is available, which differs from a streaming approach.
It will require buffering at least the amount of data required for your clip in one go before the parser is given the buffer. That might be workable for you if you are looking to grab short sections and the buffering is not too cumbersome. You may be subject to IO exceptions and the like if the read from the InputStream has issues, especially if it is remote content, whereas you really aren't expecting that with a file on a modern system.
There is also MemoryFile to consider which provides an ashmem backed file-like object. I think somehow that could be worked in.

Next a snipped shows how to open a MediaStore Uri with IsoFile from Mp4Parser. So, you can see how to get a FileChannel from a Uri.
public void test(#NonNull final Context context, #NonNull final Uri uri) throws IOException
{
ParcelFileDescriptor fileDescriptor = null;
try
{
final ContentResolver resolver = context.getContentResolver();
fileDescriptor = resolver.openFileDescriptor(uri, "rw");
if (fileDescriptor == null)
{
throw new IOException("Failed to open Uri.");
}
final FileDescriptor fd = fileDescriptor.getFileDescriptor();
final FileInputStream inputStream = new FileInputStream(fd);
final FileChannel fileChannel = inputStream.getChannel();
final DataSource channel = new FileDataSourceImpl(fileChannel);
final IsoFile isoFile = new IsoFile(channel);
... do what you need ....
}
finally
{
if (fileDescriptor != null)
{
fileDescriptor.close();
}
}
}

How can I trim a video from Uri, including files that `mp4parser` library can handle, but using Android's framework instead?

Background
Over the past few days, I've worked on making a customizable, more updated version of a library for video trimming, here (based on this library)
The problem
While for the most part, I've succeeded making it customizable and even converted all files into Kotlin, it had a major issue with the trimming itself.
It assumes the input is always a File, so if the user chooses an item from the apps chooser that returns a Uri, it crashes. The reason for this is not just the UI itself, but also because a library that it uses for trimming (mp4parser) assumes an input of only File (or filepath) and not a Uri (wrote about it here). I tried multiple ways to let it get a Uri instead, but failed. Also wrote about it here.
That's why I used a solution that I've found on StackOverflow (here)for the trimming itself. The good thing about it is that it's quiet short and uses just Android's framework itself. However, it seems that for some video files, it always fails to trim them. As an example of such files, there is one on the original library repository, here (issue reported here).
Looking at the exception, this is what I got:
E: Unsupported mime 'audio/ac3'
E: FATAL EXCEPTION: pool-1-thread-1
Process: life.knowledge4.videocroppersample, PID: 26274
java.lang.IllegalStateException: Failed to add the track to the muxer
at android.media.MediaMuxer.nativeAddTrack(Native Method)
at android.media.MediaMuxer.addTrack(MediaMuxer.java:626)
at life.knowledge4.videotrimmer.utils.TrimVideoUtils.genVideoUsingMuxer(TrimVideoUtils.kt:77)
at life.knowledge4.videotrimmer.utils.TrimVideoUtils.genVideoUsingMp4Parser(TrimVideoUtils.kt:144)
at life.knowledge4.videotrimmer.utils.TrimVideoUtils.startTrim(TrimVideoUtils.kt:47)
at life.knowledge4.videotrimmer.BaseVideoTrimmerView$initiateTrimming$1.execute(BaseVideoTrimmerView.kt:220)
at life.knowledge4.videotrimmer.utils.BackgroundExecutor$Task.run(BackgroundExecutor.java:210)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:458)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:301)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1167)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:641)
at java.lang.Thread.run(Thread.java:764)
What I've found
Reported about the issue here. I don't think it will get an answer, as the library hasn't updated in years...
Looking at the exception, I tried to also trim without sound. This works, but it's not a good thing, because we want to trim normally.
Thinking that this code might be based on someone else's code, I tried to find the original one. I've found that it is based on some old Google code on its gallery app, here, in a class called "VideoUtils.java" in package of "Gallery3d". Sadly, I don't see any new version for it. Latest one that I see is of Gingerbread, here.
The code that I've made out of it looks as such:
object TrimVideoUtils {
private const val DEFAULT_BUFFER_SIZE = 1024 * 1024
#JvmStatic
#WorkerThread
fun startTrim(context: Context, src: Uri, dst: File, startMs: Long, endMs: Long, callback: VideoTrimmingListener) {
dst.parentFile.mkdirs()
//Log.d(TAG, "Generated file path " + filePath);
val succeeded = genVideoUsingMuxer(context, src, dst.absolutePath, startMs, endMs, true, true)
Handler(Looper.getMainLooper()).post { callback.onFinishedTrimming(if (succeeded) Uri.parse(dst.toString()) else null) }
}
//https://stackoverflow.com/a/44653626/878126 https://android.googlesource.com/platform/packages/apps/Gallery2/+/634248d/src/com/android/gallery3d/app/VideoUtils.java
#JvmStatic
#WorkerThread
private fun genVideoUsingMuxer(context: Context, uri: Uri, dstPath: String, startMs: Long, endMs: Long, useAudio: Boolean, useVideo: Boolean): Boolean {
// Set up MediaExtractor to read from the source.
val extractor = MediaExtractor()
// val isRawResId=uri.scheme == "android.resource" && uri.host == context.packageName && !uri.pathSegments.isNullOrEmpty())
val fileDescriptor = context.contentResolver.openFileDescriptor(uri, "r")!!.fileDescriptor
extractor.setDataSource(fileDescriptor)
val trackCount = extractor.trackCount
// Set up MediaMuxer for the destination.
val muxer = MediaMuxer(dstPath, MediaMuxer.OutputFormat.MUXER_OUTPUT_MPEG_4)
// Set up the tracks and retrieve the max buffer size for selected tracks.
val indexMap = SparseIntArray(trackCount)
var bufferSize = -1
try {
for (i in 0 until trackCount) {
val format = extractor.getTrackFormat(i)
val mime = format.getString(MediaFormat.KEY_MIME)
var selectCurrentTrack = false
if (mime.startsWith("audio/") && useAudio) {
selectCurrentTrack = true
} else if (mime.startsWith("video/") && useVideo) {
selectCurrentTrack = true
}
if (selectCurrentTrack) {
extractor.selectTrack(i)
val dstIndex = muxer.addTrack(format)
indexMap.put(i, dstIndex)
if (format.containsKey(MediaFormat.KEY_MAX_INPUT_SIZE)) {
val newSize = format.getInteger(MediaFormat.KEY_MAX_INPUT_SIZE)
bufferSize = if (newSize > bufferSize) newSize else bufferSize
}
}
}
if (bufferSize < 0)
bufferSize = DEFAULT_BUFFER_SIZE
// Set up the orientation and starting time for extractor.
val retrieverSrc = MediaMetadataRetriever()
retrieverSrc.setDataSource(fileDescriptor)
val degreesString = retrieverSrc.extractMetadata(MediaMetadataRetriever.METADATA_KEY_VIDEO_ROTATION)
if (degreesString != null) {
val degrees = Integer.parseInt(degreesString)
if (degrees >= 0)
muxer.setOrientationHint(degrees)
}
if (startMs > 0)
extractor.seekTo(startMs * 1000, MediaExtractor.SEEK_TO_CLOSEST_SYNC)
// Copy the samples from MediaExtractor to MediaMuxer. We will loop
// for copying each sample and stop when we get to the end of the source
// file or exceed the end time of the trimming.
val offset = 0
var trackIndex: Int
val dstBuf = ByteBuffer.allocate(bufferSize)
val bufferInfo = MediaCodec.BufferInfo()
// try {
muxer.start()
while (true) {
bufferInfo.offset = offset
bufferInfo.size = extractor.readSampleData(dstBuf, offset)
if (bufferInfo.size < 0) {
//InstabugSDKLogger.d(TAG, "Saw input EOS.");
bufferInfo.size = 0
break
} else {
bufferInfo.presentationTimeUs = extractor.sampleTime
if (endMs > 0 && bufferInfo.presentationTimeUs > endMs * 1000) {
//InstabugSDKLogger.d(TAG, "The current sample is over the trim end time.");
break
} else {
bufferInfo.flags = extractor.sampleFlags
trackIndex = extractor.sampleTrackIndex
muxer.writeSampleData(indexMap.get(trackIndex), dstBuf,
bufferInfo)
extractor.advance()
}
}
}
muxer.stop()
return true
// } catch (e: IllegalStateException) {
// Swallow the exception due to malformed source.
//InstabugSDKLogger.w(TAG, "The source video file is malformed");
} catch (e: Exception) {
e.printStackTrace()
} finally {
muxer.release()
}
return false
}
}
The exception is thrown on val dstIndex = muxer.addTrack(format) . For now, I've wrapped it in try-catch, to avoid a real crash.
I tried to search for newer versions of this code (assuming that it got fixed later), but failed.
Searching on the Internet and here, I've found only one similar question, here, but it's not the same at all.
The questions
Is it possible to use Android's framework to trim such problematic files? Maybe there is a newer version of the trimming of the videos code? I'm interested of course only for the pure implementation of video trimming, like the function I wrote above, of "genVideoUsingMuxer" .
As a temporary solution, is it possible to detect problematic input videos, so that I won't let the user start to trim them, as I know they will fail?
Is there maybe another alternative to both of those, that have a permissive license and doesn't bloat the app? For mp4parser, I wrote a separate question, here.

Why does it occur?
audio/ac3 is an unsupported mime type.
MediaMuxer.addTrack() (native) calls MPEG4Writer.addSource(), which prints this log message before returning an error.
EDIT
My aim was not to provide an answer to each of your sub-questions, but to give you some insight into the fundamental problem. The library you have chosen relies on the Android's MediaMuxer component. For whatever reason, the MediaMuxer developers did not add support for this particular audio format. We know this because the software prints out an explicit message to that effect, then immediately throws the IllegalStateException mentioned in your question.
Because the issue only involves a particular audio format, when you provide a video-only input, everything works fine.
To fix the problem, you can either alter the library to provide for the missing functionality, or find a new library that better suits your needs. sannies/mp4parser may be one such alternative, although it has different limitations (if I recall correctly, it requires all media to be in RAM during the mastering process). I do not know if it supports ac3 explicitly, but it should provide a framework to which you can add support for arbitrary mime types.
I would encourage you to wait for a more complete answer. There may be far better ways to do what you are trying to do. But it is apparent that the library you are using simply does not support all possible mime types.

Copy image to resources Xamarin

I have a Xamarin Forms project in which I need the user to be able to "load" an image. I can already press a button and search for a file using FilePicker like this:
async void OnUpload(object sender, EventArgs e)
{
try
{
FileData filedata = await CrossFilePicker.Current.PickFile();
// the dataarray of the file will be found in filedata.DataArray
// file name will be found in filedata.FileName;
//etc etc.
}
catch (Exception ex)
{
}
}
What I would need now is to copy that "filedata" (the image) to the resource folder of the project in order to access to the file easily. I have tried:
await CrossFilePicker.Current.SaveFile(filedata.FileName);
but it doesn't save any file into the project folder.
Moreover, I only need it to work on UWP and Android.

The SaveFile method saves it in a very specific folder.
If you want to save it somewhere of your choosing you have to implement it with the DependencyService. IO operations are very specific to the OS, so are the filepaths. I will give you a simple example for you to build on.
Start with defining an interface in your shared code, like so:
public interface IFileManager
{
void SaveFile(Stream stream);
}
Of course, it can have other methods as well, or extra parameters if you would like to specify things like filename, that is up to you. You would also probably like some kind of return value to know what happened.
Now, per platform implement this interface. For example for Android, it could look like this:
[assembly: Xamarin.Forms.Dependency (typeof (FileManager_Android))]
public class FileManager_Android : IFileManager
{
public void SaveFile(Stream stream)
{
var dir = Android.OS.Environment.GetExternalStoragePublicDirectory(Android.OS.Environment.DirectoryDcim);
        string filename = System.DateTime.Now.ToString("yyyyMMddHHmmssfff") + ".jpg";
string filePath = System.IO.Path.Combine(dir, name);
try
        {
         System.IO.File.WriteAllBytes(filePath, imageData);
}
catch (Exception ex)
{
System.Console.WriteLine(e.ToString());
}
}
}
(Saving code inspired by this link)
This will take the stream and save it to a path of your choosing.
For UWP you will need to implement it as well, which is quite similar, except for the implementation of SaveFile. As far as I know there is no plugin yet which makes this easier for you. There is PCLStorage, but this plugin only seems to work with text files. You could still look into it for inspiration though.

Read Json into APK Asobe AIR as3

I need to read a .json that I have within my .APK. I have tried many times I've even spend more than 1 day in it.
I think the problem is that FLHAS PROFESSIONAL use, but not want to give surrendered.
Nor loaded pictures new URLRequest(pictURL) :
Here are some codes that do not work on your phone (Android):
var pictLdr:Loader = new Loader();
var pictURL:String = "basecon/avatar3d.jpg";
var pictURLReq:URLRequest = new URLRequest(pictURL);
pictLdr.load(pictURLReq);
this.addChild(pictLdr);
And so I read the JSON and does not work
var tempFiles:File = File.desktopDirectory;
tempFiles = tempFiles.resolvePath("basecon/conversaciones.json");
trace(tempFiles.url); // app-storage:/images
//file:///storage/sdcard1/basecon/conversaciones.json
Why? How Can I read my JSON ?

The PO has done his best to ask a question in English but it did end up being a little off. What he meant is "how to read a json file", it's not that he can't read it, it's that he doesn't know how.
A File object gives you information about a file but not about its contents so trying to read a file with a File object won't work. You simply need to load that file and read its contents.
var tempFiles:File = File.applicationDirectory;
var jsonFile:File = tempFiles.resolvePath("basecon/conversaciones.json");
var fileLoader:URLLoader = new URLLoader();
fileLoader.addEventListener(Event.COMPLETE, handleFile);
fileLoader.load(new URLRequest(jsonFile.url));
Then in the handleFile listener:
var jsonData:String = String(fileLoader.data);
var jsonObject:Object = JSON.parse(jsonData);
Pretty simple.

Can't read lines from file by using StreamReader on Unity3d (Android)

I need to read a text stream by using StreamReader from file on android platform. File is about 100k lines, so even editor is getting stuck if i try to load it all to TextAsset or if i use WWW.
I simply need to read that file line by line without loading it all to a string. Then i'll do a tree generation from the lines that i got from the file. (But probably that part doesn't matter, i just need help on file reading part.)
I'm giving the code that i wrote down below. It works perfectly on editor, but fails on android.
I would be glad if anyone tell me, what am i missing.
(ps. english is not my native and this is my first question on the site. so sorry for the any mistakes that i may have done.)
private bool Load(string fileName)
{
try
{
string line;
string path = Application.streamingAssetsPath +"/";
StreamReader theReader = new StreamReader(path + fileName +".txt", Encoding.UTF8);
using (theReader)
{
{
line = theReader.ReadLine();
linesRead++;
if (line != null)
{
tree.AddWord(line);
}
}
while (line != null);
theReader.Close();
return true;
}
}
catch (IOException e)
{
Debug.Log("{0}\n" + e.Message);
exception = e.Message;
return false;
}
}

You can't use Application.streamingAssetsPath as a path on Android because streaming assets are stored within the JAR file with the application.
From http://docs.unity3d.com/Manual/StreamingAssets.html:
Note that on Android, the files are contained within a compressed .jar
file (which is essentially the same format as standard zip-compressed
files). This means that if you do not use Unity’s WWW class to
retrieve the file then you will need to use additional software to see
inside the .jar archive and obtain the file.
Use WWW like this in a coroutine:
WWW data = new WWW(Application.streamingAssetsPath + "/" + fileName);
yield return data;
if(string.IsNullOrEmpty(data.error))
{
content = data.text;
}
Or, if you really want to keep it simple (and your file is only a few 100k, stick it in a resource folder:
TextAsset txt = (TextAsset)Resources.Load(fileName, typeof(TextAsset));
string content = txt.text;