I have ported ffmpeg library to Android. Using JNI interface, I am able to run ffmpeg commands by giving arguments to ffmpeg's main method, just like from command line.
In order to get a specific part of a video, I use this command :
ffmpeg -i /mnt/sdcard/input_video.mp4 -ss 00:00:12 -t 00:00:10 -an /mnt/sdcard/output_video.mp4
and it works great. The video is split from 12. seconds to 22. seconds and the video is saved, the method returns normally (as 0).
However, if I make a second similar call (different start time for example) just after the first one is completed, ffmpeg is not able to process the request and it throws a segmentation fault.
For the first call, it gives such an info :
Guessed Channel Layout for Input Stream #0.0 : mono
and works. But for the second, the message is like this one:
Guessed Channel Layout for Input Stream #1.0 : mono
and it doesn't work. I don't know if it has something to do with the error.
The problem, in general, should be related to static global variables (I think) but I could not manage to reset them properly. What might be the solution to make multiple successful calls to the main method of ffmpeg?
https://github.com/jhotovy/android-ffmpeg
Calling ffmpeg’s main() more than once from the same Activity causes segfaults. This is still an issue with libffmpeg, but libffmpeginvoke at least applies the band-aid solution described here: Calling native method twice of third party library in an Activity causes the Android application to close down.
This example from github made by Hiko can help others fixing the problem.
JNI invoke ffmpeg more than once
what it does is it re-initialize static variables in the begining of main() methode like this:
int main(int argc, char **argv)
{
LOGI("start run in main.");
received_sigterm = 0;
received_nb_signals = 0;
transcode_init_done = 0;
ffmpeg_exited = 0;
main_return_code = 0;
run_as_daemon = 0;
nb_frames_dup = 0;
nb_frames_drop = 0;
nb_input_streams = 0;
nb_input_files = 0;
nb_output_streams = 0;
nb_output_files = 0;
nb_filtergraphs = 0;
int ret;
int64_t ti;
register_exit(ffmpeg_cleanup);
............................................
............................................
........... The rest of the code ...........
By adding those lines you will never get the segfault again.
Related
I'm curious about this.
I wanted to check which function was faster, so I create a little code and I executed a lot of times.
public static void main(String[] args) {
long ts;
String c = "sgfrt34tdfg34";
ts = System.currentTimeMillis();
for (int k = 0; k < 10000000; k++) {
c.getBytes();
}
System.out.println("t1->" + (System.currentTimeMillis() - ts));
ts = System.currentTimeMillis();
for (int i = 0; i < 10000000; i++) {
Bytes.toBytes(c);
}
System.out.println("t2->" + (System.currentTimeMillis() - ts));
}
The "second" loop is faster, so, I thought that Bytes class from hadoop was faster than the function from String class. Then, I changed the order of the loops and then c.getBytes() got faster. I executed many times, and my conclusion was, I don't know why, but something happen in my VM after the first code execute so that the results become faster for the second loop.
This is a classic java benchmarking issue. Hotspot/JIT/etc will compile your code as you use it, so it gets faster during the run.
Run around the loop at least 3000 times (10000 on a server or on 64 bit) first - then do your measurements.
You know there's something wrong, because Bytes.toBytes calls c.getBytes internally:
public static byte[] toBytes(String s) {
try {
return s.getBytes(HConstants.UTF8_ENCODING);
} catch (UnsupportedEncodingException e) {
LOG.error("UTF-8 not supported?", e);
return null;
}
}
The source is taken from here. This tells you that the call cannot possibly be faster than the direct call - at the very best (i.e. if it gets inlined) it would have the same timing. Generally, though, you'd expect it to be a little slower, because of the small overhead in calling a function.
This is the classic problem with micro-benchmarking in interpreted, garbage-collected environments with components that run at arbitrary time, such as garbage collectors. On top of that, there are hardware optimizations, such as caching, that skew the picture. As the result, the best way to see what is going on is often to look at the source.
The "second" loop is faster, so,
When you execute a method at least 10000 times, it triggers the whole method to be compiled. This means that your second loop can be
faster as it is already compiled the first time you run it.
slower because when optimised it doesn't have good information/counters on how the code is executed.
The best solution is to place each loop in a separate method so one loop doesn't optimise the other AND run this a few times, ignoring the first run.
e.g.
for(int i = 0; i < 3; i++) {
long time1 = doTest1(); // timed using System.nanoTime();
long time2 = doTest2();
System.out.printf("Test1 took %,d on average, Test2 took %,d on average%n",
time1/RUNS, time2/RUNS);
}
Most likely, the code was still compiling or not yet compiled at the time the first loop ran.
Wrap the entire method in an outer loop so you can run the benchmarks a few times, and you should see more stable results.
Read: Dynamic compilation and performance measurement.
It simply might be the case that you allocate so much space for objects with your calls to getBytes(), that the JVM Garbage Collector starts and cleans up the unused references (bringing out the trash).
Few more observations
As pointed by #dasblinkenlight above, Hadoop's Bytes.toBytes(c); internally calls the String.getBytes("UTF-8")
The variant method String.getBytes() which takes Character Set as input is faster than the one that does not take any character set. So for a given string, getBytes("UTF-8") would be faster than getBytes(). I have tested this on my machine (Windows8, JDK 7). Run the two loops one with getBytes("UTF-8") and other with getBytes() in sequence in equal iterations.
long ts;
String c = "sgfrt34tdfg34";
ts = System.currentTimeMillis();
for (int k = 0; k < 10000000; k++) {
c.getBytes("UTF-8");
}
System.out.println("t1->" + (System.currentTimeMillis() - ts));
ts = System.currentTimeMillis();
for (int i = 0; i < 10000000; i++) {
c.getBytes();
}
System.out.println("t2->" + (System.currentTimeMillis() - ts));
this gives:
t1->1970
t2->2541
and the results are same even if you change order of executions of loop. To discount any JIT optimizations, I would suggest run the tests in separate methods to confirm this (as suggested by #Peter Lawrey above)
So, Bytes.toBytes(c) should always be faster than String.getBytes()
I know that OpenMP is included in NDK (usage example here: http://recursify.com/blog/2013/08/09/openmp-on-android ). I've done what it says on that page but when I use: #pragma omp for on a simple for loop that scans a vector, the app crashes with the famous "fatal signal 11".
What am I missing here? Btw I use a modified example from the Android samples, it's Tutorial 2 Mixed Processing. All I want is to parallelize (multithread) some of the for loops and nested for loops that I have in the jni c++ file while using OpenCV.
Any help/suggestion is appreciated!
Edit: sample code added:
#pragma omp parallel for
Mat tmp(iheight, iwidth, CV_8UC1);
for (int x = 0; x < iheight; x++) {
for (int y = 0; y < iwidth; y++) {
int value = (int) buffer[x * iwidth + y];
tmp.at<uchar>(x, y) = value;
}
}
Based on this: http://www.slideshare.net/noritsuna/how-to-use-openmp-on-native-activity
Thanks!
I think this is a known issue in GOMP, see Bug 42616 and Bug 52738.
It's about your app will crash if you try to use OpenMP directives or functions on a non-main thread, and can be traced back to the gomp_thread() function (see libgomp/libgomp.h # line 362 and 368) which returns NULL for threads you create:
#ifdef HAVE_TLS
extern __thread struct gomp_thread gomp_tls_data;
static inline struct gomp_thread *gomp_thread (void)
{
return &gomp_tls_data;
}
#else
extern pthread_key_t gomp_tls_key;
static inline struct gomp_thread *gomp_thread (void)
{
return pthread_getspecific (gomp_tls_key);
}
#endif
As you can see GOMP uses different implementation depending on whether or not thread-local storage (TLS) is available.
If it is available, then HAVE_TLS flag is set, and a global variable is used to track the state of each thread,
Otherwise, thread-local data will be managed via the function pthread_setspecific.
In the earlier version of NDKs the thread-local storage (the __thread keyword) isn't supported so HAVE_TLS won't be defined, therefore pthread_setspecific will be used.
Remark: I'm not sure whether __thread is supported or not in the last version of NDK, but here you can read the same answers about Android TLS.
When GOMP creates a worker thread, it sets up the thread specific data in the function gomp_thread_start() (line 72):
#ifdef HAVE_TLS
thr = &gomp_tls_data;
#else
struct gomp_thread local_thr;
thr = &local_thr;
pthread_setspecific (gomp_tls_key, thr);
#endif
But, when the application creates a thread independently, the thread specific data isn't set, and so the gomp_thread() function returns NULL. This causes the crash and this isn't a problem when TLS is supported, since the global variable that's used will always be available
I remember that this issue had been fixed android-ndk-r10d, but it only works with background processes (no Java). It means when you enable OpenMP and create a native thread from JNI (what is called from Java Android) then your app will crash remains.
The Android systrace logging system is fantastic, but it only works in the Java portion of the code, through Trace.beginSection() and Trace.endSection(). In a C/C++ NDK (native) portion of the code it can only be used through JNI, which is slow or unavailable in threads without a Java environment...
Is there any way of either adding events to the main systrace trace buffer, or even generating a separate log, from native C code?
This older question mentions atrace/ftrace as being the internal system Android's systrace uses. Can this be tapped into (easily)?
BONUS TWIST: Since tracing calls would often be in performance-critical sections, it should ideally be possible to run the calls after the actual event time. i.e. I for one would prefer to be able to specify the times to log, instead of the calls polling for it themselves. But that would just be icing on the cake.
Posting a follow-up answer with some code, based on fadden's pointers. Please read his/her answer first for the overview.
All it takes is writing properly formatted strings to /sys/kernel/debug/tracing/trace_marker, which can be opened without problems. Below is some very minimal code based on the cutils header and C file. I preferred to re-implement it instead of pulling in any dependencies, so if you care a lot about correctness check the rigorous implementation there, and/or add your own extra checks and error-handling.
This was tested to work on Android 4.4.2.
The trace file must first be opened, saving the file descriptor in an atrace_marker_fd global:
#include <sys/types.h>
#include <sys/stat.h>
#include <fcntl.h>
#define ATRACE_MESSAGE_LEN 256
int atrace_marker_fd = -1;
void trace_init()
{
atrace_marker_fd = open("/sys/kernel/debug/tracing/trace_marker", O_WRONLY);
if (atrace_marker_fd == -1) { /* do error handling */ }
}
Normal 'nested' traces like the Java Trace.beginSection and Trace.endSection are obtained with:
inline void trace_begin(const char *name)
{
char buf[ATRACE_MESSAGE_LEN];
int len = snprintf(buf, ATRACE_MESSAGE_LEN, "B|%d|%s", getpid(), name);
write(atrace_marker_fd, buf, len);
}
inline void trace_end()
{
char c = 'E';
write(atrace_marker_fd, &c, 1);
}
Two more trace types are available, which are not accessible to Java as far as I know: trace counters and asynchronous traces.
Counters track the value of an integer and draw a little graph in the systrace HTML output. Very useful stuff:
inline void trace_counter(const char *name, const int value)
{
char buf[ATRACE_MESSAGE_LEN];
int len = snprintf(buf, ATRACE_MESSAGE_LEN, "C|%d|%s|%i", getpid(), name, value);
write(atrace_marker_fd, buf, len);
}
Asynchronous traces produce non-nested (i.e. simply overlapping) intervals. They show up as grey segments above the thin thread-state bar in the systrace HTML output. They take an extra 32-bit integer argument that "distinguishes simultaneous events". The same name and integer must be used when ending traces:
inline void trace_async_begin(const char *name, const int32_t cookie)
{
char buf[ATRACE_MESSAGE_LEN];
int len = snprintf(buf, ATRACE_MESSAGE_LEN, "S|%d|%s|%i", getpid(), name, cookie);
write(atrace_marker_fd, buf, len);
}
inline void trace_async_end(const char *name, const int32_t cookie)
{
char buf[ATRACE_MESSAGE_LEN];
int len = snprintf(buf, ATRACE_MESSAGE_LEN, "F|%d|%s|%i", getpid(), name, cookie);
write(atrace_marker_fd, buf, len);
}
Finally, there indeed seems to be no way of specifying times to log, short of recompiling Android, so this doesn't do anything for the "bonus twist".
I don't think it's exposed from the NDK.
If you look at the sources, you can see that the android.os.Trace class calls into native code to do the actual work. That code calls atrace_begin() and atrace_end(), which are declared in a header in the cutils library.
You may be able to use the atrace functions directly if you extract the headers from the full source tree and link against the internal libraries. However, you can see from the header that atrace_begin() is simply:
static inline void atrace_begin(uint64_t tag, const char* name)
{
if (CC_UNLIKELY(atrace_is_tag_enabled(tag))) {
char buf[ATRACE_MESSAGE_LENGTH];
size_t len;
len = snprintf(buf, ATRACE_MESSAGE_LENGTH, "B|%d|%s", getpid(), name);
write(atrace_marker_fd, buf, len);
}
}
Events are written directly to the trace file descriptor. (Note that the timestamp is not part of the event; that's added automatically.) You could do something similar in your code; see atrace_init_once() in the .c file to see how the file is opened.
Bear in mind that, unless atrace is published as part of the NDK, any code using it would be non-portable and likely to fail in past or future versions of Android. However, as systrace is a debugging tool and not something you'd actually want to ship enabled in an app, compatibility is probably not a concern.
For anybody googling this question in the future.
Native trace events are supported since API Level 23, check out the docs here https://developer.android.com/topic/performance/tracing/custom-events-native.
I am working on Android platform, and I wonder if it is possible to start an Android app from the kernel source code. For example, at certain point along the linux kernel resume path, I want to start a specific app, say my custom lock screen app. Is that possible?
Edit:
the call_usermodehelper does not work with the "am" utility.
I have code like this in a kernel module:
int result = 0;
char *argv[] = { "/system/bin/am", "start", "-n", "com.twitter.android/com.twitter.applib.HomeTabActivity", NULL};
char *argv[] = {"/system/bin/ls", NULL};
static char *envp[] = {"HOME=/", "PATH=/sbin:/system/sbin:/system/bin:/system/xbin", NULL };
result = call_usermodehelper(argv[0], argv, envp, 1);
but when I insmod, the nothing happens, and result = -8
anyone can help?
I can't speak for certain about Android, but in vanilla Linux, there's a bunch of API's in kmod.h which can do what you want. See this article for details.
Is there any known problem with using time(NULL) on Android?
I tried running the following piece of code:
int32_t now1 = time(NULL);
int64_t now1_6 = (int64_t)time(NULL);
int32_t nt = (time_t)-1;
int64_t nt6 = (int64_t)-1;
And then log the result using the following format:
"Now1 is %d. Now1_6 is %lld. NT is %d. NT6 is %lld.\n", now1, now1_6, nt, nt6
This is the output I got:
01-05 19:10:15.354: I/SMOS(11738): Now1 is 1533390320. Now1_6 is 6585861276402981128. NT is 0. NT6 is 283493768396.
There were other problems as well, such as getting the same time values on different loop iterations.
I encountered these issues on both a virtual and a physical device running Android 4.0.3 (API 15), both of which were configured with the correct time. The output above is from the physical device.
I am led to believe there is a problem with this particular POSIX function in Bionic, but I could not find any reference to such, either online or in the Bionic docs.
You are trying to cast an int64_t into a long long, and they might not be equivalent on Android. If you want to print out int64_t values using printf and %lld, either convert the number first to long long, or use the correct format modifier:
printf("%lld", (long long)now1_6);
or
printf("%" PRId64, now1_6);
For the time(NULL) giving false times, try using gettimeofday instead:
struct timeval tm;
gettimeofday( &tm, NULL);