Categorization and grouping of Android and iOS crash reports - android

Various mobile applications we support have crash reporting as an added feature which submits more data to us than the normal device-provided method. We support both iOS and Android apps. This information is sent to us and we shove it into a MySQL database.
This was the first step of the design. Now we want the ability to categorize, group, and count these crash reports by stack trace, device type, app version, OS version, and so on.
We currently are using a MySQL database, as mentioned, but there is no reason we could not move to a different database if it provides better support for what we're trying to do. We are moving our system to AWS, so DynamoDB would be the obvious second choice.
So, before I go any further, if you have any suggestions, please answer now.
More details:
We currently have the following data sent to us:
signal (eg. SIGSEGV)
exception name (eg. java.lang.NullPointerException or NSInvalidArgumentException
exception description (eg. "Unable to instantiate activity..." or "The string argument is NULL")
application name
handheld device type (eg. samsung/m0/GT-I9300 or iPad)
native stack trace (for Android crashes in native code)
OS version (eg. 4.1.1 (SDK Level 16) or 6.1.3)
User ID (if available)
Application version
crash timestamp
stack trace
submission date
other irrelevant data
I am able to group Java stack traces together to some degree using GROUP BY which works surprisingly well... for smaller datasets. But when you have ~300,000 crash logs, it sort of grinds to a halt.
My first thought is to create a separate table for stack traces, include an SHA hash column and add an index to it, which would just be a hash of the stack trace. I could then find or create a stack trace row as necessary. I don't know if this will be faster than simply relying on the database server to do the comparison on the stack trace strings directly. I could include a counter column to count how often each stack trace occurs, although it may be better to simply keep count of those by select count(*) FROM crash_reports GROUP BY fkStackTraceID, so that I could additionally filter by date or application versions.
Currently, this all falls apart when trying to do the same thing with iOS crash logs, or with native Android crash logs. Each one is distinct, due to the inclusion of the memory location of each stack trace each element. I can go to the trouble of finding the offset (which is also included) and subtracting it, which will help.
So some questions:
Are there any other methods of filtering the data to be more easily queryable in whatever way you think would be useful for stack traces? I want to get things right the first time, so any additional ways to separate the data for querying that I can't think of now would be to get in right away.
Is MySQL the best option for this, or would a NoSQL option (i.e. DynamoDB) be more useful?
My previous question again: Are there any prepackaged solutions that do this (or help do this), which function in a manner similar to the crash log sections of the Google Play console and/or the iTunes Connect site?

Found a recent pre-built solution:
http://www.hockeyapp.net

Related

Whats the best way to create a debugging history log for an Android application

So I have an android application that has a myriad of activities (intents?) which all do their own thing. We were wanting to create a sort of debug log though, one where you can see what the user has been through and error that occurs. You may have had to send one before if you've run into a bug with a program you've used.
My initial thoughts are to just create a class where I can send information/data to and it just writes it onto a text file. It would need to be accessible across all the activities so that I can easily write to it and re-use it.
I do wonder whether that's a good way to go through, noted that it doesn't really save any actual errors but only data I tell it to. And I'm not sure if its a great idea to be constantly opening->writing->closing a file for a debug log.
Is there a smarter way? Or a common pattern that would be good to use?
Thanks so much!
Sentry's Android SDK, will automatically report errors and exceptions in your application.
The Sentry SDK catches the exception right before the crash and builds a crash report that will persist to the disk. The SDK will try to send the report right after the crash, but since the environment may be unstable at the crash time, the report is guaranteed to send once the application is started again.
You can see full documentation here

Is it feasible to develop in Android what would be an equivalent of logcat output in little time?

The question may sound stupid, but I'm explaining:
I'm not 100% sure how logcat Works, but I think it's safe to guess that it reads internal produced messages from the app it's monitoring.
I think it would be possible to also get to read these messages in the way that logcat "would be added" to the app and it would be possible for example to put a thread that would search patterns in that "logcat added to the app" and produce some action, same way a developer searchs for certain patterns to make the app work. This way it would be possible to get a detailed log of what have been the actions of an user that may cause an error in the app for example, withouth the need of having to plug the pone in the computer and trying to reproduce the error.
But maybe logcat is something extremely complex on its own, and what I'm saying is total nonsense for any practical purpose.
Is it feasible what I'm mentioning?
logcat is reading stack traces. You can place your project inside a try and in the catch, redirect the stack trace to a database or whatever. In our project, we have Acralyzer added as a dependency and it sends all errors to our server. Note that we aren't using that horrid CouchDB-we are custom-parsing the json ourselves. We do this on deployed apps so we have a stack trace of exactly what line in the program failed. This is almost always enough information. "null reference" is pretty explanatory for example.

Crashlytics: module-relative stack addresses?

We have a shared library which we do not wish to share the symbol names with anything - including Crashlytics.
As such we really want either:
1. The base address at which each .so has been loaded or
2. Module-relative addresses shown in addition to or instead of absolute addresses in stack traces
Either of those would allow us to produce post-processing tooling (ala addr2line, etc) that turn the crash report into something meaningful without sharing library symbols externally.
Is there some way already to get at this information with Crashlytics? If not, any chance it could be added in the near future? What about the crash reporting via Google Play Console?
Crashlytics does not offer a viable solution for this right now. Our symbolication is all or nothing. Thanks!

How do I provide a full stack trace with an ACRA error report in Android?

I have an app where I use ACRA for error reporting.
I recently decided to send a caught exception with ErrorReporter, and it works, albeit without any kind of stack trace. All I get is the usual system dump info, and I really need the stack traces from my users.
So, my question is, how do I have the app send the full details?
(Note: When the app actually crashes completely, the error report sent by ACRA does include the stack trace.)
Here's a screenshot: screenshot
EDIT
I kept looking deeper in the reports, and as it turns out, ACRA's just reporting the stack trace to another column that's never really been used before: tags. So, perhaps just a bug or something in ACRA.
You should follow the ACRA tutorial and full instruction guide page.
All you need is to create a from in google docs account . After creating a form a key will be generated that you have to save it and apply it in your application java file of you android application project.With minor changes in manifest and application java file you can successfully integrate ACRA.
Follow the given link.
https://github.com/ACRA/acra/wiki/BasicSetup
it will come in STACK_TRACE column of your form like these
ACRA.getErrorReporter().handleException(caughtException);
See https://github.com/ACRA/acra/wiki/AdvancedUsage#wiki-Sending_reports_for_caught_exceptions

How can I catch SIGSEGV (segmentation fault) and get a stack trace under JNI on Android?

I'm moving a project to the new Android Native Development Kit (i.e. JNI) and I'd like to catch SIGSEGV, should it occur (possibly also SIGILL, SIGABRT, SIGFPE) in order to present a nice crash reporting dialog, instead of (or before) what currently happens: the immediate unceremonious death of the process and possibly some attempt by the OS to restart it. (Edit: The JVM/Dalvik VM catches the signal and logs a stack trace and other useful information; I just want to offer the user the option to email that info to me really.)
The situation is: a large body of C code which I didn't write does most of the work in this application (all the game logic) and although it's well-tested on numerous other platforms, it's entirely possible that I, in my Android port, will feed it garbage and cause a crash in native code, so I want the crash dumps (both native and Java) that currently show up in the Android log (I guess it would be stderr in a non-Android situation). I'm free to modify both C and Java code arbitrarily, although the callbacks (both going in and coming out of JNI) number about 40 and obviously, bonus points for small diffs.
I've heard of the signal chaining library in J2SE, libjsig.so, and if I could safely install a signal handler like that on Android, that would solve the catching part of my question, but I see no such library for Android/Dalvik.
Edit: From Jelly Bean onwards you can't get the stack trace, because READ_LOGS went away. :-(
I actually got a signal handler working without doing anything too exotic, and have released code using it, which you can see on github (edit: linking to historical release; I removed the crash handler since then). Here's how:
Use sigaction() to catch the signals and store the old handlers. (android.c:570)
Time passes, a segfault happens.
In the signal handler, call up to JNI one last time and then call the old handler. (android.c:528)
In that JNI call, log any useful debugging info, and call startActivity() on an activity that is flagged as needing to be in its own process. (SGTPuzzles.java:962, AndroidManifest.xml:28)
When you come back from Java and call that old handler, the Android framework will connect to debuggerd to log a nice native trace for you, and then the process will die. (debugger.c, debuggerd.c)
Meanwhile, your crash-handling activity is starting up. Really you should pass it the PID so it can wait for step 5 to complete; I don't do this. Here you apologise to the user and ask if you can send a log. If so, gather the output of logcat -d -v threadtime and launch an ACTION_SEND with recipient, subject and body filled in. The user will have to press Send. (CrashHandler.java, SGTPuzzles.java:462, strings.xml:41
Watch out for logcat failing or taking more than a few seconds. I have encountered one device, the T-Mobile Pulse / Huawei U8220, where logcat immediately goes into the T (traced) state and hangs. (CrashHandler.java:70, strings.xml:51)
In a non-Android situation, some of this would be different. You'd need to gather your own native trace, see this other question, depending on what sort of libc you have. You'd need to handle dumping that trace, launching your separate crash-handler process, and sending the email in some appropriate ways for your platform, but I imagine the general approach should still work.
I'm a little bit late, but I had the exact same need, and I've developed a small library to address it, by catching common crashes (SEGV, SIBGUS, etc.) inside JNI code, and replace them by regular java.lang.Error exceptions. Bonus, if the client is running on Android >= 4.1.1, the stack trace embeds the resolved backtrace of the crash (a pseudo-trace containing the full native stack trace). You will not recover from vicious crashes (ie. if you corrupt the allocator, for example), but at least it should allows you to recover from most of them. (please report successes and failures, the code is brand new)
More info at https://github.com/xroche/coffeecatch
(code is BSD 2-Clauses license)
FWIW, Google Breakpad works fine on Android. I did the porting work, and we're shipping it as part of Firefox Mobile. It requires a little setup, since it doesn't give you stack traces on the client-side, but sends you the raw stack memory and does the stack walking server-side (so you don't have to ship debug symbols with your app).
In my limited experience (non-Android), SIGSEGV in JNI code will generally crash the JVM before control is returned to your Java code. I vaguely recall hearing about some non-Sun JVM which lets you catch SIGSEGV, but AFAICR you can't expect to be able to do so.
You can try to catch them in C (see sigaction(2)), although you can do very little after a SIGSEGV (or SIGFPE or SIGILL) handler as the ongoing behaviour of a process is officially undefined.

Categories

Resources