Develop Reference

Method to discover encryption pattern using a small piece of the decrypted text?

I've been trying to decompile and extract useful data from an APK for some time now. This data is stored in CSV files inside an "assets" folder. Unfortunately, the developers got smart, and have begun encrypting these CSVs starting in July. I've exhausted every way I know of to try and turn these files into readable versions of themselves without any success. But then, I realized, there are a few files in the assets folder that haven't changed since well before July—thus, I have both the decrypted and encrypted versions of these files. Using this knowledge, is it possible to predict the encryption pattern that all other files in the directory went through?
I'm fairly sure that it was encrypted bit-level, not byte-level since there are a lot of unknown characters (represented as special question marks) while trying to read these CSVs using Notepad/TextEdit/Atom in UTF-8 mode (or any other mode except UTF-16, really).

You're talking about a "known plain text" attack. No modern, widely used
method is vulnerable to this kind of attack, but many home grown encryption
methods are. Even with known text, you need to know or guess a lot about
the details of the encryption algorithm.
A better plan might be to hack the software that you know is doing the
decrypting, which must contain both the algorithm and the key.

You'd have better luck simply guessing based on the encrypted output. You'll need to familiarize yourself with characteristics of the output of algorithms and compare against what you see. This is probably a lot easier for hashes but you're talking about encryption. To answer your question though, it's unlikely that you're going to be able to use an unencrypted version of a file to break the encrypted one. You might try encrypting that file using different algorithms and comparing the results. That might give you the algo but could take longer.
Alternatively, here are some tools I found that might be able to automate the process for you...
https://code.google.com/archive/p/aligot/
https://bitbucket.org/daniel_plohmann/simplifire.idascope
https://www.aldeid.com/wiki/IDA-Pro/plugins/FindCrypt2
To crack it, you're also going to need to find the key that was used to encrypt it. Since it's a program that obvious must be decrypted to use, that key shouldn't be impossible to find. It's either in the apk or on a server somewhere in which case use wireshark but I'm guessing it's embedded.
They might be usig DexGuard or ProGuard. Here's a related post What methods are being used to protect this Android APK: Reflection? Encryption? How do I reverse engineer it and analyze it?
If it's ProGuard you might start with something like this: http://proguard.sourceforge.net/manual/retrace/examples.html
Here's some info on that: How to decode ProGuard's obfuscated code precisely?

Permanently delete files Android

I found an android app named Super Erase that deletes files and folder permanently from android device so that the file deleted cant be recovered anymore..here is the application i am talking about ...but i was wondering how to that and i know it is made with android studio ..i tried the regular way to delete file.delete() but still the file can be recovered.can i have any help .

For starters, secure file deletion on flash media is a complex problem, with no quick and easy answers. The paper Reliably Erasing Data From Flash-Based Solid State Drives gives a good overview of the problems, the potential solutions, and their limitations. They conclude that
For sanitizing entire disks, ... software techniques work most, but not
all, of the time. We found that none of the available software
techniques for sanitizing individual files were effective. [emphasis added]
NIST 800-88 also has a good overview of the technology trends contributing to the problem, along with some minimum recommendations (appendix A) for Android devices. However they tend to be either whole-disk erasure (factory reset), or rely on cryptographic erasure (CE), rather than being general file erasure methods.
But all is not lost. Even if you can't sanitize individual files, you could hope to wipe all the unallocated space after deleting files. The article Secure Deletion on Log-structured File Systems (Reardon, et al.) describes a fairly promising way to do that in user-mode software. Android's internal memory uses (always?) a log-structured file system.
This paper's "purging" method does not require kernel-level access, and doesn't seem to require any native code on Android. (Note that the term "purging" is used a little differently in documents like NIST 800-88.) The basic idea is to delete all the sensitive data, then fill in the remaining space on the drive with a junk data file, and finally delete the junk data file.
While that takes more time and effort than just overwriting the deleted files themselves (several times in different patterns), it seems to be very robust even when you have to deal with the possibility of wear-leveling and log-structure FS.
Caveat and Further Measures
The main caveat for me is about the conditions mentioned by Reardon et al. in the above paper:
Purging will work for any log-structured file system provided both the
user’s disk quota is unlimited and the file system always performs
garbage collection to reclaim even a single chunk of memory before
declaring that the drive is unwritable. [emphasis mine]
The second condition seems pretty likely to be fulfilled, but I don't know about the first one. Does Android (or some manufacturers' versions of it) enforce quotas on disk space used by apps? I have not found any info about user quotas, but there are quotas for other niches like browser persistent storage. Does Android reserve some space for system use, or for each app's caching, for example, that can't be used for other things? If so, it should help (albeit with no guarantees) if we begin purging immediately after the sensitive files are deleted, so there is little time for other filesystem activity to stake a claim to the recently freed space.
Maybe we could mitigate these risks by cyclical purging:
Determine the remaining space available (call it S) on the relevant partition, e.g. using File.getUsableSpace()
Write a series of files to the partition; each one is, say, 20% of the initial S (subject to file size limits).
When we run out of space, delete the first couple of files that we created, then write another file or two as space allows.
Repeat that last step a few times, until you've reached a threshold you're satisfied with. Maybe up to the point where you've written 2*S worth of filler files; tweak that number to balance speed against thoroughness. How much you actually need to do this would be an area for more research.
Delete the remaining filler files.
The idea with cyclical purging is that if we run out of quota to overwrite all free space, deleting the filler files just written will free up more quota; and then the way log-structured filesystems allocate new blocks should allow us to continue overwriting the remaining blocks of free space in sequence, rather than rewriting the same space again.
I'm implementing this method in a test app, and will post it when it's working.
What about FAT-formatted microSD cards?
Would the same methods work on external storage or microSD cards? FAT is block-structured, so would the purge method apply to FAT-formatted SD cards?
On most contemporary flash memory devices, such as CompactFlash and
Secure Digital cards, [wear leveling] techniques are implemented in
hardware by a built-in microcontroller. On such devices, wear leveling
is transparent and most conventional file systems can be used on them
as-is. (https://en.wikipedia.org/wiki/Wear_leveling)
...which suggests to me that even on a FAT-formatted SD card, wear leveling means that the traditional Gutmann methods would not work (see his "Even Further Epilogue") and that a method like "purging" would be necessary.
Whether purging is sufficient, depends on your security parameters. Wear leveling seems to imply that a block could potentially be "retired" at any time, in which case there is no way to erase it without bypassing the microcontroller's wear leveling. AFAIK this can't be done in software, even if you had kernel privileges; you'd have to design special hardware.
However, "retiring" a bad block should be a fairly rare event relative to the life of the media, so for many scenarios, a purging method would be secure enough.
Erasing the traces
Note that Gutmann's method has an important strength, namely, to erase possible traces of old data on the storage media that could remain even after a block is overwritten with new data. These traces could theoretically be read by a determined attacker with lots of resources. A truly thorough approach to secure deletion would augment a method like Gutmann's with purging, rather than replacing it.
However, on log-structured and wear-leveled filesystems, the much bigger problem is trying to ensure that the sensitive blocks get overwritten at all.
Do existing apps use these methods?
I don't have any inside information about apps in the app store, but looking at reviews for apps like iShredder would suggest that at best, they use methods like Reardon's "purging." For example, they can take several hours to do a single-pass wipe of 32GB of free space.
Also note limitations: The reviews on some of the secure deletion apps say that in some cases, the "deleted" files were still accessible after running the "secure delete" operation. Of course we take these reviews with a grain of salt -- there is a possibility of user error. Nevertheless, I wouldn't assume these apps are effective, without good testing.
iShredder 4 Enterprise helpfully names some of the algorithms they use, in their app description:
Depending on the edition, the iShredder™ package comes with deletion
algorithms such as DoD 5220.22-M E, US Air Force (AFSSI-5020), US Army
AR380-19, DoD 5220.22-M ECE, BSI/VS-ITR TL-03423 Standard,
BSI-VS-2011, NATO Standard, Gutmann, HMG InfoSec No.5, DoD 5220.22 SSD
and others.
This impressive-sounding list gives us some pointers for further research. It's not clear how these methods are used -- singly or in combination -- and in particular whether any of them are represented as being effective on their own. We know that Gutmann's method would not be. Similarly, DoD 5220.22-M, AFSSI-5020, AR380-19, and Infosec No. 5 specify Gutmann-like procedures for overwriting sectors on hard drives, which would not be effective for flash-based media. In fact, "The U.S. Department of Defense no longer references DoD 5220.22-M as a method for secure HDD erasure", let alone for flash-based media, so this reference is misleading to the uninformed. (The DoD is said to reference NIST 800-88 instead.) "DoD 5220.22 SSD" sounds promising, but I can't find any informative references for it. I haven't chased down the other algorithms listed, but the results so far are not encouraging.

When you delete file with standard methods like file.delete() or runtime.exec("rm -f my_file") the only job that kernel does is removing info about file from auxiliary filesystem structures. But storage sectors that contain actual data remain untouched. And because of this recovering is possible.
This gives an idea about how we can try to remove file entirely - we should erase all sectors somehow. Easiest approach is to rewrite all file content with random data few times. After each pass we must flush file buffers to ensure that new content is written to storage. All existing methods of secure file removal spin around above principle. For example this one. Note that there is no universal method that works well across all storage types and filesystems. I guess you should experiment by yourself and try to implement some of the existing approaches or design your own. E.g. you can start from next:
Overwrite and flush file 10 times with random data (use FileOutputStream methods). Note!!! Don't use zeros or another low entropy data. Some filesystems may optimize such sparse files and leave some sectors with original content. You can use /dev/urandom file as source of random data (this is a virtual file and it is endless). It gives better results and works faster then well-known Random class.
Rename and move file 10 times. Choose new file names randomly.
Then truncate file with FileChannel.truncate().
And finally remove file with File.delete().
Of course you can write all logic in native code, it may be even somewhat easier than in Java. Described algorithm is just an example. Try doing in that way.

The standard filesystem API won't give you a simple function call for that.
You will have to use the underlaying native API for FileIO. Although I have never used it, theres a library for that:
https://github.com/johanneslumpe/react-native-fs

There are two answers to this question.
First, to answer the direct question of how some of these apps might be doing secure single file delete: what you do is actually open the file and replace the contents with zeros many times. The method sounds stupid, but I have worked with filesystem-level encryption on Android in the past and I found that the above holds true for many secure file delete solutions out there. For a seemingly compliant security, you can repeat writing zeros 7 times (or whatever the NIST standards specify for your hardware type).
Charset charset = StandardCharsets.UTF_8;
String content = new String(Files.readAllBytes(path), charset);
content = content.replaceAll("*", "0");
Files.write(path, content.getBytes(charset));
The right answer to this question is however different. On modern SSD drives and operating systems, it is insecure to delete single files. Therefore, these apps don't really offer a compelling product. Modern operating systems store fragments of the file in different places, and it is possible that even after you have zeroed out the most recent version of the file block-by-block and also overwrote all metadata, that a fragment from an older version of the file might be left over in another part of the drive.
The only secure way to delete sensitive content from a disk is to zero out the entire disk multiple times before discarding the disk.
#LarsH's answer about wiping all unallocated space after deleting files is compelling, but perhaps impractical. If you simply want to secure delete files so no one can scan the disk to recover it, then a better solution is the full-disk encryption. This was in-fact the entire appeal of full-disk encryption. This is why Apple stopped supporting secure file delete in their Mac OSX and iOS, and switched to full-disk encryption as default on all iPhones. Android phones have full-disk encryption as well now.
EDIT:
If you are looking for a true solution for a customer, your best bet is to use single file encryption. Once you destroy your key which only your app would know, there is no way to decrypt the file even if someone found it on the disk.
There exists no real solution for deleting files securely on SSDs. You can only give a false sense of security to non-technical people who still remember the old HDD days.

How to secure the assets stored in Android APK

i believe this question is already asked but i am not satisfied with their answers and posting it again here.
can someone please tell me how to safeguard my android app assets from copy cats who want to build similar app?

As always there is a trade-off between convenience and security. The more secure you want your app the less convenient it will be for you to develop.
The source code is inherently insecure due to ease of decompiling especially with rooted phone. To protect your source code you can obfuscate and/or encrypt your code which will prevent decompiling. Not exactly sure what tools are available for Android, but I am sure it will complicate your build process. If you just obfuscate, decompiling may still be possible, but will be much more difficult and will likely require the person attempting to decompile your code to know and understand Bytecode if a strong level of obfuscation is used.
To protect your assets, I believe your only option is to use encryption. Again this will complicate the app and/or build process depending on where you implement.
Even if you use encryption to protect your assets, you must protect the encryption key within your source code. Obviously, it does not matter what encryption scheme you use if your encryption key is in plaintext in the source code then anybody can grab the key and your asset and decrypt. All this does is add one more small fence to jump over.
However, if you correctly protect the encryption key and use a good encryption algorithm you should have less to worry about. This is a fairly complicated process though, it is difficult to use a key for encryption within your code and not keep it in plaintext. Even if you don't keep it in plaintext within the code, at some point it must be in memory to perform decryption. So if somebody can attach a debugger or dump memory at the right time, it will compromise the key. Of course, this requires a much more skilled adversary.
Overall, you need to decide exactly who you are worried about stealing your assets. If you are worried about the average Joe copying them, then you should be ok. If you are worried about a professional hacker, script kiddie, etc. gaining access to them then you are probably out of luck.

can someone please tell me how to safeguard my android app assets from copy cats who want to build similar app?
Generally, you can't. If it's in the app, anyone who wants to can get to them.
You are welcome to roll your own encryption scheme, or use tools like DexGuard. However, since the decryption engine and key must be in the app itself, all these do is increase the level of effort required to get to your assets. Making it more difficult will reduce the odds that somebody grabs the assets out of your APK, but it does not prevent the possibility. And, of course, there are other ways to get at much of this stuff (e.g., screenshots and image editors, recording audio played back by the app).

You can secure assets folder contents by encrypt it using strong encryption algorithms and decrypt them at runtime. Copycats cannot easily decrypt and get assets folder contents by simply extract apk using zip tools.

Android obfuscation hide salt value

When following the Application Licensing document on the developer page for Android to use ServerManagedPolicy for licensing, then the section Implementing an Obfuscator says to declare a private static final array of 20 with random bytes called SALT. This is passed to the constructor of AESObfuscator and the description says it's "an array of random bytes to use for each (un)obfuscation". I am new to this, but I guess that is for obfuscating preference values.
When later I obfuscate the code itself using the ProGuard option delivered with the Android SDK for Eclipse by exporting the apk I get the final apk. But using a reverse engineer application like apktool on my apk reveals the SALT array in plain bytes. Now, like I said I am new to this and my question might seem a bit naive... but isn't that a bad thing? Shouldn't the byte array be a bit more hidden?

A salt value is not a secret, so it's not really a problem if it is disclosed. That said, the obfuscator mangles code (mostly variable and method names), not values. So anything you have stored as is (strings, byte arrays, your obfuscation key) will be recoverable by decompilation.
Obfuscating makes it a bit harder to find, but if you are purposefully looking for a random-looking 16-byte array or a 128-bit key, it not too hard to find.
BTW, that example doesn't really promote best practices -- you should use a randomly generated
new salt value for every encryption operation, and store it along with the encrypted data. Not hard-code it your encryption code and use it every time. Then again, that example assumes you will be encrypting (for obfuscation purposes) a single preference only.

How to make apk Secure. Protecting from Decompile

I am developing an application that has SQLite database to store personal information that must be protected. What are some ways of protecting these personal data? An APK can easily be de-compiled completely, so how can we secure an APK? Additionally, how can a database of a mobile application be protected?

Basically, there are 5 methods to protect your APK being cracking/ reversing/ repackaging:
1. Isolate Java Program
The easiest way is to make users unable to access to the Java Class program. This is the most fundamental way, and it has a variety of specific ways to achieve this. For example, developers can place the key Java Class on the server, clients acquire services by access relevant interfaces of the server rather than access to the Class file directly. So there is no way for hackers to decompile Class files. Currently, there are more and more standards and protocols services provided through interfaces, such as HTTP, Web Service, RPC, etc. But there are lots of applications are not suitable for this protection. For example, Java programs in stand-alone programs are unable to isolate.
2. Encrypt Class Files
To prevent Class files from being decompiled directly, many developers will encrypt some key Class files, such as registration number, serial number management and other related classes. Before using these encrypted classes, the program needs to decrypt these classes first, then loading these classes into JVM. These classes can be decrypted by hardware, or software.
Developers often loading cryptographic classes through a customed ClassLoader class (Applet does not support customed ClassLoader because of security). Customed ClassLoader will find cryptographic classes first, then decrypt them. And finally loading the decrypted classes to JVM. Customed ClassLoader is a very important class in this protect method. Because it itself is not encrypted, it may be the first target of a hacker. If the relevant decryption key and algorithm have been overcome, then the encrypted classes can easily be decrypted.
3. Convert to Native Codes
Convert program to native codes is also an effective way to prevent decompilation. Because native codes are often difficult to be decompiled. Developers can convert the entire application to native codes, or they can also convert only key modules. If just convert key part of the modules, it will need JNI technology to call when Java programs are using these modules. It abandoned Java's cross-platform feature when using this mothod to protect Java programs. For different platforms, we need to maintain different versions of the native codes, which will increase software support and maintenance workload. But for some key modules, sometimes this solution is often necessary. In order to guarantee these native codes will not be modified or replaced, developers often need to digitally sign these codes. Before using these native codes, developers often need to authenticate these local codes to ensure that these codes have not changed by hackers. If the signature check is passed, then developers can call relevant JNI methods.
4. Code Obfuscation
Code obfuscation is to re-organize and process Class file, making the treated codes accomplish the same function (semantics) with the untreated codes. But the obfuscated codes are difficult to be decompiled, i.e., the decompiled codes are very difficult to understand, therefore decompile staffs are hard to understand the really semantics. Theoretically, if hackers have enough time, obfuscated codes may still be cracked. Even some people are developing de-obfuscate tool. But from the actual situation, since the diversified development of obfuscation, the mature of obfuscation theory, obfuscated Java codes can well prevent decompilation.
5. Online Encryption
APK Protect was an online encryption website for APK, but activity has apparently been discontinued since 2013 or so. It provided Java codes and C++ codes protection to achieve anti-debugging and decompile effects.
I originally suggested you use this last method for it could save you more time. Based on my experience, it was very simple to operate and it wouldn't take long time.

With Jellybean this has now become a possibility.
$ openssl enc -aes-128-cbc -K 000102030405060708090A0B0C0D0E0F
-iv 000102030405060708090A0B0C0D0E0F -in my-app.apk -out my-app-enc.apk
$ adb install --algo 'AES/CBC/PKCS5Padding' --key 000102030405060708090A0B0C0D0E0F
--iv 000102030405060708090A0B0C0D0E0F my-app-enc.apk
pkg: /data/local/tmp/my-app-enc.apk
Success
Please read the following blog post for further details

If this is secret information that must not fall into the hands of your users, you cannot secure it. It is fundamentally impossible to put information on a device (code or data), and have your application access it, but not allow someone with the device to have access to that information.
Encrypting the information is pointless from a security point of view, because your application has to contain whatever is needed to decrypt it in order to use it, and a sufficiently motivated attacker can always extract that and decrypt it on their own.
All you can do is make it more annoying and time consuming to get access to that information, which only helps if there's not really that much of a need to keep it secret. This is what using proguard to obfuscate your .apk file can do.

Have you considered sqlite encryption? See this thread - sqlite encryption for android
As for protecting your .apk, try obfuscating your code using proguard. See http://developer.android.com/guide/developing/tools/proguard.html

You can try 'Anti Decompiler(Android)Trial'
https://play.google.com/store/apps/details?id=com.tth.AntilDecompilerTrial
It makes something Proguard doesn't:
Hide all const values (string, character), you will never see clear text like "my key", "my val"... in your apk file
Obfuscate file name, which is referenced in AndroidManifest.xml
Add fake code to your source code. Event the powerful decompilers likes: dex2jar, jd-gui,... can't reverse exactly your apk file. Most of functions will show with comment 'Error'.
=====
After transforming, if you give someone your source project, it will be nearly impossible to read and understand.
This solution doesn't exclude Proguard, You can combine them together. (function, field Obfuscation of Proguard is better than Obfuscation features of this solution)

You may read my post at: http://www.androidpit.com/en/android/forum/thread/567093/An-Analysis-of-Android-APK-Protect-Shell-APKProtect. The APK added with protect shell of APK Protect is seems unable decompile. I mean, the encrypt method is very advanced. Even a master hacker need long time to crack it.

If it is the database that contains sensitive data you can encrypt the values of several columns or the full database like mentioned in the other answer and make sure that the password is not stored on the device but has to be entered by the user on accessing the data.
If there are pieces of code you need to protect there's really no good way of securing it. All you can for a limited amount of use-cases is to create a dependency to an online service and secure the server. But for a lot of applications this would not be an option.

First, make apk that can never be modified and used. I do it by temper detection from the server. I use root check emulator check. Then on the important activity, it checks root and emulator on every oncreate and on resume, deletes important data on onpause, Great. Now encrypt data and place license to server, use SSL server. It app can not be modified and run, everything is safe for ever. Well, how to avoid decompiler and online tamper detection. I do placing a huge code to generate some sample string from apk file, and compare it with an apk copy placed on the server. I have converted apk file to string. Just enjoy.