I have an XML file that I want to extract data from using XQuery.
So far, the only library I found that allows such thing is MXQuery but the project seems to be abandonned.
Is there any other way to make XQuery work on Android ?
I have started building an XQuery app months ago. You would enter a query directly in the app and have it evaluated there. I was thinking it could become almost like an IDE, with one big query text field filling the screen, and then you click run and it shows the XML result.
Unfortunately urgent matters have happened and I have not gotten further than setting up the compiler for an empty project directory :( (huge annoyance, I set it up with ant and Sherlock-activity, and then those became deprecated and I had to start over with gradle and ActionbarActivity)
Earlier I made a command line tool that seems to run on Android, but you need a terminal emulator app. (afair I have fixed the memory issue mentioned in the later comments)
I also made an app for public libraries to automatically renew all books that you have lend from the library. It keeps a history of lend books and you can search that history, e.g. to get a list of all books that you have lend about a certain topic. Or find the book that you have lend the most often. The twist is you search with XQuery. Normal people enter $books[title = "foo"], but you can write serialize(<foo>bar</foo>) or doc("file:///whatever")/foo to run any XQuery on any local file inside the app. However, the entire query has to fit in one line, is not saved and the app is in German. (it will ask for the username/password of your public library account, but accepts an empty username, too).
I found an android app named Super Erase that deletes files and folder permanently from android device so that the file deleted cant be recovered anymore..here is the application i am talking about ...but i was wondering how to that and i know it is made with android studio ..i tried the regular way to delete file.delete() but still the file can be recovered.can i have any help .
For starters, secure file deletion on flash media is a complex problem, with no quick and easy answers. The paper Reliably Erasing Data From Flash-Based Solid State Drives gives a good overview of the problems, the potential solutions, and their limitations. They conclude that
For sanitizing entire disks, ... software techniques work most, but not
all, of the time. We found that none of the available software
techniques for sanitizing individual files were effective. [emphasis added]
NIST 800-88 also has a good overview of the technology trends contributing to the problem, along with some minimum recommendations (appendix A) for Android devices. However they tend to be either whole-disk erasure (factory reset), or rely on cryptographic erasure (CE), rather than being general file erasure methods.
But all is not lost. Even if you can't sanitize individual files, you could hope to wipe all the unallocated space after deleting files. The article Secure Deletion on Log-structured File Systems (Reardon, et al.) describes a fairly promising way to do that in user-mode software. Android's internal memory uses (always?) a log-structured file system.
This paper's "purging" method does not require kernel-level access, and doesn't seem to require any native code on Android. (Note that the term "purging" is used a little differently in documents like NIST 800-88.) The basic idea is to delete all the sensitive data, then fill in the remaining space on the drive with a junk data file, and finally delete the junk data file.
While that takes more time and effort than just overwriting the deleted files themselves (several times in different patterns), it seems to be very robust even when you have to deal with the possibility of wear-leveling and log-structure FS.
Caveat and Further Measures
The main caveat for me is about the conditions mentioned by Reardon et al. in the above paper:
Purging will work for any log-structured file system provided both the
user’s disk quota is unlimited and the file system always performs
garbage collection to reclaim even a single chunk of memory before
declaring that the drive is unwritable. [emphasis mine]
The second condition seems pretty likely to be fulfilled, but I don't know about the first one. Does Android (or some manufacturers' versions of it) enforce quotas on disk space used by apps? I have not found any info about user quotas, but there are quotas for other niches like browser persistent storage. Does Android reserve some space for system use, or for each app's caching, for example, that can't be used for other things? If so, it should help (albeit with no guarantees) if we begin purging immediately after the sensitive files are deleted, so there is little time for other filesystem activity to stake a claim to the recently freed space.
Maybe we could mitigate these risks by cyclical purging:
Determine the remaining space available (call it S) on the relevant partition, e.g. using File.getUsableSpace()
Write a series of files to the partition; each one is, say, 20% of the initial S (subject to file size limits).
When we run out of space, delete the first couple of files that we created, then write another file or two as space allows.
Repeat that last step a few times, until you've reached a threshold you're satisfied with. Maybe up to the point where you've written 2*S worth of filler files; tweak that number to balance speed against thoroughness. How much you actually need to do this would be an area for more research.
Delete the remaining filler files.
The idea with cyclical purging is that if we run out of quota to overwrite all free space, deleting the filler files just written will free up more quota; and then the way log-structured filesystems allocate new blocks should allow us to continue overwriting the remaining blocks of free space in sequence, rather than rewriting the same space again.
I'm implementing this method in a test app, and will post it when it's working.
What about FAT-formatted microSD cards?
Would the same methods work on external storage or microSD cards? FAT is block-structured, so would the purge method apply to FAT-formatted SD cards?
On most contemporary flash memory devices, such as CompactFlash and
Secure Digital cards, [wear leveling] techniques are implemented in
hardware by a built-in microcontroller. On such devices, wear leveling
is transparent and most conventional file systems can be used on them
as-is. (https://en.wikipedia.org/wiki/Wear_leveling)
...which suggests to me that even on a FAT-formatted SD card, wear leveling means that the traditional Gutmann methods would not work (see his "Even Further Epilogue") and that a method like "purging" would be necessary.
Whether purging is sufficient, depends on your security parameters. Wear leveling seems to imply that a block could potentially be "retired" at any time, in which case there is no way to erase it without bypassing the microcontroller's wear leveling. AFAIK this can't be done in software, even if you had kernel privileges; you'd have to design special hardware.
However, "retiring" a bad block should be a fairly rare event relative to the life of the media, so for many scenarios, a purging method would be secure enough.
Erasing the traces
Note that Gutmann's method has an important strength, namely, to erase possible traces of old data on the storage media that could remain even after a block is overwritten with new data. These traces could theoretically be read by a determined attacker with lots of resources. A truly thorough approach to secure deletion would augment a method like Gutmann's with purging, rather than replacing it.
However, on log-structured and wear-leveled filesystems, the much bigger problem is trying to ensure that the sensitive blocks get overwritten at all.
Do existing apps use these methods?
I don't have any inside information about apps in the app store, but looking at reviews for apps like iShredder would suggest that at best, they use methods like Reardon's "purging." For example, they can take several hours to do a single-pass wipe of 32GB of free space.
Also note limitations: The reviews on some of the secure deletion apps say that in some cases, the "deleted" files were still accessible after running the "secure delete" operation. Of course we take these reviews with a grain of salt -- there is a possibility of user error. Nevertheless, I wouldn't assume these apps are effective, without good testing.
iShredder 4 Enterprise helpfully names some of the algorithms they use, in their app description:
Depending on the edition, the iShredder™ package comes with deletion
algorithms such as DoD 5220.22-M E, US Air Force (AFSSI-5020), US Army
AR380-19, DoD 5220.22-M ECE, BSI/VS-ITR TL-03423 Standard,
BSI-VS-2011, NATO Standard, Gutmann, HMG InfoSec No.5, DoD 5220.22 SSD
and others.
This impressive-sounding list gives us some pointers for further research. It's not clear how these methods are used -- singly or in combination -- and in particular whether any of them are represented as being effective on their own. We know that Gutmann's method would not be. Similarly, DoD 5220.22-M, AFSSI-5020, AR380-19, and Infosec No. 5 specify Gutmann-like procedures for overwriting sectors on hard drives, which would not be effective for flash-based media. In fact, "The U.S. Department of Defense no longer references DoD 5220.22-M as a method for secure HDD erasure", let alone for flash-based media, so this reference is misleading to the uninformed. (The DoD is said to reference NIST 800-88 instead.) "DoD 5220.22 SSD" sounds promising, but I can't find any informative references for it. I haven't chased down the other algorithms listed, but the results so far are not encouraging.
When you delete file with standard methods like file.delete() or runtime.exec("rm -f my_file") the only job that kernel does is removing info about file from auxiliary filesystem structures. But storage sectors that contain actual data remain untouched. And because of this recovering is possible.
This gives an idea about how we can try to remove file entirely - we should erase all sectors somehow. Easiest approach is to rewrite all file content with random data few times. After each pass we must flush file buffers to ensure that new content is written to storage. All existing methods of secure file removal spin around above principle. For example this one. Note that there is no universal method that works well across all storage types and filesystems. I guess you should experiment by yourself and try to implement some of the existing approaches or design your own. E.g. you can start from next:
Overwrite and flush file 10 times with random data (use FileOutputStream methods). Note!!! Don't use zeros or another low entropy data. Some filesystems may optimize such sparse files and leave some sectors with original content. You can use /dev/urandom file as source of random data (this is a virtual file and it is endless). It gives better results and works faster then well-known Random class.
Rename and move file 10 times. Choose new file names randomly.
Then truncate file with FileChannel.truncate().
And finally remove file with File.delete().
Of course you can write all logic in native code, it may be even somewhat easier than in Java. Described algorithm is just an example. Try doing in that way.
The standard filesystem API won't give you a simple function call for that.
You will have to use the underlaying native API for FileIO. Although I have never used it, theres a library for that:
https://github.com/johanneslumpe/react-native-fs
There are two answers to this question.
First, to answer the direct question of how some of these apps might be doing secure single file delete: what you do is actually open the file and replace the contents with zeros many times. The method sounds stupid, but I have worked with filesystem-level encryption on Android in the past and I found that the above holds true for many secure file delete solutions out there. For a seemingly compliant security, you can repeat writing zeros 7 times (or whatever the NIST standards specify for your hardware type).
Charset charset = StandardCharsets.UTF_8;
String content = new String(Files.readAllBytes(path), charset);
content = content.replaceAll("*", "0");
Files.write(path, content.getBytes(charset));
The right answer to this question is however different. On modern SSD drives and operating systems, it is insecure to delete single files. Therefore, these apps don't really offer a compelling product. Modern operating systems store fragments of the file in different places, and it is possible that even after you have zeroed out the most recent version of the file block-by-block and also overwrote all metadata, that a fragment from an older version of the file might be left over in another part of the drive.
The only secure way to delete sensitive content from a disk is to zero out the entire disk multiple times before discarding the disk.
#LarsH's answer about wiping all unallocated space after deleting files is compelling, but perhaps impractical. If you simply want to secure delete files so no one can scan the disk to recover it, then a better solution is the full-disk encryption. This was in-fact the entire appeal of full-disk encryption. This is why Apple stopped supporting secure file delete in their Mac OSX and iOS, and switched to full-disk encryption as default on all iPhones. Android phones have full-disk encryption as well now.
EDIT:
If you are looking for a true solution for a customer, your best bet is to use single file encryption. Once you destroy your key which only your app would know, there is no way to decrypt the file even if someone found it on the disk.
There exists no real solution for deleting files securely on SSDs. You can only give a false sense of security to non-technical people who still remember the old HDD days.
I am attempting to create a free, configurable version of my EGMaps Android app which anyone can use to easily create their own map-based apps. The goal is to provide a framework so people with little programming knowledge can just fill in blanks, provide their data, and have it work.
There are two apps involved:
App #1 (EGMaps) does pretty much all the work. It needs access to data provided, or pointed to, by other App #2. I'm the only one working on this one.
App #2 will be created by multiple, maybe lots of other people, all with different app signatures. This is a very small, simple app which does very little other than passing data to EGMaps. I'll be providing source code and instructions on what to fill in. The other programmers can either use it directly, or modify it however they want for their app, which will then eventually call EGMaps.
App #2 needs to pass a lot of data like GPS coordinates, GPS tracks, marker locations, etc, which it's already doing. It also needs to pass an unknown, but potentially large number of small drawables. Due to space considerations, I'd prefer to use the drawables directly from the calling app, rather than copying them over or downloading and storing them inside of EGMaps. These drawables will eventually be Google Maps Marker icons.
Since the apps are written by different programmers, the app signatures are different, so setting the same user ID doesn't help.
This is as close as I've come:
iconString=callingPackage+":drawable/"+iconName;
iconValue=getResources().getIdentifier(iconString, "drawable", null);
callingPackage = name of calling app (ie: App #2). I have verified this is correct.
iconName = name of icon, as found in the drawable resources.
Without the callingPackage part, and with the drawables saved directly in the app, this works fine. It's just accessing the external drawable that doesn't work. iconValue always returns 0. I have also tried putting callingPackage into the third parameter of getIdentifier, with and without adding it to iconString, but that didn't make any difference.
Is there any way to directly (or indirectly, I suppose) access these drawables from the calling app without actually copying them from somewhere?
I would have expected getResources().getIdentifier(iconName, "drawable", callingPackage) to have worked, assuming that callingPackage is an actual package name.
You can try createPackageContext() and calling getResources() on it to get a Resources object referencing the other package.
I have released an android app downloadable in the UK, but I wish to release a (very slightly different) version of the app for the U.S. Currently I believe I need to do the following for the new version of the app ...
1. Change the 'package' attribute in the manifest file to something different.
2. Ensure that on Google Play only the US can download the app.
Is there anything else I am yet to discover I need to do, or any other problems I am yet to consider?
Edit 19th July 13:19 GMT - I am currently warming towards using TelephonyManager.getNetworkCountryIso() as an initial country guess, and falling back on the locale if this fails (because of no SIM card). What do people think? Using GPS is also an option though, but is that overkill?
Since you say you're just changing a couple of strings (we'll think of the URL as just a string, too), I would suggest the following:
Create the resource directories res/values and res/values-en-rUS.
In each of these, create a strings.xml resource file
Define your default (UK specific) string values in the res/values directory, and your US specific string values in the res/values-en-rUS folder.
Then, to refer to these strings, simply use #string/my_string_name when referring to them from an XML document, or getResources().getString(R.string.my_string_name) when referring to them from code.
For more details on the types of resource qualifiers, check out this page, also for the list of qualified country codes you can use, see this page.
I'm sure anyone determined enough could change their region to US -- I'm honestly not sure offhand how the region is determined -- but for all intents and purposes this should do the trick without having to maintain two separate applications. I would just evaluate how critical it is that UK users be unable to access the US specific functions of the application, and with that in mind determine whether it is worth the maintenance of two applications.
EDIT Some more additional searching leads me to believe the region is locked into the build.prop file, which is not editable outside of rooting your device (i.e. it is not going to happen accidentally). That said, if it's still imperative that they have the correct option, I might suggest a popup dialog only on the first run of the application that confirms the locale with the user, saves it as a SharedPreference, and then choose your Strings programmatically based on that preference. That way you're still only maintaining one application, but you still have the flexibility of handling the two regions.
This is correct. The Play Store goes off of two things for who can download and if they can update. The first is the packagename com.andya.myukapp -> com.andya.myusapp should work, as long as none of your existing customers are expecting to switch freely between the two (assuming it's a paid app)
I eventually decided that the best policy was to use TelephonyManager to check the country of the Network.
TelephonyManager tm = (TelephonyManager)context.getSystemService(Context.TELEPHONY_SERVICE);
String networkCountryCode = tm.getNetworkCountryIso();
If that failed, I got the locale.
String locale = context.getResources().getConfiguration().locale.getCountry();
This would happen once at the start of the app. The results of this would then be saved and be configurable in the settings screen.
I have queried the call log on Android. Some calls have 0 and other have 1 for the CallLog.Calls.CACHED_NUMBER_TYPE field. What do these numbers mean? Does 1 mean "Home"? Where is this documented?
AFAIK, it is not explicitely documented. But if you read the source code of android you'll see that what is used are integers defined in
http://developer.android.com/reference/android/provider/ContactsContract.CommonDataKinds.Phone.html
(See allowed DATA2).
About how to retreive that in the relevant android source : for example in the tests =>
https://github.com/android/platform_packages_providers_contactsprovider/blob/c085b3eeebf13ebdfb197444747354a1d6eced2b/tests/src/com/android/providers/contacts/CallLogProviderTest.java#L81
If you want to do more things with call logs and callers infos that I've extracted a standalone version of Android's CallerInfo class :
http://code.google.com/p/csipsimple/source/browse/trunk/CSipSimple/src/com/csipsimple/models/CallerInfo.java
It allows me to cache by my own display name (which is not necessarily done by all android contact apps of all manufacturers -- HTC sense).
Also, keep in mind that all these Cached values may be erased by the contact app when it will refresh the screen. If you want to make sure your value remain the only way I found for now is to create an associated contact.
See https://github.com/Wysie/android_packages_apps_Contacts/blob/c3772f17c37817ebb4eb925146c3a633aa258aa2/src/com/android/contacts/RecentCallsListActivity.java#L364
(The source code of the call log app, that automatically refresh cached values).
Warning this code may differ on custom distrib from manufacturers. From example, as I said, HTC do that differently in their HTC Sense. And even inside android AOSP versions it changes. And no doubt samsung do things their own way in their UI for example.