Develop Reference

Crawling Android File System gets stuck in possible SymLink loop

I'm trying to crawl the entire file system of an android device, both directories and files, without the benefit of NIO, to build a tree of it. If I had NIO then I could use WalkTree or similar, but I don't.
The problem I am having (on the Nexus 5 API 23 x86 emulator) is in /sys/bus/pci/devices and possibly other directories (eg /proc/self) - it doesn't complete before the app times out/quits/crashes (unknown which), possibly getting into some kind of loop or something (the path may change in a repetitive fashion but the canonical path varies little or not at all) .
However if I rule out Symbolic links then that problem goes away but I get what is only some of the files on the device rather than all - for example lacking files under /data (or /data/media/0) and those files not showing up elsewhere - not to mention it looks completely different from the file system that most file managers show. The former is strange as I'd understood Symbolic Links pointed to files and folders that were still present in the file system, but just made them look as if they were elsewhere.
What's the solution? Do I have to code exceptions or special handling for /sys/bus/pci/devices, /proc/self and others? I'd prefer to keep Symbolic Links being followed if I can, and I'd prefer to crawl as many files and folders as I can (so starting in a sub-folder is not preferred).
And a few related questions that might affect the approach I eventually take - if I DO keep SymLinks then does that mean that some things will be crawled twice or more? Is there a way to avoid that? Is there a way to detect when something is the TARGET of a SymLink, other than following the SymLink and checking the CanonicalPath?
Here's my code:
I get the root (I understand that in Android, the first and likely only root is the valid one):
File[] roots = File.listRoots();
String rootPath = "";
try {
rootPath = roots[0].getCanonicalPath();
} catch (IOException e) {
// do something
}
Then I start the crawl (note the boolean to choose whether to ignore simlinks or not):
try {
// check if the rootPath is null or empty, and then...
File rootFile = new File(rootPath);
rootNode = new FileFolderNode(rootFile, null, true, false); // last param may be true to ignore sim links
//FileFolderNode(String filePath, FileFolderNode parent, boolean addChildren, boolean ignoreSimLinks)
} catch (Exception e) {
// do something
}
That uses the FileFolderNode, which has constructor:
public FileFolderNode(File file, FileFolderNode parent, boolean addChildren, boolean ignoreSimLinks) throws IOException {
if (file == null)
throw new IOException("File is null in new FileFolderNode");
if (!file.exists())
throw new IOException("File '" + file.getName() + "' does not exist in new FileFolderNode");
// for now this uses isSymLink() from https://svn.apache.org/repos/asf/commons/_moved_to_git/io/trunk/src/main/java/org/apache/commons/io/FileUtils.java adjusted a bit to remove Java 7 and Windows mentions
if (!ignoreSimLinks)
if (FileUtils.isSymlink(file))
return;
this.name = file.getName();
if (this.name.equals("") && ! file.getCanonicalPath().equals("/"))
throw new IOException("Name is empty in new FileFolderNode");
this.isDirectory = file.isDirectory();
if (this.isDirectory) {
this.children = new ArrayList<FileFolderNode>();
if (addChildren) {
File[] files = file.listFiles();
if (files == null) {
// do something
} else {
// add in children
for (File f : files) {
FileFolderNode child = null;
try {
child = new FileFolderNode(f, this, addChildren, ignoreSimLinks);
} catch (Exception e) {
child = null;
}
if (child != null)
children.add(child);
}
}
}
}
}
Given the lack of answers here, I've broken this question down into areas needing clarification, and am trying to get answers to those - please do see if you can help with those:
Get Android Filing System root
Android SymLinks to hidden or separate locations or partitions
Avoiding Android Symbolic Link loop

Dropbox API Android listing contents of contents from directory

I would like to list the content of the content of a directory from Dropbox using its Android API.
I tried using dirEntry.contents() in a for loop, like this:
Entry rootDirEnt = mApi.metadata(mPath, 1000, null,
true, null);
if (!rootDirEnt.isDir || rootDirEnt.contents == null) {
// It's not a directory, or there's nothing in it
mErrorMsg = "No files available in Dropbox";
return false;
}
for (Entry childDirEnt : rootDirEnt.contents) {
// check if it still exists
if (childDirEnt.isDir && !childDirEnt.isDeleted
&& childDirEnt.contents != null) {
// childDirEnt contents is already null, even though there are files inside this directory
for (Entry fileEnt : childDirEnt.contents) {
// do smth with file
if (isCancelled()) {
return false;
} else {
publishProgress();
}
}
}
}
So I thought of usign mApi.metadata() again with the new path which works, but my question is: Can't I do it without calling medata() for every directory inside root dir ? (maybe using the contents call or smth.. )

You can call metadata again on each directory you find. Generally, Dropbox discourages calling metadata recursively like that unless driven by user action. See https://www.dropbox.com/developers/core/bestpractices.
You might instead use the delta API. The first time you call that, it will return a full list of all the content your app has access to in a user's Dropbox.

Not finding local data files saved in my application

The process seemed quite simplistic at first, but there must be something that I am missing going forward with this task. There was a settings file that I wanted to create local to my application for storing a whole bunch of data (not preference worthy). I ended up saving the file with the following code snippet.
protected File createSettingsFileLocation(String fileNameF)
{
File directoryFile = context_.getDir("settings", Context.MODE_PRIVATE);
File settingsFile;
settingsFile = new File(directoryFile, fileNameF);
if (!settingsFile.exists())
{
try
{
settingsFile.createNewFile();
} catch(IOException e)
{
Log.e(MyConstants.LOG_TAG, "Could not create the file as intended within internal storage.");
return null;
}
}
return settingsFile;
}
and then proceeded to retrieve the file later by looking for it locally with the following code snippets.
public String getCurrentFileContainingSettings()
{
List<String >settingFilesInFolder = getLocalStorageFileNames();
if (settingFilesInFolder == null || settingFilesInFolder.isEmpty())
{
return null;
}
String pathToCurrentSettingsFile = settingFilesInFolder.get(0);
return pathToCurrentSettingsFile;
}
protected List<String> getLocalStorageFileNames()
{
return Arrays.asList(context_.fileList());
}
However, the settingFilesInFolder always returns no entries, so I get null back from the getCurrentFileContainingSettings(). As what I could see from the documentation it seems as thought I was doing it right. But, I must be missing something, so I was hoping that someone could point something out to me. I could potentially hard-code the file name once it has been created within the system in a preference file for access later the first time that the settings are created, but I shouldn't have to do something like that I would think.

fileList() only looks in getFilesDir(), not in its subdirectories, such as the one you created via getDir(). Use standard Java file I/O (e.g., list()) instead.

Copy the shared preferences XML file from /data on Samsung device failed

There's an exporting feature in my application. It's just a copy operation since all my settings are store in shared preference.
I just copy the xml file from /data/data/package.name/shared_prefs/settings.xml to SD card. It works fine on my HTC desire. However, it might not work on Samsung devices, and i got the following error while I try to copy the file.
I/System.out( 3166): /data/data/package.name/shared_prefs/settings.xml (No such file or directory)
in the directory.
Anyone know how to fix it, or is there another simple way to store the shared preference ?
Thanks.

Never never never never never never never never never hardwire paths.
Unfortunately, there's no getSharedPreferenceDir() anywhere that I can think of. The best solution I can think of will be:
new File(getFilesDir(), "../shared_prefs")
This way if a device manufacturer elects to change partition names, you are covered.
Try this and see if it helps.

CommonsWare's suggestion would a be clever hack, but unfortunately it won't work.
Samsung does not always put the shared_prefs directory in the same parent directory as the getFilesDir().
I'd recommend testing for the existence of (hardcode it, except for package name):
/dbdata/databases/<package_name>/shared_prefs/package.name_preferences.xml and if it exists use it, otherwise fall back to either CommonsWare's suggestion of new File(getFilesDir(), "../shared_prefs") or just /data/data/<package_name>/shared_prefs/package.name_preferences.xml.
A warning though that this method could potentially have problems if a user switched from a Samsung rom to a custom rom without wiping, as the /dbdata/databases file might be unused but still exist.
More details
On some Samsung devices, such as the Galaxy S series running froyo, the setup is this:
/data/data/<package_name>/(lib|files|databases)
Sometimes there's a shared_prefs there too, but it's just Samsung's attempt to confuse you! Don't trust it! (I think it can happen as a left over from a 2.1 upgrade to 2.2, but it might be a left over from users switching roms. I don't really know, I just have both included in my app's bug report interface and sometimes see both files).
And:
/dbdata/databases/<package_name>/shared_prefs
That's the real shared_prefs directory.
However on the Galaxy Tab on Froyo, it's weird. Generally you have: /data/data/<package_name>/(lib|shared_prefs|files|databases)
With no /dbdata/databases/<package_name> directory, but it seems the system apps do have:
/dbdata/databases/<package_name>/yourdatabase.db
And added bonus is that /dbdata/databases/<package_name> is not removed when your app is uninstalled. Good luck using SharedPreferences if the user ever reinstalls your app!

Try using
context.getFilesDir().getParentFile().getAbsolutePath()

Best way to get valid path on all devices - run method Context.getSharedPrefsFile defined as:
/**
* {#hide}
* Return the full path to the shared prefs file for the given prefs group name.
*
* <p>Note: this is not generally useful for applications, since they should
* not be directly accessing the file system.
*/
public abstract File getSharedPrefsFile(String name);
Because of it hidden need use reflection and use fallback on fail:
private File getSharedPrefsFile(String name) {
Context context = ...;
File file = null;
try {
if (Build.VERSION.SDK_INT >= 24) {
try {
Method m = context.getClass().getMethod("getSharedPreferencesPath", new Class[] {String.class});
file = (File)m.invoke(context, new Object[]{name});
} catch (Throwable e) {
Log.w("App TAG", "Failed call getSharedPreferencesPath", e);
}
}
if (file == null) {
Method m = context.getClass().getMethod("getSharedPrefsFile", new Class[] {String.class});
file = (File)m.invoke(context, new Object[]{name});
}
} catch (Throwable e) {
Log.w("App TAG", "Failed call getSharedPrefsFile", e);
file = new File(context.getFilesDir(), "../shared_prefs/" + name + ".xml");
}
return file;
}
On some Samsungs implements like this:
public File getSharedPrefsFile(String paramString) {
return makeFilename(getPreferencesDir(), paramString + ".xml");
}
private File getPreferencesDir() {
synchronized (this.mSync) {
if (this.mPreferencesDir == null) {
this.mPreferencesDir = new File("/dbdata/databases/" + getPackageName() + "/", "shared_prefs");
}
File localFile = this.mPreferencesDir;
return localFile;
}
}
On other Android like this:
public File getSharedPrefsFile(String name) {
return makeFilename(getPreferencesDir(), name + ".xml");
}
private File getPreferencesDir() {
synchronized (mSync) {
if (mPreferencesDir == null) {
mPreferencesDir = new File(getDataDirFile(), "shared_prefs");
}
return mPreferencesDir;
}
}
private File getDataDirFile() {
if (mPackageInfo != null) {
return mPackageInfo.getDataDirFile();
}
throw new RuntimeException("Not supported in system context");
}
After while Google change API for level 24 and later:
https://android.googlesource.com/platform/frameworks/base/+/6a6cdafaec56fcd793214678c7fcc52f0b860cfc%5E%21/core/java/android/app/ContextImpl.java

I've tested in Samsung P1010 with:
//I'm in a IntentService class
File file = this.getDir("shared_prefs", MODE_PRIVATE);
I got:
"/data/data/package.name/app_shared_prefs"
It works fine to me. I can run ffmpeg in this folder.
Look:
Context.getDir

You have to create the shared_prefs directory:
try{
String dir="/data/data/package.name/shared_prefs";
// Create one directory
boolean success = (new File(dir)).mkdirs();
if (success) {
// now copy the file
}
}catch (Exception e){//Catch exception if any
System.err.println("Error: " + e.getMessage());
}
Also... the package of your app is package.name? Make sure you are referring to the right package.

Android Webview - Completely Clear the Cache

I have a WebView in one of my Activities, and when it loads a webpage, the page gathers some background data from Facebook.
What I'm seeing though, is the page displayed in the application is the same on each time the app is opened and refreshed.
I've tried setting the WebView not to use cache and clear the cache and history of the WebView.
I've also followed the suggestion here: How to empty cache for WebView?
But none of this works, does anyone have any ideas of I can overcome this problem because it is a vital part of my application.
mWebView.setWebChromeClient(new WebChromeClient()
{
public void onProgressChanged(WebView view, int progress)
{
if(progress >= 100)
{
mProgressBar.setVisibility(ProgressBar.INVISIBLE);
}
else
{
mProgressBar.setVisibility(ProgressBar.VISIBLE);
}
}
});
mWebView.setWebViewClient(new SignInFBWebViewClient(mUIHandler));
mWebView.getSettings().setJavaScriptEnabled(true);
mWebView.clearHistory();
mWebView.clearFormData();
mWebView.clearCache(true);
WebSettings webSettings = mWebView.getSettings();
webSettings.setCacheMode(WebSettings.LOAD_NO_CACHE);
Time time = new Time();
time.setToNow();
mWebView.loadUrl(mSocialProxy.getSignInURL()+"?time="+time.format("%Y%m%d%H%M%S"));
So I implemented the first suggestion (Although changed the code to be recursive)
private void clearApplicationCache() {
File dir = getCacheDir();
if (dir != null && dir.isDirectory()) {
try {
ArrayList<File> stack = new ArrayList<File>();
// Initialise the list
File[] children = dir.listFiles();
for (File child : children) {
stack.add(child);
}
while (stack.size() > 0) {
Log.v(TAG, LOG_START + "Clearing the stack - " + stack.size());
File f = stack.get(stack.size() - 1);
if (f.isDirectory() == true) {
boolean empty = f.delete();
if (empty == false) {
File[] files = f.listFiles();
if (files.length != 0) {
for (File tmp : files) {
stack.add(tmp);
}
}
} else {
stack.remove(stack.size() - 1);
}
} else {
f.delete();
stack.remove(stack.size() - 1);
}
}
} catch (Exception e) {
Log.e(TAG, LOG_START + "Failed to clean the cache");
}
}
}
However this still hasn't changed what the page is displaying. On my desktop browser I am getting different html code to the web page produced in the WebView so I know the WebView must be caching somewhere.
On the IRC channel I was pointed to a fix to remove caching from a URL Connection but can't see how to apply it to a WebView yet.
http://www.androidsnippets.org/snippets/45/
If I delete my application and re-install it, I can get the webpage back up to date, i.e. a non-cached version. The main problem is the changes are made to links in the webpage, so the front end of the webpage is completely unchanged.

I found an even elegant and simple solution to clearing cache
WebView obj;
obj.clearCache(true);
http://developer.android.com/reference/android/webkit/WebView.html#clearCache%28boolean%29
I have been trying to figure out the way to clear the cache, but all we could do from the above mentioned methods was remove the local files, but it never clean the RAM.
The API clearCache, frees up the RAM used by the webview and hence mandates that the webpage be loaded again.

The edited code snippet above posted by Gaunt Face contains an error in that if a directory fails to delete because one of its files cannot be deleted, the code will keep retrying in an infinite loop. I rewrote it to be truly recursive, and added a numDays parameter so you can control how old the files must be that are pruned:
//helper method for clearCache() , recursive
//returns number of deleted files
static int clearCacheFolder(final File dir, final int numDays) {
int deletedFiles = 0;
if (dir!= null && dir.isDirectory()) {
try {
for (File child:dir.listFiles()) {
//first delete subdirectories recursively
if (child.isDirectory()) {
deletedFiles += clearCacheFolder(child, numDays);
}
//then delete the files and subdirectories in this dir
//only empty directories can be deleted, so subdirs have been done first
if (child.lastModified() < new Date().getTime() - numDays * DateUtils.DAY_IN_MILLIS) {
if (child.delete()) {
deletedFiles++;
}
}
}
}
catch(Exception e) {
Log.e(TAG, String.format("Failed to clean the cache, error %s", e.getMessage()));
}
}
return deletedFiles;
}
/*
* Delete the files older than numDays days from the application cache
* 0 means all files.
*/
public static void clearCache(final Context context, final int numDays) {
Log.i(TAG, String.format("Starting cache prune, deleting files older than %d days", numDays));
int numDeletedFiles = clearCacheFolder(context.getCacheDir(), numDays);
Log.i(TAG, String.format("Cache pruning completed, %d files deleted", numDeletedFiles));
}
Hopefully of use to other people :)

I found the fix you were looking for:
context.deleteDatabase("webview.db");
context.deleteDatabase("webviewCache.db");
For some reason Android makes a bad cache of the url which it keeps returning by accident instead of the new data you need. Sure, you could just delete the entries from the DB but in my case I am only trying to access one URL so blowing away the whole DB is easier.
And don't worry, these DBs are just associated with your app so you aren't clearing the cache of the whole phone.

To clear all the webview caches while you signOUT form your APP:
CookieSyncManager.createInstance(this);
CookieManager cookieManager = CookieManager.getInstance();
cookieManager.removeAllCookie();
For Lollipop and above:
CookieSyncManager.createInstance(this);
CookieManager cookieManager = CookieManager.getInstance();
cookieManager.removeAllCookies(ValueCallback);

To clear cookie and cache from Webview,
// Clear all the Application Cache, Web SQL Database and the HTML5 Web Storage
WebStorage.getInstance().deleteAllData();
// Clear all the cookies
CookieManager.getInstance().removeAllCookies(null);
CookieManager.getInstance().flush();
webView.clearCache(true);
webView.clearFormData();
webView.clearHistory();
webView.clearSslPreferences();

The only solution that works for me
if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.LOLLIPOP_MR1) {
CookieManager.getInstance().removeAllCookies(null);
CookieManager.getInstance().flush();
}

This should clear your applications cache which should be where your webview cache is
File dir = getActivity().getCacheDir();
if (dir != null && dir.isDirectory()) {
try {
File[] children = dir.listFiles();
if (children.length > 0) {
for (int i = 0; i < children.length; i++) {
File[] temp = children[i].listFiles();
for (int x = 0; x < temp.length; x++) {
temp[x].delete();
}
}
}
} catch (Exception e) {
Log.e("Cache", "failed cache clean");
}
}

webView.clearCache(true)
appFormWebView.clearFormData()
appFormWebView.clearHistory()
appFormWebView.clearSslPreferences()
CookieManager.getInstance().removeAllCookies(null)
CookieManager.getInstance().flush()
WebStorage.getInstance().deleteAllData()

Simply using below code in Kotlin works for me
WebView(applicationContext).clearCache(true)

CookieSyncManager.createInstance(this);
CookieManager cookieManager = CookieManager.getInstance();
cookieManager.removeAllCookie();

CookieSyncManager.createInstance(this);
CookieManager cookieManager = CookieManager.getInstance();
cookieManager.removeAllCookie();
It can clear google account in my webview

To clear the history, simply do:
this.appView.clearHistory();
Source: http://developer.android.com/reference/android/webkit/WebView.html

Make sure you use below method for the form data not be displayed as autopop when clicked on input fields.
getSettings().setSaveFormData(false);

context.deleteDatabase("webview.db");
context.deleteDatabase("webviewCache.db")
Did the trick

to completely clear the cache in kotlin you can use:
context.cacheDir.deleteRecursively()
Just in case someone needs the kotlin code (:

Previous code has been deprecated. So, you can try this one in Kotlin base android projects:
CookieManager.getInstance().removeAllCookies {
// Do your work here.
}