Hej,
I have some data shipped out with the app which shall be copied on the external storage. It's nested in a couple of subfolders and I'd like to copy the whole structure.
I'm having a hard time getting a File object for any ressource in /assets. But I think I'm depended on that 'cause I need something like File.isDirectory() to determine if I have to start copying or dive deeper into the system.
My first approach was using Assets Manager but it seems that class is not providing the information I need. The most promising why was to obtain an AssetFileDescriptorand go down to a [FileDescriptor][2]. However non of them seems to have a isDirectory-method.
So my other approach is straight forward: Creating a File Object and be happy. However it seems like I'm running in this problem of lacking a proper path to instance the file object. I'm aware of file://android_asset but it doesn't seem to work for the fileconstructor.
My last idea would to utilise the InputStream (which I need for copying anyway) and somehow filter the byte for a significant bit which indicates this resource to be a directory. That's a pretty hacky solution and probably right in the hell of ineffectiveness but I don't see another way to get around that.
I had the same problem. At some point I realized that list() is really slow (50ms on every call), so i'm using a different approach now:
I have an (eclipse) ant-builder which creates an index-file everytime my asset-folder changes. The file just contains one file-name per line, so directories are listed implicitely (if they are not empty).
The Builder:
<?xml version="1.0"?>
<project default="createAssetIndex">
<target name="createAssetIndex">
<fileset id="assets.fileset" dir="assets/" includes="**"
excludes="asset.index" />
<pathconvert pathsep="${line.separator}" property="assets"
refid="assets.fileset">
<mapper>
<globmapper from="${basedir}/assets/*" to="*"
handledirsep="yes" />
</mapper>
</pathconvert>
<echo file="assets/asset.index">${assets}</echo>
</target>
</project>
The class which loads asset.index into a List of Strings, so you can do arbitrary stuff with it, fast:
import android.content.ContextWrapper;
import com.google.common.collect.ImmutableList;
import java.io.BufferedInputStream;
import java.io.File;
import java.io.IOException;
import java.io.InputStream;
import java.util.Scanner;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
/**
* uses asset.index file (which is pregenerated) since asset-list()s take very long
*
*/
public final class AssetIndex {
//~ Static fields/initializers -------------------------------------------------------------------------------------
private static final Logger L = LoggerFactory.getLogger(AssetIndex.class);
//~ Instance fields ------------------------------------------------------------------------------------------------
private final ImmutableList<String> files;
//~ Constructors ---------------------------------------------------------------------------------------------------
public AssetIndex(final ContextWrapper contextWrapper) {
ImmutableList.Builder<String> ib = ImmutableList.builder();
L.debug("creating index from assets");
InputStream in = null;
Scanner scanner = null;
try {
in = contextWrapper.getAssets().open("asset.index");
scanner = new Scanner(new BufferedInputStream(in));
while (scanner.hasNextLine()) {
ib.add(scanner.nextLine());
}
scanner.close();
in.close();
} catch (final IOException e) {
L.error(e.getMessage(), e);
} finally {
if (scanner != null) {
scanner.close();
}
if (in != null) {
try {
in.close();
} catch (final IOException e) {
L.error(e.getMessage(), e);
}
}
}
this.files = ib.build();
}
//~ Methods --------------------------------------------------------------------------------------------------------
/* returns the number of files in a directory */
public int numFiles(final String dir) {
String directory = dir;
if (directory.endsWith(File.separator)) {
directory = directory.substring(0, directory.length() - 1);
}
int num = 0;
for (final String file : this.files) {
if (file.startsWith(directory)) {
String rest = file.substring(directory.length());
if (rest.charAt(0) == File.separatorChar) {
if (rest.indexOf(File.separator, 1) == -1) {
num = num + 1;
}
}
}
}
return num;
}
}
list() on AssetManager will probably give a null / zero length array / IOException if you try to get a list on a file, but a valid response on a directory.
But otherwise it should be file:///android_asset (with 3 /)
In my specific case, regular files have a name like filename.ext, while directories only have a name, without extension, and their name never contains the "." (dot) character.
So a regular file can be distinguished from a directory by testing its name as follows:
filename.contains(".")
If this your case too, the same solution should work for you.
Related
I want to repackage apache's httpclient lib to ship it with an android app (like https://code.google.com/p/httpclientandroidlib/ but with HttpClient 4.3.1)
Therefore, I downloaded the httpclient 4.3.1 jar (includes all its dependencies) by hand and used jarjar to repackage it:
x#x$: cd libs && for f in *.jar; do java -jar ../jarjar-1.4.jar process ../rules.txt $f out/my-$f; done
with rules.txt:
rule org.apache.http.** my.repackaged.org.apache.http.#1
Then I used ant to put the output together:
<project name="MyProject" default="merge" basedir=".">
<target name="merge">
<zip destfile="my-org-apache-httpclient-4.3.1.jar">
<zipgroupfileset dir="libs/out" includes="*.jar"/>
</zip>
</target>
</project>
I can use that file to develop and test my app, but if I deploy it on android, it throws an exception s/th like that it cannot find my.repackaged.org.apache.logging.log4j.something referenced by my.package.org.apache.logging.whatEver.
So, now I want to strip out any dependency on commons-logging by using bytecode manipulation. This has been done before: http://sixlegs.com/blog/java/dependency-killer.html
But I wonder how I actually do it? There are only dependencies on org.apache.commons.logging.Log:
x$x$: java -jar jarjar-1.4.jar find jar my-org-apache-httpclient-4.3.1.jar commons-logging-1.1.3.jar
my/http/impl/execchain/ServiceUnavailableRetryExec -> org/apache/commons/logging/Log
my/http/impl/execchain/RetryExec -> org/apache/commons/logging/Log
my/http/impl/execchain/RedirectExec -> org/apache/commons/logging/Log
my/http/impl/execchain/ProtocolExec -> org/apache/commons/logging/Log
...
I think the way to go is, to remove these dependencies and replace it with an own implementation like he did here https://code.google.com/p/httpclientandroidlib/ . Therefore, I made a new maven project with only one class with provided scope for the commons-logging that implements org.apache.commons.logging.Log interface and just delefates to the android.utils.Log:
MyLog implements org.apache.commons.logging.Log {}
in the package my.log and I packaged that in my-log-1.0.0.jar. I put that jar into the same folder as the repackaged httpclient-jars and used ant as mentioned above to package all together in my-org-apache-httpclient-4.3.1.jar.
Approach 1
I tried to use jarjar again:
java -jar jarjar-1.4.jar process rules2.txt my-org-apache-httpclient-4.3.1.jar my-org-apache-httpclient-4.3.1-without-logging-dep.jar
with rules2.txt:
rule my.repackaged.commons.logging.** my.log.#1
but that does not work. The exception that it cannot find my.repackaged.org.apache.logging.log4j.something referenced by my.package.org.apache.logging.whatEver is still thrown.
Approach 2
I also tried to delete the logging stuff from the final jar and/or repackage the my.repackaged.org.apache.log4j and logging to its original packages:
rules2.txt v2:
rule my.repackaged.org.apache.log4j.** org.apache.log4j.#1
rule my.repackaged.org.apache.logging.** org.apache.logging.#1
but that also is still throwing the excpetion: my.repackaged.org.apache.logging.log4j.something referenced by my.package.org.apache.logging.whatEver
QUESTION
How can I kill/replace that commons-logging dependencies and get rid of the Exception?
Introduction
If a program depends on a library it usually means that it uses methods of the library. Removing a dependency is therefore not a simple task. You effectively want to take away code that is - at least formally - required by the program.
There are three ways of removing dependencies:
Adapt the source code to not depend on the library and compile it from scratch.
Modify the bytecode to remove references to the library the project depends on.
Manipulate the runtime to not require the dependency. The easiest way is to recreate the required classes and to put them into the jar file.
None of these ways are really pretty. All of them can require a lot of work. None are guaranteed to work without side effects.
Solution
I will describe my solution by presenting the files and steps I used to solve the problem. To reproduce, you will need the following files (in a single directory):
lib/xxx-v.v.v.jar: The library jars (httpclient and dependencies, excluding commons-logging-1.1.3.jar)
jarjar-1.4.jar: Used for repackaging the jars
rules.txt: The jarjar rules
rule org.apache.http.** my.http.#1
rule org.apache.commons.logging.** my.logging.#1
build.xml: Ant build configuration
<project name="MyProject" basedir=".">
<target name="logimpl">
<javac srcdir="java/src" destdir="java/bin" target="1.5" />
<jar jarfile="out/logimpl.jar" basedir="java/bin" />
</target>
<target name="merge">
<zip destfile="httpclient-4.3.1.jar">
<zipgroupfileset dir="out" includes="*.jar"/>
</zip>
</target>
</project>
java/src/Log.java
package my.logging;
public interface Log {
public boolean isDebugEnabled();
public void debug(Object message);
public void debug(Object message, Throwable t);
public boolean isInfoEnabled();
public void info(Object message);
public void info(Object message, Throwable t);
public boolean isWarnEnabled();
public void warn(Object message);
public void warn(Object message, Throwable t);
public boolean isErrorEnabled();
public void error(Object message);
public void error(Object message, Throwable t);
public boolean isFatalEnabled();
public void fatal(Object message);
public void fatal(Object message, Throwable t);
}
java/src/LogFactory.java
package my.logging;
public class LogFactory {
private static Log log;
public static Log getLog(Class<?> clazz) {
return getLog(clazz.getName());
}
public static Log getLog(String name) {
if(log == null) {
log = new Log() {
public boolean isWarnEnabled() { return false; }
public boolean isInfoEnabled() { return false; }
public boolean isFatalEnabled() { return false; }
public boolean isErrorEnabled() {return false; }
public boolean isDebugEnabled() { return false; }
public void warn(Object message, Throwable t) {}
public void warn(Object message) {}
public void info(Object message, Throwable t) {}
public void info(Object message) {}
public void fatal(Object message, Throwable t) {}
public void fatal(Object message) {}
public void error(Object message, Throwable t) {}
public void error(Object message) {}
public void debug(Object message, Throwable t) {}
public void debug(Object message) {}
};
}
return log;
}
}
do_everything.sh
#!/bin/sh
# Repackage library
mkdir -p out
for jf in lib/*.jar; do
java -jar jarjar-1.4.jar process rules.txt $jf `echo $jf | sed 's/lib\//out\//'`
done
# Compile logging implementation
mkdir -p java/bin
ant logimpl
# Merge jar files
ant merge
That's it. Open up a console and execute
cd my_directory && ./do_everything.sh
This will create a folder "out" containing single jar files and "httpclient-4.3.1.jar" which is the final, independent and working jar file. So, what did we just do?
Repackaged httpclient (now in my.http)
Modified the library to use my.logging instead of org.apache.commons.logging
Compiled required classes to be used by the library (my.logging.Log and my.logging.LogFactory).
Merged the repackaged libraries and the compiled classes into a single jar file, httpclient-4.3.1.jar.
Pretty simple, isn't it? Just read the shell script line by line to discover the single steps. To check whether all dependencies were removed you can run
java -jar jarjar-1.4.jar find class httpclient-4.3.1.jar commons-logging-1.1.3.jar
I tried the generated jar file with SE7 and Android 4.4, it worked in both cases (see below for remarks).
Class file version
Every class file has a major version and a minor version (both depend on the compiler). The Android SDK requires class files to have a major version less than 0x33 (so everything pre 1.7 / JDK 7). I added the target="1.5" attribute to the ant javac task so the generated class files have a major version of 0x31 and can therefore be included in your Android app.
Alternative (bytecode manipulation)
You're lucky. Logging is (almost always) a one-way operation. It barely causes side effects affecting the main program. That means that removing commons-logging should be possible as it won't affect the functionality of the program.
I chose the second way, bytecode manipulation, which you suggested in your question. The concept is basically just this (A is httpclient, B is commons-logging):
If the return type of a method of A is part of B, the return type will be changed to java.lang.Object.
If any argument of a method of A has a type that is part of B, the argument type will be changed to java.lang.Object.
Invocations of methods belonging to B are removed entirely. pop and constant instructions are inserted to repair the VM stack.
Types belonging to B are removed from descriptors of methods called from A. This requires the target class (the class containing the called method) to be processed. All object types belonging to B will be replaced with java.lang.Object.
Instructions that attempt to access fields of classes belonging to B are removed. pop and constant instructions are inserted to repair the VM stack.
If a method tries to access a field of a type that belongs to B, the field signature referenced by the instruction is changed to java.lang.Object. This requires the target class (the class containing the accessed field) to be processed.
Fields of a type contained in B but belonging to classes of A are modified so that their type is java.lang.Object.
As you can see, the idea behind this is to replace all referenced classes with java.lang.Object and to remove all accesses to class members belonging to commons-logging.
I don't know whether this is reliable and I did not test the library after applying the manipulator. But from what I saw (the disassembled class files and no VM errors while loading the class files) I am fairly sure the code works.
I tried to document almost everything the program does. It uses the ASM Tree API which provides rather simple access to the class file structure. And - to avoid unnecessary negative reviews - this is "quick 'n' dirty" code. I did not really test it a lot and I bet there are faster ways of bytecode manipulation. But this program seems to fulfill the OP's needs and that's all I wrote it for.
import java.io.File;
import java.io.FileOutputStream;
import java.io.IOException;
import java.io.InputStream;
import java.io.OutputStream;
import java.util.Enumeration;
import java.util.List;
import java.util.jar.JarEntry;
import java.util.jar.JarFile;
import java.util.jar.JarOutputStream;
import org.objectweb.asm.ClassReader;
import org.objectweb.asm.ClassWriter;
import org.objectweb.asm.Opcodes;
import org.objectweb.asm.Type;
import org.objectweb.asm.tree.AbstractInsnNode;
import org.objectweb.asm.tree.ClassNode;
import org.objectweb.asm.tree.FieldInsnNode;
import org.objectweb.asm.tree.FieldNode;
import org.objectweb.asm.tree.InsnList;
import org.objectweb.asm.tree.InsnNode;
import org.objectweb.asm.tree.MethodInsnNode;
import org.objectweb.asm.tree.MethodNode;
public class DependencyFinder {
public static void main(String[] args) throws IOException {
if(args.length < 2) return;
DependencyFinder df = new DependencyFinder();
df.analyze(new File(args[0]), new File(args[1]), "org.apache.http/.*", "org.apache.commons.logging..*");
}
#SuppressWarnings("unchecked")
public void analyze(File inputFile, File outputFile, String sClassRegex, String dpClassRegex) throws IOException {
JarFile inJar = new JarFile(inputFile);
JarOutputStream outJar = new JarOutputStream(new FileOutputStream(outputFile));
for(Enumeration<JarEntry> entries = inJar.entries(); entries.hasMoreElements();) {
JarEntry inEntry = entries.nextElement();
InputStream inStream = inJar.getInputStream(inEntry);
JarEntry outEntry = new JarEntry(inEntry.getName());
outEntry.setTime(inEntry.getTime());
outJar.putNextEntry(outEntry);
OutputStream outStream = outJar;
// Only process class files, copy all other resources
if(inEntry.getName().endsWith(".class")) {
// Initialize class reader and writer
ClassReader classReader = new ClassReader(inStream);
ClassWriter classWriter = new ClassWriter(0);
String className = classReader.getClassName();
// Check whether to process this class
if(className.matches(sClassRegex)) {
System.out.println("Processing " + className);
// Parse entire class
ClassNode classNode = new ClassNode(Opcodes.ASM4);
classReader.accept(classNode, 0);
// Check super class and interfaces
String superClassName = classNode.superName;
if(superClassName.matches(dpClassRegex)) {
throw new RuntimeException(className + " extends " + superClassName);
}
for(String iface : (List<String>) classNode.interfaces) {
if(iface.matches(dpClassRegex)) {
throw new RuntimeException(className + " implements " + superClassName);
}
}
// Process methods
for(MethodNode method : (List<MethodNode>) classNode.methods) {
Type methodDesc = Type.getMethodType(method.desc);
boolean changed = false;
// Change return type if necessary
Type retType = methodDesc.getReturnType();
if(retType.getClassName().matches(dpClassRegex)) {
retType = Type.getObjectType("java/lang/Object");
changed = true;
}
// Change argument types if necessary
Type[] argTypes = methodDesc.getArgumentTypes();
for(int i = 0; i < argTypes.length; i++) {
if(argTypes[i].getClassName().matches(dpClassRegex)) {
argTypes[i] = Type.getObjectType("java/lang/Object");
changed = true;
}
}
if(changed) {
// Update method descriptor
System.out.print("Changing " + method.name + methodDesc);
methodDesc = Type.getMethodType(retType, argTypes);
method.desc = methodDesc.getDescriptor();
System.out.println(" to " + methodDesc);
}
// Remove method invocations
InsnList insns = method.instructions;
for(int i = 0; i < insns.size(); i++) {
AbstractInsnNode insn = insns.get(i);
// Ignore all other nodes
if(insn instanceof MethodInsnNode) {
MethodInsnNode mnode = (MethodInsnNode) insn;
Type[] cArgTypes = Type.getArgumentTypes(mnode.desc);
Type cRetType = Type.getReturnType(mnode.desc);
if(mnode.owner.matches(dpClassRegex)) {
// The method belongs to one of the classes we want to get rid of
System.out.println("Removing method call " + mnode.owner + "." +
mnode.name + " in " + method.name);
boolean isStatic = (mnode.getOpcode() == Opcodes.INVOKESTATIC);
if(!isStatic) {
// pop instance
insns.insertBefore(insn, new InsnNode(Opcodes.POP));
}
for(int j = 0; j < cArgTypes.length; j++) {
// pop argument on stack
insns.insertBefore(insn, new InsnNode(Opcodes.POP));
}
// Insert a constant value to repair the stack
if(cRetType.getSort() != Type.VOID) {
InsnNode valueInsn = getValueInstruction(cRetType);
insns.insertBefore(insn, valueInsn);
}
// Remove the actual method call
insns.remove(insn);
// Go back one instruction to not skip the next one
i--;
} else {
changed = false;
if(cRetType.getClassName().matches(dpClassRegex)) {
// Change return type
cRetType = Type.getObjectType("java/lang/Object");
changed = true;
}
for(int j = 0; j < cArgTypes.length; j++) {
if(cArgTypes[j].getClassName().matches(dpClassRegex)) {
// Change argument type
cArgTypes[j] = Type.getObjectType("java/lang/Object");
changed = true;
}
}
if(changed) {
// Update method invocation
System.out.println("Patching method call " + mnode.owner + "." +
mnode.name + " in " + method.name);
mnode.desc = Type.getMethodDescriptor(cRetType, cArgTypes);
}
}
} else if(insn instanceof FieldInsnNode) {
// Yeah I lied... we must not ignore all other instructions
FieldInsnNode fnode = (FieldInsnNode) insn;
Type fieldType = Type.getType(fnode.desc);
if(fnode.owner.matches(dpClassRegex)) {
System.out.println("Removing field access to " + fnode.owner + "." +
fnode.name + " in " + method.name);
// Patch code
switch(fnode.getOpcode()) {
case Opcodes.PUTFIELD:
case Opcodes.GETFIELD:
// Pop instance
insns.insertBefore(insn, new InsnNode(Opcodes.POP));
if(fnode.getOpcode() == Opcodes.PUTFIELD) break;
case Opcodes.GETSTATIC:
// Repair stack
insns.insertBefore(insn, getValueInstruction(fieldType));
break;
default:
throw new RuntimeException("Invalid opcode");
}
// Remove instruction
insns.remove(fnode);
i--;
} else {
if(fieldType.getClassName().matches(dpClassRegex)) {
// Change field type
System.out.println("Patching field access to " + fnode.owner +
"." + fnode.name + " in " + method.name);
fieldType = Type.getObjectType("java/lang/Object");
}
// Update field type
fnode.desc = fieldType.getDescriptor();
}
}
}
}
// Process fields
for(FieldNode field : (List<FieldNode>) classNode.fields) {
Type fieldType = Type.getType(field.desc);
if(fieldType.getClassName().matches(dpClassRegex)) {
System.out.print("Changing " + fieldType.getClassName() + " " + field.name);
fieldType = Type.getObjectType("java/lang/Object");
field.desc = fieldType.getDescriptor();
System.out.println(" to " + fieldType.getClassName());
}
}
// Class processed
classNode.accept(classWriter);
} else {
// Nothing changed
classReader.accept(classWriter, 0);
}
// Write class to JAR entry
byte[] bClass = classWriter.toByteArray();
outStream.write(bClass);
} else {
// Copy file
byte[] buffer = new byte[1024 * 64];
int read;
while((read = inStream.read(buffer)) != -1) {
outStream.write(buffer, 0, read);
}
}
outJar.closeEntry();
}
outJar.flush();
outJar.close();
inJar.close();
}
InsnNode getValueInstruction(Type type) {
switch(type.getSort()) {
case Type.INT:
case Type.BOOLEAN:
return new InsnNode(Opcodes.ICONST_0);
case Type.LONG:
return new InsnNode(Opcodes.LCONST_0);
case Type.OBJECT:
case Type.ARRAY:
return new InsnNode(Opcodes.ACONST_NULL);
default:
// I am lazy, I did not implement all types
throw new RuntimeException("Type not implemented: " + type);
}
}
}
The process seemed quite simplistic at first, but there must be something that I am missing going forward with this task. There was a settings file that I wanted to create local to my application for storing a whole bunch of data (not preference worthy). I ended up saving the file with the following code snippet.
protected File createSettingsFileLocation(String fileNameF)
{
File directoryFile = context_.getDir("settings", Context.MODE_PRIVATE);
File settingsFile;
settingsFile = new File(directoryFile, fileNameF);
if (!settingsFile.exists())
{
try
{
settingsFile.createNewFile();
} catch(IOException e)
{
Log.e(MyConstants.LOG_TAG, "Could not create the file as intended within internal storage.");
return null;
}
}
return settingsFile;
}
and then proceeded to retrieve the file later by looking for it locally with the following code snippets.
public String getCurrentFileContainingSettings()
{
List<String >settingFilesInFolder = getLocalStorageFileNames();
if (settingFilesInFolder == null || settingFilesInFolder.isEmpty())
{
return null;
}
String pathToCurrentSettingsFile = settingFilesInFolder.get(0);
return pathToCurrentSettingsFile;
}
protected List<String> getLocalStorageFileNames()
{
return Arrays.asList(context_.fileList());
}
However, the settingFilesInFolder always returns no entries, so I get null back from the getCurrentFileContainingSettings(). As what I could see from the documentation it seems as thought I was doing it right. But, I must be missing something, so I was hoping that someone could point something out to me. I could potentially hard-code the file name once it has been created within the system in a preference file for access later the first time that the settings are created, but I shouldn't have to do something like that I would think.
fileList() only looks in getFilesDir(), not in its subdirectories, such as the one you created via getDir(). Use standard Java file I/O (e.g., list()) instead.
For the life of me, I cannot seem to write a simple tsv file on Android 4.2.2 using Processing for Android.
This code simply fails silently...
try {
saveTable(tsv,"data.tsv");
}
catch (Exception e) {
println(e);
}
I have set the permission: WRITE_EXTERNAL_STORAGE but this made no difference.
I am going crazy!!
Thanks for any help!
Bob
Sorry for the noise.... I found the solution finally:
using a terminal app on tyhe Nexus 4 (Android 4.2.2), I created a directory:
$ mkdir /sdcard/MyStuff
the code should now read
saveTable(tsv,"//sdcard/MyStuff/data.tsv");
Note the // before "sdcard" directory name - I guess that somehow makes the system consider the path as "external storage" and so it's ok!
What complexity!
You can also use the GetExternalStorageDirectory method:
String basePath = Environment.getExternalStorageDirectory().getAbsolutePath();
then concatenate the string to match your path:
basePath += "MyStuff";
This requires that you import the Environment Class from the Android SDK:
import android.os.Environment;
Also, in Processing, you can just use the 'sdcard' nomenclature - ie.:
basePath = "//sdcard//MyStuff";
Also, a related answer about making a directory with terminal - here is how you would do it from Processing:
import java.io.File;
import java.io.IOException;
String dirName;
// Create Directory
try{
dirName = "//sdcard//MyStuff"; // Or use te Environment Class -- see above
File newFile = new File(dirName);
newFile.mkdirs();
if(newFile.exists()) {
//
if(newFile.isDirectory()) {
//
}
else {
}
}
else {
println("Directory Doesn't Exist... Creating");
}
}
catch(Exception e) {
e.printStackTrace();
}
This way your code knows about the path automatically... Hope that helps!
I'm writing a word game in Android. It's my first app so my knowledge is almost non-existent.
What I would like to do is use JWI to access the WordNet dictionary. This requires specifying the WordNet dictionary's file path.
From what I've read, Android "assets" are not available via a simple file path, but what JWI requires to initialize the WordNet dictionary API is a URL to the disk location of the dictionary files.
So, what is the best course of action? Should I copy the assets at startup-time into a known folder on the android device? I can't think of a better way but that seems entirely stupid to me.
Any help gratefully received.
I have the same problem (for a jetty webapp however and not android) and tried those two approaches, however unsuccessfully:
JWNL.initialize(this.getClass().getClassLoader().getResourceAsStream("wordnet_properties.xml");
dict = Dictionary.getInstance();
Here it successfully loads wordnet_properties.xml but it cannot access the dictionary which is pointed to by the properties file.
Using the dictionary folder directly:
String dictPath = "models/en/wordnet/dict/";
URL url = this.getClass().getClassLoader().getResource(dictPath);
System.out.println("loading wordnet from "+url);
dict = new RAMDictionary(url, ILoadPolicy.NO_LOAD);
Here I get the dictionary URL to be jar:file:/home/myusername/.m2/repository/package/1.0-SNAPSHOT/commons-1.0-SNAPSHOT.jar!/models/en/wordnet/dict/. WordNet however doesn't accept the jar protocol and gives me the error:
java.lang.IllegalArgumentException: URL source must use 'file' protocol
at edu.mit.jwi.data.FileProvider.toFile(FileProvider.java:693)
at edu.mit.jwi.data.FileProvider.open(FileProvider.java:304)
at edu.mit.jwi.DataSourceDictionary.open(DataSourceDictionary.java:92)
at edu.mit.jwi.RAMDictionary.open(RAMDictionary.java:216)
My next investigation will be to create a subclass to RAMDictionary or something similar, please tell me if you have found a solution in the meantime.
P.S.: I just wrote the developer a mail asking for help after I tried to rewrite the FileProvider to use resources instead but after one or two hours I gave up because the code calls so much other code that also only works with files. I will keep you up to date!
P.P.S.: I received an answer from the developer saying that it is principially not possible with streams because they don't offer random access which is necessary. However, he offered to implement a solution to load it all in RAM, if really necessary, but that would use up about 500 MB and I guess that is too much for android apps so I guess it is still best to unpack it somewhere.
P.S.: Here is my unpacking solution (you can replace the System.out.println statements with logger statements if you use logging or remove them if you don't like them):
import java.io.File;
import java.io.FileOutputStream;
import java.io.IOException;
import java.io.InputStream;
import java.net.URISyntaxException;
import java.util.Enumeration;
import java.util.jar.JarEntry;
import java.util.jar.JarFile;
/** Allows WordNet to be run from within a jar file by unpacking it to a temporary directory.**/
public class WordNetUnpacker
{
static final String ID = "178558556719"; // minimize the chance of interfering with an existing directory
static final String jarDir = "models/en/wordnet/dict";
/**If running from within a jar, unpack wordnet from the jar to a temp directory (if not already done) and return that.
* If not running from a jar, just return the existing wordnet directory.
* #see getUnpackedWordNetDir(Class)*/
static File getUnpackedWordNetDir() throws IOException
{return getUnpackedWordNetDir(WordNetUnpacker.class);}
/**If running from within a jar, unpack wordnet from the jar to a temp directory (if not already done) and return that.
* If not running from a jar, just return the existing wordnet directory.
* #param clazz the class in whose classloader the wordnet resources are found.
* #see getUnpackedWordNetDir()**/
static File getUnpackedWordNetDir(Class clazz) throws IOException
{
String codeSource = clazz.getProtectionDomain().getCodeSource().getLocation().getPath();
System.out.println("getUnpackedWordNetDir: using code source "+codeSource);
if(!codeSource.endsWith(".jar"))
{
System.out.println("not running from jar, no unpacking necessary");
try{return new File(WordNetUnpacker.class.getClassLoader().getResource(jarDir).toURI());}
catch (URISyntaxException e) {throw new IOException(e);}
}
try(JarFile jarFile = new JarFile(codeSource))
{
String tempDirString = System.getProperty("java.io.tmpdir");
if(tempDirString==null) {throw new IOException("java.io.tmpdir not set");}
File tempDir = new File(tempDirString);
if(!tempDir.exists()) {throw new IOException("temporary directory does not exist");}
if(!tempDir.isDirectory()) {throw new IOException("temporary directory is a file, not a directory ");}
File wordNetDir = new File(tempDirString+'/'+"wordnet"+ID);
wordNetDir.mkdir();
System.out.println("unpacking jarfile "+jarFile.getName());
copyResourcesToDirectory(jarFile, jarDir, wordNetDir.getAbsolutePath());
return wordNetDir;
}
}
/** Copies a directory from a jar file to an external directory. Copied from Stack Overflow. */
public static void copyResourcesToDirectory(JarFile fromJar, String jarDir, String destDir) throws IOException
{
int copyCount = 0;
for (Enumeration<JarEntry> entries = fromJar.entries(); entries.hasMoreElements();)
{
JarEntry entry = entries.nextElement();
if(!entry.getName().contains("models")) continue;
if (entry.getName().startsWith(jarDir) && !entry.isDirectory()) {
copyCount++;
File dest = new File(destDir + "/" + entry.getName().substring(jarDir.length() + 1));
File parent = dest.getParentFile();
if (parent != null) {
parent.mkdirs();
}
FileOutputStream out = new FileOutputStream(dest);
InputStream in = fromJar.getInputStream(entry);
try {
byte[] buffer = new byte[8 * 1024];
int s = 0;
while ((s = in.read(buffer)) > 0) {
out.write(buffer, 0, s);
}
} catch (IOException e) {
throw new IOException("Could not copy asset from jar file", e);
} finally {
try {
in.close();
} catch (IOException ignored) {}
try {
out.close();
} catch (IOException ignored) {}
}
}
}
if(copyCount==0) System.out.println("Warning: No files copied!");
}
}
You can just copy all dict files from "assets" to the internal directory of your app. Just do it once, on the first app launch.
Since then you can use JWI in a causual way like this:
String path = getFilesDir() + "/dict";
URL url = new URL("file", null, path);
IDictionary dict = new Dictionary(url);
There's an exporting feature in my application. It's just a copy operation since all my settings are store in shared preference.
I just copy the xml file from /data/data/package.name/shared_prefs/settings.xml to SD card. It works fine on my HTC desire. However, it might not work on Samsung devices, and i got the following error while I try to copy the file.
I/System.out( 3166): /data/data/package.name/shared_prefs/settings.xml (No such file or directory)
in the directory.
Anyone know how to fix it, or is there another simple way to store the shared preference ?
Thanks.
Never never never never never never never never never hardwire paths.
Unfortunately, there's no getSharedPreferenceDir() anywhere that I can think of. The best solution I can think of will be:
new File(getFilesDir(), "../shared_prefs")
This way if a device manufacturer elects to change partition names, you are covered.
Try this and see if it helps.
CommonsWare's suggestion would a be clever hack, but unfortunately it won't work.
Samsung does not always put the shared_prefs directory in the same parent directory as the getFilesDir().
I'd recommend testing for the existence of (hardcode it, except for package name):
/dbdata/databases/<package_name>/shared_prefs/package.name_preferences.xml and if it exists use it, otherwise fall back to either CommonsWare's suggestion of new File(getFilesDir(), "../shared_prefs") or just /data/data/<package_name>/shared_prefs/package.name_preferences.xml.
A warning though that this method could potentially have problems if a user switched from a Samsung rom to a custom rom without wiping, as the /dbdata/databases file might be unused but still exist.
More details
On some Samsung devices, such as the Galaxy S series running froyo, the setup is this:
/data/data/<package_name>/(lib|files|databases)
Sometimes there's a shared_prefs there too, but it's just Samsung's attempt to confuse you! Don't trust it! (I think it can happen as a left over from a 2.1 upgrade to 2.2, but it might be a left over from users switching roms. I don't really know, I just have both included in my app's bug report interface and sometimes see both files).
And:
/dbdata/databases/<package_name>/shared_prefs
That's the real shared_prefs directory.
However on the Galaxy Tab on Froyo, it's weird. Generally you have: /data/data/<package_name>/(lib|shared_prefs|files|databases)
With no /dbdata/databases/<package_name> directory, but it seems the system apps do have:
/dbdata/databases/<package_name>/yourdatabase.db
And added bonus is that /dbdata/databases/<package_name> is not removed when your app is uninstalled. Good luck using SharedPreferences if the user ever reinstalls your app!
Try using
context.getFilesDir().getParentFile().getAbsolutePath()
Best way to get valid path on all devices - run method Context.getSharedPrefsFile defined as:
/**
* {#hide}
* Return the full path to the shared prefs file for the given prefs group name.
*
* <p>Note: this is not generally useful for applications, since they should
* not be directly accessing the file system.
*/
public abstract File getSharedPrefsFile(String name);
Because of it hidden need use reflection and use fallback on fail:
private File getSharedPrefsFile(String name) {
Context context = ...;
File file = null;
try {
if (Build.VERSION.SDK_INT >= 24) {
try {
Method m = context.getClass().getMethod("getSharedPreferencesPath", new Class[] {String.class});
file = (File)m.invoke(context, new Object[]{name});
} catch (Throwable e) {
Log.w("App TAG", "Failed call getSharedPreferencesPath", e);
}
}
if (file == null) {
Method m = context.getClass().getMethod("getSharedPrefsFile", new Class[] {String.class});
file = (File)m.invoke(context, new Object[]{name});
}
} catch (Throwable e) {
Log.w("App TAG", "Failed call getSharedPrefsFile", e);
file = new File(context.getFilesDir(), "../shared_prefs/" + name + ".xml");
}
return file;
}
On some Samsungs implements like this:
public File getSharedPrefsFile(String paramString) {
return makeFilename(getPreferencesDir(), paramString + ".xml");
}
private File getPreferencesDir() {
synchronized (this.mSync) {
if (this.mPreferencesDir == null) {
this.mPreferencesDir = new File("/dbdata/databases/" + getPackageName() + "/", "shared_prefs");
}
File localFile = this.mPreferencesDir;
return localFile;
}
}
On other Android like this:
public File getSharedPrefsFile(String name) {
return makeFilename(getPreferencesDir(), name + ".xml");
}
private File getPreferencesDir() {
synchronized (mSync) {
if (mPreferencesDir == null) {
mPreferencesDir = new File(getDataDirFile(), "shared_prefs");
}
return mPreferencesDir;
}
}
private File getDataDirFile() {
if (mPackageInfo != null) {
return mPackageInfo.getDataDirFile();
}
throw new RuntimeException("Not supported in system context");
}
After while Google change API for level 24 and later:
https://android.googlesource.com/platform/frameworks/base/+/6a6cdafaec56fcd793214678c7fcc52f0b860cfc%5E%21/core/java/android/app/ContextImpl.java
I've tested in Samsung P1010 with:
//I'm in a IntentService class
File file = this.getDir("shared_prefs", MODE_PRIVATE);
I got:
"/data/data/package.name/app_shared_prefs"
It works fine to me. I can run ffmpeg in this folder.
Look:
Context.getDir
You have to create the shared_prefs directory:
try{
String dir="/data/data/package.name/shared_prefs";
// Create one directory
boolean success = (new File(dir)).mkdirs();
if (success) {
// now copy the file
}
}catch (Exception e){//Catch exception if any
System.err.println("Error: " + e.getMessage());
}
Also... the package of your app is package.name? Make sure you are referring to the right package.