I need to implement a service in android that must be able to monitor a folder to detect a certain file and read what it contains. I'm having a strange behavior with my code and I can't find the reason. This is my relevant code.
public void onCreate(){
lectorFichCSV = new LectorFichCSV(); //object to read CSV files
ftpFileObserver = new FileObserver(filePath.getAbsolutePath()){
public void onEvent(int event, String file) {
if((FileObserver.CREATE & event) != 0){
Log.i("INFO: ", filePath.getAbsolutePath() + "/" + file + " is created");
if(file.substring(0,3).equals("RVE")){ //If file is created and the one I expect
try{
Log.i("INFO: ", "We have a RVE answer");
is = new FileInputStream(filePath + "/" + file);
lineaVent = lectorFichCSV.parseCSVFileAsList(is); //Get information in a list
//Get dao from ORMLite
dao = getHelper().getLineaVentDao();
Iterator<String[]> iterator = lineaVent.iterator();
if(iterator.hasNext()){
String[] aux = iterator.next();
Log.i("INFO:", "CodLineaVent "+aux[0]);
if(aux[2].equals("S")){
//Update DB information accordin to my file
UpdateBuilder<LineaVent, Integer> updateBuilder = dao.updateBuilder();
updateBuilder.where().eq("_id", aux[0]);
updateBuilder.updateColumnValue("valido", true);
updateBuilder.updateColumnValue("saldo", true);
updateBuilder.update();
lineaVent.clear();
}else if(aux[2].equals("N")){
UpdateBuilder<LineaVent, Integer> updateBuilder = dao.updateBuilder();
updateBuilder.where().eq("_id", aux[0]);
updateBuilder.updateColumnValue("saldo", false);
updateBuilder.update();
lineaVent.clear();
}
File fileToDel = new File(filePath + "/" + file);
fileToDel.delete();
}
}catch(FileNotFoundException e){
e.printStackTrace();
}catch(SQLException e){
e.printStackTrace();
}
}
I debugged the code and sometimes is working and sometimes I get lineaVent.size() == 0. I'm going crazy with this, I'm thinking, is it possible that events occurs faster than the creation of my file? that would be the reason when I tried to parse my CSV file into my List object is size = 0? In that case I'm not getting any FileNotFoundException.
Any help will be appreciate. Thank you.
I am not an expert with the inotify POSIX API that, IIRC, underlies FileObserver. However, given that there are separate events for CREATE, MODIFY, and CLOSE_WRITE, it stands to reason that the CREATE event is solely for file creation -- in other words, allocating a new entry in the filesystem for the file. That would either create an empty file, or perhaps a file with some initial load of bytes, but where other MODIFY calls might be needed to write out the full contents. CLOSE_WRITE would then be called to indicate that whoever was writing to the file has now closed their file handle.
Hence, if you are watching for some file to be created, to read it in, watch for CREATE, then watch for CLOSE_WRITE on that same file, and then try to read it, and see if that works better.
Related
I'm trying to crawl the entire file system of an android device, both directories and files, without the benefit of NIO, to build a tree of it. If I had NIO then I could use WalkTree or similar, but I don't.
The problem I am having (on the Nexus 5 API 23 x86 emulator) is in /sys/bus/pci/devices and possibly other directories (eg /proc/self) - it doesn't complete before the app times out/quits/crashes (unknown which), possibly getting into some kind of loop or something (the path may change in a repetitive fashion but the canonical path varies little or not at all) .
However if I rule out Symbolic links then that problem goes away but I get what is only some of the files on the device rather than all - for example lacking files under /data (or /data/media/0) and those files not showing up elsewhere - not to mention it looks completely different from the file system that most file managers show. The former is strange as I'd understood Symbolic Links pointed to files and folders that were still present in the file system, but just made them look as if they were elsewhere.
What's the solution? Do I have to code exceptions or special handling for /sys/bus/pci/devices, /proc/self and others? I'd prefer to keep Symbolic Links being followed if I can, and I'd prefer to crawl as many files and folders as I can (so starting in a sub-folder is not preferred).
And a few related questions that might affect the approach I eventually take - if I DO keep SymLinks then does that mean that some things will be crawled twice or more? Is there a way to avoid that? Is there a way to detect when something is the TARGET of a SymLink, other than following the SymLink and checking the CanonicalPath?
Here's my code:
I get the root (I understand that in Android, the first and likely only root is the valid one):
File[] roots = File.listRoots();
String rootPath = "";
try {
rootPath = roots[0].getCanonicalPath();
} catch (IOException e) {
// do something
}
Then I start the crawl (note the boolean to choose whether to ignore simlinks or not):
try {
// check if the rootPath is null or empty, and then...
File rootFile = new File(rootPath);
rootNode = new FileFolderNode(rootFile, null, true, false); // last param may be true to ignore sim links
//FileFolderNode(String filePath, FileFolderNode parent, boolean addChildren, boolean ignoreSimLinks)
} catch (Exception e) {
// do something
}
That uses the FileFolderNode, which has constructor:
public FileFolderNode(File file, FileFolderNode parent, boolean addChildren, boolean ignoreSimLinks) throws IOException {
if (file == null)
throw new IOException("File is null in new FileFolderNode");
if (!file.exists())
throw new IOException("File '" + file.getName() + "' does not exist in new FileFolderNode");
// for now this uses isSymLink() from https://svn.apache.org/repos/asf/commons/_moved_to_git/io/trunk/src/main/java/org/apache/commons/io/FileUtils.java adjusted a bit to remove Java 7 and Windows mentions
if (!ignoreSimLinks)
if (FileUtils.isSymlink(file))
return;
this.name = file.getName();
if (this.name.equals("") && ! file.getCanonicalPath().equals("/"))
throw new IOException("Name is empty in new FileFolderNode");
this.isDirectory = file.isDirectory();
if (this.isDirectory) {
this.children = new ArrayList<FileFolderNode>();
if (addChildren) {
File[] files = file.listFiles();
if (files == null) {
// do something
} else {
// add in children
for (File f : files) {
FileFolderNode child = null;
try {
child = new FileFolderNode(f, this, addChildren, ignoreSimLinks);
} catch (Exception e) {
child = null;
}
if (child != null)
children.add(child);
}
}
}
}
}
Given the lack of answers here, I've broken this question down into areas needing clarification, and am trying to get answers to those - please do see if you can help with those:
Get Android Filing System root
Android SymLinks to hidden or separate locations or partitions
Avoiding Android Symbolic Link loop
I want to export my realm database to CSV/JSON in Android. Is there some in-build method in the realm database which can do this?
There is a iOS way of converting realm to CSV link. I want a similar method in Android.
I was able to cobble together the following solution in my project:
// Grab all data from the DB in question (TaskDB):
RealmResults<TaskDB> resultsDB = realm.where(TaskDB.class).findAll();
// Here we need to put in header fields
String dataP = null;
String header = DataExport.grabHeader(realm, "TaskDB");
// We write the header to file
savBak(header);
// Now we write all the data corresponding to the fields grabbed above:
for (TaskDB taskitems: resultsDB) {
dataP = taskitems.toString();
// We process the data obtained and add commas and formatting:
dataP = dataProcess(dataP);
// Workaround to remove the last comma from final string
int total = dataP.length() - 1;
dataP = dataP.substring(0,total);
// We write the data to file
savBak(dataP);
}
I will explain what it is doing as best I can and include all corresponding code(all in reference to the first code block).
The first I did is grab the header using the following method I wrote in a separate class (DataExport.grabHeader). It takes 2 arguments: the realm object in question and the DB object model name:
public static String grabHeader(Realm realm, String model){
final RealmSchema schema = realm.getSchema();
final RealmObjectSchema testSchema = schema.get(model);
final String header = testSchema.getFieldNames().toString();
String dataProcessed = new String();
Pattern p = Pattern.compile("\\[(.*?)\\]");
Matcher m = p.matcher(header);
while(m.find()) {
dataProcessed += m.group(1).trim().replaceAll("\\p{Z}","");
}
return dataProcessed;
Within grabHeader, I apply some regex magic and spit out a string that will be used as the header with the appropriate commas in place (String dataProcessed).
In this scenario, after I obtained the data needed, I used another method (savBak) to write the information to a file which takes 1 string argument:
#Override
public void savBak(String data){
FileOutputStream fos = null;
try {
fos = openFileOutput(FILE_NAME, MODE_PRIVATE | MODE_APPEND);
fos.write(data.getBytes());
fos.write("\n".getBytes());
Log.d("tester", "saved to: " + getFilesDir() + "/" + FILE_NAME);
} catch (FileNotFoundException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
} finally {
if (fos != null) {
try {
fos.close();
} catch (IOException e) {
e.printStackTrace();
}
}
}
}
The "savBak" method writes the information to a FILE_NAME specified in a variable and we have our header information. After the header is written, we do the basically the same process with the DB using a forloop but I also had to include 2 lines to remove the trailing comma after the line was processed. Each line is appended to the file and viola, CSV formatted goodness.
From here, you can use other existing methods of converting CSV to JSON and whatever else as well as putting the information back into realm via JSON. When it comes to more advanced elements like primary keys and such, I am not sure but it worked for my particular project needs.
Please excuse any "bad code" practice as I'm new to Java/Android in general coming from a "barely intermediate" Python background so hopefully this makes sense.
I got a reply from Realm support via email.
Unfortunately, we do not have this feature yet. You can see it tracked here: https://github.com/realm/realm-java/issues/2880
You could use a dynamic API and write a script yourself to perform a similar feature.
I'm trying to implement a method that will and detect when a screenshot has been created and immediately delete it. I'm using FileObserver to observe the screenshot directory and flag when a new file is created in said directory. I don't know that the syntax is entirely correct, so any help would be appreciated. Thanks!
private void deleteMostRecentScreenshot() {
/**
* Set the path for the screenshot directory.
*/
String path = Environment.getExternalStoragePublicDirectory(Environment.DIRECTORY_PICTURES).toString() + "Screenshots";
OLog.d(TAG, path);
/**
* Array of the files in our screenshot directory, sorted so that the oldest pictures are first.
*/
final File[] screenshots = new File(path).listFiles();
Arrays.sort(screenshots, new Comparator<File>() {
#Override
public int compare(File f1, File f2) {
return Long.valueOf(f1.lastModified()).compareTo(f2.lastModified());
}
});
/**
* Watch for when a new file is created in our directory, ignoring "probes" created when the camera
* is launched. If a new file is created, delete the last file in the array, which should be
* the newest picture.
*/
FileObserver fileObserver = new FileObserver(path) {
#Override
public void onEvent(int event, String path) {
OLog.d(TAG, event + "" + path);
if (event == FileObserver.CREATE && !path.equals(".probe")) {
OLog.d(TAG, "File Created ["
+ Environment.getExternalStorageState()
+ "Screenshots"
+ path
+ "]");
screenshots[screenshots.length - 1].delete();
}
}
};
fileObserver.startWatching();
I had asked a few of the other programmers I work with if they saw anything out of place, looked at numerous other SO articles and still wasn't sure why it wasn't working as intended.
It is unclear exactly what you are intending. Here are two oddities that I see:
Your FileObserver will randomly stop observing files, as it will be eligible for garbage collection as soon as deleteMostRecentScreenshot() returns. And, as the documentation points out, "If a FileObserver is garbage collected, it will stop sending events. To ensure you keep receiving events, you must keep a reference to the FileObserver instance from some other live object."
Your statement that "the last file in the array... should be the newest picture" is incorrect. At best, it will have been the most-recently-modified file as of the time when you called listFiles(). Changes that occur after listFiles() is called will not affect your screenshots array. Since your FileObserver is given the path to the file, you might consider just using that path.
When another application is sending a file to my app, I get a Uri via the intent.getExtras().get(EXTRA_STREAM) property. I can then get the bytes of the file using an inputstream : new BufferedInputStream(activity.getContentResolver().openInputStream(uri));
Everything's OK and working so far. Now I'd like to show some kind of progress to my user, but I'm not sure of how to get the total number of bytes of the file without reading the stream completely beforehand (which would defeat the whole purpose of the progress bar) ...
I tried ParcelFileDescriptor fileDesc = activity.getContentResolver().openFileDescriptor(uri, "r"); but this only works with uris of type file://....
For example If I receive a file from Skydrive I get a content://.. Uri, as in : content://com.microsoft.skydrive.content.external/external_property/10C32CC94ECB90C4!155/Sunset.480p.mp4
On such Uri I get (unsurprisingly) a "FileNotFoundException : Not a whole file" exception.
Any sure fire way to get the total size of the stream of data I will get ?
Even though InputStream.available() is (almost) never a recommended way of getting file size, it might be a viable solution in your case.
The content is already available locally. A server is not involved. So, the following should return the exact file size:
try {
InputStream inputStream = getContentResolver().openInputStream(uri);
Log.i("TEST", "File Size: " + inputStream.available());
} catch (FileNotFoundException fnfe) {
fnfe.printStackTrace();
} catch (IOException ioe) {
ioe.printStackTrace();
}
I tested this with SkyDrive and Dropbox. The file sizes returned were correct.
There is no general solution for getting the size of a stream, other than reading the entire stream. This is easily proven: One could create a web server that, for some URL, generates a random stream of text that is terminated at a random time. (In fact, I'm sure such URLs exist, whether by design or not :-) In such a case, the size of the stream isn't known until the last byte has been generated, never mind received.
So, the stream size, if it is sent by the server at all, has to be sent in an application-specific manner.
I've never worked with SkyDrive, but a google search for its API turned up this link, which has the following example for Android Java apps:
public void readFile() {
String fileId = "file.a6b2a7e8f2515e5e.A6B2A7E8F2515E5E!141";
client.getAsync(fileId, new LiveOperationListener() {
public void onError(LiveOperationException exception, LiveOperation operation) {
resultTextView.setText("Error reading file: " + exception.getMessage());
}
public void onComplete(LiveOperation operation) {
JSONObject result = operation.getResult();
String text = "File info:" +
"\nID = " + result.optString("id") +
"\nName = " + result.optString("name");
resultTextView.setText(text);
}
});
}
Based on other examples on that page, I would guess that something like result.optString("size") (or maybe result.optInt("size") ?) would give you the size of the file.
I am planning to automate the testing of an application by creating a log to store some results of execution of the app and latter on parse it using a piece of python code and plot a graph.
The application is a WiFi fingerprinter i.e it collects info such as mac id, rss(recieved signal strength and rank(normalized rss) about the wifi devices in the surrounding environment. So to test this application I would have to take it to the location and record the results(as of now manually). So logcat wouldn't serve the purpose.
Automation requires
1. Storing the log in the device
2. Access to the log file in the system through usb
Format of the Log file:
Snapshot: 1
Fingerprint: 1, Rank: 0.23424, Boolean: true
Fingerprint: 2, Rank: 0.42344, Boolean: false
Fingerprint: 3, Rank: 0.23425, Boolean: true
Snapshot: 2
Fingerprint: 1, Rank: 0.75654, Boolean: false
Fingerprint: 2, Rank: 0.23456, Boolean: true
Fingerprint: 3, Rank: 0.89423, Boolean: true
................
Now I know there are basically 3 approaches for persistent storage(SharedPrefs wouldn't suit this scenario anyway). I tried Internal Storage, but even after setting the mode of the file as MODE_WORLD_READABLE it was impossible to read the file using Device File Explorer in Eclipse.
I am still wary of using external storage for storing the log. Any tutorial on how to write to a file in usb of the device will definitely help.
I thought of structuring the data to be stored so as to use SQLite for storage. But this establishing many unnecessary relations(foreign and domestic) between data and make it complex. If there is no way around, then here be dragons.
Basically I want to write to a file(easier I suppose) in the device and latter on read it in my system by connecting to it via usb. Any help on how to do it would be much appreciated.
Wary or not, External Storage still may be the only way to go. Without root access on the device, you can't really get at anything "Internal" unless you're going to be okay with reading within an application on the device. The docs provide pretty solid guidelines for where to create external files, and if you are using API Level 8 or higher, there are a couple of extra functions that can be used. I'm sure you know this page, but here it is anyway: http://developer.android.com/guide/topics/data/data-storage.html#filesExternal
If you're in need of any file io example code... I think I could dig some up...
EDIT - I would start by following the guidelines in the above docs to first confirm the state of the storage. I unfortunately don't have any experience with appending a file in Java, so someone else would definitely be more qualified to answer. This doesn't cover appending, but I have a backup routine in one of my personal apps that looks something like this.
File backupPath = Environment.getExternalStorageDirectory();
backupPath = new File(backupPath.getPath() + "/Android/data/com.maximusdev.bankrecord/files");
if(!backupPath.exists()){
backupPath.mkdirs();
}
FileOutputStream fos;
try {
fos = new FileOutputStream(backupPath.getPath() + "/recordsbackup.txt");
if(okaytowrite){
for(int i = 0; i < count; ++i){
entry = adapter.getItem(i);
fos.write(entry.toString().getBytes());
fos.write("\n".getBytes());
fos.write(String.valueOf(entry.dateTime).getBytes());
fos.write("\n".getBytes());
fos.write(String.valueOf(entry.sign).getBytes());
fos.write("\n".getBytes());
fos.write(String.valueOf(entry.cleared).getBytes());
fos.write("\n".getBytes());
fos.write(String.valueOf(entry.transDate).getBytes());
fos.write("\n".getBytes());
fos.write(entry.category.getBytes());
fos.write("\n".getBytes());
}
}
fos.close();
Toast.makeText(this, "Backup Complete", Toast.LENGTH_SHORT).show();
} catch (FileNotFoundException e) {
e.printStackTrace();
AlertDialog.Builder delmessagebuilder = new AlertDialog.Builder(this);
delmessagebuilder.setCancelable(false);
delmessagebuilder.setMessage("File Access Error");
delmessagebuilder.setNeutralButton("Okay", new DialogInterface.OnClickListener() {
public void onClick(DialogInterface dialog, int id) {
dialog.dismiss();
}
});
delmessagebuilder.create().show();
} catch (IOException e) {
e.printStackTrace();
AlertDialog.Builder delmessagebuilder = new AlertDialog.Builder(this);
delmessagebuilder.setCancelable(false);
delmessagebuilder.setMessage("File Access Error");
delmessagebuilder.setNeutralButton("Okay", new DialogInterface.OnClickListener() {
public void onClick(DialogInterface dialog, int id) {
dialog.dismiss();
}
});
delmessagebuilder.create().show();
}
Once I'm ready to write, I'm pulling a custom object (entry) out of an ArrayAdapter (adapter) and converting field valuse to strings and using getBytes() to pass to the FileOutputStream write function. I've done some research and there are quite a few other options for file writing in Java/Android... the FileWriter Class for instance, so it bears further research.
I used a very simple approach to write String messages to the log file by creating a FileWriter object.
public static BufferedWriter out;
private void createFileOnDevice(Boolean append) throws IOException {
/*
* Function to initially create the log file and it also writes the time of creation to file.
*/
File Root = Environment.getExternalStorageDirectory();
if(Root.canWrite()){
File LogFile = new File(Root, "Log.txt");
FileWriter LogWriter = new FileWriter(LogFile, append);
out = new BufferedWriter(LogWriter);
Date date = new Date();
out.write("Logged at" + String.valueOf(date.getHours() + ":" + date.getMinutes() + ":" + date.getSeconds() + "\n"));
out.close();
}
}
Now the function to write a new message to the log file.
public void writeToFile(String message){
try {
out.write(message+"\n");
out.close();
} catch (IOException e) {
e.printStackTrace();
}