Develop Reference

Is there a way to export realm database to CSV/JSON?

I want to export my realm database to CSV/JSON in Android. Is there some in-build method in the realm database which can do this?
There is a iOS way of converting realm to CSV link. I want a similar method in Android.

I was able to cobble together the following solution in my project:
// Grab all data from the DB in question (TaskDB):
RealmResults<TaskDB> resultsDB = realm.where(TaskDB.class).findAll();
// Here we need to put in header fields
String dataP = null;
String header = DataExport.grabHeader(realm, "TaskDB");
// We write the header to file
savBak(header);
// Now we write all the data corresponding to the fields grabbed above:
for (TaskDB taskitems: resultsDB) {
dataP = taskitems.toString();
// We process the data obtained and add commas and formatting:
dataP = dataProcess(dataP);
// Workaround to remove the last comma from final string
int total = dataP.length() - 1;
dataP = dataP.substring(0,total);
// We write the data to file
savBak(dataP);
}
I will explain what it is doing as best I can and include all corresponding code(all in reference to the first code block).
The first I did is grab the header using the following method I wrote in a separate class (DataExport.grabHeader). It takes 2 arguments: the realm object in question and the DB object model name:
public static String grabHeader(Realm realm, String model){
final RealmSchema schema = realm.getSchema();
final RealmObjectSchema testSchema = schema.get(model);
final String header = testSchema.getFieldNames().toString();
String dataProcessed = new String();
Pattern p = Pattern.compile("\\[(.*?)\\]");
Matcher m = p.matcher(header);
while(m.find()) {
dataProcessed += m.group(1).trim().replaceAll("\\p{Z}","");
}
return dataProcessed;
Within grabHeader, I apply some regex magic and spit out a string that will be used as the header with the appropriate commas in place (String dataProcessed).
In this scenario, after I obtained the data needed, I used another method (savBak) to write the information to a file which takes 1 string argument:
#Override
public void savBak(String data){
FileOutputStream fos = null;
try {
fos = openFileOutput(FILE_NAME, MODE_PRIVATE | MODE_APPEND);
fos.write(data.getBytes());
fos.write("\n".getBytes());
Log.d("tester", "saved to: " + getFilesDir() + "/" + FILE_NAME);
} catch (FileNotFoundException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
} finally {
if (fos != null) {
try {
fos.close();
} catch (IOException e) {
e.printStackTrace();
}
}
}
}
The "savBak" method writes the information to a FILE_NAME specified in a variable and we have our header information. After the header is written, we do the basically the same process with the DB using a forloop but I also had to include 2 lines to remove the trailing comma after the line was processed. Each line is appended to the file and viola, CSV formatted goodness.
From here, you can use other existing methods of converting CSV to JSON and whatever else as well as putting the information back into realm via JSON. When it comes to more advanced elements like primary keys and such, I am not sure but it worked for my particular project needs.
Please excuse any "bad code" practice as I'm new to Java/Android in general coming from a "barely intermediate" Python background so hopefully this makes sense.

I got a reply from Realm support via email.
Unfortunately, we do not have this feature yet. You can see it tracked here: https://github.com/realm/realm-java/issues/2880
You could use a dynamic API and write a script yourself to perform a similar feature.

Output an object to the Logcat console

I would like to take a look at an object, similar to a print_r(an_object) in php, or a console.log(an_object) in javascript (in browser), but for Android.
I tried this
public void a_method( SomeClass the_arg )
{
Log.d( "an_id" , the_arg );
}
This generates an error message:
Error:(23, 10) error: no suitable method found for d(String,View)
method Log.d(String,String,Throwable) is not applicable
(actual and formal argument lists differ in length)
method Log.d(String,String) is not applicable
(actual argument View cannot be converted to String by method invocation conversion)

You cannot print an object to the console in Java as you would in javascript.
You have three solutions.
1) Use debugger. Place a breakpoint and debug in android studio. Then you can inspect the full object in scope.
2) Serialize your object (for example to JSON) and print the result to console.
3) Override toString method of your object to give you all the information you want and call Log.d("myTag", yourObj.toString())
I highly recommend first method. I used to avoid Debugger but learning how to use the debugger was the best thing I did. It increases your efficiency and makes debugging super easy

Convert object to JSON. You can use Gson.
val gson = Gson()
val json = gson.toJson(yourObject)
Log.e(TAG, json)
Or
Log.e(TAG, Gson().toJson(yourObject))

The second argument must be a String.
Log.d("an_id", String.valueOf(the_arg));
Your error says you can't log a View class
no suitable method found for d(String,View)
Don't be surprised when you see some nonsense in the console when you print that View object through using String.valueOf

It's a bit easy to output the data object in the logcat.
The correct way is to override the toString() inside the object.
It can be generated by the Android Studio itself by following this simple steps:
1- Open the DataClass
2- Press Alt+Insert OR right click and click on Generate...
3- In Generate window select toString()
4- Select all the variables in the next window and click OK
5- toString() method would override in your data class returning the String template of your data model.
Cheers!

just wrote one generic method, which makes it possible trough reflection:
#Override
public String toString() {
StringBuilder sb = new StringBuilder();
for (Field field : this.getClass().getDeclaredFields()) {
String item;
if(! field.getType().getSimpleName().equals("ArrayList")) {
try {
Object value = field.get(this);
item = String.format("%s %s %s: %s%n", Modifier.toString(field.getModifiers()), field.getType().getSimpleName(), field.getName(), String.valueOf(value));
sb.append(item);
} catch (IllegalAccessException e) {
e.printStackTrace();
}
} else {
item = String.format("%s %s %s: ArrayList<>%n", Modifier.toString(field.getModifiers()), field.getType().getSimpleName(), field.getName());
sb.append(item);
}
}
return sb.toString();
}

You can try using Gson class as shown below :
public void yourMethod(SomeClass theObject) {
Gson gson = new Gson();
Log.d( "sample" , gson.toJson(theObject));
}
Here is its Gradle repository
implementation 'com.google.code.gson:gson:2.8.6'

SharedPreferences in Android not persisted to disk when key contains newline

In Android, I'd like to write SharedPreferences key-value pairs where the keys are Base64 strings.
// get a SharedPreferences instance
SharedPreferences prefs = getSharedPreferences("some-name", Context.MODE_PRIVATE);
// generate the base64 key
String someKey = new String(Base64.encode("some-key".getBytes("UTF-8"), Base64.URL_SAFE), "UTF-8");
// write the value for the generated key
prefs.edit().putBoolean(someKey, true).commit();
In the last line, the call to commit returns true. So this key-value pair should have been saved successfully.
When I close and destroy the Activity where this piece of code was used and then re-create the Activity (running this code again), the specified value is returned for the key that we used.
But it turns out that, when I destroy the whole application/process (e.g. using "Force stop" in app settings), the value for our key is lost on the next launch of the Activity.
When I don't use Base64.URL_SAFE but Base64.URL_SAFE | Base64.NO_WRAP as the flags for the Base64 encoding, it works fine.
So this problem has been caused by the newlines at the end of the Base64 keys. Keys like abc can be written without any problems. But when the key is abc\n, it fails.
The problem is that it appears to work without problems first, returning true on commit() and returning the correct preference value on subsequent calls. But when the whole application is destroyed and re-started, the value has not been persisted.
Is this intended behaviour? A bug? Does the documentation say anything about valid key names?

I took a look at GrepCode and see that the operations will be the following (I do not mention useless ones) :
android.app.SharedPreferencesImpl.commit()
android.app.SharedPreferencesImpl.commitToMemory()
android.app.SharedPreferencesImpl.queueDiskWrite(MemoryCommitResult,Runnable)
3.1. XmlUtils.writeMapXml(Map, OutputStream)
3.2. XmlUtils.writeMapXml(Map, String, XmlSerializer)
3.3. XmlUtils.writeValueXml(Object v, String name, XmlSerializer ser)
First : how your data are converted ?
The method XmlUtils.writeValueXml writes the Object value in a XML tag with the attribute name set to the String value. This String value contains exactly the value you specified at the SharedPreference's name.
(And I confirmed this by doing a step-by-step debug with your piece of code).
The XML will be with an unescaped line break character. Actually, the XmlSerializer instance is a FastXmlSerializer instance and it does not escape the \n character (see the link for this class at the end if you want to read the source code)
Interesting piece of code :
writeValueXml(Object v, String name, XmlSerializer out) {
// -- "useless" code skipped
out.startTag(null, typeStr);
if (name != null) {
out.attribute(null, "name", name);
}
out.attribute(null, "value", v.toString());
out.endTag(null, typeStr);
// -- "useless" code skipped
}
Second : why the result is true ?
The commit method has the following code :
public boolean commit() {
MemoryCommitResult mcr = commitToMemory();
SharedPreferencesImpl.this.enqueueDiskWrite(
mcr, null /* sync write on this thread okay */);
try {
mcr.writtenToDiskLatch.await();
} catch (InterruptedException e) {
return false;
}
notifyListeners(mcr);
return mcr.writeToDiskResult;
}
So it returns the mcr.writeToDiskResult which is set in the SharedPreferencesImpl.writeToFile(MemoryCommitResult) method. Interesting piece of code :
writeToFile(MemoryCommitResult mcr) {
// -- "useless" code skipped
try {
FileOutputStream str = createFileOutputStream(mFile);
if (str == null) {
mcr.setDiskWriteResult(false);
return;
}
XmlUtils.writeMapXml(mcr.mapToWriteToDisk, str);
FileUtils.sync(str);
str.close();
ContextImpl.setFilePermissionsFromMode(mFile.getPath(), mMode, 0);
try {
final StructStat stat = Libcore.os.stat(mFile.getPath());
synchronized (this) {
mStatTimestamp = stat.st_mtime;
mStatSize = stat.st_size;
}
} catch (ErrnoException e) {
// Do nothing
}
// Writing was successful, delete the backup file if there is one.
mBackupFile.delete();
mcr.setDiskWriteResult(true);
return;
} catch (XmlPullParserException e) {
Log.w(TAG, "writeToFile: Got exception:", e);
} catch (IOException e) {
Log.w(TAG, "writeToFile: Got exception:", e);
}
// -- "useless" code skipped
}
As we see at the previous point : the XML writing is "ok" (do not throw anything, do not fails), so the sync in the file will be too (just a copy of a Stream in another one, nothing checks the XML content here !).
Currently : your key was converted to (badly formatted) XML and correctly wrote in a File. The result of the whole operation is true as everything went OK. Your changes are comitted to the disk and in the memory.
Third and last : why do I get back the correct value the first time and a bad one the second time
Take a quick look to what happen when we commit the changes to memory in SharedPreferences.Editor.commitToMemory(...) method (interesting part only... :)):
for (Map.Entry<String, Object> e : mModified.entrySet()) {
String k = e.getKey();
Object v = e.getValue();
if (v == this) { // magic value for a removal mutation
if (!mMap.containsKey(k)) {
continue;
}
mMap.remove(k);
} else {
boolean isSame = false;
if (mMap.containsKey(k)) {
Object existingValue = mMap.get(k);
if (existingValue != null && existingValue.equals(v)) {
continue;
}
}
mMap.put(k, v);
}
mcr.changesMade = true;
if (hasListeners) {
mcr.keysModified.add(k);
}
}
Important point : the changes are commited to the mMap attribute.
Then, take a quick look to how we get back a value :
public boolean getBoolean(String key, boolean defValue) {
synchronized (this) {
awaitLoadedLocked();
Boolean v = (Boolean)mMap.get(key);
return v != null ? v : defValue;
}
}
We are taking back the key from mMap (no reading of the value in the file for now). So we have the correct value for this time :)
When you reload your application, you will load the data back from the disk, and so the SharedPreferencesImpl constructor will be called, and it will call the SharedPreferencesImpl.loadFromDiskLocked() method. This method will read the file content and load it in the mMap attribute (I let you see the code by yourself, link provided at the end).
A step-by-step debug shown me that the abc\n was written as abc (with a whitespace character). So, when you will try to get it back, you will never succeed.
To finish, thank you to #CommonsWare to give me a hint about the file content in the comment :)
Links
XmlUtils
FastXmlSerializer
SharedPreferencesImpl
SharedPreferencesImpl.EditorImpl.commit()
SharedPreferencesImpl.EditorImpl.commitToMemory()
SharedPreferencesImpl.enqueueDiskWrite(MemoryCommitResult, Runnable)
SharedPreferencesImpl.writeToFile(MemoryCommitResult)
SharedPreferencesImpl.loadFromDiskLocked()

Not finding local data files saved in my application

The process seemed quite simplistic at first, but there must be something that I am missing going forward with this task. There was a settings file that I wanted to create local to my application for storing a whole bunch of data (not preference worthy). I ended up saving the file with the following code snippet.
protected File createSettingsFileLocation(String fileNameF)
{
File directoryFile = context_.getDir("settings", Context.MODE_PRIVATE);
File settingsFile;
settingsFile = new File(directoryFile, fileNameF);
if (!settingsFile.exists())
{
try
{
settingsFile.createNewFile();
} catch(IOException e)
{
Log.e(MyConstants.LOG_TAG, "Could not create the file as intended within internal storage.");
return null;
}
}
return settingsFile;
}
and then proceeded to retrieve the file later by looking for it locally with the following code snippets.
public String getCurrentFileContainingSettings()
{
List<String >settingFilesInFolder = getLocalStorageFileNames();
if (settingFilesInFolder == null || settingFilesInFolder.isEmpty())
{
return null;
}
String pathToCurrentSettingsFile = settingFilesInFolder.get(0);
return pathToCurrentSettingsFile;
}
protected List<String> getLocalStorageFileNames()
{
return Arrays.asList(context_.fileList());
}
However, the settingFilesInFolder always returns no entries, so I get null back from the getCurrentFileContainingSettings(). As what I could see from the documentation it seems as thought I was doing it right. But, I must be missing something, so I was hoping that someone could point something out to me. I could potentially hard-code the file name once it has been created within the system in a preference file for access later the first time that the settings are created, but I shouldn't have to do something like that I would think.

fileList() only looks in getFilesDir(), not in its subdirectories, such as the one you created via getDir(). Use standard Java file I/O (e.g., list()) instead.

Using java.util.regex in Android apps - are there issues with this?

In an Android app I have a utility class that I use to parse strings for 2 regEx's. I compile the 2 patterns in a static initializer so they only get compiled once, then activities can use the parsing methods statically.
This works fine except that the first time the class is accessed and loaded, and the static initializer compiles the pattern, the UI hangs for close to a MINUTE while it compiles the pattern! After the first time, it flies on all subsequent calls to parseString().
My regEx that I am using is rather large - 847 characters, but in a normal java webapp this is lightning fast. I am testing this so far only in the emulator with a 1.5 AVD.
Could this just be an emulator issue or is there some other reason that this pattern is taking so long to compile?
private static final String exp1 = "(insertratherlong---847character--regexhere)";
private static Pattern regex1 = null;
private static final String newLineAndTagsExp = "[<>\\s]";
private static Pattern regexNewLineAndTags = null;
static {
regex1 = Pattern.compile(exp1, Pattern.CASE_INSENSITIVE);
regexNewLineAndTags = Pattern.compile(newLineAndTagsExp);
}
public static String parseString(CharSequence inputStr) {
String replacementStr = "replaceMentText";
String resultString = "none";
try {
Matcher regexMatcher = regex1.matcher(inputStr);
try {
resultString = regexMatcher.replaceAll(replacementStr);
} catch (IllegalArgumentException ex) {
} catch (IndexOutOfBoundsException ex) {
}
} catch (PatternSyntaxException ex) {
}
return resultString;
}

please file a reproduceable test case at http://code.google.com/p/android/issues/entry and i'll have a look. note that i will need a regular expression that reproduces the problem. (our regular expressions are implemented by ICU4C, so the compilation actually happens in native code and this may end up being an ICU bug, but if you file an Android bug i'll worry about upstream.)

If you launched with debugging you can expect it to be about twice as slow as a regular launch. However a minute does seem extraordinary. Some things to suggest, i. look at the console output to see if warnings are being spat out, ii. when it is doing the compile, in the debugger press 'pause' and just see what it is doing. There are ways to get the source, but even so just looking at the call stack may reveal something.