Develop Reference

Is there a way to export realm database to CSV/JSON?

I want to export my realm database to CSV/JSON in Android. Is there some in-build method in the realm database which can do this?
There is a iOS way of converting realm to CSV link. I want a similar method in Android.

I was able to cobble together the following solution in my project:
// Grab all data from the DB in question (TaskDB):
RealmResults<TaskDB> resultsDB = realm.where(TaskDB.class).findAll();
// Here we need to put in header fields
String dataP = null;
String header = DataExport.grabHeader(realm, "TaskDB");
// We write the header to file
savBak(header);
// Now we write all the data corresponding to the fields grabbed above:
for (TaskDB taskitems: resultsDB) {
dataP = taskitems.toString();
// We process the data obtained and add commas and formatting:
dataP = dataProcess(dataP);
// Workaround to remove the last comma from final string
int total = dataP.length() - 1;
dataP = dataP.substring(0,total);
// We write the data to file
savBak(dataP);
}
I will explain what it is doing as best I can and include all corresponding code(all in reference to the first code block).
The first I did is grab the header using the following method I wrote in a separate class (DataExport.grabHeader). It takes 2 arguments: the realm object in question and the DB object model name:
public static String grabHeader(Realm realm, String model){
final RealmSchema schema = realm.getSchema();
final RealmObjectSchema testSchema = schema.get(model);
final String header = testSchema.getFieldNames().toString();
String dataProcessed = new String();
Pattern p = Pattern.compile("\\[(.*?)\\]");
Matcher m = p.matcher(header);
while(m.find()) {
dataProcessed += m.group(1).trim().replaceAll("\\p{Z}","");
}
return dataProcessed;
Within grabHeader, I apply some regex magic and spit out a string that will be used as the header with the appropriate commas in place (String dataProcessed).
In this scenario, after I obtained the data needed, I used another method (savBak) to write the information to a file which takes 1 string argument:
#Override
public void savBak(String data){
FileOutputStream fos = null;
try {
fos = openFileOutput(FILE_NAME, MODE_PRIVATE | MODE_APPEND);
fos.write(data.getBytes());
fos.write("\n".getBytes());
Log.d("tester", "saved to: " + getFilesDir() + "/" + FILE_NAME);
} catch (FileNotFoundException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
} finally {
if (fos != null) {
try {
fos.close();
} catch (IOException e) {
e.printStackTrace();
}
}
}
}
The "savBak" method writes the information to a FILE_NAME specified in a variable and we have our header information. After the header is written, we do the basically the same process with the DB using a forloop but I also had to include 2 lines to remove the trailing comma after the line was processed. Each line is appended to the file and viola, CSV formatted goodness.
From here, you can use other existing methods of converting CSV to JSON and whatever else as well as putting the information back into realm via JSON. When it comes to more advanced elements like primary keys and such, I am not sure but it worked for my particular project needs.
Please excuse any "bad code" practice as I'm new to Java/Android in general coming from a "barely intermediate" Python background so hopefully this makes sense.

I got a reply from Realm support via email.
Unfortunately, we do not have this feature yet. You can see it tracked here: https://github.com/realm/realm-java/issues/2880
You could use a dynamic API and write a script yourself to perform a similar feature.

Copy image to resources Xamarin

I have a Xamarin Forms project in which I need the user to be able to "load" an image. I can already press a button and search for a file using FilePicker like this:
async void OnUpload(object sender, EventArgs e)
{
try
{
FileData filedata = await CrossFilePicker.Current.PickFile();
// the dataarray of the file will be found in filedata.DataArray
// file name will be found in filedata.FileName;
//etc etc.
}
catch (Exception ex)
{
}
}
What I would need now is to copy that "filedata" (the image) to the resource folder of the project in order to access to the file easily. I have tried:
await CrossFilePicker.Current.SaveFile(filedata.FileName);
but it doesn't save any file into the project folder.
Moreover, I only need it to work on UWP and Android.

The SaveFile method saves it in a very specific folder.
If you want to save it somewhere of your choosing you have to implement it with the DependencyService. IO operations are very specific to the OS, so are the filepaths. I will give you a simple example for you to build on.
Start with defining an interface in your shared code, like so:
public interface IFileManager
{
void SaveFile(Stream stream);
}
Of course, it can have other methods as well, or extra parameters if you would like to specify things like filename, that is up to you. You would also probably like some kind of return value to know what happened.
Now, per platform implement this interface. For example for Android, it could look like this:
[assembly: Xamarin.Forms.Dependency (typeof (FileManager_Android))]
public class FileManager_Android : IFileManager
{
public void SaveFile(Stream stream)
{
var dir = Android.OS.Environment.GetExternalStoragePublicDirectory(Android.OS.Environment.DirectoryDcim);
        string filename = System.DateTime.Now.ToString("yyyyMMddHHmmssfff") + ".jpg";
string filePath = System.IO.Path.Combine(dir, name);
try
        {
         System.IO.File.WriteAllBytes(filePath, imageData);
}
catch (Exception ex)
{
System.Console.WriteLine(e.ToString());
}
}
}
(Saving code inspired by this link)
This will take the stream and save it to a path of your choosing.
For UWP you will need to implement it as well, which is quite similar, except for the implementation of SaveFile. As far as I know there is no plugin yet which makes this easier for you. There is PCLStorage, but this plugin only seems to work with text files. You could still look into it for inspiration though.

In Unity, how can I populate this allRoundData array from a json file in a way that works on Android?

I am new to Unity and Android development, but please do not mark this as a duplicate - I've looked all over the stacks for an answer to this, but similar topics and questions haven't yielded any working solutions, or are wanting on details, or outdated, or seem not to fit my specific need.
Ok, I have a quiz app built from following part 1 & 2 of this tutorial. That link contains all the source files anyone might need for reference, and everything works fine on ios and in the unity editor, fyi.
The trouble is with the loadGameData function from the DataController script in Android. Again, everything works fine in iOS and the unity editor, but when I make an Android sdk, the quiz is blank and the console says the data couldn't be loaded.
Here is how the function is currently written (full file in tutorial link):
private void LoadGameData ()
{
string filePath = Path.Combine (Application.streamingAssetsPath, gameDataFileName);
if (File.Exists (result))
{
string dataAsJson = File.ReadAllText (result);
GameData loadedData = JsonUtility.FromJson<GameData> (dataAsJson);
allRoundData = loadedData.allRoundData;
} // #if(File.Exists...
else
{
Debug.LogError ("Cannot load game data!");
} // #else
} // #LoadGameData
If you check the same tutorial on youtube, you'll see lots of people have noted the same problem with the Android build and have been left unanswered. Same goes with unity forums - that's one reason why I don't think this question is a duplicate and could be helpful to others in a similar situation.
I've found that Android has always been sorta tricky with this and that there used to different ways of accessing a file based on platform, but these days "Application.streamingAssetsPath" should find the streaming assets directory on any platform, even Android.
What I've also learned is that in android, even if the path is correct, the file is compressed and will only return a url. So the url needs to be converted using unity's WWW class. I get that, but as of yet, I haven't been able to re-write my loadGameData function to work properly and load the allRoundData array.
Here's an example of some things I've tried:
IEnumerator androidData()
{
string filePath = Path.Combine (Application.streamingAssetsPath, gameDataFileName);
if (filePath.Contains("://"))
{
WWW www = new WWW(filePath);
yield return www;
result = www.text;
} // #if(filePath.Contains
} // #androidData
private void LoadGameData ()
{
androidData();
if (File.Exists (result))
{
string dataAsJson = File.ReadAllText (result);
GameData loadedData = JsonUtility.FromJson<GameData> (dataAsJson);
allRoundData = loadedData.allRoundData;
} // #if(File.Exists...
else
{
Debug.LogError ("Cannot load game data!");
} // #else
} // #LoadGameData
I know I'm close, and this is probably simple -- but I just can't seem to get to the finish line on this. Can some one help me figure out how to write the loadGameData function so it will load this allRoundData array on android?
An example code would be awesome and much appreciated, not just by me, but I'm sure many others would appreciate it also - Thank you!
UPDATE:
Based on the first answer, I've tested some code that works on the unity editor, but crashes in Android. In the Unity editor, I get the "file already exists" message. Here is the code I've tested:
Already had: private string gameDataFileName = "data.json";
I added the copyFile call above loadGameDate in Start() and wrote the copy file and loadGameData functions like so ..
int copyFileToPersistentDataPath(string gameDataFileName)
{
string persistentPath = Application.persistentDataPath + "/" + gameDataFileName;
try
{
//Copy only if gameDataFileName does not exist
if (!System.IO.File.Exists(persistentPath))
{
string path = System.IO.Path.Combine(Application.streamingAssetsPath, gameDataFileName);
WWW www = new WWW(path);
while (!www.isDone) { }
System.IO.File.WriteAllBytes(persistentPath, www.bytes);
Debug.Log(gameDataFileName + " Successfully Copied File to " + persistentPath);
return 1;
}
else
{
Debug.Log(gameDataFileName + " File already exist here. There is no need to copy it again");
return 0;
}
}
catch (Exception e)
{
Debug.Log(gameDataFileName + " Failed To Copy File. Reason: " + e.Message);
return -1;
}
}
private void LoadGameData ()
{
string tempPath = Path.Combine(Application.persistentDataPath, gameDataFileName);
string dataAsJson = File.ReadAllText(tempPath);
GameData loadedData = JsonUtility.FromJson<GameData>(dataAsJson);
allRoundData = loadedData.allRoundData;
} // #LoadGameData
This works with or without the call to copy the file in the editor, but crashes either way in Android.

I ended up putting the files in a Resources folder and going the resources.load a json file into a text asset and throw that into a string to parse route. I now have two different load functions, one that works in ios etc. and one that works in android. Here is the android function (the resourcesGameDataFile does not have the .json extension):
public void LoadDataForAndroid()
{
TextAsset dataFile = Resources.Load(resourcesGameDataFile) as TextAsset;
string dataAsJson = dataFile.ToString();
GameData loadedData = JsonUtility.FromJson<GameData>(dataAsJson);
allRoundData = loadedData.allRoundData;
Debug.Log ("Android data loaded with" + resourcesGameDataFile);
} // #LoadDataForAndroid
And this works in the unity editor and in Bluestacks (android simulator).

As for loading and saving games, this is a duplicate. I marked and remove this as a duplicate because the answer in the duplicated questions did not explain how to read from the StreamingAssets folder. It only talked about saving and loading data.
Make sure that Write Permission is set to External (SDCard).
The first thing to do is to copy the file from StreamingAssets to the persistentDataPath location.
I've found reading data from Application.streamingAssetsPath problematic but I use two methods to solve this.
int copyFileToPersistentDataPath(string fileNameWithExtensionName)
{
string persistentPath = Application.persistentDataPath + "/" + fileNameWithExtensionName;
try
{
//Copy only if fileNameWithExtensionName does not exist
if (!System.IO.File.Exists(persistentPath))
{
string path = System.IO.Path.Combine(Application.streamingAssetsPath, fileNameWithExtensionName);
WWW www = new WWW(path);
while (!www.isDone) { }
System.IO.File.WriteAllBytes(persistentPath, www.bytes);
Debug.Log(fileNameWithExtensionName + " Successfully Copied File to " + persistentPath);
return 1;
}
else
{
Debug.Log(fileNameWithExtensionName + " File already exist here. There is no need to copy it again");
return 0;
}
}
catch (Exception e)
{
Debug.Log(fileNameWithExtensionName + " Failed To Copy File. Reason: " + e.Message);
return -1;
}
}
If that does not work for you, use the method with WebClient below:
void copyFileToPersistentDataPath(string fileNameWithExtensionName)
{
string path = System.IO.Path.Combine(Application.streamingAssetsPath, fileNameWithExtensionName);
string persistentPath = Application.persistentDataPath + "/" + fileNameWithExtensionName;
Debug.Log("Dir: " + persistentPath);
WebClient webClient = new WebClient();
webClient.Proxy = null;
webClient.DownloadFileCompleted += new AsyncCompletedEventHandler(OnDownloadComplete);
webClient.DownloadProgressChanged += new DownloadProgressChangedEventHandler(OnUpdateDownloadProgress);
Uri uri = new Uri(path);
webClient.DownloadFileAsync(uri, persistentPath);
}
void OnDownloadComplete(object sender, AsyncCompletedEventArgs e)
{
Debug.Log("Finished Downloading: " + e.Error.Message);
}
void OnUpdateDownloadProgress(object sender, DownloadProgressChangedEventArgs e)
{
Debug.Log("Uploading Progreess: " + e.ProgressPercentage);
}
File Copy Usage:
copyFileToPersistentDataPath("yourFileName.txt");
After copying the file, you can then read and convert it to Json like this:
string fileNameWithExtensionName = "questionfile.txt";
string tempPath = Path.Combine(Application.persistentDataPath, fileNameWithExtensionName);
string dataAsJson = File.ReadAllText(fileNameWithExtensionName);
GameData loadedData = JsonUtility.FromJson<GameData>(dataAsJson);

Can't read lines from file by using StreamReader on Unity3d (Android)

I need to read a text stream by using StreamReader from file on android platform. File is about 100k lines, so even editor is getting stuck if i try to load it all to TextAsset or if i use WWW.
I simply need to read that file line by line without loading it all to a string. Then i'll do a tree generation from the lines that i got from the file. (But probably that part doesn't matter, i just need help on file reading part.)
I'm giving the code that i wrote down below. It works perfectly on editor, but fails on android.
I would be glad if anyone tell me, what am i missing.
(ps. english is not my native and this is my first question on the site. so sorry for the any mistakes that i may have done.)
private bool Load(string fileName)
{
try
{
string line;
string path = Application.streamingAssetsPath +"/";
StreamReader theReader = new StreamReader(path + fileName +".txt", Encoding.UTF8);
using (theReader)
{
{
line = theReader.ReadLine();
linesRead++;
if (line != null)
{
tree.AddWord(line);
}
}
while (line != null);
theReader.Close();
return true;
}
}
catch (IOException e)
{
Debug.Log("{0}\n" + e.Message);
exception = e.Message;
return false;
}
}

You can't use Application.streamingAssetsPath as a path on Android because streaming assets are stored within the JAR file with the application.
From http://docs.unity3d.com/Manual/StreamingAssets.html:
Note that on Android, the files are contained within a compressed .jar
file (which is essentially the same format as standard zip-compressed
files). This means that if you do not use Unity’s WWW class to
retrieve the file then you will need to use additional software to see
inside the .jar archive and obtain the file.
Use WWW like this in a coroutine:
WWW data = new WWW(Application.streamingAssetsPath + "/" + fileName);
yield return data;
if(string.IsNullOrEmpty(data.error))
{
content = data.text;
}
Or, if you really want to keep it simple (and your file is only a few 100k, stick it in a resource folder:
TextAsset txt = (TextAsset)Resources.Load(fileName, typeof(TextAsset));
string content = txt.text;

Copy the shared preferences XML file from /data on Samsung device failed

There's an exporting feature in my application. It's just a copy operation since all my settings are store in shared preference.
I just copy the xml file from /data/data/package.name/shared_prefs/settings.xml to SD card. It works fine on my HTC desire. However, it might not work on Samsung devices, and i got the following error while I try to copy the file.
I/System.out( 3166): /data/data/package.name/shared_prefs/settings.xml (No such file or directory)
in the directory.
Anyone know how to fix it, or is there another simple way to store the shared preference ?
Thanks.

Never never never never never never never never never hardwire paths.
Unfortunately, there's no getSharedPreferenceDir() anywhere that I can think of. The best solution I can think of will be:
new File(getFilesDir(), "../shared_prefs")
This way if a device manufacturer elects to change partition names, you are covered.
Try this and see if it helps.

CommonsWare's suggestion would a be clever hack, but unfortunately it won't work.
Samsung does not always put the shared_prefs directory in the same parent directory as the getFilesDir().
I'd recommend testing for the existence of (hardcode it, except for package name):
/dbdata/databases/<package_name>/shared_prefs/package.name_preferences.xml and if it exists use it, otherwise fall back to either CommonsWare's suggestion of new File(getFilesDir(), "../shared_prefs") or just /data/data/<package_name>/shared_prefs/package.name_preferences.xml.
A warning though that this method could potentially have problems if a user switched from a Samsung rom to a custom rom without wiping, as the /dbdata/databases file might be unused but still exist.
More details
On some Samsung devices, such as the Galaxy S series running froyo, the setup is this:
/data/data/<package_name>/(lib|files|databases)
Sometimes there's a shared_prefs there too, but it's just Samsung's attempt to confuse you! Don't trust it! (I think it can happen as a left over from a 2.1 upgrade to 2.2, but it might be a left over from users switching roms. I don't really know, I just have both included in my app's bug report interface and sometimes see both files).
And:
/dbdata/databases/<package_name>/shared_prefs
That's the real shared_prefs directory.
However on the Galaxy Tab on Froyo, it's weird. Generally you have: /data/data/<package_name>/(lib|shared_prefs|files|databases)
With no /dbdata/databases/<package_name> directory, but it seems the system apps do have:
/dbdata/databases/<package_name>/yourdatabase.db
And added bonus is that /dbdata/databases/<package_name> is not removed when your app is uninstalled. Good luck using SharedPreferences if the user ever reinstalls your app!

Try using
context.getFilesDir().getParentFile().getAbsolutePath()

Best way to get valid path on all devices - run method Context.getSharedPrefsFile defined as:
/**
* {#hide}
* Return the full path to the shared prefs file for the given prefs group name.
*
* <p>Note: this is not generally useful for applications, since they should
* not be directly accessing the file system.
*/
public abstract File getSharedPrefsFile(String name);
Because of it hidden need use reflection and use fallback on fail:
private File getSharedPrefsFile(String name) {
Context context = ...;
File file = null;
try {
if (Build.VERSION.SDK_INT >= 24) {
try {
Method m = context.getClass().getMethod("getSharedPreferencesPath", new Class[] {String.class});
file = (File)m.invoke(context, new Object[]{name});
} catch (Throwable e) {
Log.w("App TAG", "Failed call getSharedPreferencesPath", e);
}
}
if (file == null) {
Method m = context.getClass().getMethod("getSharedPrefsFile", new Class[] {String.class});
file = (File)m.invoke(context, new Object[]{name});
}
} catch (Throwable e) {
Log.w("App TAG", "Failed call getSharedPrefsFile", e);
file = new File(context.getFilesDir(), "../shared_prefs/" + name + ".xml");
}
return file;
}
On some Samsungs implements like this:
public File getSharedPrefsFile(String paramString) {
return makeFilename(getPreferencesDir(), paramString + ".xml");
}
private File getPreferencesDir() {
synchronized (this.mSync) {
if (this.mPreferencesDir == null) {
this.mPreferencesDir = new File("/dbdata/databases/" + getPackageName() + "/", "shared_prefs");
}
File localFile = this.mPreferencesDir;
return localFile;
}
}
On other Android like this:
public File getSharedPrefsFile(String name) {
return makeFilename(getPreferencesDir(), name + ".xml");
}
private File getPreferencesDir() {
synchronized (mSync) {
if (mPreferencesDir == null) {
mPreferencesDir = new File(getDataDirFile(), "shared_prefs");
}
return mPreferencesDir;
}
}
private File getDataDirFile() {
if (mPackageInfo != null) {
return mPackageInfo.getDataDirFile();
}
throw new RuntimeException("Not supported in system context");
}
After while Google change API for level 24 and later:
https://android.googlesource.com/platform/frameworks/base/+/6a6cdafaec56fcd793214678c7fcc52f0b860cfc%5E%21/core/java/android/app/ContextImpl.java

I've tested in Samsung P1010 with:
//I'm in a IntentService class
File file = this.getDir("shared_prefs", MODE_PRIVATE);
I got:
"/data/data/package.name/app_shared_prefs"
It works fine to me. I can run ffmpeg in this folder.
Look:
Context.getDir

You have to create the shared_prefs directory:
try{
String dir="/data/data/package.name/shared_prefs";
// Create one directory
boolean success = (new File(dir)).mkdirs();
if (success) {
// now copy the file
}
}catch (Exception e){//Catch exception if any
System.err.println("Error: " + e.getMessage());
}
Also... the package of your app is package.name? Make sure you are referring to the right package.