I want to export my realm database to CSV/JSON in Android. Is there some in-build method in the realm database which can do this?
There is a iOS way of converting realm to CSV link. I want a similar method in Android.
I was able to cobble together the following solution in my project:
// Grab all data from the DB in question (TaskDB):
RealmResults<TaskDB> resultsDB = realm.where(TaskDB.class).findAll();
// Here we need to put in header fields
String dataP = null;
String header = DataExport.grabHeader(realm, "TaskDB");
// We write the header to file
savBak(header);
// Now we write all the data corresponding to the fields grabbed above:
for (TaskDB taskitems: resultsDB) {
dataP = taskitems.toString();
// We process the data obtained and add commas and formatting:
dataP = dataProcess(dataP);
// Workaround to remove the last comma from final string
int total = dataP.length() - 1;
dataP = dataP.substring(0,total);
// We write the data to file
savBak(dataP);
}
I will explain what it is doing as best I can and include all corresponding code(all in reference to the first code block).
The first I did is grab the header using the following method I wrote in a separate class (DataExport.grabHeader). It takes 2 arguments: the realm object in question and the DB object model name:
public static String grabHeader(Realm realm, String model){
final RealmSchema schema = realm.getSchema();
final RealmObjectSchema testSchema = schema.get(model);
final String header = testSchema.getFieldNames().toString();
String dataProcessed = new String();
Pattern p = Pattern.compile("\\[(.*?)\\]");
Matcher m = p.matcher(header);
while(m.find()) {
dataProcessed += m.group(1).trim().replaceAll("\\p{Z}","");
}
return dataProcessed;
Within grabHeader, I apply some regex magic and spit out a string that will be used as the header with the appropriate commas in place (String dataProcessed).
In this scenario, after I obtained the data needed, I used another method (savBak) to write the information to a file which takes 1 string argument:
#Override
public void savBak(String data){
FileOutputStream fos = null;
try {
fos = openFileOutput(FILE_NAME, MODE_PRIVATE | MODE_APPEND);
fos.write(data.getBytes());
fos.write("\n".getBytes());
Log.d("tester", "saved to: " + getFilesDir() + "/" + FILE_NAME);
} catch (FileNotFoundException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
} finally {
if (fos != null) {
try {
fos.close();
} catch (IOException e) {
e.printStackTrace();
}
}
}
}
The "savBak" method writes the information to a FILE_NAME specified in a variable and we have our header information. After the header is written, we do the basically the same process with the DB using a forloop but I also had to include 2 lines to remove the trailing comma after the line was processed. Each line is appended to the file and viola, CSV formatted goodness.
From here, you can use other existing methods of converting CSV to JSON and whatever else as well as putting the information back into realm via JSON. When it comes to more advanced elements like primary keys and such, I am not sure but it worked for my particular project needs.
Please excuse any "bad code" practice as I'm new to Java/Android in general coming from a "barely intermediate" Python background so hopefully this makes sense.
I got a reply from Realm support via email.
Unfortunately, we do not have this feature yet. You can see it tracked here: https://github.com/realm/realm-java/issues/2880
You could use a dynamic API and write a script yourself to perform a similar feature.
I have this method:
public static Object parseStringToObject(String json) {
String Object = json;
Gson gson = new Gson();
Object objects = gson.fromJson(object, Object.class);
parseConfigFromObjectToString(object);
return objects;
}
And I want to parse a JSON with:
public static void addObject(String IP, Object addObject) {
try {
String json = sendPostRequest("http://" + IP + ":3000/config/add_Object", ConfigJSONParser.parseConfigFromObjectToString(addObject));
addObject = ConfigJSONParser.parseStringToObject(json);
} catch (Exception ex) {
ex.printStackTrace();
}
}
But I get an error message:
com.google.gson.JsonSyntaxException: java.lang.IllegalStateException:
Expected BEGIN_OBJECT but was STRING at line 1 column 1
Even without seeing your JSON string you can tell from the error message that it is not the correct structure to be parsed into an instance of your class.
Gson is expecting your JSON string to begin with an object opening brace. e.g.
{
But the string you have passed to it starts with an open quotes
"
Invalid JSON from the server should always be an expected use case. A million things can go wrong during transmission. Gson is a bit tricky, because its error output will give you one problem, and the actual exception you catch will be of a different type.
With all that in mind, the proper fix on the client side is
try
{
gson.fromJSON(ad, Ad.class);
//...
}
catch (IllegalStateException | JsonSyntaxException exception)
{
//...
If you want to know why the JSON you received from the server is wrong, you can look inside your catch block at the exception. But even if it is your problem, it's not the client's responsibility to fix JSON it is receiving from the internet.
Either way, it is the client's responsibility to decide what to do when it gets bad JSON. Two possibilities are rejecting the JSON and doing nothing, and trying again.
If you are going to try again, I highly recommend setting a flag inside the try / catch block and then responding to that flag outside the try / catch block. Nested try / catch is likely how Gson got us into this mess with our stack trace and exceptions not matching up.
In other words, even though I'll admit it doesn't look very elegant, I would recommend
boolean failed = false;
try
{
gson.fromJSON(ad, Ad.class);
//...
}
catch (IllegalStateException | JsonSyntaxException exception)
{
failed = true;
//...
}
if (failed)
{
//...
I had a similar problem recently and found an interesting solution. Basically I needed to deserialize following nested JSON String into my POJO:
"{\"restaurant\":{\"id\":\"abc-012\",\"name\":\"good restaurant\",\"foodType\":\"American\",\"phoneNumber\":\"123-456-7890\",\"currency\":\"USD\",\"website\":\"website.com\",\"location\":{\"address\":{\"street\":\" Good Street\",\"city\":\"Good City\",\"state\":\"CA\",\"country\":\"USA\",\"postalCode\":\"12345\"},\"coordinates\":{\"latitude\":\"00.7904692\",\"longitude\":\"-000.4047208\"}},\"restaurantUser\":{\"firstName\":\"test\",\"lastName\":\"test\",\"email\":\"test#test.com\",\"title\":\"server\",\"phone\":\"0000000000\"}}}"
I ended up using regex to remove the open quotes from beginning and the end of JSON and then used apache.commons unescapeJava() method to unescape it. Basically passed the unclean JSON into following method to get back a cleansed one:
private String removeQuotesAndUnescape(String uncleanJson) {
String noQuotes = uncleanJson.replaceAll("^\"|\"$", "");
return StringEscapeUtils.unescapeJava(noQuotes);
}
then used Google GSON to parse it into my own Object:
MyObject myObject = new.Gson().fromJson(this.removeQuotesAndUnescape(uncleanJson));
In Retrofit2, When you want to send your parameters in raw you must use Scalars.
first add this in your gradle:
compile 'com.squareup.retrofit2:retrofit:2.3.0'
compile 'com.squareup.retrofit2:converter-gson:2.3.0'
compile 'com.squareup.retrofit2:converter-scalars:2.3.0'
public interface ApiInterface {
String URL_BASE = "http://10.157.102.22/rest/";
#Headers("Content-Type: application/json")
#POST("login")
Call<User> getUser(#Body String body);
}
my SampleActivity :
public class SampleActivity extends AppCompatActivity implements Callback<User> {
#Override
protected void onCreate(#Nullable Bundle savedInstanceState) {
super.onCreate(savedInstanceState);
setContentView(R.layout.activity_sample);
Retrofit retrofit = new Retrofit.Builder()
.baseUrl(ApiInterface.URL_BASE)
.addConverterFactory(ScalarsConverterFactory.create())
.addConverterFactory(GsonConverterFactory.create())
.build();
ApiInterface apiInterface = retrofit.create(ApiInterface.class);
// prepare call in Retrofit 2.0
try {
JSONObject paramObject = new JSONObject();
paramObject.put("email", "sample#gmail.com");
paramObject.put("pass", "4384984938943");
Call<User> userCall = apiInterface.getUser(paramObject.toString());
userCall.enqueue(this);
} catch (JSONException e) {
e.printStackTrace();
}
}
#Override
public void onResponse(Call<User> call, Response<User> response) {
}
#Override
public void onFailure(Call<User> call, Throwable t) {
}
}
Reference: [How to POST raw whole JSON in the body of a Retrofit request?
I have come to share an solution. The error happened to me after forcing the notbook to hang up. possible solution clean preject.
Maybe your JSON Object is right,but the response that you received is not your valid data.Just like when you connect the invalid WiFi,you may received a strange response < html>.....< /html> that GSON can not parse.
you may need to do some try..catch.. for this strange response to avoid crash.
Make sure you have DESERIALIZED objects like DATE/DATETIME etc. If you are directly sending JSON without deserializing it then it can cause this problem.
In my situation, I have a "model", consist of several String parameters, with the exception of one: it is byte array byte[].
Some code snippet:
String response = args[0].toString();
Gson gson = new Gson();
BaseModel responseModel = gson.fromJson(response, BaseModel.class);
The last line above is when the
java.lang.IllegalStateException: Expected BEGIN_OBJECT but was STRING at line 1 column
is triggered. Searching through the SO, I realised I need to have some form of Adapter to convert my BaseModel to and fro a JsonObject. Having mixed of String and byte[] in a model does complicate thing. Apparently, Gson don't really like the situation.
I end up making an Adapter to ensure byte[] is converted to Base64 format. Here is my Adapter class:
public class ByteArrayToBase64Adapter implements JsonSerializer<byte[]>, JsonDeserializer<byte[]> {
#Override
public byte[] deserialize(JsonElement json, Type typeOfT, JsonDeserializationContext context) throws JsonParseException {
return Base64.decode(json.getAsString(), Base64.NO_WRAP);
}
#Override
public JsonElement serialize(byte[] src, Type typeOfSrc, JsonSerializationContext context) {
return new JsonPrimitive(Base64.encodeToString(src, Base64.NO_WRAP));
}
}
To convert JSONObject to model, I used the following:
Gson customGson = new GsonBuilder().registerTypeHierarchyAdapter(byte[].class, new ByteArrayToBase64Adapter()).create();
BaseModel responseModel = customGson.fromJson(response, BaseModel.class);
Similarly, to convert the model to JSONObject, I used the following:
Gson customGson = new GsonBuilder().registerTypeHierarchyAdapter(byte[].class, new ByteArrayToBase64Adapter()).create();
String responseJSon = customGson.toJson(response);
What the code is doing is basically to push the intended class/object (in this case, byte[] class) through the Adapter whenever it is encountered during the convertion to/fro JSONObject.
Don't use jsonObject.toString on a JSON object.
In my case, I am Returning JSON Object as
{"data":"","message":"Attendance Saved
Successfully..!!!","status":"success"}
Resolved by changing it as
{"data":{},"message":"Attendance Saved
Successfully..!!!","status":"success"}
Here data is a sub JsonObject and it should starts from { not ""
Don't forget to convert your object into Json first using Gson()
val fromUserJson = Gson().toJson(notificationRequest.fromUser)
Then you can easily convert it back into an object using this awesome library
val fromUser = Gson().fromJson(fromUserJson, User::class.java)
if your json format and variables are okay then check your database queries...even if data is saved in db correctly the actual problem might be in there...recheck your queries and try again.. Hope it helps
I had a case where I read from a handwritten json file. The json is perfect. However, this error occurred. So I write from a java object to json file, then read from that json file. things are fine. I could not see any difference between the handwritten json and the one from java object. Tried beyondCompare it sees no difference.
I finally noticed the two file sizes are slightly different, and I used winHex tool and detected extra stuff.
So the solution for my situation is, make copy of the good json file, paste content into it and use.
In my case, my custom http-client didn't support the gzip encoding. I was sending the "Accept-Encoding: gzip" header, and so the response was sent back as a gzip string and couldn't be decoded.
The solution was to not send that header.
I was making a POST request with some parameters using Retrofit in Android
WHAT I FACED:
The error I was getting in Android Studio logcat:
java.lang.IllegalStateException: Expected BEGIN_OBJECT but was STRING
at line 2 column 1 path $
[but it was working fine with VOLLY library]
when I googled it...
you know[ Obviously json is expecting a OBJECT but...]
BUT when I changed my service to return a simple string [ like print_r("don't lose hope") ] or
Noting at all
It was getting printed fine in Postman
but in Android studio logcat, it was still SAME ERROR [
java.lang.IllegalStateException: Expected BEGIN_OBJECT but was STRING
at line 2 column 1 path $
]
Hold up now, I am sending a simple message or not sending anything in response and still studio is
telling me "...Expected BEGIN_OBJECT but was STRING..."
SOMETHING IS WRONG
On 4th day:
I finally stopped for looking "QUICK SOLUTIONS" and REALLY READ some stack overflow questions
and articles carefully.
WHAT I GOT:
Logging interceptor
It will show you whatever data comes from your server[even eco messages] which are not shown in
Andorid studios logcat,
that way you can FIND THE PROBLEM.
What I found is I was sending data with #Body like-
#Headers("Content-Type: application/json")
#POST("CreateNewPost")
Call<Resp> createNewPost(#Body ParaModel paraModel);
but no parameter was reaching to server, everything was null [I found using Logging interceptor]
then I simply searched an article "how to make POST request using Retrofit"
here's one
SOLUTION:
from here I changed my method to:
#POST("CreateNewPost")
#FormUrlEncoded
Call<Resp> createNewPost(
#Field("user_id") Integer user_id,
#Field("user_name") String user_name,
#Field("description") String description,
#Field("tags") String tags);
and everything was fine.
CONCLUSION:
I don't understand why Retrofit gave this error
java.lang.IllegalStateException: Expected BEGIN_OBJECT but was STRING
at line 2 column 1 path $
it doesn't make any sense at all.
So ALWAYS DEBUG in detail then find WHERE THINGS ARE LEAKING and then FIX.
This error solved for by replacing .toString method to .string on the response
toString => string (add in try{...code..}catche(IOException e))
below code is working for me
try {
MainModelResponse model;
Gson gson = new GsonBuilder().create();
if (response.code() == ConstantValues.SUCCESS_OK) {
model = gson.fromJson(response.body().string(), MainModelResponse.class);
} else {
model = gson.fromJson(response.errorBody().string(), MainModelResponse.class);
}
moduleData.postValue(model);
}catch (IllegalStateException | JsonSyntaxException | IOException exception){
exception.printStackTrace();
}
}
use a string begin & end with {}.
such as
final String jsStr = "{\"metric\":\"opentsdb_metric\",\"tags\":{\"testtag\":\"sunbotest\"},\"aggregateTags\":[],\"dps\":{\"1483399261\":18}}";
DataPoint dataPoint = new Gson().fromJson(jsStr, DataPoint.class);
this works for me.
In my case the object was all fine even the Json Validator was giving it a valid resposne but I was using Interface like this
#POST(NetworkConstants.REGISTER_USER)
Call<UserResponse> registerUser(
#Query("name") String name,
#Query("email") String email,
#Query("password") String password,
#Query("created_date") Long creationDate
);
Then I changed the code to
#FormUrlEncoded
#POST(NetworkConstants.REGISTER_USER)
Call<UserResponse> registerUser(
#Field("name") String name,
#Field("email") String email,
#Field("password") String password,
#Field("created_date") Long creationDate
);
And everything was resolved.
my problem not related to my codes
after copy some files from an other project got this issue
in the stack pointed to Gson library
in android studio 4.2.1 this problem not solved when I try file-> invalidate and restart
and
after restart in first time build got same error but in second build this problem solved
I don't understand why this happened
I was using an old version of retrofit library. So what I had to do was to change my code from this after upgrading it to com.squareup.retrofit2:retrofit:2.9.0:
#POST(AppConstants.UPLOAD_TRANSACTION_DETAIL)
fun postPremiumAppTransactionDetail(
#Query("name") planName:String,
#Query("amount") amount:String,
#Query("user_id") userId: String,
#Query("sub_id") planId: String,
#Query("folder") description:String,
#Query("payment_type") paymentType:String):
Call<TransactionResponseModel>
To this:
#FormUrlEncoded
#POST(AppConstants.UPLOAD_TRANSACTION_DETAIL)
fun postPremiumAppTransactionDetail(
#Field("name") planName:String,
#Field("amount") amount:String,
#Field("user_id") userId: String,
#Field("sub_id") planId: String,
#Field("folder") description:String,
#Field("payment_type") paymentType:String):
Call<TransactionResponseModel>
For me it turned out that I was trying to deserialize to an object that used java.time.ZonedDateTime for one of the properties. It worked as soon as I changed it to a java.util.Date instead.
I`m trying to convert the next JSON to object:
{
AccountName:"temnoi",
Parts:{
part-0:{
Name:"HOME",
UptimeSeconds:"2143943",
},
part-1:{
Name:"WORK",
UptimeSeconds:"2276958",
}
}
}
The problem is that Parts isn't an array so I don't have any idea how
to obtain them as List or any other data structure.
For now I have such DTO class
public class Info {
private String AccountName;
private List<Parts> Parts;
}
But obviously program crash as there are no array. I use Retrofit2 with GsonConverter.
Can anyone suggest something to solve this problem?
Unfortunately, as I don't have a lot of time, I came with the next solution.
I replace Retrofit2 by OkHTTP and Gson with built-in JSON parser.
After I get a response with OkHttpClient I manually convert JSON to my object.
JSONObject root = new JSONObject(responseFromServer);
JSONObject parts = root.getJSONObject("Parts");
Iterator<String> jsonPartsIterator = parts.keys();
List<Part> partsList = new ArrayList<>();
while (jsonPartsIterator.hasNext()) {
try{
String key = jsonRootIterator.next();
partsList.add(convertPartJsonToObject(computers.getJSONObject(key)));
} catch(Exception e){
// in case if there will be number '0' return empty List
}
}
Here Part convertPartJsonToObject(JSONObject object) is method to convert part-0, part-1... to object which I need.
Normally I use Realm as:
RealmConfiguration config = new RealmConfiguration.Builder(applicationContext).deleteRealmIfMigrationNeeded().build();
How can I add to the assets folder of my project a database with data and read it?
Since Realm Java 0.91.0 there has been an assetFile(String) option on the RealmConfiguration that automatically will copy a file from assets and use that if needed (e.g. if the Realm is opened the first time or has been deleted for some reason):
RealmConfiguration config = new RealmConfiguration.Builder()
.assetFile("path/to/file/in/assets") // e.g "default.realm" or "lib/data.realm"
.deleteRealmIfMigrationNeeded()
.build()
The above will copy the file from assets the first time the Realm is opened or if it has been deleted due to migrations (remember to update the asset Realm in that case).
OLD ANSWER:
It is possible to bundle a Realm database in the assets folder, but then you just need to copy it from there when starting the app the first time.
We have an example of how to copy the files here: https://github.com/realm/realm-java/blob/master/examples/migrationExample/src/main/java/io/realm/examples/realmmigrationexample/MigrationExampleActivity.java#L101-Lundefined
copyBundledRealmFile(this.getResources().openRawResource(R.raw.default_realm), "default.realm");
private String copyBundledRealmFile(InputStream inputStream, String outFileName) {
try {
File file = new File(this.getFilesDir(), outFileName);
FileOutputStream outputStream = new FileOutputStream(file);
byte[] buf = new byte[1024];
int bytesRead;
while ((bytesRead = inputStream.read(buf)) > 0) {
outputStream.write(buf, 0, bytesRead);
}
outputStream.close();
return file.getAbsolutePath();
} catch (IOException e) {
e.printStackTrace();
}
return null;
}
Since Realm 0.89.0 RealmConfiguration.initialData(Realm.Transaction) can now be used to populate a Realm file before it is used for the first time.
RealmConfiguration conf = new RealmConfiguration.Builder(context)
.initialData(new Realm.Transaction() {
#Override
public void execute(Realm realm) {
realm.beginTransaction();
realm.createObject(....)
realm.commitTransaction();
}
}).deleteRealmIfMigrationNeeded().name("mRealm.db").build();
Realm realm = Realm.getInstance(conf);
[EDIT] See Stan's answer below. Apparently Realm now supports this directly so you can ignore this answer (unless you're using older Realm versions).
We had a similar need, and also wanted support for a read-only realm database shared with an iOS version of the app.
We created a simple library and have open-sourced it. It includes the copy code given in #christian-melchior's answer, as well as some optional extra tracking for read-only realm database(s) bundled with the APK. Comments and PRs welcomed. See:
https://github.com/eggheadgames/android-realm-asset-helper
Realm has a special parameter in its RealmConfiguration.Builder called assetFile. You could use it like:
realmConfiguration = new RealmConfiguration.Builder()
.assetFile("dataBase/default.realm") // your app's packaged DB
...
.build();
just set yer assets DB path and file name and you are good to go without any android-realm-asset-helper lib or copy-file-from-assets code. In this example my app packaged DB-file lies in "assets/dataBase/default.realm".Note, version below 2 has a bit another way to call assetFile, you should pass context additionally:
realmConfiguration = new RealmConfiguration.Builder(context)
.assetFile(context, "dataBase/default.realm")
.build();
You can use assetFile() method. Please be aware that you can't use assetFile() with deleteIfMigrationNeeded().