I'm currently working on an app that maintains a database on device that can be updated via Retrofit at a later date but I'm trying to design a solution that would allow me to cache the current database to put on device when publishing.
Currently this is what happens:
Sqlite database is manually created by hand
Sqlite database is put into assets folder in app
App is run, and database is converted to local database through SQLiteOpenHelper/SQLiteAssetHelper
At a later date, the device syncs with the back-end, grabbing a JSON file and manually updating the newly created database.
I'm trying to replace the first step with something more automated.
Is there a way, say I can create a Gradle task that would call my 'sync' code, grab that JSON, and some way to convert it to a Sqlite database to store within the assets folder?
Or perhaps a way to instead use the JSON directly when building the local database in step 3?
I would rather stay away from any other programs just to keep the project simple to use/update.
Thanks.
If you dont need to cache data page by page( it's mean ur local db save all data from server, and u just need to save current data) just save JSON as String, and on start, parse it again to show offline data, after that, fletch new data if connection is available.
I think you can go with creating the database directly using a JSON file. You can define a JSON file with multiple nodes each corresponding to a version of your database where every node contains all your DDL commands corresponding to that version of your database.
{
"patches": [
{
"version": 1,
"commands": [
"CREATE TABLE User ...",
"CREATE TABLE Login ..."
]
},
{
"version": 2,
"commands": [
"ALTER TABLE User ...",
"DROP TABLE Login ..."
]
}
]
}
This way you would only need to maintain a single JSON file and update that file for every update you publish for your app. If someone installs your app for the first time then all the DDL commands in your JSON file will be executed, otherwise only those commands will be executed that are corresponding to a higher database version than what is already present in the app before the update.
You would also have to update the onCreate and onUpgrade methods of your SQLiteOpenHelper. onCreate method will go through the entire json file and execute all the DDL commands in order(based on version). While onUpgrade will only execute the DDL commands under the nodes that have a higher database version.
#Override
public void onUpgrade(SQLiteDatabase sqLiteDatabase, int oldVersion, int newVersion) {
List<Patch> patches = parseJson(); // Parse the JSON file and get the patches
for (Patch patch : patches) {
int databaseVersion = patch.getVersion();
if (databaseVersion > oldVersion && databaseVersion <= newVersion) {
for (String command : patch.getCommands()) {
database.execSQL(command);
}
}
}
}
If your JSON tree structure will not change often, you can make a PoJo for it with a tool such as jsonschema2pojo. And convert instances using Gson.
You can store this PoJo using Google's new library: Room Persistence Library.
It can save the PoJo for you (You'll need to annotate it with #Entity) in sqlite database, and it can update you on changes in the database.
For more details you can watch the Google I/O presentations.
Related
When using SQlite database in my android app, I have two tables being created in onUpgrade method. On my test device, if I go to settings > apps > myApp > storage and press "Clear data" button, the next time I want to test the app, it crashes and complains that the two tables does not exist. I am fairly new to SQlite world, so any instruction would be appreciated.
Here is part of my DBHelper class:
#Override
public void onCreate(SQLiteDatabase sqLiteDatabase) {
sqLiteDatabase.execSQL(SQL_CREATE_MAPS_TABLE);
}
#Override
public void onUpgrade(SQLiteDatabase sqLiteDatabase, int oldVersion, int newVersion) {
if (oldVersion < 2) {
sqLiteDatabase.execSQL(SQL_CREATE_TRACK_RECORD_TABLE);
sqLiteDatabase.execSQL(SQL_CREATE_TRACK_POINT_TABLE);
}
}
If you delete the App's data, then the onUpgrade step will not run because the onCreate method runs, which will set the version to the coded version.
If you need to delete all existing data and change the schema then you code to create the new schenma (tables etc) should be in the onCreate method. The onUpgrade method is intended for introducing changes to an existing database.
Thank you very much Mike! I just don't understand the difference.
This is what happens when the database (a file) is opened (i.e. an attempt is made to access the database via an instance of the Database Helper) via a Database Helper (a class that extends SQLiteOpenHelper).
A check is made to see if the the database file (ie the database's name is the file name) exists at the given location (typically data/data/the_package_name/databases/the_database_name).
the_package_name and the_database_name will differe from app to app.
If the database file does not exist then create the file setting the user version (version) in the file's header to the version passed via the Database Helper (other things are done such as creating the sqlite_master table (done by the SQLite API) and adding the android_metadata table that store the locale). The file is now open so onCreate is called and a jump is made to step 4.
If the file does exist then get the user version stored in the file's header if it is not the same as the version number passed via the Database Helper then
if the version passed via the Database Helper is greater then the version extracted from the file's header then call onUpgrade passing the version as per the file's header and the version passed via the Database Helper and a jump is made to step 4.
if the version passsed via the Database Helper is less than the version extracted from the file's header then call onDowngrade passing the version as per the file's header and the version passed via the Database Helper and a jump is made to step 4.
The database has now been opened so return to the doing the user's request.
As such deleting the App's data takes the path 1 (check if db exists), 2 (create db, setting version and call onCreate) and then 4 (hand control back).
Hi I am developing an android app and I am using room DB and I have created one table and now I want to add another table I need to write a migration for that and in-migration I need to write full SQL queries for the new table assume I have more than 20 fields how complex will be the query.
In SQLite, we need to write such complex queries which sometimes becomes complicated to write and find errors So, room DB came to resue but now we need to do the same in the room DB migration. How does it useful then?
If we use SQLite, then if we already added one table and now We want to add another table then we can just uninstall the application and new tables will be generated, but in-room DB this is not a case I tried the same thing but it is still showing me that you need to write a migration for the new table. So, in this case, while developing there will a lot of migration scripts that will be hard to maintain at some point in the future.
How does it useful then I have to write a multiple create queries while developing the app this is a very basic flow in any application.
Once we go to prodution then it makes sense to write migration for every table but not in the developing mode.
How does room DB make developr job eaiser?
I have more than 20 fields how complex will be the query.
It can be very simple as an Entity defines an Object e.g. your 20 columns and to get the 20 columns can be as simple as
#Query(SELECT * FROM thetable)
List<Thetable> getAll();
The above being in an Interface that is annotated with #Dao and all you do in the code is retrieve an instance from the built room database and then use the getAll method which returns a List of Thetable objects. Each with all the member variables populated from the database.
e.g. you could have :-
mMyTheTableDaoObject = mMyBuiltRoomDatabase.getAll();
List<TheTable> myTheTableList = mMyTheTableDaoObject.getAll();
for(TheTable t: myTheTableList) {
int current???? = t.get????();
}
While using standard/non-room then you would have to do something along the lines of :-
SQLitedatabase db = whatever_you_need_to_do_to_get_an_SQLiteDatabase_instance;
Cursor c = db.query("theTable",null,null,null,null,null,null);
ArrayList<TheTable> myTheTableList = new ArrayList();
while(c.moveToNext()) {
currentTheTable = new TheTable();
current.TheTable.setId = c.getLong(c.getColumnIndex("id");
current.TheTable.setNextColumn1 = c.getString("next_column1");
current.TheTable.setNextColumn2 = c.getString("next_column2");
........ another 17 similar lines of code
currentTheTable.setNextColumn20 = c.getString("next_column20");
myTheTableList.add(currentTheTable);
}
for(TheTable t: myTheTableList) {
int current???? = t.get????();
}
If we use SQLite, then if we already added one table and now We want to add another table then we can just uninstall the application and new tables will be generated, but in-room DB this is not a case I tried the same thing but it is still showing me that you need to write a migration for the new table.
Once we go to production then it makes sense to write migration for every table but not in the developing mode.
Rather then migrating simply delete the database (delete the App's data or uninstall the App, the database is stored in the default location (data/data/package_name/databases)) and rerun without changing the version. The database will be created as per the new schema. Perhaps utilising temporary code to load data accordingly.
How does room DB make developr job eaiser?
In Short ROOM generates what is termed as the boilerplate code from relatively simple code e.g the #Query above writes the underlying code to extract the data and build the objects (e.g. the code as above).
Please check the official document:
https://developer.android.com/training/data-storage/room/migrating-db-versions
Actually Room using SQLITE behind the scene. It provide you plethora of other facilities. In case of Migration you have to write full code to create table.
Harsh your question is valid in some way but as you know android is maintaining SQLite database and libraries like room, greendao or even native SQLiteOpenHelper
is handling the transaction with sqllite behind the scene for the developers.
In all the earlier libraries too you have to maintain versions of your database and which fields or tables have been added to your database and write migrations for the version upgrades of the database.
The beauty of room comes in play in how easy they have made the CRUD operations on the SQLite database and getting data wrapped in LiveData or Observable, not that you don't need to write migrations.
Aim -
I am trying to store longitude and latitude data from the geolocation API using React native. I created a .db file put it into the /assets/ folder in android. First, instead of the location data, I am trying to insert or retrieve any kind of data.
ERROR -
It shows that there is no table named Test when I try to console.log() it.
Problem -
Whenever I try to insert data into the database I don't see any new data added and also When I try to read an already populated database I see nothing. Where am I wrong? I am fairly new to this.
What is the response returned by the SQL transaction. I console.log it but couldn't understand.
Where should be the transaction function be located?
I am using DBbrowser for SQLite
OS - MacOS
CODE -
I am using this function to send or receive data.
sendData = () => {
// var SQLite = require('react-native-sqlite-storage')
var db = SQLite.openDatabase({name: 'a', createFromLocation : "~test.sqlite"}, this.openCB, this.errorCB);
db.transaction(txn => {
txn.executeSql(
// "INSERT INTO Test (latitude,longitude) VALUES(?,?)",
'SELECT * FROM Test',
[], //Argument to pass for the prepared statement
(res) => {
// let row = res.rows.(1);
console.log(res);
} //Callback function to handle the result response
);
});
};
// I mapped this button to send data.
<TouchableOpacity onPress={this.sendData} style={styles.button}>
<Text> SEND DATA</Text
</TouchableOpacity>
You won't be able to modify a database stored in the assets folder. The typical method is before ever trying to open the database to insert records, you first check to see if the database exists in your apps data folder (not sure if React has the same folder structures or not). If the database doesn't exist in the data folder, then you copy your clean database from assets to the data folder, then open the database from the data folder. As long as the app isn't deleted off the device, the database in the data folder will persist, along with all of the data you have inserted into it. Once the app has been deleted and reinstalled, the database will have to be copied from assets once again and will be clean, with no new data in it.
I don't know if the same data folder structure applies to React, but in native Android I use:
"/data/data/com.mycompany.myapp/databases/"
OR
DB_PATH = myContext.getFilesDir().getParentFile().getPath() + "/databases/";
I need opinion regarding rendering of data in android app. I have all the data stored in a json file abc.json which is in res > raw folder. i have a class that then reads data from that json file and build SQLite database when the app runs and later on i'm performing all operations like searching the data using sql queries for that database. But i am afraid if thats not a good option and the code is not optimized because code now contains so many functions for adding the items to database.
For example, json file has Authors, books, keywords, references, acknowledgements, subauthors and when the database is built, data is read and a specific function is called for each item. I'm just concerned because of too many functions as one for each item. Like whenever json is parsed for an item, e.g author, it calls addAuthors function to add that to database. Following are 2 of the functions for example.
//Sample function code for adding authors to db
public void addAuthors(Integer id, String Name, String is_corresponding) {
ContentValues value = new ContentValues();
value.put("_id", id);
value.put("NAME", Name);
value.put("IS__CORRESPONDING", is_corresponding);
authors_id = database.insert(TABLENAME_AUTHOR, null, value);
}
//example function for adding keywords to db
public void addKeyWord(String KeyWords, Integer id) {
ContentValues values = new ContentValues();
values.put("KEYWORDS", KeyWords);
values.put("_id ", id);
database.insert(TABLENAME_ABSTRACT_KEY_WORDS, null, values);
}
I need help with optimizing my code. Is there any way to optimize the current code ? Kindly help me with this and suggest some improvements for it. Thanks in advance
I would recommend bundling a sqlite database as an asset in your APK instead of bundling the JSON file and then inserting the data into a database. If your data isn't changing, you can then get rid of all your insert functions. You will also save the cost of creating and populating your database dynamically.
You can use the methods described here to create your database and to copy it from the assets of your APK. Be sure to copy it first before you try to open in in your app -- you can't open it directly as an asset.
Just curious on the best practice on syncing data from a database to an android tablet.
Tables:
- Part1
- Part2
- Part3
- Part4
- Part5
Whenever I open the app on the tablet I grab the latest lists from the database, truncate the table, and re-add the records. Each table consists of 400 records. So it takes around 60.45 per table to grab the data from the server and add it. Since I have 5 tables it takes around 5 minutes. Is there a better way to achieve efficient syncing for what I am doing? After I grab the data from the server, instead of truncating the table I've tried checking if it exists firsts before adding it but that didn't help with the time.
What I am currently doing: I get the JSON list from the API server and truncate the table and add the rows back. Pretty time consuming with 5 tables of 500 records each.
libraryApp = (LibraryApp) act.getApplication();
List<Pair> technicians = getJsonData("get_technicians");
if(technicians.size() > 0) {
stiLibraryApp.getDataManager().emptyTechnicianTable(); // truncate current table
// add technicians back to database
for(Pair p : technicians) {
libraryApp.getDataManager().saveTechnician(new Technician((Integer) p.key(), (String) p.value()));
}
}
Given the limited information provided I would try the following:
(1) Have your server keep a record of when the table you are updating was last "put" on the server. I have no idea what backend language you are using so I cannot make a sugestion. But it should be really easy to keep a lastupdated timestamp.
With this timestamp you will be able to tell if the version of the table on your server is more recent than the version on your mobile device.
(2) Using an AsyncTask download the data you need. I am not sure if all 5 tables are in the same activity, in seperate activities, in fragments or something else. However, the general idea is as follows:
private class GetTableData extends AsyncTask<Void, Void, Void>{
#Override
protected Void doInBackground(){
//get data from your sever
return null;
}
protect void onPostExecute(Void result){
//update the table view if version on server is newer
}
You will place all your I/O methods, that is those that connect to your server and download data within doInBackground. You will place all methods that update the table view within onPostExecute. This seperation is necessary because while I/O functions must run in the background after Jellybean, views must be updated from the UI thread.
(3) Check the timestamp of what you downloaded. If what you downloaded is newer update your table. You can acomplish this by simply adding in a conditional statment to your onPostExecute function such that
if(lastDownloadTime < lastUpdatedOnServerTime){
//update view
}
Depending on how big the table files are you may want to add a function on your sever code that just returns the time the table was last updated. That way you can check the time it was last updated against the time you last downloaded the table. If the table was updated on the server after you downladed it you can proceed to download the new information.
That's the basic idea. You can adapt it to your own set up.