How Improve SQLite Performance? [duplicate] - android

This question already has answers here:
Android SQLite database: slow insertion
(5 answers)
Closed 8 years ago.
I want to inset more than 3000 records in android SQLite but the following code take to much time for the data inserion
Here is my code.
public boolean addSale(Sale objSale) {
ContentValues values = new ContentValues();
values.put(SALE_BRANCH, objSale.getBranch());
values.put(SALE_SUPPLIER, objSale.getSupplier());
values.put(SALE_BUYER, objSale.getBuyer());
values.put(SALE_CAT1, objSale.getCat1());
values.put(SALE_CAT2, objSale.getCat2());
values.put(SALE_CAT3, objSale.getCat3());
values.put(SALE_CAT4, objSale.getCat4());
values.put(SALE_CAT5, objSale.getCat5());
values.put(SALE_CAT6, objSale.getCat6());
values.put(SALE_DESIGNO, objSale.getDesigNo());
values.put(SALE_ITEMSIZE, objSale.getItemSize());
values.put(SALE_SALEQTY, objSale.getSaleQty());
values.put(SALE_STOCKQTY, objSale.getStockQty());
values.put(SALE_FinalProduct, objSale.getFinalProduct());
values.put(SALE_PriceRange, objSale.getPriceRange());
values.put(SALE_CoreNonCore, objSale.getCoreNonCore());
values.put(SALE_Color, objSale.getColor());
values.put(SALE_GSLCode, objSale.getGSLCode());
values.put(SALE_Wanted, objSale.getWanted());
values.put(SALE_Pqty, objSale.getPqty());
values.put(SALE_MRP, objSale.getMRP());
values.put(SALE_PRate, objSale.getPRate());
SQLiteDatabase db = this.getWritableDatabase();
db.insert(TABLE_SALE, null, values);
db.close();
return true;
}
And this one is my Asynk Task. Where i fetch data from webservice and insert into SQLite
protected void onPostExecute(String result)
{
try {
String l = result.replace("\\", "");
l = l.replace("''", "");
String sdsd = l.substring(1, l.length() - 1);
JSONArray jsonArray = new JSONArray(sdsd);
Log.i("JSON", "Number of surveys in feed: " +jsonArray.length());
/*if(db.delSaleData()){
}*/
for (int i = 0; i < jsonArray.length(); i++)
{
JSONObject jsonObject = jsonArray.getJSONObject(i);
sl.setBranch(jsonObject.getString("Branch"));
sl.setSupplier(jsonObject.getString("Supplier"));
sl.setBuyer(jsonObject.getString("Buyer"));
sl.setCat1(jsonObject.getString("Cat1"));
sl.setCat2(jsonObject.getString("Cat2"));
sl.setCat3(jsonObject.getString("Cat3"));
sl.setCat4(jsonObject.getString("Cat4"));
sl.setCat5(jsonObject.getString("Cat5"));
sl.setCat6(jsonObject.getString("Cat6"));
sl.setDesigNo(jsonObject.getString("DesigNo"));
sl.setItemSize(jsonObject.getString("ItemSize"));
sl.setSaleQty(jsonObject.getString("SaleQty"));
sl.setStockQty(jsonObject.getString("StockQty"));
sl.setFinalProduct(jsonObject.getString("FinalProduct"));
sl.setPriceRange(jsonObject.getString("PriceRange"));
sl.setCoreNonCore(jsonObject.getString("CoreNonCore"));
sl.setColor(jsonObject.getString("Color"));
sl.setGSLCode(jsonObject.getString("GSLCode"));
sl.setWanted(jsonObject.getString("Wanted"));
sl.setPqty(jsonObject.getString("Pqty"));
sl.setMRP(jsonObject.getString("MRP"));
sl.setPRate(jsonObject.getString("PRate"));
if(db.addSale(sl))
{
//Toast.makeText(getApplicationContext(), " Insert.." , Toast.LENGTH_SHORT).show();
}
}
setData();
} catch (Exception e) {
e.printStackTrace();
}
dialog.dismiss();
setTableData("All");
}

use the SQLiteStatement for example:
private void bulkInsertRecords(String[] records) {
String sql = "INSERT INTO "+ SAMPLE_TABLE_NAME +" VALUES (?,?,?);";
SQLiteStatement statement = sampleDB.compileStatement(sql); //Este é o prepare
sampleDB.beginTransaction();
for (int i = 0; i<records.length; i++) {
statement.clearBindings();
statement.bindString(1, records[0]);
statement.bindString(2, records[1]);
statement.bindString(3, records[2]);
statement.execute();
}
sampleDB.setTransactionSuccessful();
sampleDB.endTransaction();
}

Don't open and close the database every time you want to do an insert. Open it once and then close it when exiting the app.

Use the Asyncclass. Do the operation onBackground, not in the main UI thread.
UPDATE
On your code, you insert the SQLite via onPost, not onBackground. onPost do things on UI thread, while onBackground do it on the separate (background) thread so it doesnt effect the UI. So move the operation code to onBackground.

Related

Retrieve Specific Columns from SQLite Database

I have an app that gets all the data from the sqlite database and converts it into JSON, but I was wondering how would I be able to do it if I only want to get the data of specific columns?
This is the code I use to get all the data from the SQLite databse and convert it to JSON:
Cursor data = db.getCartItems();
orderNameArray = new JSONArray();
data.moveToFirst();
while(data.isAfterLast() == false)
{
int totalColumn = data.getColumnCount();
orderNameObject = new JSONObject();
for (int i = 0; i < totalColumn; i++)
{
try {
orderNameObject.put(data.getColumnName(i), data.getString(i));
} catch (JSONException e) {
e.printStackTrace();
}
}
orderNameArray.put(orderNameObject);
data.moveToNext();
}
data.close();
Thanks in advance for all the insights or help in advance! :D
If you want to fetch only selected fields of table, then use the following query:
SELECT Coulmn1, Coulmn2, Coulmn3 FROM TABLENAME;
or u want all data
SELECT * FROM TABLENAME;
for more info https://www.tutorialspoint.com/sqlite/sqlite_select_query.htm

Constantly retrieve data from database in a infinite loop

I created a database with a table named flagTable, this table only has two fields, which are id(auto increment) and an integer field. Next, in my program, I have a button that will trigger a thread to start. When the thread is starting, it constantly retrieve data from database, and check for the for the value, if the value is equal to one then it will trigger another new Thread, something like this:
private class statusOfStrummingInAnotherDevice extends Thread {
int value;
public void run() {
try{
while(true){
try{
if(flagCursor == null){
flagCursor = cdb1.getFlagAll();
}
}catch(Exception e){break;}
try{
Log.i("MAIN3ACTIVITY","getting status");
int size = cdb1.getSize(flagCursor);
Log.i("MAIN3ACTIVITY","SIZE is" + String.valueOf(xyz));
for(int i = 0 ; i < size ; i++){
flagCursor.moveToPosition(i);
Log.i("MAIN3ACTIVITY","getting status jkasdfasdf");
value = cdb1.getFlag();
if(value == 1){
Log.i("FLAGCURSOR=====>>>>","Succesful");
releasingNotes = new ReleasingNotes(IntendedChord);
releasingNotes.start();
//break;
}
cdb1.updateFlag(0);
Log.i("FLAGCURSOR=====>>>>",String.valueOf(value));
}
flagCursor = null;
}catch(Exception e){break;}
Log.i("MAIN3ACTIVITY","thread is sleeping");
try {
Thread.sleep(1000);
} catch (InterruptedException e) {
break;
}
}
}catch(Exception e){
}
}
}
In the meantime, the data that were retrieved from the database is using this function:
public Cursor getFlagAll(){
return getReadableDatabase().rawQuery(
"SELECT _ID, flag from flagTable", null);
}
And, the data that were updated to the database through this method:
public int updateFlag(int i) {
SQLiteDatabase db = this.getWritableDatabase();
ContentValues contentValues = new ContentValues();
contentValues.put("flag",i);
return db.update("flagTable" , contentValues , "_ID" + "= ?",new String[]{String.valueOf(1)});
}
Now, above codes will give no error, however, the data that were retrieved from the database is always 1, it keeps trigger a new function. In my above codes, I stated if the value is equal to 1, then the current thread will trigger a new thread to start, When its finished, the program will update the current data to 0. So that, the next round of the infinite loop can stop triggering new thread until a the conditon is met. What is problem overhere? did my codes really updated the new value? or I need to referesh the database every time I updated a new value.
Use Listeners to your database.
use SQLiteTransactionListener and do your things in onCommit()
Some guide in details here :
https://developer.android.com/reference/android/database/sqlite/SQLiteTransactionListener.html and
http://www.programcreek.com/java-api-examples/index.php?api=android.database.sqlite.SQLiteTransactionListener

Android SQLite unidirection Sync with SQL Server using Rest API webservice [closed]

Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 4 years ago.
Improve this question
I have an SQL Server Database with quotes.
The table has four fields ID primary key, Title, Description, and InsertDate.
I have created an Android application to display the quotes. The quotes are downloaded locally to the SQLite database and then displayed using ListView.
Whenever the user clicks the sync button, if any new quotes are added to the SQL Server database it should be downloaded to the SQLite database.
I have created a web service for the syncing using RestApi.
I have never used Webservice and new to Android development. Worked on Asp.net/C#.
My question is:
The android application should only download the latest records how to request only the latest inserted record from android using webservice.
If I change a record on SQL server how to identify such records and change in the SQLite database.
My current code is logically wrong and I need to change as it is deleting all the records and inserting all the records again. (Wanted to give demo to the customer)
// Create AsycHttpClient object
AsyncHttpClient client = new AsyncHttpClient();
// Http Request Params Object
RequestParams params = new RequestParams();
// Show ProgressBar
prgDialog.show();
client.get(getApplicationContext().getString(R.string.IpAdd) + "/WebApi/api/mpAudioapi/GetAllAudio", params, new AsyncHttpResponseHandler() {
#Override
public void onSuccess(int statusCode, Header[] headers, byte[] responseBody) {
// Hide ProgressBar
prgDialog.hide();
// Update SQLite DB with response sent by getusers.php
String str;
try {
str = new String(responseBody, "UTF-8");
} catch (UnsupportedEncodingException e) {
// this should never happen because "UTF-8" is hard-coded.
throw new IllegalStateException(e);
}
updateSQLite(str);
}
#Override
public void onFailure(int statusCode, Header[] headers, byte[] responseBody, Throwable error) {
// TODO Auto-generated method stub
// Hide ProgressBar
prgDialog.hide();
if (statusCode == 404) {
Toast.makeText(getApplicationContext(), "Requested resource not found", Toast.LENGTH_LONG).show();
} else if (statusCode == 500) {
Toast.makeText(getApplicationContext(), "Something went wrong at server end", Toast.LENGTH_LONG).show();
} else {
Toast.makeText(getApplicationContext(), "Unexpected Error occcured! [Most common Error: Device might not be connected to Internet]",
Toast.LENGTH_LONG).show();
}
}
});
The code to Sync
int writing_counts = dbHandler.getMpAudioRowCount();
ArrayList<HashMap<String, String>> usersynclist;
usersynclist = new ArrayList<HashMap<String, String>>();
// Create GSON object
Gson gson = new GsonBuilder().create();
try {
// Extract JSON array from the response
JSONArray arr = new JSONArray(response);
//JSONObject responseObject = new JSONObject(response);
//JSONArray arr = responseObject.getJSONArray("results");
//JSONArray arr = new JSONArray(response);
System.out.println(arr.length());
if (writing_counts == arr.length()) {
Toast.makeText(getApplicationContext(), "Sorry. No new writings to sync.", Toast.LENGTH_LONG).show();
return;
}
dbHandler.truncateTableAudio();
// If no of array elements is not zero
if (arr.length() != 0) {
// Loop through each array element, get JSON object which has Title and Des
for (int i = 0; i < arr.length(); i++) {
// Get JSON object
JSONObject obj = (JSONObject) arr.get(i);
System.out.println(obj.get("Id"));
System.out.println(obj.get("Title_Audio"));
System.out.println(obj.get("URL_Audio"));
// DB QueryValues Object to insert into SQLite
queryValues = new HashMap<String, String>();
// Add ID extracted from Object
queryValues.put("id", obj.get("Id").toString());
// Add Title extracted from Object
queryValues.put("title", obj.get("Title_Audio").toString());
// Add Des extracted from Object
queryValues.put("url", obj.get("URL_Audio").toString());
// Insert User into SQLite DB
dbHandler.insertWriteAudio(queryValues);
HashMap<String, String> map = new HashMap<String, String>();
// Add status for each User in Hashmap
map.put("id", obj.get("Id").toString());
map.put("title", obj.get("Title_Audio").toString());
map.put("url", obj.get("URL_Audio").toString());
usersynclist.add(map);
}
// Inform Remote MySQL DB about the completion of Sync activity by passing Sync status of Users
//updateMySQLSyncSts(gson.toJson(usersynclist));
// Reload the Main Activity
reloadActivity();
Toast.makeText(getApplicationContext(), "Sync activity completed successfully.", Toast.LENGTH_LONG).show();
}
} catch (JSONException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
My DBhandler Code:
public void truncateTable(){
SQLiteDatabase db = this.getWritableDatabase();
db.execSQL("delete from "+ MP_TABLE);
}
public void insertWriteAudio(HashMap<String, String> queryValues) {
SQLiteDatabase database = this.getWritableDatabase();
ContentValues values = new ContentValues();
values.put("id", queryValues.get("Id"));
values.put("title", queryValues.get("title"));
values.put("url", queryValues.get("url"));
database.insert("audios", null, values);
database.close();
}
public int getMpAudioRowCount() {
SQLiteDatabase db = this.getReadableDatabase();
long cnt = DatabaseUtils.queryNumEntries(db, MP_TABLE_Audio);
db.close();
return (int) cnt;
}
You can download only the updated content from database server. This is how you can do that. It will need some changes in your schema -
On Server side
Add one more column in the database, 'Updated_On' which will contain
the timestamp when that record was updated on the server
When you are inserting or updating any new record in the databse,
change the 'Updated_On' field accordingly
Now make some changes on the client side
On Client Side
In your Android app, keep a note of what was the last change that
was downloaded, i.e. the last download was made for 'Updated_On' =
{some past time stamp} (You can do that in SharedPreferences on cancreate a SQLite table)
When you query to download, query for all records who have
'Updated_On' > {some past time stamp}
If the record exist in your local SQLite database, update it; else insert the record in your SQLite databse
To help you get started, where you are making the AsyncHttp call
....
RequestParams params = new RequestParams();
// Show ProgressBar
prgDialog.show();
params.put("latestChange",lastUpdateTimestamp);
//where lastUpdateTimestamp is the sharedPreference value you are keeping of last update
client.get(getApplicationContext().getString(R.string.IpAdd) + "/WebApi/api/mpAudioapi/GetAllAudio", params, new AsyncHttpResponseHandler() {
....
On your server side you get that post variable $lastUpdate = $_POST['latestChange']. Now use the variable $lastUpdate in your API to query the database Select * from table where Updated_On > $lastUpdate

How to know when SQLite query is finished

Ok, I've got this Retrofit Call that receives a list of objects and insert the into a local SQLite database. I want to display a message saying that the operation was successful with a Ok button that when pressed opens a new activity.
How do I check if my Query has finished so I can show the message?
final ContactApi contactApi = retrofit.create(ContactApi.class);
Call<List<Contact>> callContact = contactApi.getContact(token);
callContact.enqueue(new Callback<List<Contact>>() {
#Override
public void onResponse(Response<List<Contact>> response, Retrofit retrofit) {
List<Contact> contactList = response.body();
if (contactList != null) {
try {
DBHelper dbHelper = new DBHelper(TokenActivity.this, token);
SQLiteDatabase conn = dbHelper.getWritableDatabase();
RepoContact repocontact = new RepoContact(conn);
// Inserts each contact into the database
for (Contatc c : contactList) {
repositorioCadastro.inserirCadastro(c);
Log.i("ACTTOKEN", "Contact insert ID: " + c.getId());
}
} catch (SQLiteException e) {
Log.i("ACTTOKEN", "Faillure on insert: " + e.getMessage());
}
}
wrap your code in try{...}finally{...} blocks with a listener ( beginTransactionWithListener(SQLiteTransactionListener transactionListener)), and use the transactionListner to check whether everything went well within the transaction, in addition to everything within the try/finally.
what you have is good, just try adding finally block..
something like this..
db.beginTransaction();
try {
...
db.setTransactionSuccessful();
} finally {
db.endTransaction();
}
You can try a different loop, something like this:
for(int i = 0; i < contactList.size(); i++) {
Contact c = contactList.get(i);
repositorioCadastro.inserirCadastro(c);
Log.i("ACTTOKEN", "Contact insert ID: " + c.getId());
if(i == (contactList.size() - 1)) {
// DO SOMETHING HERE
}
}
You may check insert statement return a long when query successfully executed then long value.
db.insert()
returns the row ID of the newly inserted row, or -1 if an error occurred

How to bulk insert from Json to Sqlite in Android [duplicate]

This question already has answers here:
Android SQLite database: slow insertion
(5 answers)
Closed 8 years ago.
How is it possible to insert bulk json data coming from server into Sqlite database in Android very efficiently. The method I use now is very inefficient and it takes almost a minute to complete insertion of about 2000 records. The method I am following is :
for (int i = 0; i < jsonObj.length(); i++)
{
JSONObject itemObj = (JSONObject) jsonObj.get(i);
ContentValues value1 = new ContentValues();
ContentValues value2 = new ContentValues();
value2.put(DbHelper.BusinessID,itemObj.getString("BusinessID"));
value2.put(DbHelper.LocationID,itemObj.getString("LocationID"));
JSONArray list = itemObj.getJSONArray("OfficeDetails");
if (list != null)
{
for (int k = 0; k < list.length(); k++)
{
JSONObject elem = list.getJSONObject(k);
if (elem != null)
{
try
{
value1.put(DbHelper.Office_Code,elem.getInt("Office_Code"));
value1.put(DbHelper.Office_District,elem.getInt("Office_District"));
db.insert(DbHelper.MessageDetail,null, value1);
}
catch (Exception e)
{
e.printStackTrace();
}
}
db.insert(DbHelper.Message,null, value2);
}
The input that is coming is a nested Json array, which itself is nested. Is there a better way to fastly insert huge amount of data in very short time ?
You could try bulkInsert as the following:
ContentResolver.bulkInsert (Uri url, ContentValues[] values); //Array of rows to be inserted
This only fits if you are going to use only 1 URI, if you are going to use multiple uris, you should use applyBatch method in your ContentResolver.
Hope it helps
First create the model class of your json data. Using Gson you can get data inside an arraylist. then you can insert data into sqlite using that arraylist. GreenDao is your best option for fast performance.
when you receive json data in a stream, use following code :
Type collectionType = new TypeToken<ArrayList<your_model>>() {}.getType();
Gson gson = new Gson();
ArrayList<your_model> yourModelList = gson.fromJson(stream, collectionType);
There is an excellent library called JSQL available in Github.
https://github.com/wenchaojiang/JSQL
Its makes it so easy to save Persist JSON string and JSON Objects to your SQLite on Database.
Create a list of your json data and then used this custom query for bulk data insertion:
/**
* insert the bulk data into database
*
* #param query to insert data into table
* #param parameter data to be inserted into table
* #return number of rows got inserted
*/
protected int insertBulk(String query, String[][] parameter) throws SQLiteConstraintException, SQLiteException {
int rowCount = -1;
SQLiteStatement statement = mSqldb.compileStatement(query);
mSqldb.beginTransaction();
try {
for (int index = 0; index < parameter.length; index++) {
statement.bindAllArgsAsStrings(parameter[index]);
statement.execute();
statement.clearBindings();
}
rowCount = parameter.length;
} finally {
mSqldb.setTransactionSuccessful();
mSqldb.endTransaction();
}
return rowCount;
}

Categories

Resources