Develop Reference

SQLite - Increase speed of insertion

I have a method which reads data from file line by line and takes value between coma, then puts this value into INSERT query. Data in file saved in this way:
–,08:10,–,20:20,08:15,08:16,20:26,20:27,08:20,08:21,20:31,20:32,08:30,08:31,20:40,20:41,08:37,08:38,20:46
20:47,08:48,08:50,20:56,20:57,09:00,09:01,21:07,21:08
08:53,–,17:43,09:01,09:03,09:13,09:15,18:02,18:04,–,–,09:19,09:25
Here is actual my code:
public void insertTime(SQLiteDatabase database, String table) throws FileNotFoundException {
BufferedReader br = null;
String line;
try {
int j = 0;
br = new BufferedReader(new InputStreamReader(context.getAssets().open("time.txt")));
database.beginTransaction();
while ((line = br.readLine()) != null) {
j++;
String query = "INSERT INTO "+table+""+j+" (arrival, departure) VALUES (?,?)";
SQLiteStatement statement = database.compileStatement(query);
// use comma as separator
String[] time = line.split(",");
for(int i = 1; i < time.length; i+=2) {
statement.bindString(1,time[i-1]);//arrival
statement.bindString(2,time[i]);//departure
statement.executeInsert();
statement.clearBindings();
}
}
database.setTransactionSuccessful();
database.endTransaction();
} catch (IOException e) {
e.printStackTrace();
} finally {
if (br != null) {
try {
br.close();
} catch (IOException e) {
e.printStackTrace();
}
}
}
}
The problem is that data insert very slow, despite I use SQLiteStatement and transactions. For example, when I insert 69000 rows it takes about 65,929 seconds.
What have I to change in my code to improve speed of insertion ?
UPDATE
OK, I have simplified my code, I got rid of BufferedReader and now it looks like this
public void insertTime(SQLiteDatabase database) throws FileNotFoundException {
database.beginTransaction();
int r = 0;
while (r < 122) {
r++;
String query = "INSERT INTO table_1 (arrival, departure) VALUES (?,?)";
SQLiteStatement statement = database.compileStatement(query);
for(int i = 1; i < 1100; i++) {
statement.bindString(1,i+"");//arrival
statement.bindString(2,i+"");//departure
statement.executeInsert();
statement.clearBindings();
}
}
database.setTransactionSuccessful();
database.endTransaction();
}
But it still so long inserts data, more than 2 min. Do you have any ideas how to increase speed of my second example ?

Here is a very very detailed post on every method of increasing SQL insertion speed.

Move beginTransaction() and setTransactionSuccessful() outside of while loop and it will be way faster.

A new transaction is started for each item in the while() loop.
It might go a bit faster if you only have 1 transaction to do all your insertions.
Also, when your data is corrupt and String.split doesn't give you at least 2 items, then your transaction will not be ended properly due to an Exception being thrown.

Every time you insert a row in a table with indexes, the indexes have to be adjusted. That operation can be costly. Indexes are kept as b-trees and if you hit the rebalance point, you're bound to have a slowdown. One thing you can do to test this is to remove your indexes. You could also drop the indexes, insert, then re-create the indexes.

For those using JDBC (Java): to be sure, do you first set the autoCommit to FALSE?
I guess so, because you work with explicit transactions.
The performace gain I got by explicitly setting the autocommit off was over 1000 times!
So:
Class.forName("org.sqlite.JDBC");
String urlInput = "jdbc:sqlite:" + databaseFile;
databaseConnection = DriverManager.getConnection(urlInput);
databaseConnection.setAutoCommit( false);
And:
String sql = "INSERT INTO " + TABLE_NAME + " ( type, bi, ci, fvi, tvi, content_type) VALUES ('V',?,?,?,?,'rtf')";
PreparedStatement psi = databaseConnection.prepareStatement(sql);
for( Item item : items) {
psi.setInt(1, item.property1);
// ....
count = psi.executeUpdate();
}
databaseConnection.commit();
databaseConnection.setAutoCommit( true);
So, when somebody forgets this, this may have a huge effect.

Update item ID when deleting

I have 5 empty TextViews where I add the names. After adding a name, it is stored in a database. The database consist on 2 columns, the item ID and the item NAME. This is an example of what I'm doing:
- Mark1 //ID=1, NAME= Mark1
- Mark2 //ID=2, NAME= Mark2
- Mark3 //ID=3, NAME= Mark3
- Empty
- Empty
I add and edit perfectly the textViews, but I'm facing a problem when deleting. This has something to do with the way I'm getting the values from the database, I'll explain:
Every time the app starts, or I edit, add or delete one element, what I do is get the items from the database, get them into a Map, and copy them into the textviews (whose at a first time are invisible) making visible just the ones that have a name setted.
This is the code I use to do that:
public void getTravelers() {
/*Create map where store items*/
Map<Integer, String> nameList = new HashMap<Integer, String>();
/*Lon in providers query() method to get database's items and save them into the map*/
Cursor c = getContentResolver().query(TravelersProvider.CONTENT_URI, PROJECTION, null, null, null);
if (c.moveToFirst()) {
do {
nameList.put(Integer.parseInt(c.getString(c.getColumnIndex(Travelers._ID))), c.getString(c.getColumnIndex(Travelers.NAME)));
}while(c.moveToNext());
}
if (c != null && !c.isClosed()) {
c.close();
}
/*Check size*/
int size = nameList.size();
if (size >= 1) {
/*Save items in TextViews*/
//TODO: This is the code I should fix
for (int i = 0; i <= size; i++) {
if (i==1) {
traveler1.setText(nameList.get(i).toString());
traveler1.setVisibility(View.VISIBLE);
}
if (i==2) {
traveler2.setText(nameList.get(i).toString());
traveler2.setVisibility(View.VISIBLE);
}
if (i==3) {
traveler3.setText(nameList.get(i).toString());
traveler3.setVisibility(View.VISIBLE);
}
if (i==4) {
traveler4.setText(nameList.get(i).toString());
traveler4.setVisibility(View.VISIBLE);
}
if (i==5) {
traveler5.setText(nameList.get(i).toString());
traveler5.setVisibility(View.VISIBLE);
}
}
}
}
The problem comes in the for loop. Let's supposse that from the items named above, I want to delete Mark2 with ID=2, so then the size of the new Map would be 2, and it would enter to (i == 1) and (i == 2). But when entering to this last one, it would do traveler2.setText(nameList.get(2).toString()); and as seen, there is no element existing with the ID=2 because that is the one that I've deleted and it throws a NPE.
So my question is, what would be the right way to do this without facing this problem?

You should go for switch case other than for loop. Than code will not be in loop.

Finally I get what I need just changing the Key value of the Map that was the same as the ID of the database:
if (c.moveToFirst()) {
int key = 0;
do {
key++;
nameList.put(key, c.getString(c.getColumnIndex(Travelers.NAME)));
}while(c.moveToNext());
}
if (c != null && !c.isClosed()) {
c.close();
}
Basically this way I don't need to change nothing more as now the key value of the Map will match with the Textview position

Receiving unknown count of results from SQLite db

Basically this is not a problem in itself, my code works so far.
What I have is a App, that lets a user log in and depending on his ID in the db, he gets displayed his saved notes. For this view I have this part of code:
title = new ArrayAdapter<String>(this, android.R.layout.simple_list_item_1);
MyDbHandler dbh = new MyDbHandler(this);
for(int i = 0; i < 999; i++) {
content = dbh.getNoteTitle(id, i); //getNoteTitle(int, int) returns String
if(content != null && content != "0")
title.add(content);
else
break;
}
list.setAdapter(title);
As I said, this works so far.
Thing is - I am very unhappy with the use of ' break; ' here, as I learned during education, this shouldn't be used.
Is there a smoother way to approach this issue?
Also ' content != "0" ' should be ' ! content.equals("0") ' normally, right? But that one doesn't work then... Why is this?

I am not sure what are you trying to do. First of all you should use "equal" method for Strings. The condition "content != "0" will always be true, because you are comparing 2 different objects. The condition "! content.equals("0")" should return true most of the time (when the value is not "0") and probably you should use the debugger to see exactly what is the value of content.
Second if you want to take all the notes from the database and show them to the user you should have first a method in the SQLiteOpenHelper similar to (it is not efficient to interrogate the database for each item, plus the separation of concerns):
public ArrayList<String> getNotesList (int userID){
SQLiteDatabase db = getWritableDatabase();
Cursor c = db.query(TABLE_NAME, new String[] {MyDbHandler.COLUMN_NOTE_TITLE}, MyDbHandler.userID + "=" + userID,null, null, null, null);
ArrayList<String> list = null;
String noteTitle;
if (c != null && c.moveToFirst())
{
list = new ArrayList<String>(c.getCount());
for (int i = 0; i < c.getCount(); i++)
{
noteTitle = c.getString(c.getColumnIndex(MyDbHandler.COLUMN_SESSION_PATH));
list.add(noteTitle);
c.moveToNext();
}
}
c.close();
db.close();
return list;
I think you should not save notes that you don't want to use (e.g. null or "0"), so instead of checking here, I would check in the addNote method.
For the list initialization you have:
MyDbHandler dbh = new MyDbHandler(this);
ArrayList listData = dbh.getNotesList(id)
if (listData != null && listData.length != 0){
title = new ArrayAdapter<String>(this, android.R.layout.simple_list_item_1, list);
listData.setAdapter(title);
}
I didn't test the code, but I hope it helps you. Good luck!

Check if the event exists before adding it to the Android calender

I have a list of events in my app. A button on the side lets the user add the event date and time to his/her calender. I use a calender intent to redirect the user to the android calender which the corresponding date and time. Now after the user adds the event to his calender, I would like to disable the 'add event' button which corresponds to the events he/she had already added(so the user avoid adding the same event again). How can I do this? I have gone through the new calender API for android 4.0 but I wasnt able to achieve what I wanted.
Basically what I want is to avoid repeated entries for the same event in the users calender.
Any help would be appreciated.

You should test, if an instance for this event exists. See the documentation of the Android's CalendarContract.Instances class.
Especially the second query method should be helpful in this case.
This examples is some code, I posted on my blog post about the CalendarContract provider - slightly altered for your needs:
long begin = // starting time in milliseconds
long end = // ending time in milliseconds
String[] proj =
new String[]{
Instances._ID,
Instances.BEGIN,
Instances.END,
Instances.EVENT_ID};
Cursor cursor =
Instances.query(getContentResolver(), proj, begin, end, "\"Your event title\"");
if (cursor.getCount() > 0) {
// deal with conflict
}
Be aware: The time is always in UTC millis since the epoch. So you might have to adjust given the user's timezone.
And the last parameter should contain the title of the event you have added to the calendar. Keep the quotes - otherwise Android looks for "your" or "event" or "title"!
And do not forget to include the necessary permissions.

Instances.query is not recommended to be run on the UI thread, but can be done efficiently by ensuring start and end time duration is minimized.
The search string will search all values, not just title, so adding a loop to check for that an exact field value is necessary.
public boolean eventExistsOnCalendar(String eventTitle, long startTimeMs, long endTimeMs) {
if (eventTitle == null || "".equals(eventTitle)) {
return false;
}
if (ActivityCompat.checkSelfPermission(this, Manifest.permission.READ_CALENDAR) != PackageManager.PERMISSION_GRANTED) {
return false;
}
// If no end time, use start + 1 hour or = 1 day. Query is slow if searching a huge time range
if (endTimeMs <= 0) {
endTimeMs = startTimeMs + 1000 * 60 * 60; // + 1 hour
}
final ContentResolver resolver = mContext.getContentResolver();
final String[] duplicateProjection = {CalendarContract.Events.TITLE}; // Can change to whatever unique param you are searching for
Cursor cursor =
CalendarContract.Instances.query(
resolver,
duplicateProjection,
startTimeMs,
endTimeMs,
'"' + eventTitle + '"');
if (cursor == null) {
return false;
}
if (cursor.getCount() == 0) {
cursor.close();
return false;
}
while (cursor.moveToNext()) {
String title = cursor.getString(0);
if (eventTitle.equals(title)) {
cursor.close();
return true;
}
}
cursor.close();
return false;
}

I have used following way to check it ...what i am passing event_id to check whether is it in calendar or not....
public boolean isEventInCal(Context context, String cal_meeting_id) {
Cursor cursor = context.getContentResolver().query(
Uri.parse("content://com.android.calendar/events"),
new String[] { "_id" }, " _id = ? ",
new String[] { cal_meeting_id }, null);
if (cursor.moveToFirst()) {
//Yes Event Exist...
return true;
}
return false;
}

Please check this, this might help:
private static boolean isEventInCalendar(Context context, String titleText, long dtStart, long dtEnd) {
final String[] projection = new String[]{CalendarContract.Instances.BEGIN, CalendarContract.Instances.END, CalendarContract.Instances.TITLE};
Cursor cursor = CalendarContract.Instances.query(context.getContentResolver(), projection, dtStart, dtEnd);
return cursor != null && cursor.moveToFirst() && cursor.getString(cursor.getColumnIndex(CalendarContract.Instances.TITLE)).equalsIgnoreCase(titleText);
}

Insert Array in SQLite Database in android

I want to save weekdays in database, so i thought to store it by assigning int value to each day. i.e
1 -> Selected, 0 -> Not Selected.
Monday = 0/1
Tuesday = 0/1
.
.
.
.
.
Sunday = 0/1.
But this will make 7 columns in DB. So I was thinking if anyone can help me with this if I should store it in a single array and retrieve the values for further use. I was reading some examples over internet but didn't get it in a easy way.

To insert 7 values in one column you can use comma separator like this
where Total_Score_P1 is an string array
//string array
String[] Total_Score = new String[] { p1e1,p1e2,p1e3,p1e4,p1e5,p1e6 };
// Convderting it into a single string
String result_ScoreP1 = ("" + Arrays.asList(Total_Score_P1)).
replaceAll("(^.|.$)", " ").replace(", ", " , " );
result_ScoreP1 will be
// output of this
result_ScoreP1 = "p1e1,p1e2,p1e3,p1e4,p1e5,p1e6";
insert it as a single string in database and
when retrieve it in again break in parts like
// a string array list
// query fired
public ArrayList<String> rulTable(String id) {
// TODO Auto-generated method stub
ArrayList<String> Ruleob = new ArrayList<String>();
Cursor c_rule;
try
{
c_rule = db.query(NameTable, new String[]{
columns1
},
Rule_COurseID + "=" + id ,
null, null,
null, null, null);
c_rule.moveToFirst();
// if there is data available after the cursor's pointer, add
// it to the ArrayList that will be returned by the method.
if (!c_rule.isAfterLast())
{
do
{
Ruleob.add(c_rule.getString(0));
}
while (c_rule.moveToNext());
}
// let java know that you are through with the cursor.
c_rule.close();
}
catch(Exception e)
{
}
return Ruleob;
}
//list to get elements
ArrayList<String> ListOne = new ArrayList<String>();
ArrayList<String> row ;
try{
// received values
row = db.TheTable(id);
String r1 = row .get(0);
}
catch(Exception e)
{
}
StringTokenizer st2 = new StringTokenizer(r1, "||");
while(st2.hasMoreTokens()) {
String Desc = st2.nextToken();
System.out.println(Desc+ "\t" );
ListOne.add(Desc);
//
}

You can use a binary integer 1= selected 0 =Not Selected (1111111) (0000000)
total seven days so index 0=mon, 1=tues, 2=wed, 3=thurs, 4=friday, 5=sat, 6=sunday..and so on..
here 1111111 means all day selected, 0000000 all day not selected, 0001000 only thursday is selected.

I have also discovered a way i.e. convert your so called values to a JSON Array and then store the complete JSON String to an entity/field in Database.
It helps in serving the values easily and effectivly.

Create another table with a column for each day, boolean value. Make an association to this table by integer id (use a foreign key) This is the relational way of solving the problem.