I am testing the capabilities of the device -- to show the customer the size of data that can be stored inside the device, how fast it can be retrieved, how fast the search works, etc.
I am using my content provider to access the product database table with few columns. I have already moved the code to the content provider to avoid the extra communication when inserting the test records. The following code is called via menu from an activity to fill the table with the test content
Uri uri = Uri.parse(DemoContentProvider.PRODUCTS_CONTENT_URI + "/insertdemo");
getContentResolver().insert(uri, null);
The URI is recognized in the .insert method of the content provider and the following private method (of the same content provider) is called to fill the table (notice the 100 thousands of items):
private void insertDemoProducts() {
for (int i = 1; i <= 100000; ++i) {
String id = Integer.toString(i);
insertProduct(id, "Test product " + id, "100", "75.50", "70.27");
}
}
The inner insertProduct() looks like that:
private void insertProduct(String code, String name, String stock,
String price, String listprice) {
SQLiteDatabase sqlDB = database.getWritableDatabase();
ContentValues values = new ContentValues();
values.put(ProductTable.COLUMN_CODE, code);
values.put(ProductTable.COLUMN_NAME, name);
values.put(ProductTable.COLUMN_STOCK, stock);
values.put(ProductTable.COLUMN_PRICE, price);
values.put(ProductTable.COLUMN_LISTPRICE, listprice);
sqlDB.insert(ProductTable.TABLE_PRODUCT, null, values);
}
It works, but it takes "forever". How can I make it faster? What is the fastest method you know to fill the table?
Just some numbers to consider: 1000 items takes about 20 seconds to be created.
You need to use transactions when writing to a sqlite-database, otherwise it will persist the data for every insert i.e save it to sd which will take "forever".
for instance, make insertProduct take a list of products and save them in one transaction:
private void insertProducts(List<Product> products) {
try {
db.beginTransaction();
for(Product product : products) {
insertProduct(...);
}
db.setTransactionSuccessful();
} finally {
db.endTransaction();
}
}
This is how you can implement it in your existing code:
private void insertDemoProducts() {
SQLiteDatabase sqlDB = database.getWritableDatabase();
try {
sqlDB.beginTransaction();
for (int i = 1; i <= 100000; ++i) {
String id = Integer.toString(i);
insertProduct(id, "Test product " + id, "100", "75.50", "70.27");
}
sqlDB.setTransactionSuccessful();
} finally {
sqlDB.endTransaction();
}
}
Anyway, I am not completely satisfied with the accepted question because I do not understand the reason why adding the transaction makes it faster.
Looking at the Android sources, I have found that the sqlDB.insert(...) calls insertWithOnConflict(...) and that one construct the string for the SQL command using the StringBuilder class (with questionmarks as placeholders for the inserted values). Only then the string is passed to the SQLiteStatement constructor together with array of the inserted values. This means that string with the SQL command is being built again and again.
Further, a string representation of an SQL command template can be precompiled thus avoiding also the repeated compilation of the command. Then .bindXxx and .execute methods can be used for inserting the wanted records into the table. When put together, I did use the followig code (iside the outer transaction as Dean suggested):
StringBuilder sql = new StringBuilder();
sql.append("INSERT INTO ");
sql.append(ProductTable.TABLENAME);
sql.append("(");
sql.append(ProductTable.COLUMN_CODE);
sql.append(",");
sql.append(ProductTable.COLUMN_NAME);
sql.append(",");
sql.append(ProductTable.COLUMN_STOCK);
sql.append(",");
sql.append(ProductTable.COLUMN_PRICE);
sql.append(",");
sql.append(ProductTable.COLUMN_LISTPRICE);
sql.append(") VALUES (?, ?, 100, 75.50, 70.27)");
SQLiteStatement insert = sqldb.compileStatement(sql.toString());
for (int i = 1; i <= 300000; ++i) {
insert.bindLong(1, i);
insert.bindString(2, "Test product " + i);
insert.execute();
}
When compared with adding the transaction only, the result is about 3-times faster. The 300 thousands records were inserted in about 3 minutes and 15 seconds on Nexus 7.
Related
How to check if table is empty using parse , I'm having a problem with the code below :
private String[] getMaxDateMessage() throws ParseException {
final String[] msgData = new String[3];
ParseObject ob = null;
String[] userIds = {currentUserId, recipientId};
ParseQuery<ParseObject> query = ParseQuery.getQuery("ParseMessage");
query.whereContainedIn("senderId", Arrays.asList(userIds));
query.whereContainedIn("recipientId", Arrays.asList(userIds));
query.orderByDescending("createdAt");
if(query.hasCachedResult())
{
ob = query.getFirst();
if (ob.isDataAvailable()) {
//for (int i = 0; i < 1; i++) {
//createdDate[0] = messageList.get(i).get("createdAt").toString();
msgData[0] = ob.getCreatedAt().toString();
msgData[1] = ob.get("senderId").toString();
msgData[2] = ob.get("recipientId").toString();
// }
}
}
The thing is that the table is empty , so the query should return null , but no exception is been throwed , it just crashes the app .
So how can I check if the table is empty before trying to fetch any data ?
Update : The solution that I have found is to use query.count().
If the count returns a value that is not 0 then the table is not empty .
Using query.count() to determine if the table is empty is not an optimal solution. While this is perfectly fine when actually run against an empty table, using query.count() will almost always result in a sub-optimal query when there's more than one object in the table. The reason for this is quite clear: you only care about the first object matched by this query, yet a query.count() will scan the whole table in order to return the total of objects that match your query.
Therefore, the ideal solution is to simply use query.getFirst() and check if you get any results. You should be able to handle the case where ob is not a ParseObject, e.g. the collection is either empty or no objects match your query.
I am new to android and maybe its a silly question but i am not getting it. See i am designing a game in which we give scores to some persons. So i want to store the names of the persons in a database while installation and then their scores set to 0 initially which will be updated according to what the users select. Here i am not able to figure out that how should i enter the data as it will be around 100 names and their scores. Using INSERT INTO() statement will make it like 100 statements. So is there any short method like can we do it through strings or something. Just guessing though. Any help would be appreciated.
You don't hard-code names or scores into your SQL statements. Instead, you use parameters.
var command = new SQLiteCommand()
command.CommandText = "INSERT INTO Scores (name, score) VALUES(#name, #score)";
command.CommandType = CommandType.Text;
foreach (var item in data)
{
command.Parameters.Add(new SQLiteParameter("#name", item.Name));
command.Parameters.Add(new SQLiteParameter("#score", item.Score));
command.ExecuteNonQuery();
}
and then just loop through all of the names and scores.
I recommend you using a transaction.
You can archive this stating you want to use a transaction with beginTransaction(), do all the inserts on makeAllInserts() with a loop and if everything works then call setTransactionSuccessful() to do it in a batch operation. If something goes wrong, on the finally section you will call endTransaction() without setting the success, this will execute a rollback.
db.beginTransaction();
try {
makeAllInserts();
db.setTransactionSuccessful();
}catch {
//Error in between database transaction
}finally {
db.endTransaction();
}
For the makeAllInserts function, something like this could work out:
public void makeAllInserts() {
for(int i = 0; i < myData.size(); i++) {
myDataBase = openDatabase();
ContentValues values = new ContentValues();
values.put("name", myData.get(i).getName());
values.put("score", myData.get(i).getScore());
myDataBase.insert("MYTABLE", nullColumnHack, values);
}
}
If you also want to know about the nullColumnHack here you have a good link -> https://stackoverflow.com/a/2663620/709671
Hope it helps.
Part of my functionality requires updating a value in every row (only happens rarely, when a user selects a certain setting).
Trouble is, the query takes a good few minutes to perform (at best), and there's only 269 test records. Is there any way this could be optimized?
String allRecords = "SELECT id, weight FROM Workout_Entry";
Cursor cursor = db.rawQuery(allRecords, null);
int rows = cursor.getCount();
int id;
double weight;
try
{
if (cursor.moveToFirst())
{
do
{
for (int i = 0; i < rows; i++)
{
id = cursor.getInt(0);
weight = cursor.getInt(1) / 2.2;
String strFilter = "id = " + id;
ContentValues args = new ContentValues();
args.put("weight", weight);
db.update("Workout_Entry", args, strFilter, null);
}
} while (cursor.moveToNext());
}
} finally
{
cursor.close();
}
(db).close();
Thanks!
Just push the work to the database engine instead of pulling the data out one row at a time and firing up a new update query each time. Replace your code with something like:
db.execSQL("UPDATE Workout_Entry SET weight=weight/2.2");
Also, since this seems to be some kind of metric/imperial unit conversion, consider keeping the data in just one format in the database and convert/format to the appropriate unit for display purposes.
You should learn to use transactions - you can see example of how you use that in this presentation of mine.
Also showing the impact of not using the transaction.
Ok, I've realised my error. I had a do while loop for every record, and inside of that was a for loop which looped through x many times, where x was the amount of existing records.
So it was going through O(n)² times instead of O(n).
Thanks to those who replied! I found both of your comments useful.
I have seen a lot of posts on optimizing SQLITE on android with bulk inserts
Currently its taking 90 seconds to do 900 inserts/updates. I added the Begin/End Transaction around them but only saw minor improvements.
So I want to add I believe SQLiteStatement
Here is my code currently
static ArrayList<ContentSystem> csList = new ArrayList<ContentSystem>();
..fill out csList..
_dh.BeginTransaction(Table);
for(int i = 0; i < csList.size(); ++i)
{
addItem(ma, csList.get(i), dh, Table, Key);
}
_dh.setTransactionSuccessful();
_dh.EndTransaction(Table);
public static void addItem(final MainActivity ma, ContentSystem cs,
final DatabaseHandler dh, String Table, String Key)
{
worked = dh.UpdateItem(cs.cv, Table, Key, cs.UUID);
if (worked == 0)
{
worked = dh.AddData(Table, cs.cv);
if (worked <= 0)
{
ErrorLogger.AddError("Error")
}
}
}
My problem is that if my csList contains ~1000 items and some are already in my list, some are not
so I am currently doing a update if the update returns 0 then I do an add
How could I get something like this to work in a bulk statement?
A bit more info
dh.Update
int wok = db.update(table, values, Key + " = ?", new String[] { Item });
dh.Add
int work = (int) db.insert(table, null, values);
ContentSystem is a list of ContentValues
Try INSERT OR REPLACE, instead of either just an update or a failed update followed by an insert.
Are you sure the instantiated SQLiteDatabase object in your DatabaseHandler class is the same as _dh?
You might have started a transaction for _dh, but that might not even be the instantiated object that is actually doing any work.
I need to process a SQLite dataset in an Android system.
In my dataBaseHelper file (DataBaseAccessor) I have the following code (which when attached to a listview shows the relevant data).
public static ArrayList<QuestionListQuestion> getQuestionListQuestions(long id){
String qry = "select QuestionListQuestionID, QuestionListQuestionQuestionListID, QuestionListQuestionQuestionID, QuestionListQuestionSortOrder, QuestionListQuestionSupplementalQuestionIDYes, QuestionListQuestionSupplementalQuestionIDNo, QuestionListQuestionSupplementalQuestionIDText, QuestionListQuestionSurveyGroupID from QuestionListQuestion where QuestionListQuestionQuestionListID=" + id;
ArrayList<QuestionListQuestion> list = new ArrayList<QuestionListQuestion>();
try{
Cursor cursor = wdb.rawQuery(qry, null);
while (cursor.moveToNext()) {
QuestionListQuestion questionlistquestion = new QuestionListQuestion();
questionlistquestion.QuestionListQuestionID = cursor.getLong(0);
questionlistquestion.QuestionListQuestionQuestionListID = cursor.getLong(1);
questionlistquestion.QuestionListQuestionQuestionID = cursor.getLong(2);
questionlistquestion.QuestionListQuestionSortOrder = cursor.getLong(3);
questionlistquestion.QuestionListQuestionSupplementalQuestionIDYes = cursor.getString(4); questionlistquestion.QuestionListQuestionSupplementalQuestionIDNo = cursor.getString(5);
questionlistquestion.QuestionListQuestionSupplementalQuestionIDText = cursor.getString(6);
questionlistquestion.QuestionListQuestionSurveyGroupID = cursor.getLong(7);
list.add(questionlistquestion);
}
cursor.close();
}
catch (Exception e) {
e.printStackTrace();
}
return list;
}
I now need to extend the system so that I can process the data and create new records in another table based on the original list returned.
I tried the following attached to a button (selecting the relevant list ID from a spinner):-
QuestionListID = (String) SiteGenerateQuestions.this.spnQuestL.getSelectedItem().toString();
long SpinnerSelectedBT;
SpinnerSelectedBT = GenerateQuestions.this.spnQuestL.getSelectedItemId();
list = DatabaseAccessor.getQuestionListQuestions(SpinnerSelectedBT);
for (int i=0; i < list.size(); i++){
Toast.makeText(SiteGenerateQuestions.this," list.get(" + i + ") = " + list.get(i) + " " , Toast.LENGTH_SHORT).show();
}
The Toast displays the following:-
list.get(0) = com.tw.question.entity.QuestionListQuestion#407a6F70
list.get(1) = com.tw.question.entity.QuestionListQuestion#407bc170
etc...
How can I get access to the actual data instead of ... .entity.QuestionListQuestion#407bc170 or am I completely off-track?
Many Thanks
I agree with #wsanville the get() method will return the object in that location of the list. When you print out an object (in a toast, log, System.out.println etc) it will use the toString() value in the printout. The default toString() is the package name followed by # which is followed by a hex representation of that object. Your class will need to override the toString() method so when you use get() it will print out whatever you put in your toString() method.
The output you're seeing is because you haven't implemented the toString() method of your QuestionListQuestion class. Other than that, it seems like you do have the data you're looking for. Just try outputting the ID of your object, rather than concatenating the object itself (which will call toString() under the hood).
Also, since it looks like you're doing a database operation when a button gets clicked, make sure to do your database operations outside the UI thread. Check out the docs for a high level overview. You might want to use an AsyncTask for your database operation in question.