Deleting large sqlite table - android

I'm stuck in unique situation. In my app, I have local database. Somehow one of the tables got filled with junk data and database grew massively large. By the time I came to know, size was over 1 gb. So I kept getting database locked exception as reading that junk rows on that table was very very slow. After debugging, I figured out which table it was and now I want to delete it (either drop whole table or delete all rows) but whenever I perform any operation on that table, database gets locked and ANR happens. So I'm not sure how to get rid of that table. I can't uninstall app.
Note that I'm using GreenDao in my android app.
I've tried following things
1) DROP TABLE mytable;
2) DELETE FROM mytable;
3) DaoSession.getMyDao().deleteAll(); // this is a greedao method but it internally performs 2nd query I mentioned I think
In all the cases. It is just producing ANR.and app database gets locked.
Edit: I tried it in separate thread also, It just avoided ANR but did not drop table and database was still locked.
After lock I get this whenever other thread tried to write to db which is obvious.
E/SQLiteDatabase: Failed to open database '/data/user/0/myapp/databases/mydb.db'.
android.database.sqlite.SQLiteDatabaseLockedException: database is locked (code 5): , while compiling: PRAGMA journal_mode
#################################################################
Error Code : 5 (SQLITE_BUSY)
Caused By : The database file is locked.
(database is locked (code 5): , while compiling: PRAGMA journal_mode)
#################################################################
at android.database.sqlite.SQLiteConnection.nativePrepareStatement(Native Method)
at android.database.sqlite.SQLiteConnection.acquirePreparedStatement(SQLiteConnection.java:1000)
at android.database.sqlite.SQLiteConnection.executeForString(SQLiteConnection.java:704)
at android.database.sqlite.SQLiteConnection.setJournalMode(SQLiteConnection.java:385)
at android.database.sqlite.SQLiteConnection.setWalModeFromConfiguration(SQLiteConnection.java:359)
at android.database.sqlite.SQLiteConnection.open(SQLiteConnection.java:248)
at android.database.sqlite.SQLiteConnection.open(SQLiteConnection.java:199)
at android.database.sqlite.SQLiteConnectionPool.openConnectionLocked(SQLiteConnectionPool.java:514)
at android.database.sqlite.SQLiteConnectionPool.open(SQLiteConnectionPool.java:206)
at android.database.sqlite.SQLiteConnectionPool.open(SQLiteConnectionPool.java:178)
at android.database.sqlite.SQLiteDatabase.openInner(SQLiteDatabase.java:934)
at android.database.sqlite.SQLiteDatabase.open(SQLiteDatabase.java:895)
at android.database.sqlite.SQLiteDatabase.openDatabase(SQLiteDatabase.java:708)
at android.app.ContextImpl.openOrCreateDatabase(ContextImpl.java:646)
at android.content.ContextWrapper.openOrCreateDatabase(ContextWrapper.java:283)
at android.database.sqlite.SQLiteOpenHelper.getDatabaseLocked(SQLiteOpenHelper.java:223)
at android.database.sqlite.SQLiteOpenHelper.getReadableDatabase(SQLiteOpenHelper.java:187)
at com.intouchapp.database.IntouchDb.getNewReadableDaoSesssion(IntouchDb.java:80)
at com.intouchapp.models.ActivityLogsDb.getCursorOfAllResults(ActivityLogsDb.java:298)
at com.intouchapp.services.ActivityLogsDbInsertionService.onHandleIntent(ActivityLogsDbInsertionService.java:73)
at android.app.IntentService$ServiceHandler.handleMessage(IntentService.java:66)
at android.os.Handler.dispatchMessage(Handler.java:102)
at android.os.Looper.loop(Looper.java:148)
at android.os.HandlerThread.run(HandlerThread.java:61)
Also I get this too,
The connection pool for database '+data+user+0+myapp+databases+mydb' has been unable to grant a connection to thread 2262 (SyncAdapterThread-1) with flags 0x2 for 12.0060005 seconds.
Connections: 1 active, 0 idle, 0 available.
Requests in progress:
executeForChangedRowCount started 12645ms ago - running, sql="DELETE FROM 'mytable'"
This clearly states that It is not able to execute "DELETE FROM 'mytable'" this particular query hence the lock.

SQLite locks database when it is used to write operation.
You should take care about some points regarding this :-
DataBase operation should use in another Thread not in UI Thread.
Make a single instance of SQLiteOpenHelper class
close all the instances of database after finishing task
always use endTransaction()
close all instance of database in finally block

Related

Move Table from One Database to another

I have two databases one is SQLite db_1 and the other SQLCipher db_2. I want to move a table from db_1 to db_2.
Since I am using SQLCipher in one of the databases, I can't ATTACH the databases and copy Table and its contents from one to the other.
Is it possible with either database queries or Android Code or both to somehow move the table to db_2 and then delete Table in db_1 in onUpgrade().
Note :
When trying to ATTACH the database I got the following error
Failure 26 (file is encrypted or is not a database) on 0xab61cb98 when executing 'ATTACH DATABASE '/data/data/com.example/databases/DATABASE.db' AS DB;'

React-Native: Async Storage no such table

At the opening of my app I'm trying to get some preferences via AsyncStorage.multiGet or AsyncStorage.getItem. Unfortunately the very first time I open the app, the SQLite table used by AsyncStorage in Android, doesn't exist. And so the app crashes. How should I handle the error to avoid having a red death screen.
FYI the exception thrown is :
no such table: catalystLocalStorage (code 1): , while compiling: INSERT OR REPLACE INTO catalystLocalStorage VALUES (?, ?);
Thanks

Attempt to write a readonly database... but I'm not

I have a read-only database connection. Sometimes, when reading data from the database with a SELECT query, it throws a SQLiteReadOnlyDatabaseException.
I open the connection like this:
return SQLiteDatabase.openDatabase(path, null, SQLiteDatabase.OPEN_READONLY);
The query is:
Select * FROM BudgetVersions WHERE entityId = ?
I read data from the database using db.rawQuery(), like this:
String query = ...;
Cursor c = db.rawQuery(query, new String[]{ activeBudgetId });
try {
if (c.moveToFirst()) {
bv.versionName = c.getString(c.getColumnIndexOrThrow("versionName"));
return bv;
} else {
return null;
}
} finally {
c.close();
}
Very rarely, I get a crash like this, inside the call to c.moveToFirst():
Caused by: android.database.sqlite.SQLiteReadOnlyDatabaseException: attempt to write a readonly database (code 776)
at android.database.sqlite.SQLiteConnection.nativeExecuteForCursorWindow(Native Method)
at android.database.sqlite.SQLiteConnection.executeForCursorWindow(SQLiteConnection.java:845)
at android.database.sqlite.SQLiteSession.executeForCursorWindow(SQLiteSession.java:836)
at android.database.sqlite.SQLiteQuery.fillWindow(SQLiteQuery.java:62)
at android.database.sqlite.SQLiteCursor.fillWindow(SQLiteCursor.java:144)
at android.database.sqlite.SQLiteCursor.getCount(SQLiteCursor.java:133)
at android.database.AbstractCursor.moveToPosition(AbstractCursor.java:197)
at android.database.AbstractCursor.moveToFirst(AbstractCursor.java:237)
As a workaround, I could try using a writable database connection instead, but I'd like to know why the crash is happening.
The table I'm reading from is a standard SQLite table:
CREATE TABLE BudgetVersions (
entityId VARCHAR PRIMARY KEY NOT NULL UNIQUE,
budgetId VARCHAR NOT NULL,
versionName VARCHAR NOT NULL,
dateFormat VARCHAR,
currencyFormat VARCHAR,
lastAccessedOn DATETIME,
isTombstone BOOL NOT NULL,
deviceKnowledge NUMERIC NOT NULL
);
I've seen the crash happen on both a KitKat emulator and a device running Lollipop.
There is a separate writeable connection open to the same database at the same time, owned by a WebView. The database is being updated by Javascript code in the WebView, and read from in the native Android/Java layer with this read-only connection.
I expect this may prove to be the ultimate cause of the problem, but I'd like to understand in detail why a read-only connection would interfere with a separate writeable connection.
I am well aware that the general advice is to use a single connection to the database, but since the writeable connection is owned by the WebView, I don't have easy access to it from the Java code.
Solved by changing it to a writeable database connection. The clue was in the documentation for the 776 error code:
(776) SQLITE_READONLY_ROLLBACK
The SQLITE_READONLY_ROLLBACK error code is an extended error code for
SQLITE_READONLY. The SQLITE_READONLY_ROLLBACK error code indicates
that a database cannot be opened because it has a hot journal that
needs to be rolled back but cannot because the database is readonly.
During development, I am frequently interrupting the currently-running app to install and run a new version. This causes the currently-running app to be force-stopped by the system. If the Javascript code in the WebView is in the middle of writing to the database via its separate writeable connection when the app is nuked, then a hot journal will be left behind.
When the new version of the app starts up, the read-only database connection in the native Java code is opened. When this connection spots the journal, it tries to roll back the journal. And because it's a read-only connection, it fails.
(This fits with the crash being observed immediately on startup after I've made a change.)
The correct fix is therefore to make the Java connection a writeable connection. This connection never attempts a write during normal operation, but it must write when recovering from a previous interrupted write through the WebView's writeable connection.

drop or delete a sqlite - table as fast as possible on Android device

I have a table with 1400 rows. Every row has a blob field which holds data between 10kb and 500kb. I need to delete that table. It takes me 3.5 minutes to delete the table and 3 minutes to drop the table. Thats too long for the users.
How can I remove that table as fast as possible ? ( No rollback needed or any security, just remove it. )
I already tried the following.
1. Set pagesize :
sqlitedatabase.setPageSize(8000);
sqlitedatabase.execSQL("DROP TABLE IF EXISTS " + sTableName);
2. deactivate journallog which did not work.
sqlitedatabase.rawQuery("PRAGMA journal_mode=OFF",null);
sqlitedatabase.execSQL("DROP TABLE IF EXISTS " + sTableName);
this doesn't work for me. journal log, which I guess takes a lot of time, is still be written on to the disk.
From the SQLite manual (with emphasis added):
SQLite is slower than the other databases when it comes to dropping tables. This probably is because when SQLite drops a table, it has to go through and erase the records in the database file that deal with that table. MySQL and PostgreSQL, on the other hand, use separate files to represent each table so they can drop a table simply by deleting a file, which is much faster.
You do have the option of creating and storing multiple database files, which you can then manage from a single connection with ATTACH and DETACH queries.
Here's an example I just ran in SQLite3's command-line client:
sqlite> ATTACH 'example.sqlite' AS example;
sqlite> CREATE TABLE example.ex ( a INTEGER );
sqlite> INSERT INTO example.ex VALUES (1),(2),(3);
sqlite> SELECT * FROM example.ex;
1
2
3
sqlite> DETACH example;
sqlite>
Since the ex table is in its own file, example.sqlite, I can simply detach that DB from the connection and delete the entire file, which will be much faster.
Bear in mind that the number of DBs you can attach is fairly low (with default compile options: 7). I've also read that foreign keys aren't supported in this scenario, though that info might be out of date.
I got a solution for me, which speeds up the deletion 6 times.
with
connection_read.enableWriteAheadLogging();
I drop my table in 30 Seconds. Without it it takes the mentoined 3 minutes.
enableWriteAheadLogging is an alternative journal log which is way faster.
Why not just put the drop table code in a separate thread? The user shouldn't have to wait for the app to drop a table.

How to delete thousands of records from sqlite android

I have more than 5000-6000 records in SQLite table. When I delete this all records it takes very long time and causes screen pause and starts releasing resources.
I tried it with AsyncTask but still the same problem. So can anyone tell how should I delete this thousands of records without blocking app.
I am no expert to Sqlite but in general there are 3 ways to do that.
As every body commented Truncate if you are going to delete all record.
If you are going to majority of the records you can store the non-delete files in tempTable and then truncate your actual table finally insert all the records from temp to actual table
This one is what I use most of the time. Use a Top XXX delete statement in your case you can delete 200 records in every 2min. (I am assuming you don t insert more than 200 records in 2 min). The AsyncTask is the way for that kind of approach.
In T-SQL I use the following sql to it is up to you
Delete From tUser
where UserId in (
Select top 200 UserId
From tUser
where LastLoggin< GetDate()-120
)
If you want to delete all records in a table you could try dropping the table:
http://www.sqlite.org/lang_droptable.html
and later re-creating an empty table:
http://www.sqlite.org/lang_createtable.html

Categories

Resources