Is there a way to access the SQLiteLog or at least pipe out errors to do with SQLite?
I would like to automatically send any errors, like the below, so I can optimise the database by adding in indexes, or making other changes as need be.
E/SQLiteLog: (284) automatic index on messages(chat_id)
It could well be that I'd only want to catch code 284, rather than getting everything. Is there a way of doing this when building the app, as would be useful to pass to crashlytics to help with development going forward
A potential alternative, would be to run the queries preceded with EXPLAIN QUERY PLAN, and to then check for AUTOMATIC COVERING INDEX in the results.
e.g.
DROP TABLE IF EXISTS table1;
DROP TABLE IF EXISTS table2;
CREATE TABLE IF NOT EXISTS table1 (column1 TEXT, column2 TEXT);
CREATE TABLE IF NOT EXISTS table2 (column3, column4);
EXPLAIN QUERY PLAN
SELECT * FROM table1, table2 WHERE column1=column3;
Results in :-
As you can see this is based upon a predication of the number of rows rather than the actual number of rows, so works if there is no data (as above), which could well be beneficial, from a development perspective, as opposed to trying to trap on a as happens basis.
Related
I have an already functioning app running on iOS whose database uses a composite primary key. For discussions sake, lets say "CID" and "RID" make up that composite pk, resulting in something that looks like:
CID-RID
F6uuDTEU1c-1
F6uuDTEU1c-2
F6uuDTEU1c-3
However, there are conditions under which the CID column is altered, resetting the RID column. For example:
CID-RID
...
F6uuDTEU1c-4
F6uuDTEU1c-5
WQq6JnyrDI-1
WQq6JnyrDI-2
WQq6JnyrDI-3
...etc
These databases are to be shared cross-platform (ios - android) and going back and editing the current ios structure is not an option. What issues am I going to run into not having an _id column as my pk running on Android?
I found this here on SO - which seems to state that the db itself does not have to have the _id column, only that ...
"The result set for the cursor must contain _id, not the cursor itself."
... but I could be reading this all wrong. Any input/help is much appreciated.
PS: I already looked at a few (what I thought were) similar questions here, here, and here.
You are free to have any database schema you want. Android doesn't impose any additional restrictrions there.
Only if you use a CursorAdapter, then the Cursor needs an _id column. Any app can be written without using CursorAdapter, it's just there to provide some convenience. sqlite tables always have a ROWID column that aliases to the INTEGER PRIMARY KEY column if the table has one. You can always select it as the _id, e.g. SELECT rowid AS _id ... if needed.
I have a table with 1400 rows. Every row has a blob field which holds data between 10kb and 500kb. I need to delete that table. It takes me 3.5 minutes to delete the table and 3 minutes to drop the table. Thats too long for the users.
How can I remove that table as fast as possible ? ( No rollback needed or any security, just remove it. )
I already tried the following.
1. Set pagesize :
sqlitedatabase.setPageSize(8000);
sqlitedatabase.execSQL("DROP TABLE IF EXISTS " + sTableName);
2. deactivate journallog which did not work.
sqlitedatabase.rawQuery("PRAGMA journal_mode=OFF",null);
sqlitedatabase.execSQL("DROP TABLE IF EXISTS " + sTableName);
this doesn't work for me. journal log, which I guess takes a lot of time, is still be written on to the disk.
From the SQLite manual (with emphasis added):
SQLite is slower than the other databases when it comes to dropping tables. This probably is because when SQLite drops a table, it has to go through and erase the records in the database file that deal with that table. MySQL and PostgreSQL, on the other hand, use separate files to represent each table so they can drop a table simply by deleting a file, which is much faster.
You do have the option of creating and storing multiple database files, which you can then manage from a single connection with ATTACH and DETACH queries.
Here's an example I just ran in SQLite3's command-line client:
sqlite> ATTACH 'example.sqlite' AS example;
sqlite> CREATE TABLE example.ex ( a INTEGER );
sqlite> INSERT INTO example.ex VALUES (1),(2),(3);
sqlite> SELECT * FROM example.ex;
1
2
3
sqlite> DETACH example;
sqlite>
Since the ex table is in its own file, example.sqlite, I can simply detach that DB from the connection and delete the entire file, which will be much faster.
Bear in mind that the number of DBs you can attach is fairly low (with default compile options: 7). I've also read that foreign keys aren't supported in this scenario, though that info might be out of date.
I got a solution for me, which speeds up the deletion 6 times.
with
connection_read.enableWriteAheadLogging();
I drop my table in 30 Seconds. Without it it takes the mentoined 3 minutes.
enableWriteAheadLogging is an alternative journal log which is way faster.
Why not just put the drop table code in a separate thread? The user shouldn't have to wait for the app to drop a table.
In Android SQLite I had one table MyTable. By mistake I dropped it after upgrade the DB.
How can I ROLL BACK that dropped table if it's possible.
Any good answer will be accepted.
Thanks.
Dropping tables is not a recoverable action, unless performed as part of a transaction that is rolled back (which appears to be not the scenario for your particular case).
From the SQLite documentation:
The DROP TABLE statement removes a table added with the CREATE TABLE statement. The name specified is the table name.
The dropped table is completely removed from the database schema and the disk file. The table can not be recovered. All indices and triggers associated with the table are also deleted.
That's not quite the complete picture, as the behaviour under rolled-back transaction can be seen with (tested on https://sqliteonline.com/):
drop table if exists paxtable;
create table paxtable (paxcolumn integer);
insert into paxtable values (42);
begin transaction;
drop table paxtable;
rollback;
select paxcolumn from paxtable;
That shows that the table still exists after the rollback. If you commit rather than roll back (or if you remove the transactional control altogether), the table has died, expired, gone to meet its maker, shuffled off this mortal coil, <insert your favourite metaphor here>.
So, since you didn't do it as part of a rolled-back transaction (as evidenced by the fact the table has actually gone), you'll need to re-create it from scratch (or from backups if possible).
CLARIFICATION:
Although you can commit or rollback DML statements like "insert" or "delete" (provided you do it within a transaction), in general you cannot rollback a DDL statement like "alter table" or "drop table".
This is true for most databases under most circumstances: Oracle, MSSQL, mySQL, etc.
There is an exception for sqlite: if you drop table in a transaction, then a rollback will restore that table.
Otherwise (per the sqlite manual):
http://sqlite.org/lang_droptable.html
The DROP TABLE statement removes a table added with the CREATE TABLE
statement. The name specified is the table name. The dropped table is
completely removed from the database schema and the disk file. The
table can not be recovered. All indices and triggers associated with
the table are also deleted.
PS:
This link discusses "DDL", "DML" and related acronyms, if you're interested:
http://www.orafaq.com/faq/what_are_the_difference_between_ddl_dml_and_dcl_commands
I think there's two different interpretations of this question and I want to make sure both get answered and demonstrated conclusively since this is still the top search result for sqlite drop table rollback (and the links to the SQLite documentation seems misleading as well).
To the first question, you can rollback a drop table DDL action that occurs within a transaction scope, i.e.,
// with connection created
var transaction = connection.BeginTransaction();
try {
// create a table
// drop a different table
transaction.Commit(); // go ahead and commit if everything is successfully
}
catch
{
transaction.Rollback(); // rollback in case of error
}
And to confirm this is the same behavior in a language-agnostic fashion, here's the same behavior using the SQ Lite command-line shell:
sqlite3 demo
// outside transaction scope
create table tbl1( col varchar(10));
// list the current tables
.tables
// start a transaction that wraps both your DDL commands
begin transaction;
enter code here
create table tbl2 (col varchar(10));
drop table tbl1;
rollback;
.tables
The expectation is that the final list tables command, should still return tbl1 since both the create table and drop table commands were both rolled back. Another way to say this is that SQLite is not bound by the same DML/DDL distinction for what operations can be rolled back that are present in Oracle.
For the second interpretation of the question, i.e., can I recover a table dropped outside of a transaction scope (which would also entail the "Oh S#%T" experience you may have had as a developer as well as disaster recovery), the references to the SQ Lite documentation are appropriate:
The dropped table is completely removed from the database schema and the disk file. The table can not be recovered. All indices and triggers associated with the table are also deleted.
I'm having an issue performing normal SELECT statements (that is, not the 'searches' they were designed to do- I'm not using MATCH). What I'm trying to do is query the table for all rows of an INTEGER column. However, the SELECT statement always returns no rows. I've inspected the database with an SQLite browser and the query should work. Here it is:
Here's the MAKE TABLE statement:
CREATE VIRTUAL TABLE FTS_journal USING fts3(journal_id INTEGER, journal_text_col TEXT)
And here's the SELECT:
SELECT journal_id FROM FTS_journal
Does it have something to do with my running a 'normal' query over a virtual table? I can't really think of any other reason
EDIT: I'm using Android's version of SQLite, meaning SQLite3. I know it supports FTS properly...
If any further information I can provide will help, please tell me and I will post it.
What you've written should work. Are you 100% positive you've actually inserted something into the table? Have you ran insert statements, and if so, are you sure they were successful?
I tested your statements, running the create statement, then the following statement:
insert into FTS_journal (journal_id, journal_text_col) values (1, 'test');
... then your select statement, and sqlite returned 1 row (journal_id, value 1) just as it should.
I have a sqlite db that at the moment has few tables where the biggest one has over 10,000 rows. This table has four columns: id, term, definition, category. I have used a FTS3 module to speed up searching which helped a lot. However, now when I try to fetch 'next' or 'previous' row from table it takes longer than it was before I started using FTS3.
This is how I create virtual table:
CREATE VIRTUAL TABLE profanity USING fts3(_id integer primary key,name text,definition text,category text);
This is how I fetch next/previous rows:
SELECT * FROM dictionary WHERE _id < "+id + " ORDER BY _id DESC LIMIT 1
SELECT * FROM dictionary WHERE _id > "+id + " ORDER BY _id LIMIT 1
When I run these statements on the virtual table:
NEXT term is fetch within ~300ms,
PREVIOUS term is fetch within ~200ms
When I do it with normal table (the one created without FTS3):
NEXT term is fetch within ~3ms,
PREVIOUS term is fetch within ~2ms
Why there is such a big difference? Is there any way I can improve this speed?
EDITED:
I still can't get it to work!
Virtual table you've created is designed to provide full text queries. It's not aimed to fast processing standard queries using PK in where condition.
In this case there is no index on your _id column, so SQLite probably performs full table scan.
Next problem is your query - it's totally inefficient. Try something like this (untested):
SELECT * FROM dictionary WHERE _id = (select max(_id) from dictionary where _id < ?)
Next thing you can consider is redesign of your app. Instead of loading 1 row you, maybe you should get let's say 40, load them into memory and make background data loading when there is less than n to one of the ends. Long SQL operation will become invisible to user even if it'll last 3s instead of 0,3s
If you're running LIMIT 1 to begin with, you can remove the order by clause completely. This may help. I'm not familiar with FTS3, however.
You could also just flat out assign your id variable a ++ or -- and assert `WHERE _id = "+id+" LIMIT 1" which would make a single lookup instead of < or >.
Edit: and now that I look back at what I typed, if you do it that way, you can just remove LIMIT 1 completely, since your _id is your pk and must be unique.
hey look, a raw where clause!