I'd like to optimize an app, kinda scheduler. I have tasks with different statuses like done, to do and archived(after delete). App logic uses the first two types the most, so should I keep them in the same table(and just selecting done & to do every time I need) or should I do a copy table just for archived items to make the select "easier" because status archived is very rarely used.
So does sqLite db size affects somehow db operations or not? I tested select operations in big tables(only one in whole db) and it take a long time to process 100K - 1M rows
SQLite has no problem with big tables and 1 millions records is not really big.
You should use only multiple tables if it makes sense logically, performance is not an issue. So your only table could look like this:
tasks table
---------------------------
id (primary key)
title (indexed)
description
status (or status_id) (indexed)
A rule of thumb is to index all columns you use in the where clauses of your queries.
If you use strings as status in your DB then you could use another table and refer to it in your tasks table with a foreign key to the id of this table:
status table
------------
id
name
Related
Is there a way to access the SQLiteLog or at least pipe out errors to do with SQLite?
I would like to automatically send any errors, like the below, so I can optimise the database by adding in indexes, or making other changes as need be.
E/SQLiteLog: (284) automatic index on messages(chat_id)
It could well be that I'd only want to catch code 284, rather than getting everything. Is there a way of doing this when building the app, as would be useful to pass to crashlytics to help with development going forward
A potential alternative, would be to run the queries preceded with EXPLAIN QUERY PLAN, and to then check for AUTOMATIC COVERING INDEX in the results.
e.g.
DROP TABLE IF EXISTS table1;
DROP TABLE IF EXISTS table2;
CREATE TABLE IF NOT EXISTS table1 (column1 TEXT, column2 TEXT);
CREATE TABLE IF NOT EXISTS table2 (column3, column4);
EXPLAIN QUERY PLAN
SELECT * FROM table1, table2 WHERE column1=column3;
Results in :-
As you can see this is based upon a predication of the number of rows rather than the actual number of rows, so works if there is no data (as above), which could well be beneficial, from a development perspective, as opposed to trying to trap on a as happens basis.
For an Android word game (with minSdkLevel=9 meaning SQLite version 3.6.22) -
I would like to deliver the dictionary as a prefilled SQLite table within the APK file (with the help of SQLiteAssetHelper).
In the SQLite database there will be just 1 table:
create table dict ( /* contains 700 000 unique words */
word text not null
);
My question please:
How to declare the table for the best performance and which kind of SQL-query to use?
(When checking if a word entered by player is present in the dict table or not - that will be the main usage of the SQLite database in the app).
Should I create index (is it possible to have index for text columns at all)?
Or should I declare the word column as primary key?
Also, some SQLite for Android guides suggest to have an _id column in each table (probably to enable fetching the last inserted record? - which I don't really need here). Should I maybe use
create table dict (
_id integer primary key,
word text unique not null
);
create index word_index on dict(word);
or will that be a waste of 4 x 700 000 bytes? (Or is it added as _rowid_ anyway?)
Quick answer: yes, you can create index on text column.
However for best performance, this may not be the best option.
Because the index created by SQLite should be simply a b-tree (binary tree), which speed up the search by binary search. i.e. with 700k words, the binary search has to run about 20 intervals. But this could be fast enough, you need to test it to actually know the performance.
Some alternative methods would be to create multiple tables (buckets), e.g. create table as wordA, wordB, wordC etc.
And use the first character to determine which table the word is put.
This drops the size of each table to contains about 27k records. (of course each bucket is not of equal size)
By doing this, it reduces the interval used performing the binary search.
And actually you should use hash function to determine the bucket, which makes the size of each buckets more balanced and you can freely control the number of buckets.
And you have to actually fine tune to know what is the optimal bucket size.
I have a table with 1400 rows. Every row has a blob field which holds data between 10kb and 500kb. I need to delete that table. It takes me 3.5 minutes to delete the table and 3 minutes to drop the table. Thats too long for the users.
How can I remove that table as fast as possible ? ( No rollback needed or any security, just remove it. )
I already tried the following.
1. Set pagesize :
sqlitedatabase.setPageSize(8000);
sqlitedatabase.execSQL("DROP TABLE IF EXISTS " + sTableName);
2. deactivate journallog which did not work.
sqlitedatabase.rawQuery("PRAGMA journal_mode=OFF",null);
sqlitedatabase.execSQL("DROP TABLE IF EXISTS " + sTableName);
this doesn't work for me. journal log, which I guess takes a lot of time, is still be written on to the disk.
From the SQLite manual (with emphasis added):
SQLite is slower than the other databases when it comes to dropping tables. This probably is because when SQLite drops a table, it has to go through and erase the records in the database file that deal with that table. MySQL and PostgreSQL, on the other hand, use separate files to represent each table so they can drop a table simply by deleting a file, which is much faster.
You do have the option of creating and storing multiple database files, which you can then manage from a single connection with ATTACH and DETACH queries.
Here's an example I just ran in SQLite3's command-line client:
sqlite> ATTACH 'example.sqlite' AS example;
sqlite> CREATE TABLE example.ex ( a INTEGER );
sqlite> INSERT INTO example.ex VALUES (1),(2),(3);
sqlite> SELECT * FROM example.ex;
1
2
3
sqlite> DETACH example;
sqlite>
Since the ex table is in its own file, example.sqlite, I can simply detach that DB from the connection and delete the entire file, which will be much faster.
Bear in mind that the number of DBs you can attach is fairly low (with default compile options: 7). I've also read that foreign keys aren't supported in this scenario, though that info might be out of date.
I got a solution for me, which speeds up the deletion 6 times.
with
connection_read.enableWriteAheadLogging();
I drop my table in 30 Seconds. Without it it takes the mentoined 3 minutes.
enableWriteAheadLogging is an alternative journal log which is way faster.
Why not just put the drop table code in a separate thread? The user shouldn't have to wait for the app to drop a table.
In my Android app, I need to temporarily store some data in a form of table such as follows:
id | column 1 | column 2 | ... | column n
The data are downloaded from a server whenever users press a button. However, the data table doesn't have a fix number of column (as well as row) every time user downloads it from the server. For example, the server may send data with 3 columns the first time. Then it might send data with 5 columns the second time, etc...
Given this scenario, I think the database is probably the right data structure to use. My plan is to create a database, then add and delete tables as necessary. So I have been reading various tutorials on Android database (one example is this one http://www.codeproject.com/Articles/119293/Using-SQLite-Database-with-Android#). It seems to me I cannot create new table with variable number of columns using the sqlite database. Is this correct? In the onCreate(SQLiteDatabase db) method, the "create table" command must be specified with known number of columns and their data types. I could provide several "create table" commands, each with different number of columns but that seems like very crude. Is there a way to create database tables with variable number of columns on the fly?
Another alternative probably using several hash tables, each storing one column of the data table. I'm seriously considering this approach if the database approach is not possible. Any better suggestion is welcomed.
There is no such thing as a variable number of columns in an SQLite data base. Also, adding and deleting tables dynamically seems like a horrible hack.
It sounds like you want to store an array of values associated with an id. I suggest you think in terms of rows, not columns. Use a table structure like (id, index, value); each array of values returned by the server results in as many rows as necessary to store the values.
I am creating a db for my android app which requires different tables for different types of users. Basically, if user type A logs in he will use Table A, user type B logs in he will use Table B and similarly Table C.
Should I go ahead and create 3 tables and use only the table which the user type has logged in? Would this have any significant memory implications (size of the db)? That is, if the table doesn't contain data, would the sqlite db still allocate some space (other than for definitions of the columns).
The other way would be to check the user type and dynamically create the table at runtime once user logs in. I prefer the first method if there isn't much space used.
Any pointers or suggestions would be very useful.
The SQLite DB will occupy some space to store the table structures and the constraints/indexes. Why don't u just create the database tables (w/o any data) using SQLite Browser and check the size of the database file.
Would help you decide.
I think instead of creating all tables at one go use SQL queries to create tables as and when needed. If its not needed anymore you can even drop the table.