I've got two SQLite databases, each with a table that I need to keep synchronized by merging rows that have the same key. The tables are laid out like this:
CREATE TABLE titles ( name TEXT PRIMARY KEY,
chapter TEXT ,
page INTEGER DEFAULT 1 ,
updated INTEGER DEFAULT 0 );
I want to be able to run the same commands on each of the two tables, with the result that for pairs of rows with the same name, whichever row has the greater value in updated will overwrite the other row completely, and rows which do not have a match are copied across, so both tables are identical when finished.
This is for an Android app, so I could feasibly do the comparisons in Java, but I'd prefer an SQLite solution if possible. I'm not very experienced with SQL, so the more explanation you can give, the more it'll help.
EDIT
To clarify: I need something I can execute at an arbitrary time, to be invoked by other code. One of the two databases is not always present, and may not be completely intact when operations on the other occur, so I don't think a trigger will work.
Assuming that you have attached the other database to your main database:
ATTACH '/some/where/.../the/other/db-file' AS other;
you can first delete all records that are to be overwritten because their updated field is smaller than the corresponding updated field in the other table:
DELETE FROM main.titles
WHERE updated < (SELECT updated
FROM other.titles
WHERE other.titles.name = main.titles.name);
and then copy all newer and missing records:
INSERT INTO main.titles
SELECT * FROM other.titles
WHERE name NOT IN (SELECT name
FROM main.titles);
To update in the other direction, exchange the main/other database names.
For this, you can use a trigger.
i.e.
CREATE TRIGGER sync_trigger
AFTER INSERT OR UPDATE OF updated ON titles
REFERENCING NEW AS n
FOR EACH ROW
DECLARE updated_match;
DECLARE prime_name;
DECLARE max_updated;
BEGIN
SET prime_name = n.name;
ATTACH database2name AS db2;
SELECT updated
INTO updated_match
FROM db2.titles t
WHERE t.name=prime_name)
IF updated_match is not null THEN
IF n.updated > updated_match THEN
SET max_updated=n.updated;
ELSE
SET max_updated=updated_match;
END IF;
UPDATE titles
SET updated=max_updated
WHERE name=prime_name;
UPDATE db2.titles
SET updated=max_updated
WHERE name=prime_name;
END IF;
END sync_trigger;
The syntax may be a little off. I don't use triggers all that often and this is a fairly complex one, but it should give you an idea of where to start at least. You will need to assign this to one database, exchanging "database2name" for the other database's name and then assign it again to the other database, swapping the "database2name" out for the other database.
Hope this helps.
Related
I have an SQLite DB where I perform a query like
Select * from table where col_name NOT IN ('val1','val2')
Basically I'm getting a huge list of values from server and I need to select the ones which is not present in the list given.
Currently its working fine, No issues. But the number of values from server becomes huge as the server DB is getting updated frequently.
So, I may get thousands of String values which I need to pass to the NOT IN
My question is, Will it cause any perfomance issue in the future? Does the NOT IN parameter have any size restriction? (like max 10000 values you can check)?
Will it cause any crash at some point?
This is an official reference about various limitation in sqlite. I think the Maximum Length Of An SQL Statement may related to your case. Default value is 1000000, and it is adjustable.
Except this I don't think any limitation existed for numbers of parameter of NOT IN clause.
With more than a few values to test for, you're better off putting them in a table that has an index on the column holding them. Then things like
SELECT *
FROM table
WHERE col_name NOT IN (SELECT value_col FROM value_table);
or
SELECT *
FROM table AS t
WHERE NOT EXISTS (SELECT 1 FROM value_table WHERE value_col = t.col_name);
will be reasonably efficient no matter how many records are in value_table because that index will be used to find entries.
Plus, of course, it makes it a lot easier to re-use prepared statements because you don't have to create a new one and re-bind every value (You are using prepared statements with placeholders for these values, right, and not trying to put their contents inline into a string?) every time you add a value to the ones you need to check. You just insert it into value_table instead.
Yes, there is a limit of 999 arguments as reported in the official documentation: https://www.sqlite.org/limits.html#max_variable_number
I have table that for the specified rows with specified ids need to change the value, while for previously selected rows should be reset.
Do I need to reset the whole table and then update for specified rows. is there any option to update table with only one query.
I'm using room persistence on android
Like any database standard, Room Update and Delete are separate operation types.
Then maybe you can try to execute Trigger if you need mixed operation (thread about trigger).
But for what reason do you have to execute this two operations in a same query ?
Say my SQLite Databate has 2 columns, the first being an auto-incrementing ID and the 2nd being some string. Say right now it's
1 random
2 jellybean
3 ImTired
if I were to delete entry 2, it would then be
1 random
3 ImTired
What I want is a way to make it so when you delete entry 2, it turns it into
1 random
2 ImTired
I thought about updating the entries to shift them all down one and delete the last one, but even if it worked(in my case, it deleted all of my entries, but whatever...), and even if I did get it to
1 random
2 ImTired
the next time I create a new entry, it'll be entry 4. I don't think this necessary to my app, but it seriously bugs me.
The ID column on your DB is working as a Primary Key, which is a column or group of columns used to uniquely identify a row. Once you set a Primary Key on a row you shouldn't change it, else you risk losing the consistency of the DB. For instance, suppose you later create another table that references the rows in your first table. That reference will be made using the Primary Key, and if you later change it your data won't make sense anymore.
If you wanted the ID column to keep changing just to reflect the number of rows in your table you can solve that problem with other methods. For instance. SQL offers a COUNT operator that will return the number of rows in your table:
SELECT COUNT(*) FROM Table_name;
I am fetching my data with id which is Integer primary key or integer.
But after deleting any row...
After that if we make select query to show all.
But it will give force close because one id is missing.
I want that id can itself take auto increment & decrement.
when i delete a record at the end(i.g. id=7) after this i add a row then id must be 7 not 8. as same when i delete a row in middle(i.g. id=3) then all the row auto specify by acceding.
your idea can help me.
Most systems with auto-incrementing columns keep track of the last value inserted (or the next one to be inserted) and do not ever reissue a number (give the same number twice), even if the last number issued has been removed from the table.
Judging from what you are asking, SQLite is another such system.
If there is any concurrency in the system, then this is risky, but for a single-user, single-app-at-a-time system, you might get away with:
SELECT MAX(id_column) + 1 FROM YourTable
to find the next available value. Depending on how SQLite behaves, you might be able to embed that in the VALUES list of an INSERT statement:
INSERT INTO YourTable(id_column, ...)
VALUES((SELECT MAX(id_column) + 1 FROM YourTable), ...);
That may not work; you may have to do this as two operations. Note that if there is any concurrency, the two statement form is a bad ideaTM. The primary key unique constraint normally prevents disaster, but one of two concurrent statements fails because it tries to insert a value that the other just inserted - so it has to retry and hope for the best. Clearly, a cell phone has less concurrency than, say, a web server so the problem is correspondingly less severe. But be careful.
On the whole, though, it is best to let gaps appear in the sequence without worrying about it. It is usually not necessary to worry about them. If you must worry about gaps, don't let people make them in the first place. Or move an existing row to fill in the gap when you do a delete that creates one. That still leaves deletes at the end creating gaps when new rows are added, which is why it is best to get over the "it must be a contiguous sequence of numbers" mentality. Auto-increment guarantees uniqueness; it does not guarantee contiguity.
I am wondering how can I insert an element at the beginning of the data base? I want to do this because:
a) I want to display the elements (ListView) as 'last-inserted on top and first-inserted on the bottom' (like a stack)
b) I want to limit the amount of elements in my db. When a new element is added (over the limit) I can put the new one (at the beginning) and delete the last one. (Don't know yet how to delete the last element).
I was searching for a solution but I am starting to wonder I have any control of how the element are inserted. If not I was thinking about displaying the database from end to bottom. But don't actually know how to do it since the cursor is always set at the beginning of the db. If could achieve this I can solve b) by deleting the first element (again don't know how to achieve this yet).
Every row of every SQLite table has a 64-bit signed integer key that uniquely identifies the row within its table. This integer is usually called the "rowid"
To get last inserted at top and first inserted at bottom....
SELECT * from mytable order by ROWID desc;
Read more about ROWID here
And, to delete the oldest:
DELETE from mytable WHERE ROWID = (SELECT MIN(ROWID) FROM mytable);
If you use the ROWID, you do not have to modify your existing table. And, the value of ROWID is guaranteed.
You can store a timestamp (as a new field) when you insert a record into your database. Then you delete the oldest record you can find if it is over the limit (e.g. using MAX SQL keyword). You can display the records in reverse-chronological order by sorting DESC with your timestamp field.