I am developing an android application which have sqlite db on my device and sql server db on the main machine, the android device contain very small part of main db, so i don't want to replicate whole db. and I have my own webserver that upload/download data between databases,
my question is how I know which part of main database is changed to only download changed entries?
One usefull technique is using rowversion data type. You can see it as the server change number. You can get the current change number by using the MIN_ACTIVE_ROWVERSION() function. Then query from changes occurred form last downloaded change number to current change number and finally store the current change number as the last downloaded change number in the local SQLite database.
Something like this:
DECLARE #CurrentServerChange binary(8)
SET #CurrentServerChange = MIN_ACTIVE_ROWVERSION()
SELECT * FROM Table1
WHERE
RecordVersion >= #LastDownloadedChangeNumber
AND RecordVersion < #CurrentServerChange
SELECT * FROM Table3
WHERE
RecordVersion >= #LastDownloadedChangeNumber
AND RecordVersion < #CurrentServerChange
SELECT #CurrentServerChange AS CurrentServerChange
I assume RecordVersion columns are of type rowversion
Related
Interesting issue while using SQLite in Android. I am seeing an inconsistency in the string length and quoting of a string between what is stored in the database and the materialized value seen in Java.
We are using an ORM called SugarORM to query the DB, but I've traced the offending code to the internal android.database.sqlite.SQLiteCursor class used within SugarORM, specifically the cursor.getString(columnIndex) method.
I have a string in the database that is an ISO data string 2019-03-25T19:19:39.664Z and is stored in a VARCHAR column . I have confirmed using DB Browser for SQLite that the length of the string as its stored in the database is indeed 24 characters. SELECT LENGTH(MyStringColumn) FROM MyTable WHERE ...
When I get the value of this string via cursor.getString(columnIndex), it is returning the string "2019-03-25T19:19:39.664Z". Notice the leading and trailing quotes. Java reports to me that the string is 26 characters long.
Any value that I store in this column that is not an ISO data does not have this behavior. I tried tracing the SQLiteCursor source back, but ultimately it ends up being a Native method and that's where my skill set stops.
Can anyone explain what might be going on here? I am probably just going to write a wrapper around my queries to get rid of the quotes, but its all very perplexing. The date string is being fed to a JavaScript interpreter and causing it to fail when creating a JavaScript Date object.
If it helps, I have replicated the behavior on both my S7 physical device and a Pixel 6 emulator.
As a quick get around you could use :-
SELECT length(replace(mystringcolumn,'"','')) FROM mytable;
or before using the original SELECT use :-
UPDATE mytable SET mystringcolumn = replace(mystringcolumn,'"','');
If this doesn't fix the issue, then for some reason it is the code that retrieves the data that is at fault.
e.g. consider :-
DROP TABLE IF EXISTS mytable;
CREATE TABLE IF NOT EXISTS mytable (mystringcolumn VARCHAR);
INSERT INTO mytable VALUES('2019-03-25T19:19:39.664Z'),('"2019-03-25T19:19:39.664Z"');
SELECT length(mystringcolumn), length(replace(mystringcolumn,'"','')) FROM mytable;
which results in :-
i.e. The 2nd row, 2nd column retrieves the appropriate value by using the replace function to strip of the quotes, if they exist.
As to why the quotes exist could depend upon either the way that the data is inserted (perhaps you have inadvertenly coded the quotes but the db being looked at isn't the actual database as copied from the App) or the way in which the data is being retrieved that for some reason adds them.
I don't believe it likely that the Cursor getString method has a bug in which the quotes are added, otherwise such an issue would likely be a recurring issue.
I'm changing my application from SQLite to Realm and I have next question.
I have next SQLite statement and I want to convert into Realm:
SELECT * from Table1 WHERE Table1.column1 = ?
AND Table1.column2 = ?
AND Table1.column3 < Table1.column4;
While that sort of query is supported by the underlying storage engine, that capability is not yet exposed in the Java API. You can follow https://github.com/realm/realm-java/issues/1615 for progress on that.
Here are my tables from SQLite :
Table1 - QUIZ
quiz_id
quiz_name
quiz_dateofcreation
quiz_key
Table2 - QUESTION
question_id
question_name
question_type (it could be S_answer or MCQ_answer)
quiz_id_fk
Table3 - S_ANSWER (standard answer)
s_answer_id
s_answer_name
question_id_fk
Table4 - MCQ_ANSWER (multiple choice answer)
mcq_answer_id
mcq_answer_name
mcq_answer_type (it could be correct or wrong)
question_id_fk
I want to send data to server for one QUIZ .Example when user click on send quiz data ,it wil be sended. DB on server is similiar, additional table for user.
First problem for me is SQL statement to get all data per QUIZ...
Second problem for me is best way to send data, i suppose
first data retrieved with sql convert to JSON like Convert SQLite to JSON
and then send them with help android volley class
If there is best way .. please let me now...
select
QUIZ.quiz_name
,QUESTION.question_name
,ANSWER.answer_name
,MCQ_ANSWER.mcq_answer_name
from QUIZ
left join QUESTION on QUESTION.quiz_id_fk = QUIZ.quiz_id
left join ANSWER on ANSWER.question_id_fk = QUESTION.question_id
left join MCQ_ANSWER on MCQ_ANSWER.question_id_fk = QUESTION.question_id
//where QUIZ.quiz_id = ...
This query will join all the tables you need so you can select any data you wish for any table. I'm not sure how you will be passing the quiz_id but where is the where clause you would use. I'm not sure how your data works so I used left joins but you can alter to your needs. I have only selected a few columns from the tables but you get the point....
Just curious on the best practice on syncing data from a database to an android tablet.
Tables:
- Part1
- Part2
- Part3
- Part4
- Part5
Whenever I open the app on the tablet I grab the latest lists from the database, truncate the table, and re-add the records. Each table consists of 400 records. So it takes around 60.45 per table to grab the data from the server and add it. Since I have 5 tables it takes around 5 minutes. Is there a better way to achieve efficient syncing for what I am doing? After I grab the data from the server, instead of truncating the table I've tried checking if it exists firsts before adding it but that didn't help with the time.
What I am currently doing: I get the JSON list from the API server and truncate the table and add the rows back. Pretty time consuming with 5 tables of 500 records each.
libraryApp = (LibraryApp) act.getApplication();
List<Pair> technicians = getJsonData("get_technicians");
if(technicians.size() > 0) {
stiLibraryApp.getDataManager().emptyTechnicianTable(); // truncate current table
// add technicians back to database
for(Pair p : technicians) {
libraryApp.getDataManager().saveTechnician(new Technician((Integer) p.key(), (String) p.value()));
}
}
Given the limited information provided I would try the following:
(1) Have your server keep a record of when the table you are updating was last "put" on the server. I have no idea what backend language you are using so I cannot make a sugestion. But it should be really easy to keep a lastupdated timestamp.
With this timestamp you will be able to tell if the version of the table on your server is more recent than the version on your mobile device.
(2) Using an AsyncTask download the data you need. I am not sure if all 5 tables are in the same activity, in seperate activities, in fragments or something else. However, the general idea is as follows:
private class GetTableData extends AsyncTask<Void, Void, Void>{
#Override
protected Void doInBackground(){
//get data from your sever
return null;
}
protect void onPostExecute(Void result){
//update the table view if version on server is newer
}
You will place all your I/O methods, that is those that connect to your server and download data within doInBackground. You will place all methods that update the table view within onPostExecute. This seperation is necessary because while I/O functions must run in the background after Jellybean, views must be updated from the UI thread.
(3) Check the timestamp of what you downloaded. If what you downloaded is newer update your table. You can acomplish this by simply adding in a conditional statment to your onPostExecute function such that
if(lastDownloadTime < lastUpdatedOnServerTime){
//update view
}
Depending on how big the table files are you may want to add a function on your sever code that just returns the time the table was last updated. That way you can check the time it was last updated against the time you last downloaded the table. If the table was updated on the server after you downladed it you can proceed to download the new information.
That's the basic idea. You can adapt it to your own set up.
I have a question for you guys.
I have been working on a project application that in one part uses an SQLite database loaded from a txt file (it has about 100k-200k rows of 5 strings separated by the ^ sign).
Now my question is, since this is my first time working with databases, how does .txt import for modifiable databases work? If I understand right, it pulls all data from the txt file once and creates a database that it keeps to work on, so when I modify the database I modify the newly created one and not the txt? Does the code try to pull info again from the txt whenever the app loads, and would loading 200k 10char words every time be too much? :)
The database consists of music bands in this format: name/genre/popular[yes/no]/selected
The selected column is the only one being modified by the user (and the app for that matter). If I use the regular approach to databases with added implementation from a txt file will the selected column reset every time (do not want that)?
Don't distribute your app with a huge txt-file and import it on the users device. This takes time and is annoying.
Rather distribute your app with a pre-populated database and copy it over from the res-folder. You can use android-sqlite-asset-helper to automate this.
Also, yes. The Database is always stored on the internal memory and you can't access it on a non-rooted device (unless you're using the AVD).
To import your txt-contents into a database, create a script or something that parses the contents and executes the corresponding SQL-queries. Again, your App should ship with the database, not the raw-file!
I was a little bored and hacked together a short Python-Script to read all entries from your txt-file and insert them into a SQLite Database:
import sqlite3
import re
counter = 0;
pattern = re.compile('^([^\^]+)\^([\w\s]+)\^(yes|no)\^\w+$');
conn = sqlite3.connect("imported.db");
cursor = conn.cursor();
# Create the Table:
conn.execute('''
CREATE TABLE Bands (
name TEXT,
genre TEXT,
popular INTEGER,
selected INTEGER
);''');
# Now, insert:
with open('bands.txt', 'r') as f:
for line in f:
match = pattern.search(line);
if match:
cursor.execute('''
INSERT INTO Bands (name, genre, popular, selected)
VALUES (?,?,?,0)''',
(
match.group(1), match.group(2),
(1 if match.group(3) == 'yes' else 0)
)
);
counter+=1;
conn.commit();
conn.close();
print "Imported ", counter, " bands!";
This will assume that the txt-file is named bands.txt, each value is separated by a / and each entry will be on it's own line. The resulting database-file is imported.db.
Also, I use INTEGER for all True|False-fields (popular, selected). These will then hold a 0 for false and a 1 for true.
Last but not least, the RegEx only allows "yes" and "no" for the popular-value.