At the moment I have apprx. 20 identical structured databases(SQLite) which I transferred from remote destinations (android phones) to my pc. Now I want to add them to a single one in order to perform data analysis on the data. How is this possible? The primary keys of the tables are auto-increment integer , so when using ATTACH I get a Primary key is not unique error. In addition it would be nice if i could somehow keep a reference between the rows and the tables which are coming from.
The way you worded the question, it sounds to me like it is a one-time operation you're doing on a home PC. I would probably just open a fresh database in the sqlite shell and attach/insert from/detach each database like they show on that link.
If you need to script this, this link has an example of embedding sqlite into shell scripts and this one shows some sql in a dos batch file. More preferably, you could code it up with your favorite language's sqlite bindings, like Python's.
Related
I have a phone application that uses a database of words and tests a user to see which words they know. I have a SQLite database with the words that I populate using a console application and this is then deployed as a resource to phones etc.
When the user runs the application then it stores pass fail data in the same database but in different tables.
When I update the application a fresh copy of the words database is installed on the phone and all the user data is lost.
How is this typically handled? Do phone applications that use SQLite have multiple databases with one being used to store user data and the other holding data which can be brought in when the application is first installed or updated?
If multiple databases are used then is it possible to create a look up from one database to the other?
Thanks in advance for any help, advice or links that point me in the right direction.
I would use a file (JSON, or plain text) to ship the words with the app. Then, when the app runs, it reads that file and adds the new words to the database. This won't affect the other tables.
Instead of having to deal with that, we hard code the values into a static method in code. Then at runtime, we see if there is any data in the table and, if not, we grab the hard coded data and do an insert.
In your case, I would also just add a version number of some kind so then, if the version was lower or the table was empty, you do a delete all and then insert your new static data.
I have a really different situation while using same database file in android app
and web app. Hope someone has idea about this problem.
I have a web application that has a database with full of data. After that, I am exporting
the file to my PC and PC to android device in .sqlite format. At this point, Let's say the last primary key id of one table in database is 25.
Now, android app will use that database file to operate. In the same table more data will
be inserted. Let's say 10 more. Now the last primary key id will be 35(25+10).
At the mean time, if I insert some more data into the database from web app, let's say 3
more. Then, the last primary key id will be 28(25+3) in the database of web app.
Now, I am pulling database file from android device and importing to the web app to keep
records
for future. The database file in the web has most of the data same as newly
imported database from android device because it was exported to android app before.
Now I want to merge the both database files to make one single database file and show record in web app. For that, what I want to do is;
Filter data from both databases not to keep same records.
Manage to replace the primary key id of android app's database followed by the last
primary key id of web app's database. Like 28+10 = 38 Now the newly modified database file has 38 records.
Please someone give me some technical idea because it is even hard for me to google. I am sorry if it is unclear. If it unclear, do not hesitate to ask.
Sounds complicated, maybe you should consider moving your data handling entirely to a cloud database: A relational database shared by an android app and a website - the easy way.
Alternatively I would suggest having two id keys on the android device, namely (_id, _id_server). This way, when importing the database from the web, the _id_server is populated and known and any future updates, from phone to web, will be easily filtered as they either have no _id_server value (new insert) or if the value is there (update existing entry).
I do not like the idea of altering id's on the web, can see a lot of problematic scenarios if doing so.
I have a bit of a theoretical question for which there is no code yet as I am still just in the thinking stage. I want to update an app to allow users to share their data with others through DropBox Datastore or something like that. However, when a user creates data which get populated into multiple sqlite tables on the device, each table has an auto-incremental integer as a primary key that is used as a foreign key in other tables to link the data.
If there is more than one user actually creating the data and sharing it then the primary key columns are obviously going to be an issue. If I download the data and store it locally I obviously can't insert user 1's key value in user 2's data table, firstly because of the auto-increment and secondly because user 2 might already have data that is not shared saved with that key value.
I have thought about a few options but nothing is particularly appealing or robust. I was thinking about creating a UUID to identify the device, that value would have to be stored in each of the tables and the primary key would be a combination of that column and the current primary key integer which would obviously have to have the auto-increment removed. So to pick up all related data from each table the id column and UUID column would both have to be used.
I feel like there must be a more robust method of achieving this though, any one have any better suggestions?
If I'm understanding well you need some sort centralised database in the cloud to communicate with your local app, is that right?
A client should never create the ids for such a system. A usual practice on these cases is to always have a remote id which is created by your DB in the cloud, and whenever you don't have this value yet, you can have a fallback value (local id created locally - which is different from the remote one).
So, to illustrate my words we could set the following example. Your app stores messages in database. Say you create messages with a local id 1,2,3. Those ids will never be meant to be unique in your central database in the cloud. Instead, you'd just use them as a local fallback. As soon as you can send those 3 messages to your centralised database, it'll give them 3 new remote ids you'll use for unique means (ie.: 35, 46, 54).
Note that when you have multiple requesters/users accessing one same database there's not such way to assure uniqueness unless you follow the explained approach, or you query a certain number of unique ids in advance and on demand to your database in the cloud.
Keep in mind, that the actual truth can be only delivered by the databases in your servers.
I am building an application which is a form generator (it creates a form with a SQLite database based on a configuration file). The problem is that the database will never be the same, so I need to make it dynamic meaning that I want to be able to specify all the table rows and tables of the database.
The problem I have is that, since it's a configuration file, when I create the database I dont know yet what are the tables and/or the table rows so I am not able to rely on the onCreate() of the database.
I was wondering if there would be a better way to proceed other than overriding the onCreate() to do nothing and making my own tableCreate() function.
I don't know if this is clear enough since english is not my native language but I will edit my question if I need to. And by the way I am new to android so snipet + explications are appreciated.
When the application loads, it creates the database using the configuration file (a simple text file) that is pushed to the application (only if the database does not exist).
Then it creates the tables based on the configuration file again (the name of the table rows and type of data).
The application builds the form based on the configuration file with an attribute which will allow me to save the answer in the database created previously.
This application's goal is to be able to create new forms efficiently and in a really quick way in order to gather some informations on given person.
Ok so I managed to build a workaround:
what I came up with is that I have a db with 4 fields (_id, farmerId, fieldName and fieldValue) this allows me to have some kind of value key for a specific farmer.
Then I builded some functions that generate a JSONObject with the different rows concerning a specific farmer and returns it to my activity. This way, it does not really matters if an "object" would need 5 or 6 fields.
We've got an android app and an iPhone app (same functionality) that use sqlite for local data storage. The apps initially come with no data, then on the first run they receive data from a remote server and store it in a sqlite database. The sqlite database is created by the server and the apps download it as one file, which is then used buy the apps. The database file is not very large by today's standards, but not a tiny one either - about 5-6 MB.
Now, once in a while, the apps need to refresh the data from the server. There a few approaches I can think of:
Download a new full database from the server and replace the existing one. This one sounds like the simplest way to deal with the problem were it not for a repeated 5-6 MB downloads. The apps do prompt the user whether they want to download the updates, so this may not be too much of a problem.
Download a delta database from the server, containing only the new/modified records and in some form information about what records to delete. This would lead to a much smaller download size, but the work on the client side is more complicated. I would need to read one database and, based on what is read, update another one. To the best of my knowledge, there's not way with sqlite to do something like insert into db1.table1 (select * from db2.table1) where db1 and db2 are two sqlite databases containing table1 of the same structure. (The full sqlite database contains about 10 tables with the largest one probably containing about 500 records or so.)
Download delta of the data in some other format (json, xml, etc.) and use this info to update the database in the app. Same as before: not to much problem on the server side, smaller download size than the full database, but quite a painful process to do the updates.
Which of the three approaches you recommend? Or maybe there's yet another way that I missed?
Many thanks in advance.
After much considerations and tries-and-errors, I went for a combination of options (2) and (3).
If no data is present at all, then the app downloads a full database file from the server.
If data is present and an update is required, the app downloads some database from the server. And checks the content of a particular value in a particular table. That value will state whether the new database is to replace the original or whether it contains deletions/updates/inserts
This turns out to be the fastest way (performance-wise) and leaves all the heavy lifting (determining whether to put everything into one database or just an update) to the server. Further, with this approach, if I need to modify the algorithm to, say, always download the full database, it would only be a change on the server without the need to re-compile and re-distribute the app.
Is there a way you can have a JSON field for each of the tables? For instance, if you got a table named users, have a column named "json" that stores the JSON for each of the users. In essence, it would contain the information the rest of the fields have.
So when you download the delta in JSON, all you got to do is insert the JSON's into the tables.
Of course with this method, you will need to do additional work in parsing the JSON and creating the model/object from it, but it's just an extra 3-4 small steps.
I will recommend approach 3, because app will download the json file more fast and local db will be updated more easily avoid overhead of more internet usages.
Just create a empty db initially according to server db and then regularly updated the same by fetching json