Not sure, if this is important: app is written with react-native.
Question is about both iOS and android apps.
I have a static database, which contains currently ~9000 rows, each row contains 45 columns and about 280 letters total in it. So, basically, database is relatively small. I'll need to perform pattern search (equivalent of ILIKE in Postgres) and sorting based on misc columns with numeric values. No insertions, no modifications, no relations with other "Tables" required.
Should i write nodejs server, instantiate PG database, connect app through web-socket and start querying data from pg, or should i just somehow create local database in app and search through it right in app? The local db is way simpler but I'm worry about performance. If nothing wrong with performance, then what if the database will grow to 15000? 50000?
If the database can be local and there's no need for it to ever be persisted on servers, then you should probably explore Realm. I have not heard of how it great it performs with a lot of data but it acts as a database engine in your phone and has indexing as well.
Related
I am pretty new to databases but I have an idea I'm trying to implement and I'm looking for some answers on if I'm going down the right path here.
I'm trying to create an app that displays locations in a list or on a map. The locations are going to be stored in a MySQL database hosted by Amazon Web Services. When the user opens the map portion of the app the database will be queried and the locations within the map view will be displayed. I figure the database won't be super big (around 10k locations) so I think it would be best to download the entire database and store it client-side on the users cell phone. There would have to be some kind of version check to determine if the user has the latest version of the database. Where would the version of the db be stored? Does the database or table automatically have a version number that increments automatically?
Second part of the question - is downloading the entire database (although small) a good idea? It seems like it would cut down on multiple database accesses per session and use less network traffic (anytime the user moves the map they would have to query for more locations within that area). I'm just trying to figure out if this is the best way to go about doing this since I'm brand new with databases. Thanks for any input.
Android has support for SQLite when it comes to embedded databases.
First you should check the compatibility between MySQL and SQLite SQL if you want to download the entire database.
Also, in android there is a class called SQLiteOpenHelper which has an onUpdate method that comes handy when one must update the database.
In my experience, I would recommend using a web service instead of downloading the entire database, mainly because of bandwidth and security issues.
I have an architecture question. If you have a web app that is storing information on a DB server, theoretically, I should be able to use the middle tier logic for a mobile app. When the mobile app starts it can connect and populate a local SQLite DB or use JSON to store information within the mobile app. What if the mobile app also needs to work in off-line mode? Do you have it sync the next time it is connected? Do you have the mobile pull down and populate a complete DB or so it available in off-line? What are the best ways to architect a mobile app that has to go from on-line to off-line?
The simplest solution would be to put a "LastEdited" column into every table in your database and then pull query all the data which has updated since the last sync ( and you can perform a check on the index to detirmine if you need to update or insert into your own local cache. )
The ability to delete rows should actually be limited to a boolean "isDeleted" flag in this case to keep the sync process nice and simple.
If you have then the ability to edit or create rows from your app then you should keep a local table of changes to sync when you can go online and may have to implement some form of "merge" logic.
Several things you need to consider.
If your app is read only, you should implement a 'delta sync' logic in your local d. Keep a timestamp of last sync and get updates from your server. Of course, you need to consider the local db size in getting too large.
If you app is read/write, when working offline, you need to consider the two way sync especially when same record can be updated in different devices/users.
I know SQLite Data Base is used in mobile devices (Android, iPhone) and it is light, takes only Kb space. Is there any limitation in SQLite? I want to know how they are different.
Every SQL database uses its own implementation of the language that varies slightly. While basic queries are almost universal, there are notable nuances between MySQL, PostgreSQL, Microsoft SQL Server, Oracle Database, etc.
What's particularly notable about SQLite is that unlike all the others mentioned above, this database software doesn't come with a daemon that queries are passed through. This means that if multiple processes are using the database at once, they will be directly altering the data through the SQLite library and making the read / write data calls to the OS themselves. It also means that the locking mechanisms don't deal with contention very well.
This isn't a problem for most applications that where one would think of using SQLite -- the small overhead benefits and easy data retrieval are worth it. However, if you'll be accessing your database with more than one process or don't consider mapping all your requests through one thread, it could be slightly troublesome.
Sqlite is very light version of SQL supporting many features of SQL. Basically is been developed for small devices like mobile phones, tablets etc.
SQLite is a third party ,open-sourced and in-process database engine. SQL Server Compact is from Microsoft, and is a stripped-down version of SQL Server.They are two competing database engines.
SQL is query language. Sqlite is embeddable relational database management system.
Edit : ( Source from following comment on my answer )
Sqlite also doesn't require a special database server or anything. It's just a direct filesystem engine that uses SQL syntax. ( By : Adam Plocher )
Techinically, SQLite is not open-source software but rather public domain. There is no license. ( By : Larry Lustig )
SQL is a query language. SQLite is an embeddable relational database management system.
Unlike other databases (like SQL Server and MySQL) SQLite does not support stored procedures.
SQLite is file-based, unlike other databases, like SQL Server and MySQL which are server-based.
What is SQLite?
SQLite is an open-source, zero-configuration, self-contained, stand-alone, transaction relational database engine designed to be embedded into an application.
Python SQLite can be defined as a C Library developed using ANSI-C, light-weight disc based database; doesn't demand for an extra or any other separate server process.
What are some SQLite characteristic?
SQLite does NOT require a server to run (RDBMS such as MySQL, PostgreSQL, etc., requires a separate server process to operate).
Is self-contained, it requires minimal support from the operating system or external library. This makes SQLite usable in any environment especially in embedded devices like iPhones, Android phones, game consoles, handheld media players.
Does not use any configuration files.
Is ACID-compliant. It means all queries and changes are Atomic, Consistent, Isolated, and Durable, all changes within a transaction take place completely or not at all even when an unexpected situation like application crash, power failure, or operating system crash occurs.
Capable of creating in-memory databases that are very fast to work with.
Uses dynamic types for tables. It means you can store any value in any column, regardless of the data type.
Allows a single database connection to access multiple database files simultaneously.
SQLite & ACID
SQLite guarantees all the transactions are ACID compliant even if the transaction is interrupted by a program crash, operation system dump, or power failure to the computer.
Atomic: a transaction should be atomic. It means that a change cannot be broken down into smaller ones. When you commit a transaction, either the entire transaction is applied or not.
Consistent: a transaction must ensure to change the database from one valid state to another. When a transaction starts and executes a statement to modify data, the database becomes inconsistent. However, when the transaction is committed or rolled back, it is important that the transaction must keep the database consistent.
Isolation: a pending transaction performed by a session must be isolated from other sessions. When a session starts a transaction and executes the INSERT or UPDATE statement to change the data, these changes are only visible to the current session, not others. On the other hand, the changes committed by other sessions after the transaction started should not be visible to the current session.
Durable: if a transaction is successfully committed, the changes must be permanent in the database regardless of the condition such as power failure or program crash. On the contrary, if the program crashes before the transaction is committed, the change should not persist.
Credit Sqlitetutorial.net
The most basic difference between SQLite and SQL is :
SQL is a query language which is used by different SQL databases. It is not a database itself.
SQLite is a database management system itself which uses SQL.
SQL is a database querying language and SQLite is a database (RDBMS) which uses SQL specifications. SQLite can be said as competitor to Microsoft's SQL Server.
The name itself suggests that it is the light version of SQL RDBMS. It is used in most of the small and portable devices like Android and iOS devices.
SQLite: Database Management System (DBMS).
SQL: Structured Query Language is a computer language, used to Create, edit and get data from DBMS via queries.
We've got an android app and an iPhone app (same functionality) that use sqlite for local data storage. The apps initially come with no data, then on the first run they receive data from a remote server and store it in a sqlite database. The sqlite database is created by the server and the apps download it as one file, which is then used buy the apps. The database file is not very large by today's standards, but not a tiny one either - about 5-6 MB.
Now, once in a while, the apps need to refresh the data from the server. There a few approaches I can think of:
Download a new full database from the server and replace the existing one. This one sounds like the simplest way to deal with the problem were it not for a repeated 5-6 MB downloads. The apps do prompt the user whether they want to download the updates, so this may not be too much of a problem.
Download a delta database from the server, containing only the new/modified records and in some form information about what records to delete. This would lead to a much smaller download size, but the work on the client side is more complicated. I would need to read one database and, based on what is read, update another one. To the best of my knowledge, there's not way with sqlite to do something like insert into db1.table1 (select * from db2.table1) where db1 and db2 are two sqlite databases containing table1 of the same structure. (The full sqlite database contains about 10 tables with the largest one probably containing about 500 records or so.)
Download delta of the data in some other format (json, xml, etc.) and use this info to update the database in the app. Same as before: not to much problem on the server side, smaller download size than the full database, but quite a painful process to do the updates.
Which of the three approaches you recommend? Or maybe there's yet another way that I missed?
Many thanks in advance.
After much considerations and tries-and-errors, I went for a combination of options (2) and (3).
If no data is present at all, then the app downloads a full database file from the server.
If data is present and an update is required, the app downloads some database from the server. And checks the content of a particular value in a particular table. That value will state whether the new database is to replace the original or whether it contains deletions/updates/inserts
This turns out to be the fastest way (performance-wise) and leaves all the heavy lifting (determining whether to put everything into one database or just an update) to the server. Further, with this approach, if I need to modify the algorithm to, say, always download the full database, it would only be a change on the server without the need to re-compile and re-distribute the app.
Is there a way you can have a JSON field for each of the tables? For instance, if you got a table named users, have a column named "json" that stores the JSON for each of the users. In essence, it would contain the information the rest of the fields have.
So when you download the delta in JSON, all you got to do is insert the JSON's into the tables.
Of course with this method, you will need to do additional work in parsing the JSON and creating the model/object from it, but it's just an extra 3-4 small steps.
I will recommend approach 3, because app will download the json file more fast and local db will be updated more easily avoid overhead of more internet usages.
Just create a empty db initially according to server db and then regularly updated the same by fetching json
I am connecting to a SQL Server database with the Android device. Yes I know not very advisable but everything works fine except ... well it's slow.
Here is what I do:
open conection
download table by table the data I need from the server database (with a connector I found online)
insert table by table into sqlite
I only select what I need; there are about 12 tables and some have like 300 items
Then I send some data to the server database (the select from 2 tables)
How could I make the thing work faster without using web services? (if possible)
If I do use web services will it be faster?
Thanks a lot in advance
PS: I know there are the risks about accessing the DB that way they are not relevant in this case.
Do you really need to replicate the same schema on Android? Try to narrow it down to only the essential data, and maybe create a new 'light' schema for the Android app. Do not use select *, only get the columns you need. Even with this, the first sync will probably be relatively slow. After you sync once, you can only get the newest items, so it should go faster.
If you want to use a webservice, you will have to rethink your model: instead of syncing tables with rows, you will work with entities (customers, orders, etc.). If you transfer the same amount of data, a webservice won't be faster, it can even be slower. The benefits of a webservice are that you don't have to know the exact schema on the server, so you don't need to change your app if it is changed (as long as it has all the data you need, of course); and that thinking in terms of a service will make you only use the data you need, making the whole interaction faster. And, it works over the Web, of course, so it will work with proxies, NAT, and what not.
After the initial sync, you could code your android app to retain the data in sqllite and only download the differences/changes on the next connection.
Depending on how changeable your data is on the server, this method could reduce you sync time.