Develop Reference

What is best way to store large data in local for iOS/Android

I'm working on an app and I have tried to use local storage but I'm not sure if it will be OK for my project.
I need to store really large data every second in my local database, and at the end of the day send to my server. This can be large data because every second write my GPS coordinates.
What is best: SQLite, local storage or websql?
I work using cordova.
I appreciate for your help.
Regards

I am an Android developer so this is how it works in Android :
Shared Preferences - simple key/value pairs specific to your application. This is probably the closest to WebStorage - only for small amount of data.
Internal Storage - read/write files(only the app can access it)
External Storage - SD card file storage(app + other apps including the user) can access it).
Databases (SQLite) - better for large amounts of structured data
Network - obviously, you can store/retrieve data remotely if needed (like Firebase).
What would I choose? if data is structured then I would choose SQLite.
More detailed information :
https://developer.android.com/training/data-storage

cordova-sqlite-storage is Native SQLite component with API based.
It will easily handle large data and easy for CRUD operation.

Yes you can use cordova-sqlite-storage for large data inputs. But I think it is also advisable not to stress your device with so much data in it because mobile is just mobile with so many limitations in terms of hardware. Maybe you can just set a treshold of data size and after reaching it, send it to your server then clear your database.
The point is don't stress you device, use minimum data as much as possible to provide a clean app for the users.

Approaches to storing data in Android

I have started learning Android development and I have a very newbie question. I understand that Android can store data in SQLite for example, but what other approaches are there to the storage of data within your application?
Do Android apps ever have data 'embedded' within the application, in which case what sort of data structure or concept would this use?
I am thinking of a scenario where the data is static but is perhaps not a large enough dataset to warrant a database..e.g. an app with general knowledge questions and answers
Any guidance much appreciated
Rowan

Yes You are correct You can use SqLite Database for storage
other ways to store data is SharedPreferences
But in your case you wanrted to save questions and answers which is static one so you can create a text file and put that in your assets folder and you can read that file as any other text file in java
Refer this link how to read file from assets folder
1.Sqlite Database
2.Shared preferences
3.Internal memory
4.external memory i.e sd card

i would suggest you to go with Database. as it will let you store as much data as your app needed, There are some other option also present like
Sharedpreference i.e. cookies in general term. It let u store only few KB data and not good to store much data. When u retrieve data from cookies. All data will be store into ram and use app memory. that is use less when u do not need all data to retrieve and store into ram and then remove
Store into file and ship that file with your app. Yeah. this could be better idea again. you need to read it byte by byte. and hence reading to mid or last line will store all data into ram and hence will take memory.
Use Web Service to download data. It will let you store Large data and you have to download using Web APi. Hence it could be better idea. But this requires active Internet connection to play game to run app.
There must be some other option also present. You can search. surely you will find them :)
Overall Database it good solution for all app. As it will let you do search store delete and let you do other operation in less amount of memory. In Mobile Development Memory is very Important thing we have to take care of.
Let me know if you have other unclear thought.
item

You could also store info in a server.
Pros :
You can change the content without needing user-side update of the app.
Cons :
Your app (mostly the UI) would need to manage connection problems.
You may need to implement async tasks for querying data from server.

How to Manage increasing size of sqlite in Mobile App?

My app are sometime needed syncing with web servers and pull the data in mobile sqlite database for offline usages, so database size is keep growing exponentially.
I want to know how the professional app like whatsapp,hike,evernote etc manage their offline sqlite database.
Please suggest me the steps to solve this problem.
PS: I am asking about offline database (i.e growing in the size after syncing) management do not confuse with database syncing with web servers.

I do not know how large is your data size is. However, I think it should not be a problem storing reasonably large data into the internal memory of an application. The internal memory is shared among all applications and hence it can grow until the storage getting filled.
In my opinion, the main problem here is the query time if you do not have the proper indexing to your database tables. Otherwise, keeping the databases in your internal storage is completely fine and I think you do not have to be worried about the amount of data which can be stored in the internal storage of an application as the newer Android devices provide better storage capability.
Hence, if your database is really big, which does not fit into the internal memory, you might consider having the data only which is being used frequently and delete otherwise. This highly depends on the use case of your application.
In one of the applications that I developed, I stored some large databases in the external memory and copied them into the internal memory whenever it was necessary. Copying the database from external storage into internal storage took some time (few seconds) though. However, once the database got copied I could run queries efficiently.
Let me know if you need any help or clarification for some points. I hope that helps you.

For max size databases. AFAIK You don't want to loose what's on the device and force a reload.
Ensure you don't drop the database with each new release of your app when a simple alter table add column will work.
What you do archive and remove from the device give the user a way to load it in the background.

There might be some Apps / databases where you can find a documentation, but probably this case is limited and an exception.
So to know exactly what's going on you need to create some snapshots of the databases. You can start with that of one app only, or do it directly with several, but without analyzing you won't get a reliable statement.
The reasons might be even different for each app as databases and app-features differ naturally too.
Faster growth in size than amount of incoming content might be related to cache-tables or indexing for searches, but perhaps there exist other reasons too. Without verification and some important basic-info about it, it's impossible to tell you a detailed reason.
It's possible that table-names of a database give already some hints, but if tablenames or even fields just use meaningless strings, then you've to analyze the data inside including the changes between snapshots.

The following link will help in understanding what exactly Whatsapp is using,
https://www.quora.com/How-is-the-Whatsapp-database-structured
Not really sure if you have to keep all the data all the time stored on the device, but if you have a choice you can always use cloud services (like FCM, AWS) to store or backup most of the data. If you need to keep all the data on the device, then perhaps one way is to use Caching mechanisms in your app.
For Example - Using LRU (Least Recently Used) to cache/store the data that you need on the device, while storing the rest on the cloud, and deleting whats unneeded from the device. If needed you can always retrieve the data on demand (i.e. if the user tries to pull to refresh or on a different action) and delete it whenever its not being used.

Android WebView: offline manifest vs local files

I'm developing a simple Android App where the user must fill in a very complex form, for which I believe it's much easier to use an HTML form than an Android Activity with tons of TextViews.
The data collected by the form must be sent to some remote database, and the application must be able to work offline.
I thought of two alternatives, the question is: which one would be better?
Let a WebView load a remote website with an offline manifest
Let a WebView load a local website in assets folder
My second question is related to the storage when offline, and once again I have two options, and I don't know which one is better:
Using the HTML5 local storage, and let HTML + javascript send data to the server when online again
Let my Android app catch the form data, and handle everything the Android way.
Any input will be very helpful. Thanks in advance.

Regarding the first question: depends on how often will you need to update your form. An online cached form can be updated quickly, while bundled pages are only updateable together with the app, and you will need to consider that both legacy and new clients can connect to your server at the same time (users will procrastinate updating).
Another aspect is portability. Do you envision an iOS version of your app, or perhaps a mobile site? If yes, then an HTML5 solution is definitely more portable. Also, debugging an app which is entirely HTML or entirely native is usually easier than a hybrid one -- you can stay within a single debugger.
Perhaps, one drawback of using HTML local storage inside WebView is that the data you save will be in a kind of a "black box" -- you will not be able to back it up easily.
[Added later] OK -- one drawback of putting your site into assets folder is that you'll have to use file: scheme in order to access it. This can lead to some cross-origin loading access related issues if you will try to mix your bundled content with content from the web. Check these WebView settings for example: setAllowFileAccessFromFileURLs, setAllowUniversalAccessFromFileURLs, setMixedContentMode.

How to build a simple offline form submission app?

I'm a Webdev beginner, have been learning some Python/PHP/Javascript and want to help a friend of mine with a project.
She's conducting a large scale survey of old houses/windows, and fills in forms (similar to Google Forms). However, she doesnt always have 3G coverage on her phone, so she wants some way of uploading the data when she gets online, and she wants to be able to upload photos and attach these to the forms.
I've found a javascript-library which makes use of html5 localstorage, but I havent understood if localstorage also accepts file uploads? Also, when you have submitted one form you would need internet access to open up the form again to submit another form.
Are there libraries or projects around that would help me with some of this, so I dont have to reinvent the wheel?

This is going to be very difficult to do with a web app. Your going to have to rely on something like AJAX to send the data and if it times out store in local storage and keep retrying every few minutes, because a web app has no access to anything like checking the wifi status etc.
Also that there is a limit to the size you can use in html5 local storage:
What is the max size of localStorage values?
, this is not going to work if you want to hold onto photos.
This would require a native or minimum a hybrid app (like phonegap) to store the data on the actual device in a file or SQLite database. This way you can store what you like, have access to the status of the internet connection and even have prompts to appear on the screen (when the app is in the background) to remind her that there are still some that need to be uploaded.
It will be much more reliable and stable this way

You can create an app which stores the data (form) in the Local DB (ie SQLite) of the App.
Once she is online (mostly wifi) she can start uploading the data.