Background: we are porting an enterprise system to have android clients. The architecture for windows and html is based around a core library that does the hard business logic but no user interaction at all, and we use programs or single page web apps to provide the user interface and simply call the core API library to actually do stuff.
The "core" is implemented as a shared library on windows and built into each app. If we mirror this and use a java library, we need to share files using external storage, which is a not permitted as data needs to be reasonably secure. (Nb data is binary data, not Sql database, in case that is relevant)
So we thought about using a bound service, and using intents, content provider etc, but it seems (from googling) we must then distribute the background service separately the user interface app, but this seems terrible experience for new users. However, a bound service seems ideal from all other angles.
We also cannot guarantee which apps a user might download, we will have at least 10 individual apps all doing logically different things, but referencing similar data.
In brief:
lots of individual apps all wanting access to same data
no control over which apps are downloaded
using external data is not permitted as data should be semi secure
using sqllite might not work as data is long binary chunks ( eg 3Mb plus ). (Ref: How to share data across a group of applications in Android )
some data files are big and do not want every app to download a private copy
some data changes dynamically, say every 15min
core business logic is big and complex, cannot be distributed in source form, lib/jar ok though.
the windows solutions all use network IO to an application server, but we want to avoid as much network traffic as possible by storing data locally.
How can we bundle a bound service in each and every user interface app we distribute? Or is there a different way to approach this whole design?
I think that there is a few number of options that you can explore:
1) I never have done this before though this seems possible as Android is package based.
First you need to use the same main package across all your apps though each app must be in a separated sub package, e.g. : main -> au.com.myapp.main and the app actually have it's first screen on app1 -> au.com.myapp.main.app1 .
Create on your main app a method(s) that will look for those extra packages (within your project), as it find something you create a trigger that will display a item on the menu. Each app should have the same main packages and main activity, as it will be responsible for enable the user have access to the others and all of them can share the same preferences, files folders and Database.
When installing the same packages should be overrides though those different ones should keep intact. You should have all the 'main' classes for each app, not the real main one declared on your manifest (that will be quite big depending on the amount of activities in all your apps) with those packages.
2) You can using Spongy Castle, create a shared zone (folder) where you create the DB and write your settings or files, encrypting everything with a key (strong one or using RSA) that might be made by the user or provided once for your company at the very first run. You must decide how to handle this.
3) You also can use the share id in all your apps and each app before run perform look up for all packages (it's possible to do) to know and if and what packages exist to check if there is a DB with data in that package.
4) Not really nice though create a background service that keep updated all tables in all apps (sharing id or using content provider), you can user AlarmManager to trigger it in intervals rather keep it on at all times and you have all apps.
My last project had a similar requirement though as the user had to login to do anything, I opted for the option 3 and each data pertinent exclusively to each app went the app DB.
I hope this helps.
Related
The Setup
I have native iOS and Android apps which sync data to and from my webserver. A requirement of the apps is that they work offline so data is stored on the apps in sqlite databases.
The apps communicate with the server with a series of REST calls which send JSON from the server for the apps to store in their databases.
My Problem
The scale of this data is very large, some tables can have a million records, and the final size of the phone databases can approach 100mb.
The REST endpoints must limit their data and have to be called many times with different offsets for a whole sync to be achieved.
So I'm looking for ways to improve the efficiency of this process.
My Idea
An idea I had was to create a script which would run on the server which would create an sqlite file from the servers database, compress it and put it somewhere for the apps to download. Effectively creating a snapshot of the server's current data.
The apps would download this snapshot but still have to call their REST methods in case something had changed since the snapshot happened.
The Question
This would add another level of complexity to my webapp and I'm wondering if this is the right approach. Are there other techniques that people use when syncing large amounts of data?
This is a complex question, as the answer should depend on your constraints:
How often will data change? If it is too often, then the snapshot will get out of date really fast, thus apps will be effectively updating data a lot. Also, with the big volume of data, an application will waste CPU time on synchronization (even if user is not actively using all of that data!), or may become quickly out of sync with the server - this is especially true for iOS where Applications have very limited background capabilities (only small window, which is throttled) compared to Android apps.
Is that DB read-only? Are you sending updates to the server? If so, then you need to prepare conflict resolution techniques and cover cases, in which data is modified, but not immediately posted to the server.
You need to support cases when DB scheme changes. Effectively in your approach, you need to have multiple (initial) databases ready for different versions of your application.
Your idea is good in case there are not too many updates done to the database and regular means of download are not efficient (which is what you generally described: sending millions of records through multiple REST calls is quite a pain).
But, beware of hitting a wall: in case data changes a lot, and you are forced to update tens/hundreds of thousands of records every day, on every device, then you probably need to consider a completely different approach: one that may require your application to support only partial offline mode (for most recent/important items) or hybrid approach to data model (so live requests performed for most recent data in case user wants to edit something).
100mb is not so big. My apps have been synching many GBs at this point. If your data can be statically generated and upated , then one thing you can do is write everything to the server, (json, images, etc...) and then sync all on your local filesystem. In my case I use S3. At a select time or when the user wants to, they sync and it only pulls/updates what's changed. AWS actually has an API call called sync on a local/remote folder or bucket. A single call. I do mine custom, but essentially it's the same, check the last update date and file size locally and if it's different, you add that to the download queue.
My goal is to have a main application and a selection of independent separate data packs, which are available on demand as APK and have their own entry in the Play Store each. It's completely up to the user which, and how many, of these they care to install. Think language packs, keyboard layouts, dictionaries, this sort of thing. The application would always check what data packs are installed and give the user the choice of which one to use.
I thought that the data APKs would not provide any activities but would contain a Service that allows access to a ContentProvider of some sort. But it's all new to me and there's a lot of terminology that confuses me: is this a receiver? A provider? Both of these seem to fit my purpose to some extent. Do I need to have this service (one per data pack) run in the background, waiting for broadcast calls, or can I just somehow inform Android it exists, so it would launch it when this particular query comes from my application?
It would be ideal if nothing would have to be run in the background, possibly even during the listing phase. Then one data file would be chosen and the corresponding APK would only run for as long as the data is being accessed. All the data packs are of the same sort (dictionaries) and need not provide any further functionality than to provide some static metadata and then give access to a raw file.
All of the information I could find relates to "extension APK"s but that's orthogonal to what I want.
I thought that the data APKs would not provide any activities but would contain a Service that allows access to a ContentProvider of some sort.
You do not need a Service to provide access to a ContentProvider.
is this a receiver? A provider?
I do not know what "this" is in that sentence. A ContentProvider uses a <provider> element in the manifest. A Service uses a <service> element in a manifest.
Do I need to have this service (one per data pack) run in the background, waiting for broadcast calls, or can I just somehow inform Android it exists, so it would launch it when this particular query comes from my application?
None of the above. Your main app finds out about the existence of these "data pack" APKs via PackageManager. For example, you might use a standard naming convention for the application ID/package names for those "data pack" APKs, then use getInstalledPackages() to find out which "data pack" APKs are installed.
From there, you could:
Use createPackageContext() to access assets and resources in the "data pack"
Generate a Uri pointing to the "data pack" APK's ContentProvider, then use a ContentResolver to access content published by the APK (e.g., openInputStream(), query())
Use other forms of IPC (broadcasts, started services, bound services) to interact with the "data pack"
All the data packs are of the same sort (dictionaries) and need not provide any further functionality than to provide some static metadata and then give access to a raw file.
I do not know why you would distribute this material in the form of APK files via the Play Store. Just download the material from your own Web server, in some convenient format (e.g., ZIP archive).
Consider the following scenario. A company releases many apps. And they want some data to be shared across all these apps. Any of these app can create or read these data, just like a common database. So company decided to create an android library which does this purpose. I searched for a few days and my analysis are given below.
SharedPreferences- not recommended and is deprecated. It does not serve the purpose too. All other apps need to know the package name of the app that created the data to create PackageContext. Here this is impractical as any app can create/update/read data and it is not possible to say who is who.
ContentProviders - This does not work for me. The reason being ContentProviders has to be present in each app. There can not be 2 content providers with same name in a device. In addition to that, ContentProviders are basically meant for one app creates data and other apps subscribe to it using Content_Uri.
Network connection - We do not want to do store data in any server.
External storage - This is the only option remaining. Should I go for this?
And interestingly the data has to be secured as well which is nowhere supported in any of the storage options.
Note: For iOS, we use keychain to implement the same functionality
Understanding the problem on Android
Ironically, due to the intense sandboxing on iOS, there's a straightforward way to make this happen there (App Groups) provided the apps that need to share data are all by the same developer. Likely because Android is more flexible on security, this actually ends up being a more difficult problem there. The Android team have so far not seen fit to provide a convenient and secure way to specifically share this kind of data because there's a low security workaround.
That said, there are plenty of ways to share data between applications without involving the cloud.
SharedPreferences
The original question states that SharedPreferences are deprecated. This isn't true, as far as I can tell, however the MODE_WORLD_READABLE and MODE_WORLD_WRITABLE contexts are deprecated which makes this approach subject to not working in the future. The mode has been deprecated for quite some time, though - since Android 4.2 (2012). There's no threat in the current Android docs to suggest they're actually phasing it out (sometimes deprecation just means "this isn't a great idea" not "this is going to be removed"). I suspect the lack of a more secure OS-level direct alternative for application data sharing at the settings level is probably the reason for preserving it in a state of deprecation for the last 5 years.
File Access
The simplest and most common way I'm aware of to implement data sharing between applications on Android is to simply request file access on the device and create a shared location on external storage for this data. (Don't be confused by the "external storage" designation - this is just how Android refers to shared data. It doesn't necessarily refer to an SD card.) You give the file a unique name, and you store it somewhere that your apps know where to look for it. Best way to get that path is something like:
Environment.getExternalStoragePublicDirectory(Environment.DIRECTORY_DOCUMENTS)
The obvious problem with this is security. While not deprecated by the OS, it introduces the same problem that the Android docs list as the reason for deprecating MODE_WORLD_* - it's inherently insecure and opens up potential exploits in your application.
You're placing your information where everything has ready access to
it.
You're asking for permissions that your app may not otherwise need.
You're reading files that you can't verify the origin of.
If your application isn't handling any sensitive data, maybe this doesn't matter to you (it might to your users). If you're planning to read data from those files, you should ensure you're providing maximum validation for that data before parsing. Check the size of the file, validate the formatting, etc.
Creating your own service
You could always create a Service or an IntentService. (There are subtle differences between the two, but IntentService is a subclass of Service that runs in a Worker thread while Service interrupts the main thread. IntentService also implements Intent support which provides the most straightforward interapplication communication on Android).
This service has its own private storage, for which it has full read/write access, but nothing else does. This service then provides an interface to receive Intents from other apps, and to return results (as Intents) to those apps. This is an extremely friendly way to implement interapplication data while maximizing data privacy and security of that data. If outlying apps mostly need to request very basic information from a central application, this is your entry-level option.
Implementing a BroadcastReceiver
Along the same lines is the BroadcastReceiver class. Depending on what sort of data you're intending to share between applications, and how familiar those applications may be with your specific approach, this another possibility. Again, you'll be managing the shared data under one application's private storage. Communication is done by Intents, so it's similar to an IntentService - except that applications may communicate with a BroadcastReceiver by issuing systemwide events (that is, they don't need to be explicitly communicating with your app or service - they're shouting out the world for a piece of info, and expecting an answer.)
Creating a ContentProvider
The Original Post seems to misunderstand what a ContentProvider is and how it works. You have to think of this type of item like you would a cloud solution - even though it's local to your device. Every app doesn't need a ContentProvider - they all need to communicate with a ContentProvider, and that ContentProvider maintains, updates and returns data.
This is probably the most "Android-y" solution for this particular usecase and offers the greatest expandability. You implement an independent process that handles data storage and responds to other applications. It is, however, a more evolved solution - and as such may be more a more challenging endeavor. If you need a real database service, rather than a fairly simple request/response type service, ContentProvider seems to be the best choice.
Seems like what you need is shared user id.
It allows application sandbox to be shared across multiple android applications if they are all signed by the same signature.
But, watch out for gotchas!
Yes. Probably using same path in external storage for all applications is the best way. A common portion of code could be used to know whether database there exists or not and therefore open or create new one. For security I recommend always to use 'user' and 'password' when connecting to DB, but if you think it is not sufficient I advise you to see this: http://www.hwaci.com/sw/sqlite/see.html
Is there anyway I can do something like this
all data I/O functions are written in a library package
data can only be shared between apps with this library (optional)
every apps with this library can initiate the "DB" at first time, and later-installed apps can access the same "DB"
I thought ContentProvider is a perfect solution for me, but it seems that condition 3 is impossible.
any suggestion plz?
all data I/O functions are written in a library package
OK.
data can only be shared between apps with this library (optional)
Perfectly fine with the proper permissions for your provider (signature).
every apps with this library can initiate the "DB" at first time, and later-installed apps can access the same "DB"
I thought ContentProvider is a perfect solution for me, but it seems that condition 3 is impossible.
It's up to you to code the underlying structure of your data. Since you already assumed that the provider will belong in a dedicated library package, a possible solution is:
Implement your provider in package com.mysuite.library.
Publish this app in the Play Store.
Make client apps A, B and C.
Publish them in the Play Store.
Require your users to download this library package whenever apps A, B or C can't find com.mysuite.library installed.
However, if you don't want to provide a central package, I believe you will need to serve a provider in each of your own apps, with different authorities (to avoid CONFLICTING_PROVIDER error). Upon initializing each client, you first check if there is another provider in your namespace (com.mysuite.provider*), assuming you either know all possible authorities you are going to create and/or iterate among them when searching (com.mysuite.provider1, 2 etc.).
However, this proposition may create problems with custom backups (say, if only one of the clients is backed up), which will force the re-creation of data. It certainly has caveats and is definitely more complex (ugly, IMHO), but it can be made to work.
Personally, I'd stick with option 1 (library package). I don't see users complaining when downloading required library packages for apps.
It's just an architectural decision, really.
There are only four ways to maintain shared data:
You have a single APK that stores the data and makes it available to other apps. If this app is uninstalled the data is gone.
Every app maintains its own copy of the data but synchronizes changes to the others.
The data is stored on the SD card. This is generally a poor solution though several apps do it.
The data is stored on a server and is only accessible when the device has a network connection.
Once you've chosen where you want your data to reside, you can transfer it around by any number of mechanisms.
ContentProvider is a good choice if you are not storing the data on SD card or on a server, but you want to be able to transfer the data to apps that don't know it exists using content: URI's.
If you have no need to ever share any of this data with anyone outside your particular suite of apps, and your data is not structured as a database, you may prefer a simpler transport.
So I have an application that logs medical data entered by user. I need to add functionality for backing up and restoring that data in case user changes their Android phone OR switches to and iPhone (there is an iPhone version of this app, which I have limited visibility into).
The data is currently stored inside a SQLite DB on the device.
I need help figuring out cleanest/easiest way to do this. So far I am thinking about these options:
add UI to email DB file to + UI to accept a DB file and copy it over current DB. Seems hacky.
create a web service which will store user data and sync on the background. I would need to build sync process and introduce some kind of account system. This seems like quite a bit of work, though likely the most flexible solution in the long run.
switch to Google Calendar as my data storage (data is essentially a set of event entries anyway). This would probably be most seamless, but iPhone option is out of the window.
Are there other pros/cons to these options that I am missing? Or perhaps there are some standard solutions to this?
Option #1 is filled with a lot of problems. Sideloading databases from files coming from external sources is no easy task. I wouldn't recommend it.
Option #3 should be a possibility - but it require the user to set up a Google account which (imho) shouldn't be required. Furthermore the addition of a new calendar and/or a lot of events wouldn't be many users favorite feature. Personally I would hate it.
The second option seems like to most promising. You can build it as simple as you like. You could even make it require only a simple identification code for backing up - or only provide "live" synchronization. It is true that you need some sort of server and accounting system - but these things come as standards and servers are cheap. This option would be the one of my choice.