I have a small Android application that uses different sets of files (a couple of images, a small SQLite DB and a couple of XML files) depending on the specific task at hand.
I know I can include my files into the main application APK using resources or assets but I would be happy to distribute them in a separated APK.
How can I create a data-only APK file?
How can I distribute it? In particular, do I have to do anything special for a data-only package (for example for associating it to the main application package in some way)?
(I'm intentioned to give the user a link to the data package and ask him to install it. No automatic installation required.)
How can I install my files into the internal or into the external storage area of my application? Is it possible at all to install files into the internal storage area created by the main application installer? Do I have to set any particular permission for this?
My approach to this would be to create a wrapper app that's nothing but a content-provider and serves up the files per request by your main app. This would allow you to supply different data packages for the user -- you could even have your main app select between those relatively easily.
It looks like that the commonly accepted way to have the same application with different contents (or styles, or configurations) is to use an Android Library Project for the common code (that is: the whole application, the "engine", the "app framework") and a standard Android Application Project for the contents (that is: an application that actually contains just data). A little bit confusing, just because the "library" here is actually the whole "app", but this seems to be the way to go.
More in detail:
Create an Android Library Application and put into it as much code as you can (all of the non-changing stuff). Please note that this library cannot be launched and cannot be distributed alone. It must be included in a hosting application.
Create a standard Android Application. Include your library into this project. Put in /res and in /asset all of your data (files, XML, etc.).
Compile everything and distribute.
Repeat this cycle every time you need a different version. Different because of data, style, configuration or anything else. Publish the resulting app with a new name.
For what regards me, I'm not completely satisfied by this approach.
A possible alternative is preprocessing the source code with Ruby, Python, Perl, GIT, bash, Ant, Maven, Rake or any other tool that is able to read a file from here, make some change here and there, and write the file there.
The general outline is something like this:
Make a "template" application. Leave your /res and /assset empty.
Run a custom-made script. The script reads a configuration file, copy the /res and /asset files from your repository into the project /res and /asset directories, changes some Java source file and creates/changes some XML file.
Compile and distribute (with a new name, of course).
Using GIT or other SCMs, you just make a new branch for every new version and compile it. Not very elegant (because it can strongly interfere with the normal use of the SCM) but...
There are a few example of these approaches on the web. I'm not completely satisfied by them, either.
Frankly, what the Android ecosystem should offer to solve this problem is some kind of "in-app package manager". Something like the Eclipse Update Manager. This would allow us to use the same application framework to handle different scenarios.
As an alternative, a solid, officially-supported, template-based code-generation mechanism would be nice. Something in the spirit of "Software Production Line": https://en.wikipedia.org/wiki/Software_production_line . Have a look at fw4spl, for example: http://code.google.com/p/fw4spl/ .
Related
The short summary is: How do I build an APK and separate libraries (by which I mean sets of classes (and ideally, resources too) in some form, such as JAR, AAR or DEX files), but not include those libraries in the APK; instead, the app loads them at run time?
Detail
So my main question is how to build such an app (e.g. Gradle configuration). How do I specify which classes go into which JAR or DEX files? Do I create an Android Studio module for each DEX file I want to end up with?
A closely related question is how the Java code should then load the external libraries and access their classes at run time. For the latter, I'm hopeful that the approach shown at accessing to classes of app from dex file by classloader would work.
I've tried the instructions at https://developer.android.com/studio/projects/android-library.html, but that builds an APK that does include the dependency library.
I've also tried Multidex (https://developer.android.com/studio/build/multidex.html), but that doesn't seem to leave the developer any control over which classes go in which DEX file, and furthermore, packages them all into a single APK. AFAICT there is no way to control the loading of these DEX files at run time.
Background
There's a possibility of the "X-Y problem" here, so I'd better explain the background.
I'm building an app for a client. It's not going to be distributed through an app store, so it won't have access to the normal mechanism for updates. Instead, the client wants the app to be able to update itself by downloading new components of itself to replace the old components, without a need to manually sideload a new APK. The primary motive here is that the updates have to be easy for non-technical users. If the app can control the update process, it can make it smooth and guide the user.
Moreover, the app will be used in areas where internet access is scarce and expensive, so the client wants to be able to issue app updates in smaller chunks (e.g. 2MB) rather than forcing the user to re-download the whole app to receive a small update.
One aspect of the requirements I should mention, in case it matters, is that the libraries to be loaded at run time are supposed to live on a microSD card. This can also help with distribution of updates without internet access.
The current status of the app is that it's about 50% written: That is, a couple of earlier versions have been released, but the app now needs to be modified (restructured) to meet the above requirements, as well as others.
This tutorial is a good start for external loading of DEX files.
Only three small files of source (MainActivity.java, LibraryInterface.java, LibraryProvider.java) and it copies secondary_dex.jar from the assets folder, into internal application storage [outdex/dex] (the internet is also stated as possible in the tutorial).
You have to build it with ant, because it uses custom build steps.
I tried it, it works fine. Worth a look.
custom class loading in Dalvik and ART
UPDATE
this code has been ported to Android Studio gradle (no need for ant).
https://github.com/timrae/custom-class-loader
Tested ok. Copies com.example.toastlib.jar from the SDcard into internal application storage [outdex/dex],(not assets folder).
( you must read the README.md file in the project to build it).
Q: How do I add an Activity, I cannot add it to the manifest ? A: Use
Fragments, they don't need entries in the manifest.
Q: A Jar with resources that is meant to be added to an existing
project needs to be able to merge its resources with the project's
own resources (R.). A: Hacks are available, Data file...Packaging Android resource files within a distributable Jar file
Q: The external file has wrong permissions. A: Import it.
Q: I need to add uses-permission. A: Use API23 you can programmatically add uses-permissions (but they still need to be declared in the Manifest, so the new permissions model is probably not much use to us).
This section is for more general users (#LarsH has more specific requirements about updates), The example above is 17kb apk and 1 kb jar. You could put the bulk of you code in the one-off jar, and updates would involve just loading an new Apk (and then importing the bulk code jar, to minimise the data transfer).
When the Apk gets too big, start again with a small Apk and everything migrated to another jar (import 2 jar's). You need to balance coding effort, user experience, maintainability, supportability, bandwidth, android rules, play store rules (if these words even exist ;O)).
NOTE Dalvik is discontinued
The successor of Dalvik is Android Runtime (ART), which uses the same bytecode and .dex files (but not .odex files), with the succession aiming at performance improvements transparent to the end users. The new runtime environment was included for the first time in Android 4.4 "KitKat" as a technology preview, and replaced Dalvik entirely in later versions; Android 5.0 "Lollipop" is the first version in which ART is the only included runtime.
You could try to build multiple apk's with the same sharedUserId and the same process.
This is the plugin mechanism used by Threema
Edit: More about Theema
Threema has one main app and two plugins:
main app: https://play.google.com/store/apps/details?id=ch.threema.app
QR-Code Plugin: https://play.google.com/store/apps/details?id=ch.threema.qrscannerplugin
Voicemessage Plugin: https://play.google.com/store/apps/details?id=ch.threema.voicemessageplugin
Doing so the main app does not need the permissions for accessing the camera or microphone
We have an Android project where we maintain a single code base for different customers, what will be the fastest/most efficient way to compile for different customers every time? Few options I found and my questions:
writing scripts: to replace resources folder and edit app name, version, etc.
Using Android Library Projects It is gonna be quite impractical to separate current project as Library projects, I am thinking whether it is possible to save some settings and resources files as a Library project and just import different library projects for different compilation?
Storing settings and resources on a remote server Is it possible to store resource files and some app settings (xml, constants, etc) on a remote server, and download them and replace to the app when the user first launch the apk? Where will these files be stored?
Any other options you would suggest?
Android Studio provides a feature called "flavors" that allow you to quickly define different configurations from a single code base. I have just learned about this in the last couple of days, so I don't know a lot more than this.
The best way I've found is a post build script step. Use a default set of resources/assets for your main build. This is your default apk, use it for default testing. Save the unsigned apk this builds. Then for the customer specific APKs, open up the unsigned apk (its just a zip file), overwrite any overwritten files, then sign the new version.
This works fine so long as you don't need to change code for different customers. It also doesn't put any unneeded assets/resources in any build, so you don't leak info to one customer about your other customers by including their files.
If you do need to change code, the best way is to do a runtime check on a variable from a settings file. And overwrite the settings file the same way you do everything else.
As an added bonus, if you need to you can write a very fancy system that would allow the customer to upload his own files to override your defaults (including allowing them to override some of your settings), so you don't need to deal with a dozen change requests. That requires a lot more work though.
TL;DR
I'm an android developer and I have essentially several copies of the same app that feature slight differences. Despite my best efforts I'm having to resort to copy-and-paste style modification for each app-copy (hereafter "flavor") whenever I need to roll out a single change to all of the apps. This is particularly heinous in Android because of the number of languages/file-types involved in a given app - C for native code, Java for regular code, .xml files for data, layouts, and os permissions, etc. Tricks I know to avoid copy-and-paste coding that work in Java or C alone won't extend to both at once, or over to the .xml files. I've been using Apache ANT with some success to create a custom build/ folder on the fly and include relevant files there for different build-targets, but it's a little cludgy, and there are some cases it just can't handle.
I'm wondering if there is a build system that could handle all of this for me; I'm open to any advice, though the solution I have in mind is particular to a sort of language-independent/filesystem #include approach, which for a given flavor at build time injects either whole files into a src/ tree, or else injects code fragments into various .java, .c, and .xml template files in that src/ tree.
A Really Long Explanation with Concrete Examples:
Examples of the differences between flavors: in the Google Store I have to use the Google in-app purchasing sdk; but in the Amazon store I have to use the Amazon in-app purchasing sdk. Different flavors are paired with different advertising partners. Certain flavors require different i18n language files; or graphics and layout assets for different devices; etc. etc. Any given flavor is essentially the base code, plus any number of these "add ons", where each add on requires a little bit of custom code and file additions to the src/ tree for that app. Whenever I need to change an add-on, I end up having to copy-and-paste the change in each flavor of the app that uses it, which is a terrible pain.
As a concrete example, imagine I have an android game and I have implemented a custom Flurry Appcircle ad view. This would typically involve the following:
The addition of "<activity>", "<service>", and/or "<uses-permission>" tags to the game's AndroidManifest.xml
The addition of various "<string>" lines for the ad in the apk's res/values-xx/strings.xml folder system, where xx is a language code (ie Spanish translations would go in res/values-es/strings.xml)
One or more image or layout resources in res/layout, or assets/, for the ad.
The actual Flurry Inc sdk in the form of a Java .jar file in libs/ - possibly multiple jar files if different hardware architectures need to be handled differently.
Java Code in the game's Activity class, e.g. in onStart or onStop, to query for an Appcircle ad and show it. To avoid code duplication, I would normally write this in a base class that surrounds all code in a "if(appcircleAdsEnabled(){...}", create a default implementation of that method that returns false, and #Override and return true in various children Activity classes.
This causes a few problems:
The Flurry .jar is now a dependency for the base class to compile whether or not the ads are enabled or not in any given flavor. I can hope that the builder notices code or resources are unused at build time and removes it from the final apk; but there are no guarantees, and failure leads to (A) bloated final binary size and (B) potential P.R. issues or worse (especially from automated scanners searching for code inclusions - in a slightly different example, I'd have a hard time assuring the Amazon App Store that "yes Google Play Billing is there but it's not used.")
Because of the way Android turns resource files into java constants, and in the same way as (1), the res/* files may also become unnecessary dependencies of all games deriving from this Base Class.
Some libraries/methodologies I've found insist on their code being added in specific places or in specific ways which precludes the base class approach altogether.
For each individual game I still have to at least edit its AndroidManifest to "turn this on" and request relevant OS permissions necessary to show the ad.
It's tedious to remember which "add on" goes with which changes in which files. For instance, in any given flavor's AndroidManifest.xml, I can't keep track of which permissions are needed by which add ons. If I remove the Flurry add on, do I still need the INTERNET permission for something else?
What I'm hoping exists, and what I'm asking about, is if there's some way to inject files and code fragments into/on-top-of some skeleton file system, each add-on composed of a collection of such files and code fragments and injected dynamically at build time, with the build system merging lines across multiple add ons, and handling dependency inclusion on the fly. IE for my Flurry Ad add-on I would have:
Some collection of files 'FILES' that includes my xml additions to various .xml files (like AndroidManifest.xml), my code additions to various .java files (like the Activity subclass), and in some cases whole files themselves to be injected in various places (like the ad layout files)
In some skeleton template file tree, incomplete versions of e.g. AndroidManifest.xml, MyActivity.java, etc. with something like a language-independent #include syntax indicating that data from FILES will go there.
Also in that template file tree, directories in src/ with 'missing files' that will be copied in dynamically at build time from FILES
Some sort of build directive script for FILES indicating what goes where when this add-on is included, to take the skeleton template file tree and turn it into a fully fleshed out build/ tree.
And again, ideally it would be able to handle multiple "add ons" at the same time; ie I might have some FILES for this Advertiser, and some FILES for that billing provider, etc. etc., and a given product is just a list of which modules it includes, the build system magically knitting them together at build time.
This seems like exactly the problem the Gradle Build Variants system was designed to solve - it handles merging multiple versions of the AndroidManifest.xml, resources, and Java packages together into one or more separate apps. Specifically, the Multi-flavor variants seem to cover your requirements for combining multiple features together in various ways.
Gradle is offered as part of Android Studio, an IntelliJ based IDE specifically built for Android Gradle projects. You may consider reviewing one of the many Gradle Build guides if you are not familiar with Gradle.
Im thinking about trying to build a complex android app structure for a game maybe or just for practice reasons. Im used to code in objective-c, so im not that much experienced in android...
Anyway in work, we structure our app on ios like this:
-core framework: handling all core items, navigation, datahandling, mechanisms, etc. its the same in all of our project
-project framework: its files are mostly relying (including) the core framework's files, extending/modifying them, and doing the project depending stuff
-skin framework: this contains all the resources and images, if we want to do a re-skinned project, we only have to alter this
-main project: this includes everything just bashing together everything into an app. just starts the application, nothing more, anything else is done by the different frameworks
So I wanted to do a similar structure on android, but I'm not sure that I'm even able to do it... I see that there is android project and library project, I can include them into eachother... but my questions are:
1: can I build a similar structure as on ios?
2: can I make for example a "core" library what contains the basics of mechanisms, and another library containing only the resources, and a third one (or the third could be the actual runnable project), what can get resources from the resource library, can distribute jobs to the core library, etc...
3: can I organize the resources as I like (so not to throw every picture into the drawable folder root for example). For example to have somehow a characters folder (i know i cant do forlders in the res folder), and map files into map folder, etc... My only chance to name them "properly"? (map_sheet_type_1, map_sheet_type_2, character_sheet_type_1, etc) (if its going to be a game, it would use opengl, lots of sprite drawing, etc)
or I should do everything in a single project, dividing everything into a lot of packages, and use libraries only for jobs like "how to transcode "A" object into "B" object" ?
Thanks for the answers in advance
although I've never developed a game before, but an app is an app:
yes
as you mention you have executable projects and libraries projects, libraries can use other libraries and the only thing that goes to the device is whatever the executable project is building. It's just important to remark that compiled libraries *.jar files resources cannot be used in your executable project (that's why the ActionBar Sherlock have to be used as a library-project). In order to use a resource placed in a library project the project must be with its full source code open in the Eclipse so it can be compiled together. That is because inside an app, there's only one R (resources) object, and during build all the resources from all the projects are put together.
unfortunately no. As you mentioned yourself the resources cannot be in subfolders and even their file names are restricted as they can only use lower case letters, numbers and _ (underline). Just be clever and organised, write a spec or something.
packages IS the way to organize a single project in Java. If you gonna use multiple or single is your choice. Usually you can encapsulate in a library-project stuff that can easily be re-used in different projects, and the final project will contain everything that is specific to that one app/game. I'll give you an example on the place I work, we have a KicthenLibrary that is a library-project that we use in every single Android app we do. That library already contains an excellent multi-threaded bitmap download and cache classes, we used to have a MapFragment (now deprecated) before Google released their MapFragment, easy Http GET/POST methods, etc. As you can see, all of those are stuff that can easily be re-used in several different projects.
And just as a last trick, http://www.eclipse.org/egit/ IMHO is much easier to use GIT directly from inside Eclipse.
Here are a couple links that should help you get started on this.
http://kasperholtze.com/android/how-to-best-organize-your-android-source/
http://bartinger.at/organization-tips-for-android-projects/
Also, when I worked at a start-up, we made an app for both iOS and Android. We started creating native apps for each, and ended up having somewhat different structure. Global information/variables were handled different, and I couldn't structure my files quite like iOS did. That said, Android structure isn't terribly hard to figure out, and I made a fair amount of sub-folders in my assets folder (for libraries and js and such). And yes, you can definitely have several libraries.
As for having several projects in several in one app, see this link How to create a single application from multiple Android projects
I am developing Android application for which I want to ship several different apks for different languages in the market (every language includes a huge bundle of files and I want to avoid creating one huge apk with all language bundles).
So what I need is to customize a bit the Manifest file for each language: e.g. the package of the application and possibly the application version etc. I know how I can template the manifest so that I can manually insert my values in certain points in the file (see this post). The problem is that I use ant for preparing my production apks, but otherwise I develop using Eclipse and so I need my project working in the IDE too. Eclipse requires complete Manifest file and will not understand of the templating I will use for the ant builds as far as I know (please somebody prove me wrong).
My problem is that I want to avoid maintaining two manifest files that are identical in large part (one templated and one complete for Eclipse). Currently I can think of two possible approaches, but I do not know how to accomplish them:
Use some kind of definition injection in the manifest file: if I am able to inject certain xml file in the body of AndroidManifest file, I can keep the identical part in one xml part and customize only the points of difference
If it is possible to configure Eclipse to use some sequence of ant tasks to build Android projects instead of the prebuild routines I might be able to integrate the way I build my production apks in the IDE.
Please if there is anyone who knows how to accomplish any of the above two, or has any other idea how can I solve my issue: help!
Take a look on ant replace task:
<replace file="${build.out}/src/config.prop" token="###" value="${build.version}-${build.type}"/>
But again you should be careful with values that they are unique.
You could also replace your eclipse manifest with generated manifest by echoxml task.
Or you could reuse this nice task about xml manipulation.
At the company I work for, we pretty much use the approach given by Eugen to automate the build process of apps that e.g. should simply be branded differently by exchanging certain assets, styles, strings and configurations. We tend to set up the project and build process as follows:
Create the Android project and set it up such that it works for a specific branding. This will make sure that you can still build and run from Eclipse. Assuming that the codebase does not change between differently branded releases, that should be fine.
Create an ant build configuration that:
copies any files that are going to be changed to a backup directory
modifies the project files according to a configuration file (see below)
compiles the project sources and signs it with a release key (if making a release build)
copies back the files from step 1, overwriting any changes and returning the project to its original state
Create configuration files for every 'branding', or language specific release in your scenario.
Basically these steps will allow you to create branded/partner/language specific builds by simply providing the appropriate configuration with the ant build command. In our case it looks somewhat like this:
ant partner-release -Dpartner=stackoverflow
where 'stackoverflow' will point to a configuration with the same name. In our case these configuration files are small xml files that contain all the parameters that are to be replaced for that specific build. It could contain strings, but might as well point to other files (e.g. logo images) that should be copied into the assets/resources. For example:
<config>
<version_name>1.00</version_name>
<version_code>1</version_code>
...
</config>
Some more pointers:
We use xmltask to modify any xml files in the project; e.g. the manifest and resources.
Since the aforementioned task is really easy to use, our build configuration files are also set up as xml files, as illustrated above. It's quite human readable and easy and straightforward to change.
You can also use replace and ReplaceRegExp tasks to change configuration-dependent variables in almost any file. The latter can be especially convenient to make build-time changes to source code.