I am working on an Android project that has a native ".so" file embedded in an apk. Whenever we need to release a bug fix we are now releasing a new apk file with ".so" which has the fix. But this is a huge file and is not efficient.
So my question is, is it possible to update the ".so" inside the ".apk" with only the relevant ".o" files that have changed? Meaning keeping the rest of the ".so" file the same, can we just update only those ".o" files that changed? Similar to how some systems push their bug fixes.
Updating parts of dynamic library (.so) is not possible. It's not like a static library (.a), which is, essentially, just a plain archive with object files. A dynamic library is more like a usual executable and can be thought of as just a program with no main. In fact, shared libraries are usually compiled as PIC and overall, if one part of the library changes, all other parts should be updated.
Say, there's one more instruction in the code of some function a, which is followed by functions b and c. Then addresses of b and c should be increased by size of the instruction which got added to a. Then all call sites of b and c inside this dynamic library should be updated with new addresses.
That said, it could be done in theory, but the cost of implementation is much higher than profit from it.
Related
I've been working with Android for quite some time, yet I have never seen any good explanations for this issue.
I'm working on an app with a number of artwork and sound files. Many of these files are temporary, are base files that future work will evolve from, or are large versions that will be split into multiple files later. I want to keep them under version control with this project, and I'd like to keep everything in the same place on my hard drive.
But I don't want these "work" files to end up in my apk. They are often huge and have nothing to do with the final product.
For most of my life (ie, before Android), I would simply have a subdirectory within my project called art_work_files. And this directory would be subdivided into all the various tasks. And git is fine with this, especially for SVG files (it even properly notes differences and changes).
But when I put these files in my Android Studio project, I get concerned that the AS will think these are somehow important and keep references to them--possibly even adding them to jar or apk files.
Yes, I realize that the final versions of the art needs to go in the res/drawing/ directories (and similar for sound and other resources). And I'm happy to do this final copy.
Am I over-thinking this, or is there a better way to organize and still keep Android Studio happy?
Since no one has answered this question in 7 months, I'll submit my solution.
I have created a directory under the main directectory (same level as java and res) called assets. Git recognizes this directory and will notice any files that are added, removed, or changed. I'm fine with using git to handle binaries btw.
For each type of asset, I make another subdirectory. Here's what it looks like:
app ->
libs
src ->
main ->
assets ->
font
ogg
png
svg
java
res
...
This seems to work fine and keeps everyone happy. I'm still curious how other people do this; I welcome your comments and answers.
The short summary is: How do I build an APK and separate libraries (by which I mean sets of classes (and ideally, resources too) in some form, such as JAR, AAR or DEX files), but not include those libraries in the APK; instead, the app loads them at run time?
Detail
So my main question is how to build such an app (e.g. Gradle configuration). How do I specify which classes go into which JAR or DEX files? Do I create an Android Studio module for each DEX file I want to end up with?
A closely related question is how the Java code should then load the external libraries and access their classes at run time. For the latter, I'm hopeful that the approach shown at accessing to classes of app from dex file by classloader would work.
I've tried the instructions at https://developer.android.com/studio/projects/android-library.html, but that builds an APK that does include the dependency library.
I've also tried Multidex (https://developer.android.com/studio/build/multidex.html), but that doesn't seem to leave the developer any control over which classes go in which DEX file, and furthermore, packages them all into a single APK. AFAICT there is no way to control the loading of these DEX files at run time.
Background
There's a possibility of the "X-Y problem" here, so I'd better explain the background.
I'm building an app for a client. It's not going to be distributed through an app store, so it won't have access to the normal mechanism for updates. Instead, the client wants the app to be able to update itself by downloading new components of itself to replace the old components, without a need to manually sideload a new APK. The primary motive here is that the updates have to be easy for non-technical users. If the app can control the update process, it can make it smooth and guide the user.
Moreover, the app will be used in areas where internet access is scarce and expensive, so the client wants to be able to issue app updates in smaller chunks (e.g. 2MB) rather than forcing the user to re-download the whole app to receive a small update.
One aspect of the requirements I should mention, in case it matters, is that the libraries to be loaded at run time are supposed to live on a microSD card. This can also help with distribution of updates without internet access.
The current status of the app is that it's about 50% written: That is, a couple of earlier versions have been released, but the app now needs to be modified (restructured) to meet the above requirements, as well as others.
This tutorial is a good start for external loading of DEX files.
Only three small files of source (MainActivity.java, LibraryInterface.java, LibraryProvider.java) and it copies secondary_dex.jar from the assets folder, into internal application storage [outdex/dex] (the internet is also stated as possible in the tutorial).
You have to build it with ant, because it uses custom build steps.
I tried it, it works fine. Worth a look.
custom class loading in Dalvik and ART
UPDATE
this code has been ported to Android Studio gradle (no need for ant).
https://github.com/timrae/custom-class-loader
Tested ok. Copies com.example.toastlib.jar from the SDcard into internal application storage [outdex/dex],(not assets folder).
( you must read the README.md file in the project to build it).
Q: How do I add an Activity, I cannot add it to the manifest ? A: Use
Fragments, they don't need entries in the manifest.
Q: A Jar with resources that is meant to be added to an existing
project needs to be able to merge its resources with the project's
own resources (R.). A: Hacks are available, Data file...Packaging Android resource files within a distributable Jar file
Q: The external file has wrong permissions. A: Import it.
Q: I need to add uses-permission. A: Use API23 you can programmatically add uses-permissions (but they still need to be declared in the Manifest, so the new permissions model is probably not much use to us).
This section is for more general users (#LarsH has more specific requirements about updates), The example above is 17kb apk and 1 kb jar. You could put the bulk of you code in the one-off jar, and updates would involve just loading an new Apk (and then importing the bulk code jar, to minimise the data transfer).
When the Apk gets too big, start again with a small Apk and everything migrated to another jar (import 2 jar's). You need to balance coding effort, user experience, maintainability, supportability, bandwidth, android rules, play store rules (if these words even exist ;O)).
NOTE Dalvik is discontinued
The successor of Dalvik is Android Runtime (ART), which uses the same bytecode and .dex files (but not .odex files), with the succession aiming at performance improvements transparent to the end users. The new runtime environment was included for the first time in Android 4.4 "KitKat" as a technology preview, and replaced Dalvik entirely in later versions; Android 5.0 "Lollipop" is the first version in which ART is the only included runtime.
You could try to build multiple apk's with the same sharedUserId and the same process.
This is the plugin mechanism used by Threema
Edit: More about Theema
Threema has one main app and two plugins:
main app: https://play.google.com/store/apps/details?id=ch.threema.app
QR-Code Plugin: https://play.google.com/store/apps/details?id=ch.threema.qrscannerplugin
Voicemessage Plugin: https://play.google.com/store/apps/details?id=ch.threema.voicemessageplugin
Doing so the main app does not need the permissions for accessing the camera or microphone
Im thinking about trying to build a complex android app structure for a game maybe or just for practice reasons. Im used to code in objective-c, so im not that much experienced in android...
Anyway in work, we structure our app on ios like this:
-core framework: handling all core items, navigation, datahandling, mechanisms, etc. its the same in all of our project
-project framework: its files are mostly relying (including) the core framework's files, extending/modifying them, and doing the project depending stuff
-skin framework: this contains all the resources and images, if we want to do a re-skinned project, we only have to alter this
-main project: this includes everything just bashing together everything into an app. just starts the application, nothing more, anything else is done by the different frameworks
So I wanted to do a similar structure on android, but I'm not sure that I'm even able to do it... I see that there is android project and library project, I can include them into eachother... but my questions are:
1: can I build a similar structure as on ios?
2: can I make for example a "core" library what contains the basics of mechanisms, and another library containing only the resources, and a third one (or the third could be the actual runnable project), what can get resources from the resource library, can distribute jobs to the core library, etc...
3: can I organize the resources as I like (so not to throw every picture into the drawable folder root for example). For example to have somehow a characters folder (i know i cant do forlders in the res folder), and map files into map folder, etc... My only chance to name them "properly"? (map_sheet_type_1, map_sheet_type_2, character_sheet_type_1, etc) (if its going to be a game, it would use opengl, lots of sprite drawing, etc)
or I should do everything in a single project, dividing everything into a lot of packages, and use libraries only for jobs like "how to transcode "A" object into "B" object" ?
Thanks for the answers in advance
although I've never developed a game before, but an app is an app:
yes
as you mention you have executable projects and libraries projects, libraries can use other libraries and the only thing that goes to the device is whatever the executable project is building. It's just important to remark that compiled libraries *.jar files resources cannot be used in your executable project (that's why the ActionBar Sherlock have to be used as a library-project). In order to use a resource placed in a library project the project must be with its full source code open in the Eclipse so it can be compiled together. That is because inside an app, there's only one R (resources) object, and during build all the resources from all the projects are put together.
unfortunately no. As you mentioned yourself the resources cannot be in subfolders and even their file names are restricted as they can only use lower case letters, numbers and _ (underline). Just be clever and organised, write a spec or something.
packages IS the way to organize a single project in Java. If you gonna use multiple or single is your choice. Usually you can encapsulate in a library-project stuff that can easily be re-used in different projects, and the final project will contain everything that is specific to that one app/game. I'll give you an example on the place I work, we have a KicthenLibrary that is a library-project that we use in every single Android app we do. That library already contains an excellent multi-threaded bitmap download and cache classes, we used to have a MapFragment (now deprecated) before Google released their MapFragment, easy Http GET/POST methods, etc. As you can see, all of those are stuff that can easily be re-used in several different projects.
And just as a last trick, http://www.eclipse.org/egit/ IMHO is much easier to use GIT directly from inside Eclipse.
Here are a couple links that should help you get started on this.
http://kasperholtze.com/android/how-to-best-organize-your-android-source/
http://bartinger.at/organization-tips-for-android-projects/
Also, when I worked at a start-up, we made an app for both iOS and Android. We started creating native apps for each, and ended up having somewhat different structure. Global information/variables were handled different, and I couldn't structure my files quite like iOS did. That said, Android structure isn't terribly hard to figure out, and I made a fair amount of sub-folders in my assets folder (for libraries and js and such). And yes, you can definitely have several libraries.
As for having several projects in several in one app, see this link How to create a single application from multiple Android projects
I have a small Android application that uses different sets of files (a couple of images, a small SQLite DB and a couple of XML files) depending on the specific task at hand.
I know I can include my files into the main application APK using resources or assets but I would be happy to distribute them in a separated APK.
How can I create a data-only APK file?
How can I distribute it? In particular, do I have to do anything special for a data-only package (for example for associating it to the main application package in some way)?
(I'm intentioned to give the user a link to the data package and ask him to install it. No automatic installation required.)
How can I install my files into the internal or into the external storage area of my application? Is it possible at all to install files into the internal storage area created by the main application installer? Do I have to set any particular permission for this?
My approach to this would be to create a wrapper app that's nothing but a content-provider and serves up the files per request by your main app. This would allow you to supply different data packages for the user -- you could even have your main app select between those relatively easily.
It looks like that the commonly accepted way to have the same application with different contents (or styles, or configurations) is to use an Android Library Project for the common code (that is: the whole application, the "engine", the "app framework") and a standard Android Application Project for the contents (that is: an application that actually contains just data). A little bit confusing, just because the "library" here is actually the whole "app", but this seems to be the way to go.
More in detail:
Create an Android Library Application and put into it as much code as you can (all of the non-changing stuff). Please note that this library cannot be launched and cannot be distributed alone. It must be included in a hosting application.
Create a standard Android Application. Include your library into this project. Put in /res and in /asset all of your data (files, XML, etc.).
Compile everything and distribute.
Repeat this cycle every time you need a different version. Different because of data, style, configuration or anything else. Publish the resulting app with a new name.
For what regards me, I'm not completely satisfied by this approach.
A possible alternative is preprocessing the source code with Ruby, Python, Perl, GIT, bash, Ant, Maven, Rake or any other tool that is able to read a file from here, make some change here and there, and write the file there.
The general outline is something like this:
Make a "template" application. Leave your /res and /assset empty.
Run a custom-made script. The script reads a configuration file, copy the /res and /asset files from your repository into the project /res and /asset directories, changes some Java source file and creates/changes some XML file.
Compile and distribute (with a new name, of course).
Using GIT or other SCMs, you just make a new branch for every new version and compile it. Not very elegant (because it can strongly interfere with the normal use of the SCM) but...
There are a few example of these approaches on the web. I'm not completely satisfied by them, either.
Frankly, what the Android ecosystem should offer to solve this problem is some kind of "in-app package manager". Something like the Eclipse Update Manager. This would allow us to use the same application framework to handle different scenarios.
As an alternative, a solid, officially-supported, template-based code-generation mechanism would be nice. Something in the spirit of "Software Production Line": https://en.wikipedia.org/wiki/Software_production_line . Have a look at fw4spl, for example: http://code.google.com/p/fw4spl/ .
The standard advice for sharing code & resources between Android projects is to use a library. Personally I find this works poorly if (a) the shared code changes a lot, or (b) your computer isn't fast enough.
I also don't want to get into deploying multiple APK's, which seems to be necessary when I use dependent projects (i.e. Java Build Path, Projects Tab).
On the other hand, sharing a folder of source code by using the Eclipse linked source feature works great (Java Build Path, Source tab, Link Source button), but for these two issues:
1) I can't use the same technique to share resources. I can create the link to the resources parent folder but then things get wonky and the shared resources don't get compiled (I'm using ADT 21).
2) So then I settle for copying the shared resources into each project, but this doesn't work because either. The shared code can't import the copy of its resources because it doesn't know the package name of the project that uses it. The solution I've been using is to access the resources dynamically, but that has become cumbersome as the number of resources grows.
So, I need a solution to either (1) or (2), or I'll have to go back to a library project. (Or maybe there is another option I haven't thought of?)
Your real problem is (2). Fixing (1) would eliminate some copying, but you would still run into problems with (2).
Unfortunately, that really isn't possible. There's a fair bit of fancy footwork that goes on to make multiple packages possible with library projects, and there's no good way to get that same result without library projects. Anything in res/ of a project is accessed via that project's R class, including your copied resources.
The solution I've been using is to access the resources dynamically
I translated that into you using getIdentifier(). That certainly works. Another approach is to having the hosting app supply resource IDs as parameters to the library code -- this is the pattern that the Android SDK itself uses. This is faster at runtime than the reflection-based getIdentifier(), and it gives the hosting app somewhat more flexibility, but you do wind up adding a bunch of parameters to your methods and constructors as needed to supply the various project-specific R values.