I've been working with Android for quite some time, yet I have never seen any good explanations for this issue.
I'm working on an app with a number of artwork and sound files. Many of these files are temporary, are base files that future work will evolve from, or are large versions that will be split into multiple files later. I want to keep them under version control with this project, and I'd like to keep everything in the same place on my hard drive.
But I don't want these "work" files to end up in my apk. They are often huge and have nothing to do with the final product.
For most of my life (ie, before Android), I would simply have a subdirectory within my project called art_work_files. And this directory would be subdivided into all the various tasks. And git is fine with this, especially for SVG files (it even properly notes differences and changes).
But when I put these files in my Android Studio project, I get concerned that the AS will think these are somehow important and keep references to them--possibly even adding them to jar or apk files.
Yes, I realize that the final versions of the art needs to go in the res/drawing/ directories (and similar for sound and other resources). And I'm happy to do this final copy.
Am I over-thinking this, or is there a better way to organize and still keep Android Studio happy?
Since no one has answered this question in 7 months, I'll submit my solution.
I have created a directory under the main directectory (same level as java and res) called assets. Git recognizes this directory and will notice any files that are added, removed, or changed. I'm fine with using git to handle binaries btw.
For each type of asset, I make another subdirectory. Here's what it looks like:
app ->
libs
src ->
main ->
assets ->
font
ogg
png
svg
java
res
...
This seems to work fine and keeps everyone happy. I'm still curious how other people do this; I welcome your comments and answers.
Related
Background
Some files of the app can only be stored in res/raw or assets folders.
Each of those folders work in a very similar way to the other. res/raw folder allows to access files easier, with all the other benefits of resource files, while assets folder allows to access them no matter the file name and structure (including folders and sub folders).
The main idea of loading files is about the same for both of them. You just have a choice of ease-of-use, depends on your needs.
The problem
I remember that a very long time ago, I've found some special behavior of both of those folders:
Each folder within the assets folder had a max number of files. I think it was about 500, but not sure. I've noticed this behavior a very long time ago,
Some said that files in the assets folder have a max size for files (example here). I never saw such a restriction. Not even on Android 2.3 at the time.
Some said (example here), and it's still believed even today (example here), that if you load a file from res/raw, it could take much more memory than if you took it from assets folder.
What I've tried
For #1, I never had to use more files anyway after the project I worked on, and at the time I worked on it, we simply split the files into more folders.
For #2 , as I wrote, I never noticed it anyway. I used much larger files sizes.
For #3, I tried to make a sample project that compares the memory usage between the 2 methods. I didn't notice any difference (memory usage or time to load) between the 2 methods. Especially not a major one. Sadly I have only one device (Nexus 5x), and it has quite a new Android version (8.1). It might be that starting from specific Android version there is no difference between the 2 methods. Another reason for this is that it's harder to measure memory usage on Java, because of the GC, and I've already noticed that on Android 8.x, memory works a bit differently than before (written about it here).
I tried to read about the differences and restrictions of the above, but all I've found are very old articles, so I think things might have changed ever since.
The questions
Actually it's just one question, but I'd like to split it in case the answer is complex:
Are there any major or unique limitations or differences between using res/raw and assets folders?
Does reading a file from the assets folder (by creating an input stream from it) really take less memory than using the res/raw? So much that even one of the most appreciated developers (here) decides to choose it, even nowadays?
Have the above restrictions existed up to specific Android versions, and then they became identical in terms of no restrictions whatsoever (except of course files naming for res/raw, but that's just how it works) ?
If so, from which Android version do they work about the same?
Are there any major or unique limitations or differences between using res/raw and assets folders?
Now, In android we don't have any restriction on max limit size for any file in assets or in raw.
Android Documentation:
Arbitrary files to save in their raw form. To open these resources
with a raw InputStream, call Resources.openRawResource() with the
resource ID, which is R.raw.filename.
However, if you need access to original file names and file hierarchy,
you might consider saving some resources in the assets/ directory
(instead of res/raw/). Files in assets/ aren't given a resource ID, so
you can read them only using AssetManager.
Does reading a file from the assets folder (by creating an input stream from it) really take less memory than using the res/raw? So much that even one of the most appreciated developers (here) decides to choose it, even nowadays?
No, I have not found any differences between memory usage. It is one of biggest mess that android is having right now, Also we don't have any official documentation about their memory limitation.
Have the above restrictions existed up to specific Android versions, and then they became identical in terms of no restrictions whatsoever (except of course files naming for res/raw, but that's just how it works) ?
Before android 2.3 we had memory restriction for asset folder, which is 1 MB. Please refer link.
If so, from which Android version do they work about the same?
From android 2.3, We don't have any memory related restriction, which they launched in December, 2010
I have a number of different Android applications building in Eclipse / ADT that share common code and assets. Sharing the common source code folder is easy as I can use Link Source in the Build Path set-up. This works great.
These programs also share the content that's in the assets folder. At the moment I'm duplicating, which is a bit inelegant and error-prone.
Is it trivially possible to tell Eclipse / ADT to use a common assets folder in a similar way that it is trivial to have it share a source code folder? I can't find any similar option or tool.
Notes:
I am not looking for a 'library' solution (indeed that seems
fraught with issues anyway, and I did try that previously and hit
issues like this one). In any case that's less relevant for
assets, and I'm looking for simplicity.
It doesn't matter if the shared assets folder has to contain a
superset of the needs of all the applications.
Assets. Not "resources".
I'm using ADT version 22.2
For many complex reasons, symbolically linking the folders will not suffice. (You do not want to know the details. You'd never believe them anyway.)
Sharing the common source code folder is easy as I can use Link Source in the Build Path set-up
I would not count on that working over the long haul.
At the moment I'm duplicating, which is a bit inelegant and error-prone.
Alas, given your other restrictions, I suspect that it is your only option, though you could take some steps to reduce the error-prone aspect (e.g., script that synchronized the contents).
Long-term, Gradle for Android will be The One True Answer for all builds, Eclipse included. At that point, I expect your "Link Source" will break. However, at that time, you should be able to cut over to Gradle for your builds.
Gradle allows developers to override the default locations of all sorts of directories, including the assets directory. You should be able to set up a build.gradle file that points to some common location for the assets directory, overriding the default location that looks inside the project itself.
It's unclear why you think that symlinks/hardlinks will not work for you as a stopgap measure. You might wish to consider asking a separate StackOverflow question, explaining what you have tried and what problems you encountered with that technique, whether you think we will believe them or not.
Bear in mind that while Eclipse handles the javac step for the Android build, everything else is largely handled by external tools (e.g., aapt). Those will be oblivious to things that exist solely in Eclipse-land. Conversely, that's why symlinks/hardlinks should work, as from the standpoint of those tools, such items should be treated the same as actual files in the filesystem. Hence, the reason why "Link Source" works is not because Android really supports it, but that Android presently delegates that one piece of the build to Eclipse itself. Once Google adds Gradle for Android support for Eclipse (which I hope will be before the end of 2014), then I would expect Eclipse to be completely cut out of the build process, and I expect "Link Source" to fail at that point.
Im thinking about trying to build a complex android app structure for a game maybe or just for practice reasons. Im used to code in objective-c, so im not that much experienced in android...
Anyway in work, we structure our app on ios like this:
-core framework: handling all core items, navigation, datahandling, mechanisms, etc. its the same in all of our project
-project framework: its files are mostly relying (including) the core framework's files, extending/modifying them, and doing the project depending stuff
-skin framework: this contains all the resources and images, if we want to do a re-skinned project, we only have to alter this
-main project: this includes everything just bashing together everything into an app. just starts the application, nothing more, anything else is done by the different frameworks
So I wanted to do a similar structure on android, but I'm not sure that I'm even able to do it... I see that there is android project and library project, I can include them into eachother... but my questions are:
1: can I build a similar structure as on ios?
2: can I make for example a "core" library what contains the basics of mechanisms, and another library containing only the resources, and a third one (or the third could be the actual runnable project), what can get resources from the resource library, can distribute jobs to the core library, etc...
3: can I organize the resources as I like (so not to throw every picture into the drawable folder root for example). For example to have somehow a characters folder (i know i cant do forlders in the res folder), and map files into map folder, etc... My only chance to name them "properly"? (map_sheet_type_1, map_sheet_type_2, character_sheet_type_1, etc) (if its going to be a game, it would use opengl, lots of sprite drawing, etc)
or I should do everything in a single project, dividing everything into a lot of packages, and use libraries only for jobs like "how to transcode "A" object into "B" object" ?
Thanks for the answers in advance
although I've never developed a game before, but an app is an app:
yes
as you mention you have executable projects and libraries projects, libraries can use other libraries and the only thing that goes to the device is whatever the executable project is building. It's just important to remark that compiled libraries *.jar files resources cannot be used in your executable project (that's why the ActionBar Sherlock have to be used as a library-project). In order to use a resource placed in a library project the project must be with its full source code open in the Eclipse so it can be compiled together. That is because inside an app, there's only one R (resources) object, and during build all the resources from all the projects are put together.
unfortunately no. As you mentioned yourself the resources cannot be in subfolders and even their file names are restricted as they can only use lower case letters, numbers and _ (underline). Just be clever and organised, write a spec or something.
packages IS the way to organize a single project in Java. If you gonna use multiple or single is your choice. Usually you can encapsulate in a library-project stuff that can easily be re-used in different projects, and the final project will contain everything that is specific to that one app/game. I'll give you an example on the place I work, we have a KicthenLibrary that is a library-project that we use in every single Android app we do. That library already contains an excellent multi-threaded bitmap download and cache classes, we used to have a MapFragment (now deprecated) before Google released their MapFragment, easy Http GET/POST methods, etc. As you can see, all of those are stuff that can easily be re-used in several different projects.
And just as a last trick, http://www.eclipse.org/egit/ IMHO is much easier to use GIT directly from inside Eclipse.
Here are a couple links that should help you get started on this.
http://kasperholtze.com/android/how-to-best-organize-your-android-source/
http://bartinger.at/organization-tips-for-android-projects/
Also, when I worked at a start-up, we made an app for both iOS and Android. We started creating native apps for each, and ended up having somewhat different structure. Global information/variables were handled different, and I couldn't structure my files quite like iOS did. That said, Android structure isn't terribly hard to figure out, and I made a fair amount of sub-folders in my assets folder (for libraries and js and such). And yes, you can definitely have several libraries.
As for having several projects in several in one app, see this link How to create a single application from multiple Android projects
I have a small Android application that uses different sets of files (a couple of images, a small SQLite DB and a couple of XML files) depending on the specific task at hand.
I know I can include my files into the main application APK using resources or assets but I would be happy to distribute them in a separated APK.
How can I create a data-only APK file?
How can I distribute it? In particular, do I have to do anything special for a data-only package (for example for associating it to the main application package in some way)?
(I'm intentioned to give the user a link to the data package and ask him to install it. No automatic installation required.)
How can I install my files into the internal or into the external storage area of my application? Is it possible at all to install files into the internal storage area created by the main application installer? Do I have to set any particular permission for this?
My approach to this would be to create a wrapper app that's nothing but a content-provider and serves up the files per request by your main app. This would allow you to supply different data packages for the user -- you could even have your main app select between those relatively easily.
It looks like that the commonly accepted way to have the same application with different contents (or styles, or configurations) is to use an Android Library Project for the common code (that is: the whole application, the "engine", the "app framework") and a standard Android Application Project for the contents (that is: an application that actually contains just data). A little bit confusing, just because the "library" here is actually the whole "app", but this seems to be the way to go.
More in detail:
Create an Android Library Application and put into it as much code as you can (all of the non-changing stuff). Please note that this library cannot be launched and cannot be distributed alone. It must be included in a hosting application.
Create a standard Android Application. Include your library into this project. Put in /res and in /asset all of your data (files, XML, etc.).
Compile everything and distribute.
Repeat this cycle every time you need a different version. Different because of data, style, configuration or anything else. Publish the resulting app with a new name.
For what regards me, I'm not completely satisfied by this approach.
A possible alternative is preprocessing the source code with Ruby, Python, Perl, GIT, bash, Ant, Maven, Rake or any other tool that is able to read a file from here, make some change here and there, and write the file there.
The general outline is something like this:
Make a "template" application. Leave your /res and /assset empty.
Run a custom-made script. The script reads a configuration file, copy the /res and /asset files from your repository into the project /res and /asset directories, changes some Java source file and creates/changes some XML file.
Compile and distribute (with a new name, of course).
Using GIT or other SCMs, you just make a new branch for every new version and compile it. Not very elegant (because it can strongly interfere with the normal use of the SCM) but...
There are a few example of these approaches on the web. I'm not completely satisfied by them, either.
Frankly, what the Android ecosystem should offer to solve this problem is some kind of "in-app package manager". Something like the Eclipse Update Manager. This would allow us to use the same application framework to handle different scenarios.
As an alternative, a solid, officially-supported, template-based code-generation mechanism would be nice. Something in the spirit of "Software Production Line": https://en.wikipedia.org/wiki/Software_production_line . Have a look at fw4spl, for example: http://code.google.com/p/fw4spl/ .
I'm porting a rather large game engine written in C++ from Windows/Mac to Android. There is a lot of pre-existing code to read assets for games. In addition, there is quite a bit of code doing file system calls (stat'ing the files to make sure they exist, looking up all of the files and directories inside of a directory, etc.)
Right now, I'm focusing on just getting something up and running as quickly as possible, so I'd prefer not to have to rewrite a lot of this. What would be a good way of getting our game assets onto the device and accessing them with minimal changes to our existing standard C++ file system API usage?
I've got some basic support implemented already using the Asset Manager API, but that doesn't support the file system calls and I'm concerned that the 1 MB asset size limit is going to bite me at some point.
I've also looked at OBB, but the tools for creating an OBB file don't look like they are part of the current SDK/NDK. Otherwise, that looks like it would be perfect.
Is it a horrible idea to package up all of the files and just extract them on the SD Card the first time the app is run? Or is there some better way of dealing with this?
Update: I'm also not very concerned on being able to run on a broad range of devices, I am specifically looking at newish tablets, probably the 10.1" Samsung Galaxy tab.
We ran into a similar problem in developing our (data-file-heavy) app, and we ended up deciding to keep the APK tiny and simply download our data files on first run; they're going to have to be downloaded either way, but a small APK works much better on older devices without a lot of internal storage. Plus, you can potentially rig up a way for people to copy over the files directly from their computer if they have a limited data plan or a slow internet connection on their phone.
The "Downloader" sample app in apps-for-android (confusingly buried under "Samples") is almost a fully-implemented solution for this - you can pretty much just plug in the particulars of your data files and let it do the rest.
I wrote an app that relies on putting a good amount of native code into the Android filesystem. I did this by packaging the files into the APK as 'resources'. Instead of pushing them to the SD card, you can put then into the application's private namespace, I.E. /data/data/com.yourdomain.yourapp/nativeFolder.
For details on how to accomplish this, you can see my answer to this question.
It's fairly simple to package to just unpack them on the first run and never worry about them again. Also, since they're under the application's namespace, they should be deleted if/when someone were to decide to delete your app.
EDIT:
This method can be used to put anything into the app's private area; /data/data/com.yourdomain.yourapp/
However, as far as I know, your application has to be the one to create all the folders and sub-folders in this area. Luckily this is fairly easy to do. For example to have your app make a folder:
Process mkdir = Runtime.getRuntime().exec("mkdir " +localPath);
That works as it would in most linux shells. I walked through the assets folder I packaged into my APK, made the corresponding directories and copied all the native files to those directories.
What you might be more concerned with is the limited Android shell. There are many commands that you might want that aren't present. stat for example isn't available, so all of this may be moot if your native code can't make it's system calls.