I'm porting a rather large game engine written in C++ from Windows/Mac to Android. There is a lot of pre-existing code to read assets for games. In addition, there is quite a bit of code doing file system calls (stat'ing the files to make sure they exist, looking up all of the files and directories inside of a directory, etc.)
Right now, I'm focusing on just getting something up and running as quickly as possible, so I'd prefer not to have to rewrite a lot of this. What would be a good way of getting our game assets onto the device and accessing them with minimal changes to our existing standard C++ file system API usage?
I've got some basic support implemented already using the Asset Manager API, but that doesn't support the file system calls and I'm concerned that the 1 MB asset size limit is going to bite me at some point.
I've also looked at OBB, but the tools for creating an OBB file don't look like they are part of the current SDK/NDK. Otherwise, that looks like it would be perfect.
Is it a horrible idea to package up all of the files and just extract them on the SD Card the first time the app is run? Or is there some better way of dealing with this?
Update: I'm also not very concerned on being able to run on a broad range of devices, I am specifically looking at newish tablets, probably the 10.1" Samsung Galaxy tab.
We ran into a similar problem in developing our (data-file-heavy) app, and we ended up deciding to keep the APK tiny and simply download our data files on first run; they're going to have to be downloaded either way, but a small APK works much better on older devices without a lot of internal storage. Plus, you can potentially rig up a way for people to copy over the files directly from their computer if they have a limited data plan or a slow internet connection on their phone.
The "Downloader" sample app in apps-for-android (confusingly buried under "Samples") is almost a fully-implemented solution for this - you can pretty much just plug in the particulars of your data files and let it do the rest.
I wrote an app that relies on putting a good amount of native code into the Android filesystem. I did this by packaging the files into the APK as 'resources'. Instead of pushing them to the SD card, you can put then into the application's private namespace, I.E. /data/data/com.yourdomain.yourapp/nativeFolder.
For details on how to accomplish this, you can see my answer to this question.
It's fairly simple to package to just unpack them on the first run and never worry about them again. Also, since they're under the application's namespace, they should be deleted if/when someone were to decide to delete your app.
EDIT:
This method can be used to put anything into the app's private area; /data/data/com.yourdomain.yourapp/
However, as far as I know, your application has to be the one to create all the folders and sub-folders in this area. Luckily this is fairly easy to do. For example to have your app make a folder:
Process mkdir = Runtime.getRuntime().exec("mkdir " +localPath);
That works as it would in most linux shells. I walked through the assets folder I packaged into my APK, made the corresponding directories and copied all the native files to those directories.
What you might be more concerned with is the limited Android shell. There are many commands that you might want that aren't present. stat for example isn't available, so all of this may be moot if your native code can't make it's system calls.
Related
I've been working with Android for quite some time, yet I have never seen any good explanations for this issue.
I'm working on an app with a number of artwork and sound files. Many of these files are temporary, are base files that future work will evolve from, or are large versions that will be split into multiple files later. I want to keep them under version control with this project, and I'd like to keep everything in the same place on my hard drive.
But I don't want these "work" files to end up in my apk. They are often huge and have nothing to do with the final product.
For most of my life (ie, before Android), I would simply have a subdirectory within my project called art_work_files. And this directory would be subdivided into all the various tasks. And git is fine with this, especially for SVG files (it even properly notes differences and changes).
But when I put these files in my Android Studio project, I get concerned that the AS will think these are somehow important and keep references to them--possibly even adding them to jar or apk files.
Yes, I realize that the final versions of the art needs to go in the res/drawing/ directories (and similar for sound and other resources). And I'm happy to do this final copy.
Am I over-thinking this, or is there a better way to organize and still keep Android Studio happy?
Since no one has answered this question in 7 months, I'll submit my solution.
I have created a directory under the main directectory (same level as java and res) called assets. Git recognizes this directory and will notice any files that are added, removed, or changed. I'm fine with using git to handle binaries btw.
For each type of asset, I make another subdirectory. Here's what it looks like:
app ->
libs
src ->
main ->
assets ->
font
ogg
png
svg
java
res
...
This seems to work fine and keeps everyone happy. I'm still curious how other people do this; I welcome your comments and answers.
Background
Some files of the app can only be stored in res/raw or assets folders.
Each of those folders work in a very similar way to the other. res/raw folder allows to access files easier, with all the other benefits of resource files, while assets folder allows to access them no matter the file name and structure (including folders and sub folders).
The main idea of loading files is about the same for both of them. You just have a choice of ease-of-use, depends on your needs.
The problem
I remember that a very long time ago, I've found some special behavior of both of those folders:
Each folder within the assets folder had a max number of files. I think it was about 500, but not sure. I've noticed this behavior a very long time ago,
Some said that files in the assets folder have a max size for files (example here). I never saw such a restriction. Not even on Android 2.3 at the time.
Some said (example here), and it's still believed even today (example here), that if you load a file from res/raw, it could take much more memory than if you took it from assets folder.
What I've tried
For #1, I never had to use more files anyway after the project I worked on, and at the time I worked on it, we simply split the files into more folders.
For #2 , as I wrote, I never noticed it anyway. I used much larger files sizes.
For #3, I tried to make a sample project that compares the memory usage between the 2 methods. I didn't notice any difference (memory usage or time to load) between the 2 methods. Especially not a major one. Sadly I have only one device (Nexus 5x), and it has quite a new Android version (8.1). It might be that starting from specific Android version there is no difference between the 2 methods. Another reason for this is that it's harder to measure memory usage on Java, because of the GC, and I've already noticed that on Android 8.x, memory works a bit differently than before (written about it here).
I tried to read about the differences and restrictions of the above, but all I've found are very old articles, so I think things might have changed ever since.
The questions
Actually it's just one question, but I'd like to split it in case the answer is complex:
Are there any major or unique limitations or differences between using res/raw and assets folders?
Does reading a file from the assets folder (by creating an input stream from it) really take less memory than using the res/raw? So much that even one of the most appreciated developers (here) decides to choose it, even nowadays?
Have the above restrictions existed up to specific Android versions, and then they became identical in terms of no restrictions whatsoever (except of course files naming for res/raw, but that's just how it works) ?
If so, from which Android version do they work about the same?
Are there any major or unique limitations or differences between using res/raw and assets folders?
Now, In android we don't have any restriction on max limit size for any file in assets or in raw.
Android Documentation:
Arbitrary files to save in their raw form. To open these resources
with a raw InputStream, call Resources.openRawResource() with the
resource ID, which is R.raw.filename.
However, if you need access to original file names and file hierarchy,
you might consider saving some resources in the assets/ directory
(instead of res/raw/). Files in assets/ aren't given a resource ID, so
you can read them only using AssetManager.
Does reading a file from the assets folder (by creating an input stream from it) really take less memory than using the res/raw? So much that even one of the most appreciated developers (here) decides to choose it, even nowadays?
No, I have not found any differences between memory usage. It is one of biggest mess that android is having right now, Also we don't have any official documentation about their memory limitation.
Have the above restrictions existed up to specific Android versions, and then they became identical in terms of no restrictions whatsoever (except of course files naming for res/raw, but that's just how it works) ?
Before android 2.3 we had memory restriction for asset folder, which is 1 MB. Please refer link.
If so, from which Android version do they work about the same?
From android 2.3, We don't have any memory related restriction, which they launched in December, 2010
I've got an Android app written in Kivy (Python), which stores local files that should survive an app update (adb install -r).
If the files are stored in a subdirectory of the current directory ("data/data/app_name/files"), I see that they are deleted after update.
However after some experiments I could "solve" this by storing the files in the "data/data/app_name/shared_prefs" directory, which seems to be persistent after updates. By the way, I didn't check but maybe the "data/data/app_name/databases" also is.
Is there a cleaner way of doing things ?
I need to test if I can create a new folder not called shared_prefs nor databases under "data/data/app_name", and if it is persistent.
(this seems kind of a hack because those directories have another dedicated purpose, even though my app is not using them for this dedicated purpose right now)
(NB: I don't want to keep the files outside the app private directory)
There is not a simple way (as in a build hook or similar) right now, but it's something we've specifically discussed in the last few days as the current situation has become a direct problem. I'm not sure what the resolution was, but there will probably be a change in python-for-android to fix it fairly soon.
If you want to keep up to date with this, ask on the kivy mailing list or irc. In particular, knapper_tech was making these changes.
I'm just wondering how the following scenario can be solved:
I want to write a very simple app for my daughter. The app displays 4 colored fields on the activity and through speech output says something like "tap green". She should then tap the green field to get positive feedback. So far, so easy.
I then thought it might be nice if the app would also "grow" as she grows older. Maybe later I want to have a game mode where I display animals, numbers, vehicles, etc., so it would be nice if I could have something like "Shape sets" - basically a set of images along with a description of what the app should say for each image. Also easy enough - all you need is a set of images and an XML file describing the images.
BUT
I'd like to be able to install these "Shape sets" as additional APKs later on, so that I don't have to modify the app every time. I'd like to install the APKs, so that the contents are added to a specific sub folder on the SD card, into which my app looks to enumerate available "Shape sets".
If I ever published the app to the Play Store, other people should also be able to download the "Shape sets" I create (no need for user contributions, though).
Is that possible? If so, what would I have to do to have the Android OS "copy" the contents of an APK to a specific folder (lets say "/TapGame/Shape Sets/Animals") on the SD card? Or is there even another way of achieving what I want that I didn't think of?
The term "plugin architecture" just came to my mind as I wrote the question. Searching using that term I found this question: Extend my android app in different APK
It seems to provide a solution to my problem - I'll investigate this further, but please feel free to suggest other possible solutions!
Is that possible?
Um, sure.
If so, what would I have to do to have the Android OS "copy" the contents of an APK to a specific folder (lets say "/TapGame/Shape Sets/Animals") on the SD card?
Android won't do any of that. You have to do that. You would have to detect that a "shape set" APK was installed (either watching for package-installed broadcasts, scanning all installed apps for ones that seem to be a "shape set", etc.). Then you would have to arrange to copy whatever you wanted to wherever you wanted it, either by:
Asking the "shape set" app to do it (e.g., send a command to some IntentService), or
Using createPackageContext() and trying to do the copying from your main app
Or is there even another way of achieving what I want that I didn't think of?
Um, just use ZIP files that your app downloads itself from a well-known location. That corresponds to Dave Smith's final paragraph of his answer on the question you just linked to in your edit.
Or, just update the main app. I'm not quite certain what effort you think that you are saving otherwise.
Or, just keep the content online, using a Web service to indicate the available "shape sets" and downloading them as needed (with optional caching).
Fascinating question. If you really want to go for plug-ins then OSGi would probably be the way to go, but it's a lot of work to get to know and to use and seems like overkill in this case.
I don't know how your shapes are defined, but they are probably each defined in a separate file-set, providing the shape (maybe a png or jpg?) as well as the audio-file that will be used as a command for this shape. If the folder in which these file-sets are stored is fixed (TapGame/Shape Sets/...) the app could scan the folder each startup and the views could be generated accordingly (in this case, the activity cannot be build entirely in the XML-File, but must be partially done programmatically).
The Plugin-Aps would be rather easy. They are an apk which includes the file sets (jpg and mp3 or whatever). Started once they deposit all these files into the specified folder (they probably check before if these files exist) and then the apk can shut down again and be uninstalled.
on the next startup the Tap Game App would find the new symbols and include them into the game.
This seems rather straight forward to me. Another way would be to actually store the shapes and audio files on the internet and with each start of the App check if the number of shapes and audio-files has changed and create local copies of new ones. This would mean no downloads of apks... probably a more usual approach to the issue.
I am very interested to hear what you make of it, seems like a different approach then the norm, which is always cool to see.
I am doing a forensic course and as a requirement I have been asked to develop a forensic investigation tool (windows based) for Google's Android OS. The requirement is such that given an image file, the tool should be able to display the databases that the applications are using, call history, messages and etc..
I have little experience in Java but I have no experience in Android development. The research so far has given me nothing on how to go about this. If anyone could point me in the right direction I would much appreciate it.
Thanks in advance.
Step 1 would be mounting the filesystem. Since Android is Linux based, there's a huge array of filesystems available, and individual vendors may or may not decide to write their own filesystems, just for the fun of it. On Windows, your options include ext2fsd or ext2read, among other possibilities.
Once you've got the filesystem mounted, then you get to deal with the per-application data storage. I'd wager a fair amount of applications use SQLite3, because it is an amazing tool. But you'll have to figure out, for each type of data you want to read, where it is stored and in what format. (The standard file(1) tool on Linux systems can come in handy, it knows heuristics that are surprisingly good at showing what type of file you might be dealing with.)
If you have the .apk of an application, a tool such as dex2jar, used in combinaison with something like jd-gui, can get you the JAVA source-code of the application (which can help, if not obfuscated).
After that, an .apk is basically a zip-file -- which means opening it with an unzip-ing application will allow you to get the images and resources it uses.
Then, databases used by Android applications tend to be SQLite, on which you can do SQL queries, using an SQLite client.