I am developing an application in which I have a database with 5000 rows with 4 columns.
problem_id (int)
problem_no (string)
problem_title (string)
dacu (int)
I need to frequently query single items in a large scale like 1000 query to fetch problem_no based on problem_id or sometimes only one item.
So I decided to query all the database rows and map them in a hashMap at runtime. I know hashmap insertion/query operation will take only O(1) or sometimes little more, so I only need 5000 operations I think. But how much space hashMap will take in this case? Would android dalvik be able to allocate them without any trouble?
How much space will hashmap take?
It's an implementation detail that can vary between versions, devices, etc. As long as we understand that and look for an estimation only, you can actually measure it very easily. Android SDK includes a powerful suite of memory analysis tools. Check out Eclipse MAT (the best one in my eyes). You can take a heap snapshot when your hashmap is fully loaded, then use MAT to see how many bytes it takes. Make sure you sum up both the hash itself, the keys and the values (if I remember correctly MAT can do the math for you too (it can handle the core collections very well).
Will dalvik be able to allocate?
For the sake of discussion let's say your hashmap takes 1MB of memory. To get a feeling if that's much, we need to understand the constraints of the system we live in. Dalvik limits the max size of your heap. The limitation varies per device. The minimum on very old devices is 16MB. Devices like Samsung Galaxy 2 have about 32MB-48MB and new devices like Galaxy 3 and 4 have more than 100MB.
The biggest memory hog in apps is usually bitmaps. Since every pixel can take as much as 4 bytes, a full screen bitmap can easily eat up a few MB of memory.
With this in mind, a toll of 1MB doesn't sound bad. It's comparable to a using a nice background image :) if your overall memory usage is low, you can distribute it as you see fit. The memory analysis tools (MAT or DDMS) let you know exactly how much memory your app is currently using, so you can easily estimate how much your total consumption will be.
Other thoughts:
Caching things in memory to improve performance is usually a good idea. So your approach is a good one in my eyes (as long as you understand the memory implications).
Since your memory hashmap is an optimization only, you can be extra careful and only do it when you have memory to spare. You can easily measure the amount of available heap (the is API for that) and make your decision accordingly. You can listen to low memory notification events (google about those). And you can even catch OutOfMemoryError exceptions of failed allocations and change your memory strategy in runtime.
You are playing in a field where exact measurements are difficult. Be sure to QA on several devices and several versions of Android. To simulate low memory settings, try to use the oldest devices you can find.
I personally think that you will have absolutely no issues handling what you want in memory. Especially if those things are just primitives (no bitmaps).
I have used queries for up to 10k rows for caching in memory and had absolutely no issues in terms of memory for them.
The issue might happen when you need to process everything. Like how fast is it to get to a specific item, get all realated items etc..
One issue i have come accross was UI related. I tried to just fill an adapter and show it in a list with all 10k of rows which took about 7 seconds to complete. It was long time ago and i don't recall why exactly that happened, but what i am saying is that i would pay more attention to keep processing outside of the UI thread and manage that as much as possible rather than memory in your case.
Related
In my android app I'm allocating a large int array. This sometimes gives me an OutOfMemory error, when I think there should be sufficient memory. This is an example of what I get:
// always this value
ActivityManager.some_instance.getMemoryClass() = 128 Mb
// always this value
Runtime.getRuntime().maxMemory()/(1024*1024) = 128 Mb
// example value
Runtime.getRuntime().freeMemory()/(1024*1024) = 81 Mb
// example value, can also be bigger than freeMemory()
arrayLengthToAllocate*4/(1024*1024) = 47 Mb
To be clear, I get the OutOfMemory error for situations where the last value is larger or smaller than freeMemory().
Why do I get the error? Is the heap size not increased when the allocation is performed? The memory use of the app just before trying to allocate is about 8 Mb, so that cannot be the problem.
PS. Other approaches than using an int[] of what this app is doing are not possible.
Generally speaking most devices have a cap on the amount of memory you can use. I know that originally this was set at 16MB per application. I think this has increased since, but it is still something you should be taking into account.
You also have to bare in mind that your applications footprint can fluctuate and that the Android Operating System has to give memory to other applications as well which are competing for resources. It may be that Android is identifying your application as being a resource hog and is cutting back it's resources.
Storing 47MB of integers in memory cannot be a very good use of an application's memory, is there no easier approach? I fail to see why you would need so many integers to be so readily available. Surely adding the integers to a SQLite database, reading them in as required, thus moving the bulk of the memory requirements away from RAM and onto physical memory would be a better approach?
When you are hitting OutOfMemory errors it is normally an indication that you are being a bit too greedy and the best way of limiting them is to scale back what you are doing, find a different approach.
The memory amount you get is the system memory size. Each application has a limited subset of that, which depends on a few system variables. In general it's something in the order of 16 - 32 MB, sometimes a bit bigger for tablets.
I figured out that the problem is heap fragmentation. Defragmentation has limits such that there are simply not enough free consecutive bytes in the heap to fit the single large array. I restructered the data into a two-dimensional array which solves the problem as each row has its own location in the heap. Downside is that retrieving the data from the array is slightly slower.
Folks, I don't think that this is a duplicate and is NOT one of those how do I avoid OOMs questions. This is a genuine quest for knowledge so hold off on those down votes please...
Imagine I have a JPEG of 500x500 pixels. I load it as ARGB_8888 which is as "bad as it gets".
I would expect Android to allocate 500x500x4 bytes = a little under 1MB however, look at a heap dump and you will see that Android allocates significantly more, often factors of 5-10 times greater.
You frequently see questions on here about OOMS where the stack trace shows a heap request of say 15MB and it is Always much larger than is required simply to hold the bytes of the image. The OP usually catches some downvotes then is bombarded with stock answers and comments about using less memory (thanks Romain!) and in scaling. I think there is more than meets the eye here.
Anybody know why this is?
If there is no apparent answer, I will put together an SSCCE if it helps.
PS. I assume that JPEG vs PNG etc is irrelevant since we're talking about the memory usage of the backing bitmap which is simply x times y times BPP - or am I being slow?
It used to be quite a common trick with memory management to grab a pool or block of memory which is parceled out into smaller requests. When I worked with embedded systems it was a common practice to maintain pools of memory of different sizes, and we just allocated a block larger than the amount requested from a pool . It is a convenient way of preventing too much memory fragmentation. Maybe this is happening here.
I need to search through text files (around 40MB) in my app with regular expressions, as you can imagine, it normally takes 1 minute or so to get it done. AND I have to do it repeatedly.
I wonder if I can keep these files in RAM after the first search. Can I possibly do that? I mean, find a way to explicitly say keep something in RAM for some time.
Consider putting your search results in a WeakHashMap, with keys that only exist for the duration that you need the values to exist, like the scope of an Activity. Watch out for memory issues though. On some devices, your application's process may only have a heap size as low as 16M.
Keep the results in a custom object that will save the search result. This will keep it in RAM (as long as you keep a reference to it).
Also keep in mind that allocating 40 MiB in RAM in Android devices is not a very good idea since RAM is quite limited in a lot of low-end devices. This can make your application a very tasty target for Android when it looks to free memory.
I've spent the last few days trying to remove memory leaks in my game, resulting in many out of memory errors. I'm on the verge of adding a significant amount of graphics, that while not hugely complicated, will add significantly to the processing requirements of my system, and I'm a bit worried about my memory usage, and I was hoping someone might have some tips for me. I don't want to go below Android 2.1, so please tailor any answers to that end.
First of all, my game consists of:
2 activities, 13 XML files (Some relating to a small part of a layout, some dialogs, and 2 directly related to activities.
A number of drawables, made in Adobe Illustrator and converted to PNG. These are probably large, but not unusually large, and for the most part, only small amounts of them are in memory at any given time.
Quite a few dialogs.
Targeted towards Android 1.6 and above.
I used the newest Admob, and as a result, I have to build against 3.2.
My default heap size for my emulators is around 24 MB.
A few sample images from my game:
What I have learned:
Despite my total app size being only around 500K, I somehow am taking up 24 Megs, as calculated by adb shell procrank.
I have done considerable optimization, but am not seeing large increases in memory.
Using tools to find what is in the Heap typically only show around 7 MB avaliable, with around 3 MB being used. Sometimes, when opening new dialogs and the like, I see an increase, but I can't say that I see it being all that large...
MAT shows that none of my classes are using an unusually large amount of memory.
So, given all of this, my questions.
Is 24 Mb an actual requirement to develop to (1.6+ android)?
Assuming it is, how can I both allow for nicer graphics for systems which can handle it, but not crash and burn for older systems?
What are some common gotchas that I can use to improve my memory usage?
Without seeing your actual code, I can't say if the following will be relevant to you or not. However, it is worth a shot.
If you are not already doing so, you can consider using something called an LruCache. http://developer.android.com/reference/android/util/LruCache.html
Using this tool, you can determine at what point your cached objects (such as Bitmaps) will become eligible for garbage collection. So, if you want to set it at 4mb (for example) the OS will deal with it should it try to grow beyond it. (See docs for implementation details and a good example).
The only downside is that that little gem only came along with 3.2, so you would have to make that your min SDK in the AndroidManifest, or do a check programatically at run time for the api level to determine if you can use it. Pre 3.2 I would say you need to call recycle() on any Bitmaps you are using, but if you have optimized already I would think the chances are good you are already doing this.
This is a nice little snippet about the difference between the heap and native memory. http://code-gotcha.blogspot.com/2011/09/android-bitmap-heap.html It may (depending on what you are doing) help you to understand why you are not seeing the drop in memory you are expecting.
And finally this post from SO should help when dealing with heap size as well:
Detect application heap size in Android
Hope that helps.
will it affect the performance if i store about 1.5mb worth 100+ string_array data inside strings.xml?
Any other best method to store them?
I don't think there would be any noticeable performance problems with 1.5MB string.xml, except that your app would take up a little more RAM for the time it runs. It's fine, as long as it doesn't grow beyond, say 2-3 MBs, which might leave less RAM for other portions of your app.
Android will throttle the RAM anyway, so, if during your testing, you don't see any OutOfMemory errors, you are good to go.
But then, there might be other approaches, depending on what exactly your requirement is.