I am thinking of storing large arrays in my app. The biggest they would be is 100000 elements. But could settle for 1000.
I have an option of exporting them to csv files. Having them in the app would be neater.
How much does having arrays of this size affect the phone's efficiency and performance?
Suggestions and thoughts appreciated.
Thanks
If you store just 100'000 numbers (as .csv files imply), this won't have any effect on your performance, since in case of int it will be just 400kb which is quite a small amount of memory for modern android phones.
P.S. I recommend you first implementing your app in any way, and then, when you will see what the problems are, be it memory consumption, battery drain or whatever, optimize it.
Related
I am developing an application in which I have a database with 5000 rows with 4 columns.
problem_id (int)
problem_no (string)
problem_title (string)
dacu (int)
I need to frequently query single items in a large scale like 1000 query to fetch problem_no based on problem_id or sometimes only one item.
So I decided to query all the database rows and map them in a hashMap at runtime. I know hashmap insertion/query operation will take only O(1) or sometimes little more, so I only need 5000 operations I think. But how much space hashMap will take in this case? Would android dalvik be able to allocate them without any trouble?
How much space will hashmap take?
It's an implementation detail that can vary between versions, devices, etc. As long as we understand that and look for an estimation only, you can actually measure it very easily. Android SDK includes a powerful suite of memory analysis tools. Check out Eclipse MAT (the best one in my eyes). You can take a heap snapshot when your hashmap is fully loaded, then use MAT to see how many bytes it takes. Make sure you sum up both the hash itself, the keys and the values (if I remember correctly MAT can do the math for you too (it can handle the core collections very well).
Will dalvik be able to allocate?
For the sake of discussion let's say your hashmap takes 1MB of memory. To get a feeling if that's much, we need to understand the constraints of the system we live in. Dalvik limits the max size of your heap. The limitation varies per device. The minimum on very old devices is 16MB. Devices like Samsung Galaxy 2 have about 32MB-48MB and new devices like Galaxy 3 and 4 have more than 100MB.
The biggest memory hog in apps is usually bitmaps. Since every pixel can take as much as 4 bytes, a full screen bitmap can easily eat up a few MB of memory.
With this in mind, a toll of 1MB doesn't sound bad. It's comparable to a using a nice background image :) if your overall memory usage is low, you can distribute it as you see fit. The memory analysis tools (MAT or DDMS) let you know exactly how much memory your app is currently using, so you can easily estimate how much your total consumption will be.
Other thoughts:
Caching things in memory to improve performance is usually a good idea. So your approach is a good one in my eyes (as long as you understand the memory implications).
Since your memory hashmap is an optimization only, you can be extra careful and only do it when you have memory to spare. You can easily measure the amount of available heap (the is API for that) and make your decision accordingly. You can listen to low memory notification events (google about those). And you can even catch OutOfMemoryError exceptions of failed allocations and change your memory strategy in runtime.
You are playing in a field where exact measurements are difficult. Be sure to QA on several devices and several versions of Android. To simulate low memory settings, try to use the oldest devices you can find.
I personally think that you will have absolutely no issues handling what you want in memory. Especially if those things are just primitives (no bitmaps).
I have used queries for up to 10k rows for caching in memory and had absolutely no issues in terms of memory for them.
The issue might happen when you need to process everything. Like how fast is it to get to a specific item, get all realated items etc..
One issue i have come accross was UI related. I tried to just fill an adapter and show it in a list with all 10k of rows which took about 7 seconds to complete. It was long time ago and i don't recall why exactly that happened, but what i am saying is that i would pay more attention to keep processing outside of the UI thread and manage that as much as possible rather than memory in your case.
Irrespective of functionality, can simply the size the database directly effects the efficiency of the device? I read Opening and Closing database is an expensive task, so is this expense depends on the size of the database?
The size of the database has no effect on the time needed to open it.
A big database becomes expensive only when you are actually accessing more data.
If you think that your app opens/closes the database too often, you should try to keep a connection around. This will trade off the additional effort against more memory use.
Of course. The only problem is that you don't want to do premature optimization.
The size itself has a very, very, very minor effect on power usage, but accessing it would. It would be more of "how often do you read/write to the db". If it's once per app, then it shouldn't be bad on a moderate-sized db (moderate in the sense of a mobile device / maybe a few hundred entries). If it's often, then you should consider other options like if you can do a read cache in RAM
I need to search through text files (around 40MB) in my app with regular expressions, as you can imagine, it normally takes 1 minute or so to get it done. AND I have to do it repeatedly.
I wonder if I can keep these files in RAM after the first search. Can I possibly do that? I mean, find a way to explicitly say keep something in RAM for some time.
Consider putting your search results in a WeakHashMap, with keys that only exist for the duration that you need the values to exist, like the scope of an Activity. Watch out for memory issues though. On some devices, your application's process may only have a heap size as low as 16M.
Keep the results in a custom object that will save the search result. This will keep it in RAM (as long as you keep a reference to it).
Also keep in mind that allocating 40 MiB in RAM in Android devices is not a very good idea since RAM is quite limited in a lot of low-end devices. This can make your application a very tasty target for Android when it looks to free memory.
Haven't been able to find this one out.
How are Bitmaps stored in memory in Android? More specifically what I'm looking for is, does it store the information pixel by pixel, or does it use any sort of algorithm to reduce the number of stored pixels, like storing a single pixel and a number for how many times to repeat it in a row. I'm wondering about this because we're having trouble fitting all the images we want into our game. If it does use some sort of algorithm, then we can do something to the original image to cut down on memory consumption, right?
Bitmaps are stored compressed, but you pretty much can't display one without assembling the pixels at some point. Your best bet is probably to save your limited CPU heap and push your images into the GPU as compressed OpenGL ES textures.
Additional:
Have a look at Displaying Bitmaps Efficiently. Also recycle your bitmaps if it helps.
While an implementation may choose to use RLE to store bitmaps in memory, I very much doubt one would do so since it would be inefficient to operate with them.
On Android 3.0 and above, you can request a larger heap.
will it affect the performance if i store about 1.5mb worth 100+ string_array data inside strings.xml?
Any other best method to store them?
I don't think there would be any noticeable performance problems with 1.5MB string.xml, except that your app would take up a little more RAM for the time it runs. It's fine, as long as it doesn't grow beyond, say 2-3 MBs, which might leave less RAM for other portions of your app.
Android will throttle the RAM anyway, so, if during your testing, you don't see any OutOfMemory errors, you are good to go.
But then, there might be other approaches, depending on what exactly your requirement is.