Develop Reference

QNetworkAccessManager - first GET very slow

I have a problem with using the QNetworkAccessManager in Qt 5.5 on android. Downloading a simple, small graphic file via http GET results in a lot of garbage collection calls and a lockup of the UI during that time. Subsequent GETs work flawlessly and without these GC calls. The code is as follows:
void DownloadManager::downloadFile(QUrl fromUrl, QString toFilePath) {
_currentFilePath = toFilePath;
QNetworkRequest request;
request.setUrl(fromUrl);
qDebug() << "before";
_currentReply = _mgr.get(request);
qDebug() << "after";
connect(_currentReply, SIGNAL(error(QNetworkReply::NetworkError)), this, SLOT(downloadError(QNetworkReply::NetworkError)));
connect(_currentReply, SIGNAL(downloadProgress(qint64, qint64)), this, SLOT(downloadProgress(qint64,qint64)));
connect(_currentReply, SIGNAL(finished()), this, SLOT(downloadFinished()));
}
DownloadManager is a custom QObject-derived class without any special features that are relevant to the get request. _mgr is a QNetworkAccessManager Object that's allocated during DownloadManagers cTor.
As you can see, this is just a textbook example of a get request, nothing too fancy about it. And as I said: it works, for the most part. Only the first get request ends up like this:
D/ .../downloadmanager.cpp:61 (void DownloadManager::downloadFile(QUrl, QString)): before
D/dalvikvm(13298): GC_CONCURRENT freed 2290K, 25% free 10911K/14407K, paused 2ms+3ms, total 29ms
D/dalvikvm(13298): GC_CONCURRENT freed 501K, 25% free 10884K/14407K, paused 13ms+2ms, total 35ms
D/dalvikvm(13298): GC_CONCURRENT freed 524K, 25% free 10892K/14407K, paused 12ms+3ms, total 36ms
D/dalvikvm(13298): WAIT_FOR_CONCURRENT_GC blocked 6ms
D/dalvikvm(13298): GC_CONCURRENT freed 537K, 25% free 10887K/14407K, paused 2ms+9ms, total 32ms
D/dalvikvm(13298): WAIT_FOR_CONCURRENT_GC blocked 14ms
D/dalvikvm(13298): GC_CONCURRENT freed 840K, 25% free 10899K/14407K, paused 12ms+3ms, total 38ms
D/dalvikvm(13298): WAIT_FOR_CONCURRENT_GC blocked 11ms
D/dalvikvm(13298): GC_CONCURRENT freed 1294K, 25% free 10901K/14407K, paused 2ms+2ms, total 27ms
D/dalvikvm(13298): WAIT_FOR_CONCURRENT_GC blocked 11ms
D/dalvikvm(13298): GC_CONCURRENT freed 1187K, 22% free 11330K/14407K, paused 2ms+2ms, total 30ms
D/dalvikvm(13298): WAIT_FOR_CONCURRENT_GC blocked 15ms
D/dalvikvm(13298): GC_CONCURRENT freed 1459K, 19% free 11919K/14535K, paused 13ms+4ms, total 64ms
D/dalvikvm(13298): WAIT_FOR_CONCURRENT_GC blocked 18ms
D/ .../downloadmanager.cpp:65 (void DownloadManager::downloadFile(QUrl, QString)): after
I simply don't understand what causes that much GC to happen - it takes about a full to one and a half seconds for everything to work out (for a download that should take a split-second and, moreover, be asynchronus and not locking up the UI).
Additional Information:
It is always only the first download that triggers this. Subsequent downloads, even for the exact same file, work flawlessly
It doesn't matter if there's a file at the exact location with the exact name or not. Downloading the file, deleting it, getting back into the application and redownloading it provides the same results - the first get is slow and has the GC, the second works perfectly fine.
I call all that from a QML File, that causes a singleton c++ object to call DownloadManager::downloadFile.
Other than the QML UI, nothing else is running within the application. No heavy data exchanges, no background loading on other threads, nothing.
I'd be thankful for any pointers towards solving this.

I didn't try on Android, but I had the same issue on Windows. Because those are the same symptoms, I would say this is likely the same reason, which is that the implementation is lazily loading some shared library on the first get() call. This is particularly true when using an encrypted connection; in my case I can see in Visual Studio that 19 DLLs are loaded on the first get() call.
One way to work around that is to pre-connect to the server using connectToHost or connectToHostEncrypted, depending on whether you are using an encrypted connection (e.g. HTTPS) or not. I am calling that at application startup, but anytime the UI is idle should be fine. Then subsequent get() calls will all have the same performance, including the first one, since the libraries have been loaded and the connection is already established. I am assuming that connecting to any server will load the libraries.
See https://forum.qt.io/topic/65201/qnetworkaccessmanager-first-get-very-slow/14 for details on the generic error (not specific to Android).

Heap not expanding on Genymotion emulator

While resizing large bitmaps for faster image upload to a server I occasionally ran into OutOfMemoryErrors.
To prevent this I calculate the required amount of memory and check if it exceeds Runtime.getRuntime().maxMemory() before trying to scale an image.
However, I still run into OOM errors even though the image should fit on the heap easily.
The emulated device (Galaxy SII API 16) gives me a max memory of 67108864 bytes using the above method.
In the following snippet, the heap size is 43975K and only < 15K of that memory is in use. For my ~31K allocation the heap should grow automatically to about 45K which is still not even close to the maximum size of 64 MiB.
But as you can see, instead of expanding the heap, the dalvik vm runs out of memory.
10-13 20:35:57.223: D/dalvikvm(1201): GC_FOR_ALLOC freed 505K, 67% free 14692K/43975K, paused 31ms, total 31ms
10-13 20:35:57.223: I/dalvikvm-heap(1201): Forcing collection of SoftReferences for 31961100-byte allocation
10-13 20:35:57.251: D/dalvikvm(1201): GC_BEFORE_OOM freed 2K, 67% free 14689K/43975K, paused 29ms, total 29ms
10-13 20:35:57.251: E/dalvikvm-heap(1201): Out of memory on a 31961100-byte allocation.
I wonder if this can happen on a real device too or if this could be a genymotion bug.
Is the heap guaranteed to expand up to maxMemory()? The JavaDoc for Runtime.getRuntime().freeMemory() says it "may" expand, whatever that means.
I just need a realiable way to calculate the amount of memory I can use, this is how I did it, please correct me if I'm wrong:
long maxMemory = Runtime.getRuntime().maxMemory();
long usedMemory = Runtime.getRuntime().totalMemory() - Runtime.getRuntime().freeMemory();
long availableMemory = maxMemory - usedMemory;
This call causes the OutOfMemoryError:
// outOptions has an appropriate inSampleSize
BitmapFactory.decodeStream(inputStream, null, outOptions);

Out of memory on a 31961100-byte allocation
Your bitmap is 32M. VM can't allocate 32M linear space to store bitmap. Heap is fragmented, so even if your heap has 32M free space it is not always possible to allocate such linear space. You can try free up as much memory as you can and call GC before decoding stream.
Try to decode your bitmap in more effective way. Or process image in parts.
If you tell us why you need this image, we can tell you how to handle it.

You have a 42MB heap , out of which 14MB is already used, 67% (28M) is free/available
D/dalvikvm(1201): GC_BEFORE_OOM freed 2K, 67% free 14689K/43975K, paused ...
E/dalvikvm-heap(1201): Out of memory on a 31961100-byte allocation.
You are trying to allocate ~31M (not 31K) , which is greater than 28M that is available, resulting in OOM.
For details on interpreting dalvikvm memory allocation log message take a look at debugging memory
There is lot of shared memory usage going on in android, to properly calculate per process memory usage refer this SO question
Android best practices on efficient bitmap memory management may be of help

One thing you might try to tweak is build.props file of the ROM.
On Genymotion emulator you can try executing the following via root shell:
cat /system/build.prop | grep dalvik
and it would display the line with dalvik settings:
dalvik.vm.heapsize=256m
dalvik.vm.lockprof.threshold=500
dalvik.vm.stack-trace-file=/data/anr/traces.txt
And maxmemory is also being reported as 268435456 bytes on the emulator I experimented with.
So, you may try playing with this setting. Also, ensure that the memory allocated in VirtualBox's settings is compatible with these values.

Bitmap OutOfMemoryError even using recycle()

I have an app that shows a lot of activities, each one showing a bitmap that fits the 100% of the width of the screen and normally more than the 100% of the height.
When you touch the screen, a new activity is created, and in the previous activity is being called finish and recycle for the bitmap:
public void clean() {
if(this.myBitmap != null){
Log.d(DEBUG_TAG, " Cleaning "+this);
this.myBitmap.recycle();
this.myBitmap=null;
System.gc();
}
}
the problem is that the memory required is growing and growing and when i am launching more than 12-13 activities these messages are being showed at logcat:
05-05 16:03:02.909: I/dalvikvm-heap(24794): Clamp target GC heap from 65.790MB to 64.000MB
05-05 16:03:02.969: D/dalvikvm(24794): GC_EXPLICIT freed 71K, 5% free 59008K/61680K, paused 2ms+13ms, total 55ms
05-05 16:03:03.559: I/dalvikvm-heap(24794): Clamp target GC heap from 67.197MB to 64.000MB
05-05 16:03:03.569: D/dalvikvm(24794): GC_EXPLICIT freed 418K, 2% free 60521K/61680K, paused 3ms+28ms, total 109ms
after a few activities more i got this crash:
05-05 16:03:05.049: E/AndroidRuntime(24794): java.lang.OutOfMemoryError
05-05 16:03:05.049: E/AndroidRuntime(24794): at android.graphics.Bitmap.nativeCreate(Native Method)
05-05 16:03:05.049: E/AndroidRuntime(24794): at android.graphics.Bitmap.createBitmap(Bitmap.java:809)
05-05 16:03:05.049: E/AndroidRuntime(24794): at android.graphics.Bitmap.createBitmap(Bitmap.java:786)
05-05 16:03:05.049: E/AndroidRuntime(24794): at android.graphics.Bitmap.createBitmap(Bitmap.java:718)
I dont know what is going wrong here, i am doing recycle() of the bitmap and also i tryed with and without System.gc(), in both cases i got the crash

Its a known bug, its not because of large files. Since Android Caches the Drawables, its going out of memory after using few images. But i found alternate way for it, by skipping the android default cache system.
Soultion:
Create a drawable folder in Assets and move the images to "drawable" folder in assets and use the following function to get BitmapDrawable
public static Drawable getAssetImage(Context context, String filename) throws IOException {
AssetManager assets = context.getResources().getAssets();
InputStream buffer = new BufferedInputStream((assets.open("drawable/" + filename + ".png")));
Bitmap bitmap = BitmapFactory.decodeStream(buffer);
return new BitmapDrawable(context.getResources(), bitmap);
}
Refrence : https://stackoverflow.com/posts/6116316/revisions
Also that add the line below in your manifest file
android:largeHeap="true"

Use DDMS to grab a heap dump and do some analysis on the dump using Eclipse Memory Analyzer (built into Eclipse ADT or can run stand-alone). What your looking for is objects that you allocated but are not being freed. One possibility is that you are leaking activities. You say the OOM error occurs after launching 12-13 activities. Only one of the activities should be visible at a time so you can free memory from the paused activities in the onPause() or onStop() method. Again, analyzing the heap dump will give you a good idea of what is eating up the heap memory.
FYI - Bitmap#recycle() is only really necessary on Gingerbread and lower devices. On newer version of the OS bitmaps are stored on the heap and are garbage collected like other objects. See, the "Manage Memory on Android 2.3.3 and Lower" section of this document: https://developer.android.com/training/displaying-bitmaps/manage-memory.html

Why so many GC_FOR_ALLOC in a simple app?

I'm getting way too many GC_FOR_ALLOC from the dalvikvm.
I'm getting XML from a REST service: in one activity I parse about 100 lines programatically(me) and in the other activity I use the SimpleXML to parse about 200 lines.
In the first one I get 50 GC_FOR_ALLOC.
In the second one I get like 300!! (I can't even post it all, the body makes 29579 characters and it's allowed only 30k)
I've searched and almost everyone complains about gc_for_"M"alloc and not gc_for_"A"lloc.
Is the SimpleXML the problem because the instances created?
I'll post the logcat dump by dalvikvm, maybe the values have some information.
Thank you very much for your help.
12-11 06:13:49.564: D/dalvikvm(6759): GC_FOR_ALLOC freed 362K, 13% free 4116K/4688K, paused 181ms, total 182ms
12-11 06:13:50.074: D/dalvikvm(6759): GC_FOR_ALLOC freed 303K, 13% free 4134K/4708K, paused 142ms, total 142ms
.... repeated many times .....
12-11 06:14:06.254: D/dalvikvm(6759): GC_FOR_ALLOC freed 73K, 13% free 4159K/4768K, paused 53ms, total 53ms
12-11 06:14:06.314: D/dalvikvm(6759): GC_FOR_ALLOC freed 103K, 13% free 4159K/4768K, paused 56ms, total 57ms
12-11 06:14:06.374: D/dalvikvm(6759): GC_FOR_ALLOC freed 29K, 12% free 4203K/4768K, paused 54ms, total 54ms
12-11 06:14:06.424: D/dalvikvm(6759): GC_FOR_ALLOC freed 73K, 13% fre

You can see the most-recently-allocated objects using the DDMS Allocation Tracker (memory debugging docs, old blog post, ddms docs). This will show you what's being allocated and give you a stack trace for the place where the allocation is being performed.
Another blog post describes MAT and other relevant tools, though heap-dump analysis is less useful for this sort of problem because it generally shows you the objects that haven't been freed, and you're more interested in the objects that are being freed.

In Android Dalvik VM, GC_FOR_ALLOC is inovked in object alloc step when dlmalloc footprint space is NOT enough for new or heap->bytesAllocated + n > hs->softLimit. You can set dalvik.system.setTargetHeapUtilization lower for more free heap space.

you can use MAT MAT tutorial
to find how many object are creating and garbage collected. so that youcan optimize your code

If you get that multiple GC_FOR_ALLOC while your app is lagging, there is a big possibility that the bug is in a loop. Check where the line of code starts to trigger the GC then start tracing the code from there. In my experience, I mistyped my inner loop iterator which causes the program to make an infinite loop. I created a bug like this:
for(int i=0; i<list.size(); i++) {
for(int j=i+1 j<list.size(); i++) {
// I mistyped the iterator of integer j with i
// making an infinite loop which triggered the GC.
//appears many times
}
}

I encounter the same problem today.
I find a not ended loop in my code such as while(i < xx), but I not implement the i++ statement in the while body.
So the messages like you meet appeared.
Check your code firstly please.

My log:
D/dalvikvm: GC_FOR_ALLOC freed 549K, 9% free 7878K/8596K, paused 30ms, total 34ms
...freed 539K, 9% free 7888K/8596K, paused 30ms, total 30ms
...freed 1856K, 21% free 8083K/10108K, paused 51ms, total 51ms
...freed 582K, 9% free 7845K/8596K, paused 38ms, total 38ms
Explain:
When your app get memory more limit per app. Dalvik/Ant call garbage collector.
What limits memory for your App decide Dalvik/Ant. As you see for my app Dalvik decide 8596K(double case) and 8083K(one case).
And limits change in runtime.
And you can not be sure when this happens. But you can reduce the likelihood. Decreasing the amount of memory that your application consumes.
PS: Decide when call GC teakes Dalvik/Ant. And you can not be sure when this happens. But you can reduce the likelihood. Decreasing the amount of memory that your application consumes.
PS: In "Monitor android" see tab "Monitors", graphics "Memory". And use buttons: "pause(enabled)", Initiate GC, "Dump Java Heap" "Start Alocation Tracking(very useful)". And use official guide for this:
https://developer.android.com/studio/profile/am-memory.html?utm_source=android-studio.
As far as I understand App must not stop/pause working or crashes when VM call GC.

Android Bitmap Memory Use

I am running a stack of image filters and seem to be hitting some memory issues.
At the beginning of the image processing I am using this much memory:
GC_FOR_MALLOC freed 3K, 45% free 3237K/5831K, external 47586K/49634K, paused 17ms
At the end I am using this much (after all processing is finished):
GC_EXTERNAL_ALLOC freed 5K, 16% free 16056K/18951K, external 51430K/52196K, paused 23ms
After I am finished with each bitmap I set it to recycle and to null:
someBitmap.recycle();
someBitmap = null;
Is there anything else I should be doing to them? Is there any cleanup I should do to the Canvas being used?
Also my filters are objects instantiated like:
BoxBlurFilter blurFilter = new BoxBlurFilter();
Is there anything I should do to release them? In iOS memory allocated with "new" I am responsible to free.
Sorry for the trivial memory management questions, but I am quite new to Android dev and things are quite different than iOS.
Thank you!
EDIT 2, I removed my full filter code.

So after setting the filter instances to null and all the byte arrays to null (in the code above and in the filter objects) I now have approximately the same size heap as I did before I run the filter:
GC_EXPLICIT freed 5126K, 77% free 3243K/13703K, external 51430K/53478K, paused 18ms
That means it went from 16mb being used to 3.2mb. Much better!
So I guess the answer to my question is make sure you set everything to null if you want it to be freed.