Develop Reference

Android Image upload/download with Base64 into JSON causes Out of memory error

I currently encode and decode images to Base64. I overcame the initial issue with OOM's with the use of streams to encode the images into strings.
My issue now is that I cannot fathom how to add multiple Base64 encoded strings for multiple resolutions images (5620 x 3747 - 4.92MB or 3264 x 1836 - 1.35MB) to a JSON Object via Gson. Currently Gson throws an OOM exception only with 2 Base64 Strings from a 5312 x 2988 - 4.95 MB Image.
I understand that android may only be able to spare 16/20Mb per application, so this conversion must be way over the limit.
How can I write the Base64 String in a stream to a JSON object that will contain the specific values needed to post into my server?
Would it be easier to change my server to accept a Multi-Part request instead of a JSON based POJO with multiple Base64 Strings? I currently use Volley and there isn't an official Multi-Part Request as well as IO streaming.
If it's a matter of compression, how much compression should I apply to the image before encoding into a Base64 String? I ideally want to lose barely any quality but have optimal compression levels.
Bit more Information
I am uploading multiple different resolution images as it is a test for compatibility. For example, all the images that I am sending up have been taken on low resolution and extremely high resolution devices as my App relies on these images for functionality. I am trying to prove that any image (to a certain extent, mainly images captured on mobile devices) can be handled by my application.
I understand that some images may be so large that by loading them into memory will cause exceptions. This is something I will try and handle later.
In some cases the images that will be uploaded can span from 1 to 200.
I'm trying to look for the most optimal solution that will scale well.

... for multiple resolutions images (5620 x 3747 - 4.92MB or 3264 x 1836 - 1.35MB)...
Not sure if this is the file size or the memory needed to allocate the image in memory, but taking a look at the following link: http://www.scantips.com/basics1d.html, I see this:
For a 4000 x 2500 pixel image,
then: 4000 x 2500 pixels = 4000x2500 = 10 megapixels
4000x2500 x 3 = 30 million bytes (if 24-bit RGB)
30,000,000 bytes / (1024 x 1024) = 28.61 megabytes (MB)
This is simply how large the data is - For ANY 24-bit 10 megapixel
image, but JPG files compress it smaller (only while in the file).
I think that the images you're handling are taking much more memory than you expect.
Also, taking a look at this question and answer: https://stackoverflow.com/a/11402374/3393666, we know that a base64 representation of an image take up to 37% more memory than the original image size.
How can I write the Base64 String in a stream to a JSON object that will contain the specific values needed to post into my server?
I think you could do this (with small images not large ones) by simply adding the base64 representation of the image in a JSON object then posting it to the server.
Would it be easier to change my server to accept a Multi-Part request instead of a JSON based POJO with multiple Base64 Strings?
In my opinion that would be your best option to implement what you're trying to achieve.
I currently use Volley and there isn't an official Multi-Part Request as well as IO streaming.
You can take a look at this answer: https://stackoverflow.com/a/16803473/3393666, you can definitely can do it with volley, but if you want an alternative you can try with retrofit (http://square.github.io/retrofit/) they support Multipart our of the box.
eg:
#Multipart
#PUT("user/photo")
Call<User> updateUser(#Part("photo") RequestBody photo, #Part("description") RequestBody description);

I looked into using Volley as a mechanism to transport large JSON objects to a server and found this answer. This answer essentially proved that Volley would be a bad idea for what I wanted.
I switched to OkHttp and now use their streaming methods allowing the JSON to be streamed to the server and then read the response with a streamlined approach. I used the GSON library to parse the response as OKHttp allows the response JSON/Object to be streamed into a reader object which Gson then uses for internal streaming and parsing to an POJO class.
The ony reason why I did not switch to a Multi-Part request was due to the server side implementation being rigid and unchangeable to cover for Multi-Part requests, it strictly expected a JSON representation of data and files.
For handling Base64 Images on Android I severely reccomend not using the String representation and merely converting to Bytes to save on using an excessive amount of memory. I read this article on String memory usage and management. With the Bytes you may easily transport the data without leaving a massive footprint on memory.
For Displaying the images I still avoid the bytes to String conversion by using the Image library Glide. They allow you to pass in a byte[] which was of massive convenience.

Sending image file using WCF

I am trying to send an image from android client to .NET based server, I am doing it using HTTP POST, will there be any significant difference if I use a stream for file upload or send the data as a base64 string, ie convert the string into base64, send and at the server side convert the base64 string back to image, what difference does it make?

There are a few (small) advantages to streaming instead of converting to base64. One is that the conversion to base64 will increase the byte load by about 30% going over the wire. Then there's the extra processing (and programming) involved in the conversion at both ends.
Having said that, I'd recommend base64 over streaming because, in the end, the programming will be much easier I think. Boxing up and sending an HTTP message, even if it contains a boat-load of base64 characters, should be child's play compared to getting that whole stream-send/stream-response business working on Android, right? How hard is it to concatenate into JSON a big string of characters? That's why I'd go with base64.

Json parsing strategy in Android

I have a large json file (~3.5 MB), with (~140) complex objects, and the max depth in the object graph is about 4-5. I use Gson to parse it, but it's really slow. I've tried some way to parse it (like mixed parsing or using stream to parse), but I couldn't increase performance.
I checked Memory Analizer, it kill memory (70-80%), if I only parse the base Id of the objects. While parsing there are 400-500k object in memory (mostly string and char).
Would be parsing more efficient if the object graph wouldn't be so deep? Do you have a good idea how could be better? I tried other libs too (like Jackson), but performance wasn't better.

I had some good performance boost by using Jackson - you should not load whole json into memory but use createJsonParser(Reader r). If it does not work for you then either try spliting data and use separate threads - it makes sense if CPU is not fully loaded by single threaded version. Otherwise try using JNI json parsing implementation.

Android uploading pictures to server in most efficient way

I need to get images along with other data (very similar to email with attachements) to the server. I also need to do it in reliable manner so I can retry, etc on failure.
Server is WCF REST server and I do lot of other communications with it(JSON) but just got this new requirement to upload images.
Since I use JSON to post data to my server - I use GSON on Android side to serialize data.
Here is how I got it implemented so far (everything else works this way but I just started with images)
User filling activity fields (text data)
User takes some picture(s) via camera intents. Currently I just use 1 file for pictures
I take picture from SDCard, load/resize it - dispaly on ImageView and store in byte[]
User submits - I take all data along with images from byte[] and put it into Java object
Call GSON converter and serialize object
Save object into SQLite
AsyncTask looks in SQLite for records, opens cursor and get's text
AsyncTask creates HttpConnection and posts text data to my server.
THE END
Now to my problems..
Obviously on #3 - I "explode" ram with my byte arrays. Sometime I even feel my Nexus S becomes sluggish. But by doing that - I avoid filling SD card or app folder with many files. I take picture and than grab it. Next picture will overwrite previous one.
Step #5 IS slow. I didn't try custom serializer on GSON and instead of serializing byte array into something like [1,-100,123,-12] I can get much smaller size with Base64 but still. It will be slow. And I can have up to 20 images...
Step #6 is no problem. But with certain size (I tried 300px image) I started to get error in step 7 on OpenCursor
07-06 20:28:47.113: ERROR/CursorWindow(16292): need to grow: mSize = 1048576, size = 925630, freeSpace() = 402958, numRows = 2
07-06 20:28:47.113: ERROR/CursorWindow(16292): not growing since there are already 2 row(s), max size 1048576
07-06 20:28:47.113: ERROR/Cursor(16292): Failed allocating 925630 bytes for text/blob at 1,1
So, this whole thing is not something I like. Ideally I want all data to be uploaded in single piece to server.
I was thinking maybe storing images timestamped on SD card and store only their name in DB. Than I would process them right before sending to server. And on success I would delete those images. This kind of logic will make SQLite schema much more complex but maybe there is no better way?!
I guess I'm looking for best practice to deal with images. How to do followin with minimal memory/CPU usage:
Take picture
Display thumbnail
Resize
Send to server
EDIT 1:
Currently I'm researching possibility of uploading whole shizang as a multi-part MIME message. That would require adding some JAR's to my Android package. Also I'm not sure how effective will be Apache code to load images and sending them(I guess better than my code)
http://okandroidletsgo.wordpress.com/2011/05/30/android-to-wcf-streaming-multi-part-binary-images/
And that I would have to deal with parsing all this on WCF side since there is no way to do it with built-on .NET framework.
http://antscode.blogspot.com/2009/11/parsing-multipart-form-data-in-wcf.html
PLEASE TELL ME IF YOU TRIED THIS!
EDIT 2:
MIME is no good. There is no point since it serializes binary using Base64 which is same thing..

Nobody answered but here is what I figured hard way:
Rule #1: When dealing with images - avoid using objects/memory. Sounds obvious but it's not. I figured that resizing image to 800x600 is OK. Anything bigger - you may consider just leaving it as is because it is possible to do http stream on bigger file but it's hard to work with OOM exceptions when you load images into memory for processing
Rule #2: When use GSON - use JsonWriter to populate stream. Otherwise memory will explode. Than pass that stream into HttpClient. JsonWriter will write in chunks and data will be sent as it process.
Rule #3: See rule #2. It will work OK for multiple small images. This way GSON will serialize them 1 by one and feed into stream. Each image WILL be loaded int memory anyway.
Rule #4: This is probably the best solution but requires more coordination with server. Images sent 1 by 1 before message sent to server. They sent as stream without any encoding. This way they don't have to be base64 encoded and they don't have to be loaded in memory on device. Size of transmission will be smaller as well. When all images sent - post main informational object and collect all package together on server.
Rule #5: Forget about storing BLOB in SQLite
Bottom line:
It is much cheaper in term of resources to send images WITHOUT any resizing. Resizing makes sense only when Image get's to about 800x600-ish
Sending multiple images in a single package makes sense when image get's small like 600x400-ish
As soon as you need to upload files - start thinking streams everywhere. DO NOT load stuff into memory.

Json read error?

Hi friends I am getting the value from webservice while json value .I am storing in stringBuffer . I am getting the lot of value from webservice .so I am getting out of memory error .can any body tell how to avoid that?
Thank you,

You can allocate the StringBuffer with more amount of size, or try to implement paging or lazy loading..
But, with a lot of value, I think it's not good to use JSon

Json is stored in memory and simply lot of data from server cause out of memory exception. Better idea is to download data in chunks and store in database.
To do it use Jackson library that support streaming data or JsonReader but it is only in Android 3.0.