I am developing an Android app that uploads image (jpg) to AWS S3. The image is being uploaded but I am unable to open it using web browser: the image is either invalid or opened as a string.
//skipping error handling for simplicity
InputStream is = Utils.streamFromUri(this, uri);
byte[] buffer = new byte[FileUtils.getInputSize(is)];
...
//is is restored at this point
is.read(buffer);
String data = new String(buffer);
S3.createObjectForBucket(bucketName, objectName, data)
Can you please share a working code for upload or give some directions on how to resolve this?
Thanks
First file is just 0 filled. Second file is not a jpeg but appears to be a result of some kind of transformation (there is some photo data in it). There should be no issue with upload under normal circumstances. If you provide specifics on how you uploaded them (code or tool) we can go from there.
Update
Your code to get data for upload is probably not going to work. I don't know specifics about your utilities but when you do
FileUtils.getInputSize(is)
it probably reads the entire stream, so you end up at the end of it and read nothing for upload. I highly recommend that you use Apache Commons IO to read data. And you can get size of it after it's read into memory.
Update 2
You are using byte to string conversion. From String(byte[]) documentation
Constructs a new String by decoding the specified array of bytes using
the platform's default charset.
Since you use binary data, it's getting transformed. That actually also can be seen from your second file which start with ef bf bd which is Unicode replacement character. So refrain from using string for storing binary data.
Two things.
In amazone S3. You need to set the privileges on the image to Everyone: read. I am using Firefox Plugin S3FOX, and then you right click, press Edit ACL and give everyone Read acces.
You might have to set a custome header. Content-Type: image/jpeg
This depends a bit on how you upload the file, normaly this is handeled by automagic.
The files you linked is ruined. Try a different tool to upload the images to S3 if you know the images is good on your computer.
http://www.s3fox.net/
Related
I have to send a DNG file which has a size around 30 mb to my server and then I have to process DNG file in matlab and after that I need to get the results back from matlab to android device. I am new to sending images to a server and I do not know if is there any special way for big size images. I saw similar questions but I could not understand what to do for sending images to a server.
Could you please help me which steps should I follow respectively and which methods or libraries do I need to use ? Thanks.
If you'd like to send big files using HTTP, chunks are the way to go.
You would need a backend server supporting this kind of operation (either with some homemade recipe or with a standardized implementation).
You'd basically need an API to create the file description (including the expected size) which would return a handle on this future file (at least an ID). Then use PUT or PATCH and send the chunks one by one.
I have encountered a strange problem I am unable to debug. An image is uploaded to Onedrive via some code very similar to the given example, and once uploaded, the image is visible in the OneDrive web interface.
http://msdn.microsoft.com/en-us/library/dn659727.aspx
Upon trying to download it, again using code from the example, the following line of code
Bitmap bMap = BitmapFactory.decodeStream(input);
also returns a null value for bMap. I know these files (which I obtain from copying from the Android Clipboard and writing to a file on disk) are valid, b/c I use them in Gridview elements and upload/download them to Dropbox in a similar way.
Is there some kind of jpg re-encoding performed in OneDrive (like RGB->CKMY conversion) what would no longer prevent them from working?
Also, is there some other type of query parameter like "/picture?type=thumbnail" or "/picture?type=normal" that needs to be appended to a file.XXXX OneDrive ID that would prevent any possible conversion?
Is there a way to debug exactly why the BitmapFactory.decodeStream() function fails, like debugging output?
EDIT: So, I came across the following SO post, and figured out this is the same problem I am having.
OneDrive - Wrong size for PNG files
For a certain file, if I download it using the Onedrive SDK and look at the stream length
public void onDownloadCompleted(LiveDownloadOperation operation) {
int length = operation.getContentLength();
}
it reports a size of 2723 bytes, but if I download the file and save it using a desktop web browser, the file is 1837 bytes. Is there something I am missing about reading the size of a stream, or is the API just broken?
This is documented behaviour as per http://msdn.microsoft.com/en-us/library/dn659726.aspx (see the first note under uploading).
You can disable conversion by adding downsize_photo_uploads=false to your query string.
The answer to this question apparently is, there is a bug in the Onedrive SDK. I ended up filing an issue on Github with Microsoft, and it turns out they discovered that with certain types of small images (and potentially PDFs), the actually file size being transmitted back after a download is wrong.
https://github.com/liveservices/LiveSDK-for-Android/issues/37#issuecomment-65457177
It's all just a weird artifact of the way I was testing my app. Since Chrome for Android is the only app I've discovered that lets you copy an image to the clipboard, I was just Google search to find images (tiny thumbnail images in the basic web search results), and copying those to the clipboard. Had I been using larger images, I may never have run across this bug.
I am developing an application where it should get data from my server with respect to the ID Number , and also I need to take an image from database , according to the id number. So I went to an option of loading url image. There I could not able to get large size images. So I went to converting the image to base64 format and sending it through webservice. Even in that I was able to retrieve small size strings , but when I go for big size of images, the base64 string is bigger. So looking for possible answers, any way thanks in advance...
Sending photos in binary form over a HTTP connection is a solved problem. I suggest you go with an off-the-shelf web server and call it a day. To send reduced-size images, install a script that (in response to a URL that indicates desire for a thumbnail) will check for thumbnails; create them if they don't exist; and then serve as images in the standard manner. Don't worry about base64 unless you're dealing with really ancient services that don't understand pure binaries. If you do go the route of base64, don't worry about file-size expansion. Unless you're sending huge numbers of images, it's not that significant.
I had this problem and had to use ftp server. I put my image files into a ftp server and download each one I need.
I am trying to download a file from server, what I have in mind to get filename in json response and the content and write same in assets or on sdcard. should i go and implement same, or there can be some other way we can achieve the same.
Actually you know you control the way you expose the content from the server - you can just make the file itself (its bytes) exposed at certain url. If you need to keep the name of the file posting as json seems to be good solution.
However, there is one trick here: I don't know how big the file you refer to is and also how reliable the network will be. For every bigger file I will recommend to implement resumable upload via byte serving. You can read about the byte serving client consumer in Android in this thread. Then just go and see on how to implement byte serving server side - it definitely depends on the platform in the only case in which I used such mechanism I actually had to mimic the byte-serving with my custom url parameter. If you need further help on server side write back and I will extend my answer.
I need to get images along with other data (very similar to email with attachements) to the server. I also need to do it in reliable manner so I can retry, etc on failure.
Server is WCF REST server and I do lot of other communications with it(JSON) but just got this new requirement to upload images.
Since I use JSON to post data to my server - I use GSON on Android side to serialize data.
Here is how I got it implemented so far (everything else works this way but I just started with images)
User filling activity fields (text data)
User takes some picture(s) via camera intents. Currently I just use 1 file for pictures
I take picture from SDCard, load/resize it - dispaly on ImageView and store in byte[]
User submits - I take all data along with images from byte[] and put it into Java object
Call GSON converter and serialize object
Save object into SQLite
AsyncTask looks in SQLite for records, opens cursor and get's text
AsyncTask creates HttpConnection and posts text data to my server.
THE END
Now to my problems..
Obviously on #3 - I "explode" ram with my byte arrays. Sometime I even feel my Nexus S becomes sluggish. But by doing that - I avoid filling SD card or app folder with many files. I take picture and than grab it. Next picture will overwrite previous one.
Step #5 IS slow. I didn't try custom serializer on GSON and instead of serializing byte array into something like [1,-100,123,-12] I can get much smaller size with Base64 but still. It will be slow. And I can have up to 20 images...
Step #6 is no problem. But with certain size (I tried 300px image) I started to get error in step 7 on OpenCursor
07-06 20:28:47.113: ERROR/CursorWindow(16292): need to grow: mSize = 1048576, size = 925630, freeSpace() = 402958, numRows = 2
07-06 20:28:47.113: ERROR/CursorWindow(16292): not growing since there are already 2 row(s), max size 1048576
07-06 20:28:47.113: ERROR/Cursor(16292): Failed allocating 925630 bytes for text/blob at 1,1
So, this whole thing is not something I like. Ideally I want all data to be uploaded in single piece to server.
I was thinking maybe storing images timestamped on SD card and store only their name in DB. Than I would process them right before sending to server. And on success I would delete those images. This kind of logic will make SQLite schema much more complex but maybe there is no better way?!
I guess I'm looking for best practice to deal with images. How to do followin with minimal memory/CPU usage:
Take picture
Display thumbnail
Resize
Send to server
EDIT 1:
Currently I'm researching possibility of uploading whole shizang as a multi-part MIME message. That would require adding some JAR's to my Android package. Also I'm not sure how effective will be Apache code to load images and sending them(I guess better than my code)
http://okandroidletsgo.wordpress.com/2011/05/30/android-to-wcf-streaming-multi-part-binary-images/
And that I would have to deal with parsing all this on WCF side since there is no way to do it with built-on .NET framework.
http://antscode.blogspot.com/2009/11/parsing-multipart-form-data-in-wcf.html
PLEASE TELL ME IF YOU TRIED THIS!
EDIT 2:
MIME is no good. There is no point since it serializes binary using Base64 which is same thing..
Nobody answered but here is what I figured hard way:
Rule #1: When dealing with images - avoid using objects/memory. Sounds obvious but it's not. I figured that resizing image to 800x600 is OK. Anything bigger - you may consider just leaving it as is because it is possible to do http stream on bigger file but it's hard to work with OOM exceptions when you load images into memory for processing
Rule #2: When use GSON - use JsonWriter to populate stream. Otherwise memory will explode. Than pass that stream into HttpClient. JsonWriter will write in chunks and data will be sent as it process.
Rule #3: See rule #2. It will work OK for multiple small images. This way GSON will serialize them 1 by one and feed into stream. Each image WILL be loaded int memory anyway.
Rule #4: This is probably the best solution but requires more coordination with server. Images sent 1 by 1 before message sent to server. They sent as stream without any encoding. This way they don't have to be base64 encoded and they don't have to be loaded in memory on device. Size of transmission will be smaller as well. When all images sent - post main informational object and collect all package together on server.
Rule #5: Forget about storing BLOB in SQLite
Bottom line:
It is much cheaper in term of resources to send images WITHOUT any resizing. Resizing makes sense only when Image get's to about 800x600-ish
Sending multiple images in a single package makes sense when image get's small like 600x400-ish
As soon as you need to upload files - start thinking streams everywhere. DO NOT load stuff into memory.