I receive a file using the following code:
byte[] fileBytes;
....
JSONObject postJSON = new JSONObject();
postJSON.put("file_name", filename);
postJSON.put("client_id", clientID);
HttpPost post = new HttpPost(fileURL);
StringEntity se = new StringEntity( postJSON.toString(), "UTF-8");
se.setContentType(new BasicHeader(HTTP.CONTENT_TYPE, "application/json"));
post.setEntity(se);
response = httpClient.execute(post);
fileBytes = EntityUtils.toByteArray(response.getEntity());
Using the debugger, I see that the response gets an entity 27136 bytes in length, which is the correct length of the test file, but the fileBytes array is only 11470 bytes long. Can anyone tell my why this truncation is taking place? When I try to get other files, a similar truncation takes place, so it is not a function of the specific file or a specific file length.
Using the following code, I get 11997 bytes for the same file:
StringBuilder stringBuilder = new StringBuilder("");
stringBuilder.append(EntityUtils.toString(response.getEntity()));
fileBytes = stringBuilder.toString().getBytes();
Reading from an InputStream, I get 12288 bytes:
fileBytes = new byte[1024];
InputStream inputStream = response.getEntity().getContent();
int bytesRead = 0;
while(true){
bytesRead = inputStream.read(fileBytes);
if (bytesRead <= 0)
break;
....
}
Changing the encoding to UTF-16 gets me an internal server error.
I also tried the following:
InputStream inputStream = response.getEntity().getContent();
response.getEntity().getContentLength()];
while ((getByte = inputStream.read()) != -1) {
bos.write(getByte);
}
bos.close();
This also gave me a file of 11470.
In all cases, the files are corrupted, and cannot be opened. When compared in a binary file viewer, the firs 11 bytes match, and then the files diverge. I could not find any pattern in the corrupted file.
OK, the answer is apparently that all of the above are fine. The problem was with the server, which was not configuring the data stream correctly: Content-type was text/plain for all files, rather than application/pdf, and so on as appropriate.
My first clue was when we put a text file on the server, and it came over successfully. At that point I started working with the server side, and we figured it out pretty quickly.
Bottom line, if you are working on a server/client application, the problem might not be on your side.
I should have mentioned various posts which helped my construct the various versions that I collected above:
including this
and this
My apologies to various other helpful people whose posts I also looked at and up-voted.
Related
How can I send large image/photo to the server using HTTP POST and JSON? I tried several methods but all methods wasn´t good (OutOfMemory Exceptions etc.).
"Classic" code:
Bitmap image;
ByteArrayOutputStream stream = new ByteArrayOutputStream();
image.compress(Bitmap.CompressFormat.PNG, 100, stream);
image.recycle();
image = null;
byte[] byteArray = stream.toByteArray();
try {
stream.close();
} catch (IOException e1) {
}
stream = null;
String encoded = Base64.encodeToString(byteArray,
Base64.DEFAULT);
HttpClient httpClient = new DefaultHttpClient();
HttpPost httppost = new HttpPost(Globals.URL + "/storageUploadFile");
httppost.setHeader("Token", Globals.Token);
String msg = "";
try {
JSONObject jo = new JSONObject();
jo.put("fileName", fileName);
jo.put("content", encoded);
httppost.setEntity(new StringEntity(jo.toString());
HttpResponse response = httpClient.execute(httppost);
HttpEntity httpentity = response.getEntity();
msg = EntityUtils.toString(httpentity);
//...
In this code I get exception here: httppost.setEntity(new StringEntity(jo.toString());
Image is saved on storage card. What do you recommend to upload the image? Send image chunk by chunk? I rather send it as one "item". I hope 2 MB is not so large. My API has parameter "content" and it´s the image in base64 encoding. Is it good way to transfer image as base64?
If you really need json and if you really need base64, you need to stream it instead of keeping all transformations in memory. If your image is 2Mb, in your method, you use:
2MB for the bytes
4.6MB for the base64 String (java strings are internally represented chars, which are 16bits)
4.6MB for the JSONObject.toString result in the String entity
That's a grand total of more than 11MB for just a simple 2MB image.
First step is to use a Json streaming API (I use Jackson)
Like so:
// The original size prevents automatic resizing which would take twice the memory
ByteArrayOutputStream baos = new ByteArrayOutputStream(byteArray.length * 1.2);
JsonGenerator jo = new JsonFactory().createGenerator(baos);
jo.writeStartObject();
jo.writeStringField("fileName", fileName);
// Jackson takes care of the base64 encoding for us
jo.writeBinaryField("content", byteArray);
jo.writeEndObject();
httppost.setEntity(new ByteArrayEntity(baos.toByteArray());
In this case, we only hold in memory byteArray and baos, with its underlying byte[] for a theoretical total of 2MB + 1.2*2MB = 4.4MB (No string representation is used, only 1 intermediate byte[]). Note that the base64 streaming to the byte[] is done transparently by Jackson.
If you still have memory issues (if you are going to send a 10MB image, for instance), you need to stream the content directly to the connection. For that, you could use HttpUrlConnection and use the connection.getOutputStream() as a parameter to createGenerator.
I have a big JSON input (download the file) API and I don´t know how to parse this data. I need:
Save this data (entire JSON input) to text file or database. What is the best way for this?
Load this data from text file or database and create JSONArray from JSON tag "list" (first tag)
The solution should be fast and support Android 2.3. What you have recomend for this? Any ideas?
My code:
HttpClient httpClient = new DefaultHttpClient();
HttpGet httpGet = new HttpGet(urls[0]);
HttpResponse httpResponse;
httpResponse = httpClient.execute(httpGet);
HttpEntity httpEntity = httpResponse.getEntity();
... and what next ?...
FYI:
EntityUtils throws OutOfMemoryException
EDIT:
I try to save data to file like this:
InputStream inputStream = httpEntity.getContent();
FileOutputStream output = new FileOutputStream(Globals.fileNews);
int bufferSize = 1024;
byte[] buffer = new byte[bufferSize];
int len = 0;
while ((len = inputStream.read(buffer)) != -1) {
output.write(buffer, 0, len);
}
And it´s OK. I load data:
FileInputStream fis = null;
StringBuffer fileContent = new StringBuffer("");
fis = new FileInputStream(Globals.fileNews);
byte[] buffer = new byte[1024];
while (fis.read(buffer) != -1) {
fileContent.append(new String(buffer));
}
But how convert StringBuffer to JSONObject? fileContent.ToString() is not ideal, sometimes I get OutOfMemoryException.
First of all: Dispose the HttpClient. Google discourages it:
Unfortunately, Apache HTTP Client does not, which is one of the many
reasons we discourage its use.
Source: developer.android.com
A good replacement is Google Volley. You have to build the JAR yourself but it just works like charm. I use for my setups Google Volley with OkHttp-Stack and GSON requests.
In your case you would write another Request which just writes the response out to the SD-card chunk by chunk. You don't buffer the string before! And some logic to open an input-stream from the file you wrote and give it to your JSON Deserializer. Jackson and GSON are able to handle streams out of the box.
Of course everything works with Android 2.3.
Don't, I repeat, don't try to dump the whole serialized stuff into a string or something. That's almost a OutOfMemoryException guarantee.
I'm using Apache-Commons and downloading file content. Everything works perfectly, but I want to add a progress indicator for which I need the incoming file's content length.
In my debugger I can see that my InputStream has an object labled 'in' of type ContentLengthInputStream, and one of this object's properties is, in fact, the length of the file! However, I don't see a way to get TO that object in my code. InputStream doesn't have all that many methods and none of them get at this inner Object or pull any values from it.
Is there some alternate way to get to this header? I can see that the data is in there, but I'm stumped as to how to access it.
In case it's useful, here's a snippet of the call...
HttpClient client = new HttpClient();
filePost = new PostMethod(URL_PATH);
InputStream ret ;
responseCode = client.executeMethod(filePost);
ret = filePost.getResponseBodyAsStream();
Get content length from headers.
Header[] headers = response.getAllHeaders();
for(Header h:headers){
if(h.getName().equals("Content-Length")){
length = h.getValue();
break;
}
}
What's the “Content-Length” field in HTTP header?
Ah! Found it...
using my code above...
HttpClient client = new HttpClient();
filePost = new PostMethod(URL_PATH);
InputStream ret ;
responseCode = client.executeMethod(filePost);
ret = filePost.getResponseBodyAsStream();
//adding this here gets you the size of the incoming file
long fileSize = filePost.getResponseContentLength();
I want to read an image using jsp and send over http to be accessed by an android application.
Code i tried for JSP is by adding data as header
String strDirectory = "D://abc.jpg";
File fp = new File(strDirectory);
int length = (int)fp.length();
buffer = new byte[length];
FileInputStream f0 = new FileInputStream(fp);
f0.read(buffer);
f0.close();
response.addHeader("image_data",new String(buffer));
I dont know if this is correct.
Whats the right way to send image bytes from a jsp page to android application
Don't think it is the right way honestly.
First of all i suggest you to use a servlet if you can otherwise
you have an implicit object called response and then
OutputStream os = response.getOutputStream();
byte[] buffer = new byte[1024];
while ( f0.read(buffer) != -1)
os.write(buffer);
.....
before this code you have to set correctly response header like:
response.setContentType("your contente type here");
Hope it helps you
I am trying to get compressed data from server. The guy that programmed the server told me, that he uses ZLIB library on his iPhone, and the gzcompress on server. I was trying to find any suitable way to get that data, but it ends up with info "java.io.IOException: unknown format (magic number 9c78)" while creating GZIPInputStream object. Finally I've reached point, where I had data as a String. It was compressed, so I used that answer to decompress: https://stackoverflow.com/a/6963668/419308 . But that code doesn't work. "in.read()" returns -1 at the beginning.
Anyone has any idea why there's -1 ? Or maybe a better way to get the compressed data?
EDIT:
I tried adding file to project and reading from that file. in.read() didn't return -1
EDIT2: According to jJ's answer I've tried this code:
HttpGet request = new HttpGet( urlTeam );
HttpResponse response = new DefaultHttpClient().execute( request );
HttpEntity entity = response.getEntity();
InputStream stream = AndroidHttpClient.getUngzippedContent( entity );
InputStreamReader reader = new InputStreamReader( stream );
BufferedReader buffer = new BufferedReader( reader );
StringBuilder sb = new StringBuilder();
sb.delete( 0, sb.length() );
String input;
while ( ( input = buffer.readLine() ) != null )
{
sb.append( input );
}
But the answer is still compressed (or unreadable)
on android you should use either HttpURLConnection which handles decompression (and http encoding headers) for you from GingerBread on or AndroidHttpClient (for older android versions) that has helper methods like getUngzippedContent and modifyRequestToAcceptGzipResponse.
This is a good summary Android HTTP clients