I'm downloading different medias files from my http server; mp3, jpg/png/ and html.
Everything worked fine when I used the now deprecated HttpClient.
I decided to use the HttpURLConnection.
But I encounter a problem with text files(html).
read() blocks on small html files, maybe waiting for a EOF or I don't know what, during few seconds and exits with the Exception "unexpected end of stream".
My code is:
URL url = new URL(urlString);
postParams = String.format("registration=%s&"....);
urlConnection = (HttpURLConnection) url.openConnection();
urlConnection.setDoOutput(true); // It's a POST request which replies sending a file
urlConnection.setChunkedStreamingMode(0);
if (fileName.contains(".htm")) { // Tried this to see...
urlConnection.setRequestProperty("Accept-Charset", "UTF-8");
urlConnection.setRequestProperty("Content-Type", "application/x-www-form-urlencoded;charset=" + "UTF-8");
}
OutputStream output = urlConnection.getOutputStream();
output.write(postParams.getBytes("UTF-8"));
is = new BufferedInputStream(urlConnection.getInputStream());
/* Get information from the HttpURLConnection automatically fires the request
* http://stackoverflow.com/questions/2793150/using-java-net-urlconnection-to-fire-and-handle-http-requests
*/
int status = urlConnection.getResponseCode();
if (status == 200) {
int len, size = 0;
byte[] buf = new byte[128 * 1024];
BufferedOutputStream os = new BufferedOutputStream(new FileOutputStream(file));
try {
while ((len = is.read(buf, 0, buf.length)) > 0) {
os.write(buf, 0, len);
size += len;
}
os.flush();
} catch (IOException e) {
Log.d(Constants.APP_TAG, "IOException." + e); // HTML files end here after few seconds
} finally {
nbFilesDownloaded++;
is.close();
os.close();
file.setReadable(true, false);
file.setWritable(true, false);
}
}
Any idea to explain why it cannot normally exit from read()??
EDIT: I verified that for the html files which cause this exception, my webserver doesn't include Content-Length in the header. Can it be the cause of the problem?
Try to use okhttp http://square.github.io/okhttp/. I have downloaded the files correctly. I hope to help.
Paul.
Related
String thisurl ="http://songolum.com/file/ppdVkTxqhwcJu-CAzCgtNeICMi8mHZnKQBgnksb5o2Q/Ed%2BSheeran%2B-%2BPerfect.mp3?r=idz&dl=311&ref=ed-sheran-perfect";
url = null;
try {
url = new URL(thisurl);
HttpURLConnection urlConnection = (HttpURLConnection) url.openConnection();
try {
// urlConnection.setRequestProperty("Transfer-Encoding", "chunked");
urlConnection.setRequestProperty("Accept-Encoding", "identity");
urlConnection.setDoOutput(true);
urlConnection.setChunkedStreamingMode(0);
int l=0;
InputStream in = new BufferedInputStream(urlConnection.getInputStream());
while(in.read()!=-1)
{
l=l+in.read();
}
System.out.println("Content-length" +l);
**I checked with other software and I found it's gzip compressed file and its with 10mb and I'm getting almost 1mb **
To answer your question directly, you were going wrong because you were calling read() twice, and also because you were adding together the values of each byte read, instead of counting them. InputStream.read() reads one byte and returns its value, or -1 on EOF. You need to read a number of bytes into a buffer and count how many bytes each read() call returned:
InputStream in = urlConnection.getInputStream();
byte[] buffer = new byte[4096];
int countBytesRead;
while((countBytesRead = in.read(buffer)) != -1) {
l += countBytesRead;
}
System.out.println("Content-length: " + l);
However, I suspect that this is not really what you need to do. The above code will simply return the size of all content in the response, including the HTTP headers and the content. Perhaps what you are looking for is the length of the document (or the file to be downloaded). You can use the Content-length HTTP header for that purpose (see other SO questions for how to get HTTP headers).
Also, note that the content may or may not be gzip-compressed. It depends on what the HTTP request says it accepts.
Please try this one hope so it will be helpful for you.
Using a HEAD request, i got my webserver to reply with the correct content-length field which otherwise was wrong. I don't know if this works in general but in my case it does:
private int tryGetFileSize(URL url) {
HttpURLConnection conn = null;
try {
conn = (HttpURLConnection) url.openConnection();
conn.setRequestMethod("HEAD");
conn.getInputStream();
return conn.getContentLength();
} catch (IOException e) {
return -1;
} finally {
conn.disconnect();
}
}
I'm trying to create an android application which depends on JSON responses. Sometimes it takes a lot of time for the server to respond and ends in a time out exception. Therefore I would like to add a restriction like my webservice call should abort after 20seconds if there is no response. Can you please help me achieving this idea.
Thanks in Advance.
You're not giving much details on the actual implementation that you have.
However, messing with the timeout seems like it may be an emergency fix to an underlying problem that should be fixed.
However, using websockets for transport could be a possible (and probably more elegant) solution. They provide a persistent connection between client and server once created.
Using websockets on Android and IOS
There are several ways to achieve the goal.
We can using HttpURLConnection to do the http request.
public String doPost() {
if (!mIsNetworkAvailable) {
return null;
}
try {
URL url = new URL(mURL);
HttpURLConnection conn = (HttpURLConnection) url.openConnection();
conn.setRequestMethod("POST");
conn.setDoOutput(true);
conn.setDoInput(true);
for (String key : mHeadersMap.keySet()) {
conn.setRequestProperty(key, mHeadersMap.get(key));
}
conn.setRequestProperty("User-Agent", "Android");
conn.setConnectTimeout(30000);
conn.setReadTimeout(30000);
conn.setRequestProperty("Content-Type", "application/json");
conn.getOutputStream().write(mContent);
conn.getOutputStream().flush();
int rspCode = conn.getResponseCode();
if (rspCode >= 400) {
return null;
}
byte[] buffer = new byte[8 * 1024];
BufferedInputStream bis = new BufferedInputStream(conn.getInputStream());
ByteArrayOutputStream baos = new ByteArrayOutputStream();
int len;
while ((len = bis.read(buffer)) > 0) {
baos.write(buffer, 0, len);
}
baos.flush();
final String result = new String(baos.toByteArray());
baos.close();
return result;
} catch (MalformedURLException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
}
return null;
}
setConnectTimeout :Sets the maximum time in milliseconds to wait while connecting.
setReadTimeout:Sets the maximum time to wait for an input stream read to complete before giving up.
Reference: http://developer.android.com/reference/java/net/URLConnection.html
I have an API on rails 4 that accepts HTTP requests in the form of a file upload. Everything works fine on Localhost but on Heroku the POST request doesn't seem to do anything.
This is what my Android POST request looks like:
public static byte[] postData(String operation, byte[] binaryData) {
String urlString = baseUrl + "/" + operation;
String boundary = "uahbkjqtjgecuaoehuaebkjahj";
byte[] postData = null;
URLConnection urlConnection;
DataInputStream responseDataInputStream;
ByteArrayOutputStream byteArrayOutputStream = new ByteArrayOutputStream();
StringBuffer startBuffer = new StringBuffer("--").append(boundary).append("\r\n");
startBuffer.append("Content-Disposition: form-data; name=\"data\"; ").append("filename=\"data.dat\"\r\n");
startBuffer.append("Content-Type: application/octet-stream\r\n\r\n");
StringBuffer endBuffer = new StringBuffer("\r\n--").append(boundary).append("--\r\n");
String startRequestData = startBuffer.toString();
String endRequestData = endBuffer.toString();
try {
URL url = new URL(urlString);
urlConnection = url.openConnection();
urlConnection.setDoInput(true);
urlConnection.setDoOutput(true);
urlConnection.setUseCaches(false);
urlConnection.setConnectTimeout(5000); //5 seconds
urlConnection.setReadTimeout(5000);//5 seconds
urlConnection.setRequestProperty("Content-Type", "multipart/form-data; boundary=" + boundary);
urlConnection.connect();
DataOutputStream _request = new DataOutputStream(urlConnection.getOutputStream());
// Write the start portion of the request
byteArrayOutputStream.write(startRequestData.getBytes());
postData = byteArrayOutputStream.toByteArray();
_request.write(postData);
// Write the Binary Packet
_request.write(binaryData);
// Write the end portion of the request
byteArrayOutputStream.reset();
byteArrayOutputStream.write(endRequestData.getBytes());
postData = byteArrayOutputStream.toByteArray();
_request.write(postData);
_request.flush();
_request.close();
// Read in the response bytes
InputStream is = urlConnection.getInputStream();
responseDataInputStream = new DataInputStream(is);
byteArrayOutputStream.reset();
byte[] buffer = new byte[responseDataInputStream.available()];
while (responseDataInputStream.read(buffer) != -1) {
byteArrayOutputStream.write(buffer);
buffer = new byte[responseDataInputStream.available()];
}
byte[] responseData = byteArrayOutputStream.toByteArray();
return responseData;
} catch (MalformedURLException e) {
e.printStackTrace();
} catch (SocketTimeoutException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
} catch (Exception e) {
e.printStackTrace();
}
return new byte[0];
}
My rails controller
skip_before_filter :verify_authenticity_token
respond_to :raw
before_filter :read_file
def read_file
data = params[:data].tempfile
data_compressed = ''
File.open(data, 'r') do |file|
file.each do |line|
data_compressed.concat(line)
end
end
#json_data = Zlib::Inflate.inflate(data_compressed)
end
def an_action
#processing stuff
response = Zlib::Deflate.deflate(j_response)
send_data #json_response.to_s
end
Heroku logs shows that the controller action is hit but nothing more from the logs
2015-05-05T22:39:29.860161+00:00 heroku[router]: at=info method=POST path="/api/login" host=[my app].herokuapp.com request_id=9d9c91b1-2ce2-4db5-9b54-3dea0322e211 fwd="197.237.24.179" dyno=web.1 connect=4ms service=12ms status=500 bytes=1683
2015-05-05T22:40:40.952219+00:00 heroku[router]: at=info method=POST path="/api/signup" host=[my app].herokuapp.com request_id=4adac44c-66e6-4001-a568-8eb913176091 fwd="197.237.24.179" dyno=web.1 connect=4ms service=8ms status=500 bytes=1683
After shifting my focus from Heroku to the Android code I figured out that there was an endless loop at this section of the code:
while (responseDataInputStream.read(buffer) != -1) {
byteArrayOutputStream.write(buffer);
buffer = new byte[responseDataInputStream.available()];
}
When theres nothing to read the return value is 0 instead of -1. So I updated it to:
while (responseDataInputStream.read(buffer) > 0) {
byteArrayOutputStream.write(buffer);
buffer = new byte[responseDataInputStream.available()];
}
Something strange is that on localhost the first piece of code works and it should work on production too. The documentation states that -1 is returned if the end of stream is reached DataInputStream.read(). Maybe thats a discussion for another day, for now I'm using the second piece of code.
EDIT
This issue has haunted me for weeks and after alot of googling and tweaking of the code I would like to point out that this approach was the WRONG route. The code worked on a WIFI connection but always failed on 3G. So i'll list the code changes that finally worked.
use HttpURLConnection instead of URLConnection. Reason here
Increased the size of Connect and Read timeouts from 5000 to 30000 and 120000 respectively.
Revert the while condition to while (responseDataInputStream.read(buffer) != -1)
#Override
public void run() {
URL imgurl;
int Read;
try {
imgurl = new URL(ServerUrl);
HttpURLConnection conn = (HttpURLConnection) imgurl.openConnection();
int len = conn.getContentLength();
Log.d("check", "ContentLength:" + len);
Log.d("check", "ServerUrl:" + ServerUrl);
Log.d("check", "LocalPath:" + LocalPath);
byte[] tmpByte = new byte[len];
InputStream is = conn.getInputStream();
File file = new File(LocalPath);
FileOutputStream fos = new FileOutputStream(file);
for (;;) {
Read = is.read(tmpByte);
if (Read <= 0) {
break;
}
fos.write(tmpByte, 0, Read);
}
is.close();
fos.flush();
fos.close();
conn.disconnect();
} catch (MalformedURLException e) {
ut.CalltoAlertDialog_ok(getString(R.string.alert), getString(R.string.setting_skin_downloadfail));
} catch (IOException e) {
ut.CalltoAlertDialog_ok(getString(R.string.alert), getString(R.string.setting_skin_downloadfail));
}
mAfterDown.sendEmptyMessage(0);
}
This is file download source.
This code prints error "NegativeArraySizeException" from here
byte[] tmpByte = new byte[len];
So, I checked len's value.
len's value was -1.
But..
When i created yesterday, This code was not print error.
I have 2 apk file.
The apk created yesterday is not a problem. Even now this apk is no problem.
But, The apk created today is problem.
I did not modify anything.
What is the cause of this?
I think your problem is here:
HttpURLConnection conn = (HttpURLConnection) imgurl.openConnection();
int len = conn.getContentLength();
Read documentation about the getContentLength method
Returns the content length in bytes specified by the response header
field content-length or -1 if this field is not set.
Returns the value of the response header field content-length.
So this case that getContentLength returned -1 seems to have happened to you. Then you use this -1 to set your Array size. => Exception thrown
Check the solution of this question about getContentLength returning -1, maybe you will have to do something similar.
But at least you will have to check that len > 0 before setting your array size
I was first using HttpURLConnection with my first test. Now I would like to also support https, but it doesn't work. I've been at it all day and so far nothing. Most of the past problems have been related to certificate issues. Weird thing is in my case it downloads the file, but its either corrupted (if its a simple file), or the zips contents are missing (empty). I will post my code to see if maybe I am doing something wrong.
try{
URL url = new URL(stuffs[0]);//<-actual url I am searching https://...
String fileName = stuffs[1];
String optionalFilePath = stuffs[2] == null ? null : stuffs[2];
HttpURLConnection connection = (HttpURLConnection) url.openConnection();
connection.setConnectTimeout(20000);
connection.connect();
if(connection.getResponseCode() != HttpURLConnection.HTTP_OK) {
checkErrorCode(connection.getResponseCode());
return false;
}
InputStream in = new BufferedInputStream(connection.getInputStream());
FileOutputStream out = null;
if(optionalFilePath == null)
out = new FileOutputStream(PATH +"/"+fileName);
else {
File newDir = new File(PATH+optionalFilePath);
newDir.mkdirs();
out = new FileOutputStream(PATH + (optionalFilePath==null?"":optionalFilePath) +"/"+fileName);
}
byte[] buffer = new byte[1024];
int count;
while((count = in.read(buffer)) != -1){
out.write(buffer, 0, count);
}
out.flush();
out.close();
in.close();
}
Upon further debugging, I found out the content length is -1. So I guess it makes sense why the zip is empty. Now I am not too sure why it returns -1. I download it on a web browser correctly. So I know it exists.
To Download file via https You should accept https certificate from application Trusting all certificates using HttpClient over HTTPS
And https file download in android causing exception
For Downlod zip or any check this out
Download a file with Android, and showing the progress in a ProgressDialog
I believe the answer is that you are calling connect().
URL url = new URL(stuffs[0]);//<-actual url I am searching https://...
HttpURLConnection connection = (HttpURLConnection) url.openConnection();
connection.setConnectTimeout(20000);
connection.connect();
if(connection.getResponseCode() != HttpURLConnection.HTTP_OK) {
checkErrorCode(connection.getResponseCode());
return false;
}
InputStream in = new BufferedInputStream(connection.getInputStream());
Try not calling connection.connect, and moving the response code check after the line that calls connection.getInputStream().