I've been through every related post I can find trying to get to the bottom of this and am no clearer - hoping someone can put me out of my misery…
I am trying to get Android 2.3 to POST over HTTPS via a Proxy. This code works perfectly on 2.2 through a proxy, and perfectly on both 2.2 and 2.3 using HTTPS when not going through a proxy, and in all cases (2.2 and 2.3) i can GET over HTTPS through a proxy. Its just 2.3 POST using HTTPS through a proxy that seems to be the issue. I get the dreaded "broken pipe" error. The error is thrown when I try and read the inputstream response from the connection - presumably because the TCP socket has been closed underneath my stream. I've tried everything I can think of, including using Connection and Proxy-connection headers (setting to both close and keep-alive) and setting big readTimeout numbers (30 seconds). From my relentless googling, I can see there are known issues with SSL on Android 2.3, but I can't seem to find anything that suggests why the POST might be an issue. Wireshark has yielded some results, but given this is SSL if just a little bit tricky to get to the issue.
Has anyone seem this. I'm using HttpsURLConnection as various posts suggest this is more stable that AndroidHttpClient. Here is my code…any help at all invaluable. Thanks
urlConnection.setSSLSocketFactory(factory);
urlConnection.setHostnameVerifier(new AllowAllHostnameVerifier() );
String dateText = "{\"loopParam\":\"" + String.valueOf(d.getHours()) + ":" + String.valueOf(d.getMinutes()) + ":" + String.valueOf(d.getSeconds()) + "\"}";
txtOutput.setText("Sending " + String.valueOf(dateText.length() ) + " bytes of JSON to /pulse/loop" );
urlConnection.addRequestProperty("Content-type", "application/json");
urlConnection.setDoOutput(true);
urlConnection.setRequestProperty("Proxy-connection", "Keep-Alive");
urlConnection.setRequestProperty("Connection", "Keep-Alive");
urlConnection.setDoInput(true);
urlConnection.setUseCaches(false);
urlConnection.setReadTimeout(30000);
urlConnection.setRequestMethod("POST");
DataOutputStream dataOut = new DataOutputStream(urlConnection.getOutputStream());
dataOut.writeBytes(dateText);
dataOut.flush();
BufferedReader bufIn = new BufferedReader(new InputStreamReader(urlConnection.getInputStream()));
String sResponse;
StringBuilder s = new StringBuilder();
//bufIn is null as error as closed urlcConnection
while ((sResponse = bufIn.readLine()) != null) {
s = s.append(sResponse);
}
Error details:
08-May-12 09:09:51 SsliferSnifferActivity Connecting through proxy INFO
08-May-12 09:09:54 SsliferSnifferActivity javax.net.ssl.SSLException: Write error: ssl=0x2d42b8: I/O error during system call, Broken pipe
at org.apache.harmony.xnet.provider.jsse.NativeCrypto.SSL_write(Native Method)
at org.apache.harmony.xnet.provider.jsse.OpenSSLSocketImpl$SSLOutputStream.write(OpenSSLSocketImpl.java:837)
at java.io.OutputStream.write(OutputStream.java:80)
at org.apache.harmony.luni.internal.net.www.protocol.http.HttpURLConnectionImpl.writeRequestHeaders(HttpURLConnectionImpl.java:799)
at org.apache.harmony.luni.internal.net.www.protocol.http.HttpURLConnectionImpl.retrieveResponse(HttpURLConnectionImpl.java:1028)
at org.apache.harmony.luni.internal.net.www.protocol.http.HttpURLConnectionImpl.getResponseCode(HttpURLConnectionImpl.java:726)
at org.apache.harmony.luni.internal.net.www.protocol.https.HttpsURLConnectionImpl.getResponseCode(HttpsURLConnectionImpl.java:110)
at uk.co.flurrished.sslifersniffer.SslifersnifferActivity.makeRequest(SslifersnifferActivity.java:236)
at uk.co.flurrished.sslifersniffer.SslifersnifferActivity.access$2(SslifersnifferActivity.java:148)
at uk.co.flurrished.sslifersniffer.SslifersnifferActivity$2.onClick(SslifersnifferActivity.java:76)
at android.view.View.performClick(View.java:2485)
at android.view.View$PerformClick.run(View.java:9080)
at android.os.Handler.handleCallback(Handler.java:587)
at android.os.Handler.dispatchMessage(Handler.java:92)
at android.os.Looper.loop(Looper.java:130)
at android.app.ActivityThread.main(ActivityThread.java:3822)
at java.lang.reflect.Method.invokeNative(Native Method)
at java.lang.reflect.Method.invoke(Method.java:507)
at com.android.internal.os.ZygoteInit$MethodAndArgsCaller.run(ZygoteInit.java:839)
at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:597)
at dalvik.system.NativeStart.main(Native Method)
EDIT : This looks like it is being caused by the server raising a 400 (Bad Request) and closing the pipe. What is it about ANdroid 2.3 that is adding extra content when routed through a proxy that causes the 400?
'Broken pipe' has exactly one meaning. You have written to a connection that has already been closed by the other end. Are you sure the peer is really speaking SSL?
Related
I am posting some data using POST API call with the following function:
conn = (HttpURLConnection) url.openConnection();
conn.setConnectTimeout(40000);
conn.setReadTimeout(40000);
conn.setDoOutput(true);
conn.setUseCaches(false);
conn.setDefaultUseCaches(false);
conn.setRequestProperty("Connection", "Keep-Alive");
conn.addRequestProperty("Cache-Control", "max-age=0");
conn.setRequestMethod("POST");
conn.setRequestProperty("Content-Type", "application/json");
conn.setRequestProperty(JsonKeys.DATE_KEY, getTimeStamp());
if (params != null) {
// Write serialized JSON data to output stream.
OutputStreamWriter wr = new OutputStreamWriter(conn.getOutputStream());
wr.write(params.toString());
wr.close();
}
int status = conn.getResponseCode();
StringBuilder sb = new StringBuilder();
try {
if (status == HttpURLConnection.HTTP_OK || status == HTTP_CREATED) {
InputStream is = conn.getInputStream();
int ch;
while ((ch = is.read()) != -1) {
sb.append((char) ch);
}
is.close();
return new JSONObject(sb.toString());
} else {
throw new IOException("We encountered some problem with the request. Please try again later");
} catch (JSONException jex) {
MessageUtils.showLog(jex);
} finally {
if (conn != null) {
MessageUtils.showLog("connection disconnected");
conn.disconnect();
}
}
This POST request needs to wait for at least 40 seconds for the response from the server. This code works perfectly fine on most of the devices but on some devices the request ends abruptly after 10 seconds.I get a 502 response (like after 10 seconds). The server log has no trace of a response being sent (confirmed by my back-end developer). The error stream for 502 that I receive in my front end contains this:
<HEAD><TITLE>Server Hangup</TITLE></HEAD>
I have no idea as to why this is behaving differently for different mobile devices or different Android APIs. Can somebody help me with an insight to this problem or a solution? I have been stuck on this problem for hours and cant find an explanation/solution to it.
This is a stack trace we are receiving in the apache access log:
[Wed May 01 13:54:54.142877 2019] [proxy:error] [pid 23000:tid 140687508961231] (111)Connection refused: AH00957: HTTP: attempt to connect to 127.0.0.1:9000 (localhost) failed
[Wed May 01 13:54:54.142995 2019] [proxy:error] [pid 23000:tid 140687508961231] AH00959: ap_proxy_connect_backend disabling worker for (localhost) for 60s
[Wed May 01 13:54:54.143001 2019] [proxy_http:error] [pid 23000:tid 140687508961231] [client 160.32.32.202:256560] AH01114: HTTP: failed to make connection to backend: localhost
The proxy configuration in our server is something like this:
<Location "/v2/">
ProxyPass http://0.0.0.0:9000/ retry=0 timeout=500
ProxyPassReverse http://0.0.0.0:9000
</Location>
ProxyPreserveHost On
ProxyRequests Off
<Proxy *>
Order deny,allow
Allow from all
</Proxy>
502 is actually a 'Bad Gateway'. This is definitely a server issue. You have some apache module that checks for stable connection and that module detects that your connection is not ok.
From the updated question it is visible that the property 'retry' on your proxy is 0. Here is the meaning from documentation:
https://httpd.apache.org/docs/2.4/mod/mod_proxy.html#proxypass
retry
Connection pool worker retry timeout in seconds. If the connection pool worker to the backend server is in the error state, Apache httpd will not forward any requests to that server until the timeout expires. This enables to shut down the backend server for maintenance and bring it back online later. A value of 0 means always retry workers in an error state with no timeout.
So definitely the problem is in different request and not in different android. Please check your requests. The requests you send might be different and server can detect connection as dead letting you think that problem is in android. You can also play with retry parameter and set it to default retry=60.
I'm trying to upload a picture file from my Android application to my JavaEE REST service, which is deployed to a JBoss Wildfly 9 server.
My understanding of Content-Disposition is that it should be defined as its own header for each part uploaded, but also can be defined in the request header - if only one file is uploaded.
So in the header I've defined Content-Type: multipart/form-data;boundary=*****, while each (for now only one) part starts with --*****, before I end the request body with --*****--. This is in case I need to upgrade to multiple file upload on a later point.
The server should be accessible from both Android and AngularJS applications. Therefore I've added a ContainerResponseFilter for the AngularJS app, with the following, but I don't see any reason this should be the reason for the blocked request.
#Override
public void filter(ContainerRequestContext requestCtx, ContainerResponseContext responseCtx)
throws IOException {
responseCtx.getHeaders().add("Access-Control-Allow-Origin", "http://localhost:8000");
responseCtx.getHeaders().add("Access-Control-Allow-Credentials", "true");
responseCtx.getHeaders().add("Access-Control-Allow-Methods", "GET, POST, DELETE, PUT");
responseCtx.getHeaders().add("Access-Control-Allow-Headers", "Host, "
+ "Accept, "
+ "Origin, "
+ "Connection, "
+ "Content-Type, "
+ "Cache-Control, "
+ "Content-Length, "
+ "Accept-Encoding, "
+ "Content-Disposition");
}
#Override
public void filter(ContainerRequestContext containerRequestContext) throws IOException {
String reqSource = "(" + servletRequest.getRemoteAddr() + ") "
+ servletRequest.getRemoteUser() + "#"
+ servletRequest.getRemoteHost() + ":"
+ servletRequest.getRemotePort();
LOGGER.trace(" :: Source :: [{}]", reqSource);
String userId = (securityContext.getUserPrincipal() != null ?
securityContext.getUserPrincipal().getName() : "unknown");
LOGGER.trace(" :: User :: [{}]", userId);
String reqUri = servletRequest.getRequestURI();
String reqType = servletRequest.getMethod();
LOGGER.trace(" :: Boundary :: [{}] - [{}]", reqUri, reqType);
}
These filters are the only code added by me that interacts with the request and reponse between the applications and the end point. I've also tried to remove these filters, without any luck. Removing the responseFilter breaks the communication with the AngularJS app, while removing the requestFilter only stops the logging.
#POST
#Consumes({MULTIPART_FORM_DATA})
public Response createPicture(MultipartFormDataInput input) {
for (InputPart inputPart : input.getParts()) {
try {
fileController.saveFile(inputPart);
return Response.ok().build();
} catch (FileNotSavedException e) {
return Response.serverError(e.getMessage()).build();
}
}
return badRequestNullResponse();
}
Code for uploading the picture (Android):
public static void uploadBitmap(Bitmap bitmap, String filename)
throws IOException {
URL url = new URL(URL_REST_API + FILE);
HttpURLConnection httpUrlConnection = (HttpURLConnection) url.openConnection();
httpUrlConnection.setUseCaches(false);
httpUrlConnection.setDoOutput(true);
httpUrlConnection.setDoInput(true);
httpUrlConnection.setRequestMethod("POST");
httpUrlConnection.setRequestProperty("Connection", "Keep-Alive");
httpUrlConnection.setRequestProperty("Cache-Control", "no-cache");
httpUrlConnection.setRequestProperty("Accept", "multipart/form-data");
httpUrlConnection.setRequestProperty("Content-Type", "multipart/form-data;boundary=" + BOUNDARY);
// EDIT 2: This following log statement was omitted in the first post.
// I extracted the setRequestProperty statements from another method
// due readability of this question, but I had missed to copy this:
Log.e(LOG_TAG, "Headers: \n" + httpUrlConnection.getHeaderFields());
DataOutputStream request = new DataOutputStream(httpUrlConnection.getOutputStream());
// Part [start]
request.writeBytes(DOUBLE_HYPHEN + BOUNDARY + CR_LF);
request.writeBytes("Content-Disposition: form-data;filename=\"" + filename + "\"" + CR_LF);
request.writeBytes(CR_LF);
// Part [content]
bitmap.compress(Bitmap.CompressFormat.JPEG, IMAGE_QUALITY_PERCENTAGE, request);
// part [end]
request.writeBytes(CR_LF);
request.writeBytes(DOUBLE_HYPHEN + BOUNDARY + DOUBLE_HYPHEN + CR_LF);
request.flush();
request.close();
httpUrlConnection.disconnect();
}
As mentioned I'm starting "each" part with double hyphens, the boundary and a line break. Since the headers are created by setRequestProperty I would assume the header are ended correctly. So is it caused by a missing body, in that case; why isn't the Content-Disposition or the picture file written to the request?
07:55:28,479 WARN [org.apache.james.mime4j.parser.MimeEntity] (default task-36) Unexpected end of headers detected. Higher level boundary detected or EOF reached.
07:55:28,479 WARN [org.apache.james.mime4j.parser.MimeEntity] (default task-36) Invalid header encountered
07:55:28,479 WARN [org.apache.james.mime4j.parser.MimeEntity] (default task-36) Body part ended prematurely. Boundary detected in header or EOF reached.
07:55:28,479 WARN [org.jboss.resteasy.core.ExceptionHandler] (default task-36) Failed executing POST /file: org.jboss.resteasy.spi.ReaderException: java.lang.RuntimeException: Could find no Content-Disposition header within part
at org.jboss.resteasy.core.MessageBodyParameterInjector.inject(MessageBodyParameterInjector.java:183)
at org.jboss.resteasy.core.MethodInjectorImpl.injectArguments(MethodInjectorImpl.java:89)
at org.jboss.resteasy.core.MethodInjectorImpl.invoke(MethodInjectorImpl.java:112)
at org.jboss.resteasy.core.ResourceMethodInvoker.invokeOnTarget(ResourceMethodInvoker.java:296)
at org.jboss.resteasy.core.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:250)
at org.jboss.resteasy.core.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:237)
at org.jboss.resteasy.core.SynchronousDispatcher.invoke(SynchronousDispatcher.java:356)
at org.jboss.resteasy.core.SynchronousDispatcher.invoke(SynchronousDispatcher.java:179)
at org.jboss.resteasy.plugins.server.servlet.ServletContainerDispatcher.service(ServletContainerDispatcher.java:220)
at org.jboss.resteasy.plugins.server.servlet.HttpServletDispatcher.service(HttpServletDispatcher.java:56)
at org.jboss.resteasy.plugins.server.servlet.HttpServletDispatcher.service(HttpServletDispatcher.java:51)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:790)
at io.undertow.servlet.handlers.ServletHandler.handleRequest(ServletHandler.java:86)
at io.undertow.servlet.handlers.security.ServletSecurityRoleHandler.handleRequest(ServletSecurityRoleHandler.java:62)
at io.undertow.servlet.handlers.ServletDispatchingHandler.handleRequest(ServletDispatchingHandler.java:36)
at org.wildfly.extension.undertow.security.SecurityContextAssociationHandler.handleRequest(SecurityContextAssociationHandler.java:78)
at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)
at io.undertow.servlet.handlers.security.SSLInformationAssociationHandler.handleRequest(SSLInformationAssociationHandler.java:131)
at io.undertow.servlet.handlers.security.ServletAuthenticationCallHandler.handleRequest(ServletAuthenticationCallHandler.java:57)
at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)
at io.undertow.security.handlers.AbstractConfidentialityHandler.handleRequest(AbstractConfidentialityHandler.java:46)
at io.undertow.servlet.handlers.security.ServletConfidentialityConstraintHandler.handleRequest(ServletConfidentialityConstraintHandler.java:64)
at io.undertow.security.handlers.AuthenticationMechanismsHandler.handleRequest(AuthenticationMechanismsHandler.java:58)
at io.undertow.servlet.handlers.security.CachedAuthenticatedSessionHandler.handleRequest(CachedAuthenticatedSessionHandler.java:72)
at io.undertow.security.handlers.NotificationReceiverHandler.handleRequest(NotificationReceiverHandler.java:50)
at io.undertow.security.handlers.SecurityInitialHandler.handleRequest(SecurityInitialHandler.java:76)
at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)
at org.wildfly.extension.undertow.security.jacc.JACCContextIdHandler.handleRequest(JACCContextIdHandler.java:61)
at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)
at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)
at io.undertow.servlet.handlers.ServletInitialHandler.handleFirstRequest(ServletInitialHandler.java:282)
at io.undertow.servlet.handlers.ServletInitialHandler.dispatchRequest(ServletInitialHandler.java:261)
at io.undertow.servlet.handlers.ServletInitialHandler.access$000(ServletInitialHandler.java:80)
at io.undertow.servlet.handlers.ServletInitialHandler$1.handleRequest(ServletInitialHandler.java:172)
at io.undertow.server.Connectors.executeRootHandler(Connectors.java:199)
at io.undertow.server.HttpServerExchange$1.run(HttpServerExchange.java:774)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.RuntimeException: Could find no Content-Disposition header within part
at org.jboss.resteasy.plugins.providers.multipart.MultipartFormDataInputImpl.extractPart(MultipartFormDataInputImpl.java:68)
at org.jboss.resteasy.plugins.providers.multipart.MultipartInputImpl.extractParts(MultipartInputImpl.java:229)
at org.jboss.resteasy.plugins.providers.multipart.MultipartInputImpl.parse(MultipartInputImpl.java:198)
at org.jboss.resteasy.plugins.providers.multipart.MultipartFormDataReader.readFrom(MultipartFormDataReader.java:52)
at org.jboss.resteasy.plugins.providers.multipart.MultipartFormDataReader.readFrom(MultipartFormDataReader.java:20)
at org.jboss.resteasy.plugins.providers.multipart.MultipartFormDataReader$Proxy$_$$_WeldClientProxy.readFrom(Unknown Source)
at org.jboss.resteasy.core.interception.AbstractReaderInterceptorContext.readFrom(AbstractReaderInterceptorContext.java:59)
at org.jboss.resteasy.core.interception.ServerReaderInterceptorContext.readFrom(ServerReaderInterceptorContext.java:62)
at org.jboss.resteasy.core.interception.AbstractReaderInterceptorContext.proceed(AbstractReaderInterceptorContext.java:51)
at org.jboss.resteasy.security.doseta.DigitalVerificationInterceptor.aroundReadFrom(DigitalVerificationInterceptor.java:32)
at org.jboss.resteasy.core.interception.AbstractReaderInterceptorContext.proceed(AbstractReaderInterceptorContext.java:53)
at org.jboss.resteasy.plugins.interceptors.encoding.GZIPDecodingInterceptor.aroundReadFrom(GZIPDecodingInterceptor.java:59)
at org.jboss.resteasy.plugins.interceptors.encoding.GZIPDecodingInterceptor$Proxy$_$$_WeldClientProxy.aroundReadFrom(Unknown Source)
at org.jboss.resteasy.core.interception.AbstractReaderInterceptorContext.proceed(AbstractReaderInterceptorContext.java:53)
at org.jboss.resteasy.core.MessageBodyParameterInjector.inject(MessageBodyParameterInjector.java:150)
... 38 more
Also; I found some WARN messages regarding WFLYWELD0052: (...) Package-private access will not work. So I fixed these by adding core and spi as dependencies in the warned modules, as suggested in this issue. It removed the warning messages, but did not fix the problem.
EDIT:
Here are the logs from an upload request by the android application, where I'm printing the headers sent with the request:
(default task-42) Header: [Accept]
(default task-42) Value: [multipart/form-data]
(default task-42) Header: [Cache-Control]
(default task-42) Value: [no-cache]
(default task-42) Header: [Connection]
(default task-42) Value: [Keep-Alive]
(default task-42) Header: [User-Agent]
(default task-42) Value: [Dalvik/2.1.0 (Linux; U; Android 5.0.2; SM-A500FU Build/LRX22G)]
(default task-42) Header: [Host]
(default task-42) Value: [***.**.***.***:8080]
(default task-42) Header: [Accept-Encoding]
(default task-42) Value: [gzip]
(default task-42) Header: [Content-Length]
(default task-42) Value: [0]
(default task-42) Header: [Content-Type]
(default task-42) Value: [multipart/form-data;boundary=*****]
(default task-42) :: Source :: [(*.***.**.**) null#*.***.**.**:28686]
(default task-42) :: User :: [unknown]
(default task-42) :: Boundary :: [/upload/api/file/] - [POST]
(default task-42) Unexpected end of headers detected. Higher level boundary detected or EOF reached.
(default task-42) Invalid header encountered
(default task-42) Body part ended prematurely. Boundary detected in header or EOF reached.
When uploading through Postman, I'm able to access the end point, as expected - without adding more than the file (perhaps Postman creates the content-disposition tags automatically?).
EDIT 2:
I have updated the Android code with the reason of error (see Edit 2-comment). It was actually a log statement that caused the incorrect behaviour. I'll add an answer to explain why!
Reason of failure:
The reason this happened was caused by a log statement I added, to verify the headers sent to the server. In my original question this was omitted, since the following was originally extracted from another method. When I copied the snippet out from this method I didn't think of the log statement as related to the problem.
httpUrlConnection.setUseCaches(false);
httpUrlConnection.setDoOutput(true);
httpUrlConnection.setDoInput(true);
httpUrlConnection.setRequestMethod("POST");
httpUrlConnection.setRequestProperty("Connection", "Keep-Alive");
httpUrlConnection.setRequestProperty("Cache-Control", "no-cache");
httpUrlConnection.setRequestProperty("Accept", "multipart/form-data");
httpUrlConnection.setRequestProperty("Content-Type",
"multipart/form-data;boundary=" + BOUNDARY);
The following line were also in the extracted method, causing the problem.
Log.e(LOG_TAG, "Headers: \n" + httpUrlConnection.getHeaderFields());
Explaination:
Why didn't you notice this before?
The method uploadBitmap just throws IOException and the caller method just swallowed the exception and printed out "Could not upload picture; server responded with 406". This bad exception handling made me focus too much on thinking the server caused the issue.
What actually happened?
The exception message that I missed in the Android application was cannot write request body after response has been read, which made me look through each line of the code, till I found that the log statement tried to read the headers.
Why did this cause error?
I'm not 100% sure about the actual reason for this, but I can imagine that reading from the connection tells the server;
"I'm done with what ever I want to say to you, now I want to know what you have to say to me".
So when I read from the connection after writing the headers, it seems that I'm basically sending the request to get the response. This then caused the error where the server couldn't find the Content-Disposition, since there never were any Content-Dispositions, parts or even a body for the server to process.
Once again; I'm not 100% sure about this, since I havn't looked it up yet.
Conclusion:
Well, after too many hours wasted on this error, I discovered that I should be more aware of my log statements. I should also stop using log statements to get values I just as easily can get by simple, plain debugging.
I would like to know if there are known issues on Android with HttpUrlConnection and POST requests. We are experiencing intermittent EOFExceptions when making POST requests from an Android client. Retrying the same request will eventually work. Here is a sample stack trace:
java.io.EOFException
at libcore.io.Streams.readAsciiLine(Streams.java:203)
at libcore.net.http.HttpEngine.readResponseHeaders(HttpEngine.java:579)
at libcore.net.http.HttpEngine.readResponse(HttpEngine.java:827)
at libcore.net.http.HttpURLConnectionImpl.getResponse(HttpURLConnectionImpl.java:283)
at libcore.net.http.HttpURLConnectionImpl.getResponseCode(HttpURLConnectionImpl.java:497)
at libcore.net.http.HttpsURLConnectionImpl.getResponseCode(HttpsURLConnectionImpl.java:134)
There are many similar bug reports and posts to stack overflow but I cannot understand if there really is an issue and if so, what versions of Android are affected and what the proposed fix/work around is.
Here are some of the similar reports I am referring to:
Android HttpsUrlConnection eofexception
Android HttpURLConnection throwing EOFException
EOFException and FileNotFoundException in HttpURLConnection getInputStream()
https://code.google.com/p/google-http-java-client/issues/detail?id=213
https://code.google.com/p/android/issues/detail?id=29509
https://code.google.com/p/google-http-java-client/issues/detail?id=230
https://code.google.com/p/android/issues/detail?id=41576
Here is a potential Android framework fix
https://android.googlesource.com/platform/libcore/+/19aa40c81c48ff98ccc7272f2a3c41479b806376
I do know there was an issue with poisoned connections in the connection pool in pre-Froyo but these issues are occurring on new ICS+ devices exclusively. If there were a problem on later devices I would expect some kind of official Android documentation of the issue.
Our conclusion is that there is an issue in the Android platform. Our workaround was to catch the EOFException and retry the request N number of times. Below is the pseudo code:
private static final int MAX_RETRIES = 3;
private ResponseType fetchResult(RequestType request) {
return fetchResult(request, 0);
}
private ResponseType fetchResult(RequestType request, int reentryCount) {
try {
// attempt to execute request
} catch (EOFException e) {
if (reentryCount < MAX_RETRIES) {
fetchResult(request, reentryCount + 1);
}
}
// continue processing response
}
HttpURLConnection library internally maintains a pool of Connections. So, whenever a request is send, it first checks if there is an existing connection already present in the pool, based on which it decides to create a new one.
These connections are nothing but sockets, and this library by default does not closes these sockets. It may sometimes happen that a connection (socket) which is not currently being used and is present in the pool is no longer usable as the Server may choose to terminate the connection after some time. Now, since the connection even though is closed by the server, the library does not knows about it and assumes the connection/socket to be still connected. Thus it sends the new request using this stale connection and hence we get EOFException.
The best way to handle this is to check the Response Headers after each request you send. The server always sends a "Connection: Close" before terminating a connection (HTTP 1.1). So, you can use getHeaderField() and check for "Connection" field. Another thing to note is that server ONLY sends this connection field when it is about to terminate the connection. So, you need to code around this with the possibility of getting a "null" in the normal case (when server is not closing the connection)
This workaround tends to be reliable and performant:
static final int MAX_CONNECTIONS = 5;
T send(..., int failures) throws IOException {
HttpURLConnection connection = null;
try {
// initialize connection...
if (failures > 0 && failures <= MAX_CONNECTIONS) {
connection.setRequestProperty("Connection", "close");
}
// return response (T) from connection...
} catch (EOFException e) {
if (failures <= MAX_CONNECTIONS) {
disconnect(connection);
connection = null;
return send(..., failures + 1);
}
throw e;
} finally {
disconnect(connection);
}
}
void disconnect(HttpURLConnection connection) {
if (connection != null) {
connection.disconnect();
}
}
This implementation relies on the fact that the default number of connections that can be opened with a server is 5 (Froyo - KitKat). This means that up to 5 stale connections may exist, each of which will have to be closed.
After each failed attempt, the Connection:close request property will cause the underlying HTTP engine to close the socket when connection.disconnect() is called. By retrying up to 6 times (max connections + 1), we ensure that the last attempt will always be given a new socket.
The request may experience additional latency if no connections are alive, but that is certainly better than an EOFException. In that case, the final send attempt won't immediately close the freshly opened connection. That's the only practical optimization that can be made.
Instead of relying on the magic default value of 5, you may be able to configure the system property yourself. Keep in mind that this property is accessed by a static initializer block in KitKat's ConnectionPool.java, and it works like this in older Android versions too. As a result, the property may be used before you have a chance to set it.
static final int MAX_CONNECTIONS = 5;
static {
System.setProperty("http.maxConnections", String.valueOf(MAX_CONNECTIONS));
}
Yes. There is a problem in the Android platform, specifically, in Android libcore with version 4.1-4.3.
The problem is introduced in this commit: https://android.googlesource.com/platform/libcore/+/b2b02ac6cd42a69463fd172531aa1f9b9bb887a8
Android 4.4 switched http lib to "okhttp" which doesn't have this problem.
Problem explained as follow:
On android 4.1-4.3, when you are using URLConnection/HttpURLConnection to POST with "ChunkedStreamingMode" or "FixedLengthStreamingMode" set, URLConnection/HttpURLConnection will not do silent retry if the reused connection is stale. You should retry POST at most "http.maxConnections+1" times in your code, just as previous answers suggest.
I suspect it might be the server that is at fault here, and the HttpURLConnection is not as forgiving as other implementations. That was the cause of my EOFException. I suspect in my case this would not be intermittent (fixed it before testing the N retry workaround), so the answers above relate to other issues and be a correct solution in those cases.
My server was using python SimpleHTTPServer and I was wrongly assuming all I needed to do to indicate success was the following:
self.send_response(200)
That sends the initial response header line, a server and a date header, but leaves the stream in the state where you are able to send additional headers too. HTTP requires an additional new line after headers to indicate they are finished. It appears if this new line isn't present when you attempt to get the result body InputStream or response code etc with HttpURLConnection then it throws the EOFException (which is actually reasonable, thinking about it). Some HTTP clients did accept the short response and reported the success result code which lead to me perhaps unfairly pointing the finger at HttpURLConnection.
I changed my server to do this instead:
self.send_response(200)
self.send_header("Content-Length", "0")
self.end_headers()
No more EOFException with that code.
This worked for me.
public ResponseObject sendPOST(String urlPrefix, JSONObject payload) throws JSONException {
String line;
StringBuffer jsonString = new StringBuffer();
ResponseObject response = new ResponseObject();
try {
URL url = new URL(POST_URL);
HttpURLConnection connection = (HttpURLConnection) url.openConnection();
connection.setDoInput(true);
connection.setDoOutput(true);
connection.setReadTimeout(10000);
connection.setConnectTimeout(15000);
connection.setRequestMethod("POST");
connection.setRequestProperty("Accept", "application/json");
connection.setRequestProperty("Content-Type", "application/json; charset=UTF-8");
OutputStream os = connection.getOutputStream();
os.write(payload.toString().getBytes("UTF-8"));
os.close();
BufferedReader br = new BufferedReader(new InputStreamReader(connection.getInputStream()));
while ((line = br.readLine()) != null) {
jsonString.append(line);
}
response.setResponseMessage(connection.getResponseMessage());
response.setResponseReturnCode(connection.getResponseCode());
br.close();
connection.disconnect();
} catch (Exception e) {
Log.w("Exception ",e);
return response;
}
String json = jsonString.toString();
response.setResponseJsonString(json);
return response;
}
connection.addRequestProperty("Accept-Encoding", "gzip");
is the answer
I'm receiving java.io.EOFException's when using Spring REST template on Android.
The stacktrace cause reads like this:
Caused by: java.io.EOFException
at libcore.io.Streams.readAsciiLine(Streams.java:203)
at libcore.net.http.HttpEngine.readResponseHeaders(HttpEngine.java:560)
at libcore.net.http.HttpEngine.readResponse(HttpEngine.java:813)
at libcore.net.http.HttpURLConnectionImpl.getResponse(HttpURLConnectionImpl.java:274)
at libcore.net.http.HttpURLConnectionImpl.getResponseCode(HttpURLConnectionImpl.java:486)
at org.springframework.http.client.SimpleClientHttpResponse.getRawStatusCode(SimpleClientHttpResponse.java:49)
at org.springframework.http.client.SimpleClientHttpResponse.getStatusCode(SimpleClientHttpResponse.java:55)
at org.springframework.http.client.BufferingClientHttpResponseWrapper.getStatusCode(BufferingClientHttpResponseWrapper.java:47)
at com.company.util.LoggingClientHttpRequestInterceptor.intercept(LoggingClientHttpRequestInterceptor.java:33)
at org.springframework.http.client.InterceptingClientHttpRequest$RequestExecution.execute(InterceptingClientHttpRequest.java:81)
at com.company.api.interceptor.AuthTokenInterceptor.intercept(AuthTokenInterceptor.java:51)
at org.springframework.http.client.InterceptingClientHttpRequest$RequestExecution.execute(InterceptingClientHttpRequest.java:81)
at org.springframework.http.client.InterceptingClientHttpRequest.executeInternal(InterceptingClientHttpRequest.java:67)
at org.springframework.http.client.AbstractBufferingClientHttpRequest.executeInternal(AbstractBufferingClientHttpRequest.java:46)
at org.springframework.http.client.AbstractClientHttpRequest.execute(AbstractClientHttpRequest.java:63)
at org.springframework.web.client.RestTemplate.doExecute(RestTemplate.java:475)
... 14 more
Another similar stacktrace:
org.springframework.web.client.ResourceAccessException: I/O error: null; nested exception is java.io.EOFException
at org.springframework.web.client.RestTemplate.doExecute(RestTemplate.java:490)
at org.springframework.web.client.RestTemplate.execute(RestTemplate.java:438)
at org.springframework.web.client.RestTemplate.exchange(RestTemplate.java:414)
at com.company.api.ApiClient_.logLoginAttempt(ApiClient_.java:299)
at com.company.security.CompanyAuthenticationService$2.onCreateCall(CompanyAuthenticationService.java:206)
at com.company.api.SafeApiCall.doInBackground(SafeApiCall.java:49)
at com.company.api.SafeApiCall.doInBackground(SafeApiCall.java:22)
at android.os.AsyncTask$2.call(AsyncTask.java:287)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:305)
at java.util.concurrent.FutureTask.run(FutureTask.java:137)
at android.os.AsyncTask$SerialExecutor$1.run(AsyncTask.java:230)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1076)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:569)
at java.lang.Thread.run(Thread.java:856)
Caused by: java.io.EOFException
at libcore.io.Streams.readAsciiLine(Streams.java:203)
at libcore.net.http.HttpEngine.readResponseHeaders(HttpEngine.java:560)
at libcore.net.http.HttpEngine.readResponse(HttpEngine.java:813)
at libcore.net.http.HttpURLConnectionImpl.getResponse(HttpURLConnectionImpl.java:274)
at libcore.net.http.HttpURLConnectionImpl.getResponseCode(HttpURLConnectionImpl.java:486)
at org.springframework.http.client.SimpleClientHttpResponse.getRawStatusCode(SimpleClientHttpResponse.java:49)
at org.springframework.http.client.SimpleClientHttpResponse.getStatusCode(SimpleClientHttpResponse.java:55)
at org.springframework.http.client.BufferingClientHttpResponseWrapper.getStatusCode(BufferingClientHttpResponseWrapper.java:47)
at org.springframework.web.client.DefaultResponseErrorHandler.hasError(DefaultResponseErrorHandler.java:46)
at org.springframework.web.client.RestTemplate.doExecute(RestTemplate.java:476)
... 13 more
This is all happening on Android 4.1.2, installed on my Xoom tablet.
The problem appears and disappears. It's not triggered by long requests either. The server part is running on a machine within the local network. When I try to run the API Calls through curl, it works just fine.
AuthTokenInterceptor:
#Override
public ClientHttpResponse intercept(HttpRequest request, byte[] data, ClientHttpRequestExecution execution) throws IOException {
HttpHeaders headers = request.getHeaders();
if (!StringUtils.isEmpty(mAuthToken)) {
headers.add((mIsOAuth ? "Authorization" : "authToken"), (mIsOAuth ? "Bearer " : "") + mAuthToken);
}
return execution.execute(request, data);
}
LoggingClientHttpRequestInterceptor:
/** {#inheritDoc} */
#Override
public ClientHttpResponse intercept(HttpRequest httpRequest, byte[] bytes, ClientHttpRequestExecution clientHttpRequestExecution) throws IOException {
Log.d(TAG, "To : " + httpRequest.getURI());
Log.d(TAG, "Method : " + httpRequest.getMethod().name());
Log.d(TAG, "Data : " + new String(bytes));
for (Object key : httpRequest.getHeaders().keySet()) {
Log.d(TAG, "Header <" + key + ">: " + httpRequest.getHeaders().get(key));
}
final ClientHttpResponse response = clientHttpRequestExecution.execute(httpRequest, bytes);
if (response != null) {
Log.d(TAG, "Response: " + response.getStatusCode());
if (response.getBody() != null) {
Log.d(TAG, "Response: " + convertStreamToString(response.getBody()));
}
} else {
Log.d(TAG, "Response: " + response);
}
return response;
}
The Rest Template is configured like this:
final RestTemplate template = new RestTemplate(false);
template.getMessageConverters().add(new MappingJacksonHttpMessageConverter());
template.setRequestFactory(new BufferingClientHttpRequestFactory(template.getRequestFactory()));
ApiUtils.addAuthTokenHeaderToRestTemplate(template, mAuthToken, false);
ApiUtils.addRequestLoggingToRestTemplate(template);
The API call in question that crashed here is described in the Android annotations based interface:
#Post("/user/memberships")
#Accept(MediaType.APPLICATION_JSON)
CompanyApiResponse saveGroupMembership(UserGroupMembership membership) throws RestClientException;
Things I've tried:
Removed LoggingInterceptor
Called all API calls by CURL
Removed call BufferingClientHttpRequestFactory - Helped a little but the error still occurs.
Tested it on Android 2.3 - the error cannot be reproduced
I've been reading various forums posts, the EOF exception seems to appear if URLs are incorrect, which I double checked in this case.
Also of note, once the EOF Exception occurs, the call not even reaches the server side.
Where would be a good point to continue the search for a fix? Is this a Android 4.1 inconvenience?
While debugging this issue, I also found https://jira.springsource.org/browse/ANDROID-102 which prevented me from seeing the real error (EOF) before.
Update: Just found http://code.google.com/p/google-http-java-client/issues/detail?id=116 - it might be related.
The fix is also outlined in https://codereview.appspot.com/6225045/ - so it might've been merged for 4.1.
This one bit me as well, running Jelly Bean 4.2. After researching, it seems that it's happening because of a combination of Keep-Alive being set and using the standard J2SE HTTP Client, which I believe is HttpURLConnection.
There are 2 solutions that I can confirm are correct.
1) Switch off Keep-Alive.
For me, the solution given in Sebastian's answer, System.setProperty("http.keepAlive", "false"); didn't work. I had to use
HttpHeaders headers = new HttpHeaders();
headers.set("Connection", "Close");
and send those headers in an HttpEntity in the RestTemplate.
As mentioned, this solution could have an impact on performance
2) Change the HTTP Client.
In Spring for Android (tested on 1.0.1.RELEASE, but could be in earlier releases too) the default HTTP Client for a RestTemplate instance is determined by the version of Android on the device. API 9 or newer uses HttpURLConnection, older uses HTTPClient. To explicitly set the client to the old one, use
restTemplate.setRequestFactory(new HttpComponentsClientHttpRequestFactory());
More info can be found here: http://static.springsource.org/spring-android/docs/1.0.1.RELEASE/reference/htmlsingle/#d4e34
I'm not sure what impact this will have on performance, but I guess it's more performant than an app that doesn't work.
Anyway, hope that helps someone. I just wasted a week wild-goose-chasing this one down.
http://code.google.com/p/google-http-java-client/issues/detail?id=116 contains a workaround in the latest comment:
This is defenetly somehow connected with keepAlive connections.
When I use: System.setProperty("http.keepAlive", "false"); problems
disappears.
But from my understanding keep alive connections are greatly increase
performance so it is better not to disable them.
Im also awere that keep alive should be disabled for old versions, but
my device is Jelly Bean.
Once applied the error disappeared.
Seems it's not entirely related to Spring, but a JB problem.
Recently I faced this issue and will able to resolved this issue after setting headers with following piece of code :
headers.set("Accept-Language", "en-US,en;q=0.8");
RestTemplate restTemplate = new RestTemplate();
((SimpleClientHttpRequestFactory)restTemplate.getRequestFactory()).setOutputStreaming(false);
restTemplate.postForObject......
I know this issue should be fixed with System.setProperty("http.keepAlive", "false"); before openConnection, but that didn't work to me. First try on this code works, second one fails. Even if i try this request after less than 5 seconds, it also works. If i wait more than that, it fails again
This is my code:
System.setProperty("http.keepAlive", "false");
HttpURLConnection conn = (HttpURLConnection) mURL.openConnection();
conn.setUseCaches(false);
conn.setRequestProperty("Connection","Keep-Alive");
conn.setRequestProperty("User-Agent", useragent);
conn.setConnectTimeout (30000) ;
conn.setDoOutput(true);
conn.setDoInput(true);
consumer.sign(conn);
InputSource is = new InputSource(conn.getInputStream());
I get the exception on last line:
java.io.IOException: Write error: I/O error during system call, Broken pipe
W/System.err( 2164): at org.apache.harmony.xnet.provider.jsse.OpenSSLSocketImpl.nativewrite(Native Method)
W/System.err( 2164): at org.apache.harmony.xnet.provider.jsse.OpenSSLSocketImpl.access$600(OpenSSLSocketImpl.java:55)
W/System.err( 2164): at org.apache.harmony.xnet.provider.jsse.OpenSSLSocketImpl$SSLOutputStream.write(OpenSSLSocketImpl.java:583)
W/System.err( 2164): at java.io.OutputStream.write(OutputStream.java:82)
W/System.err( 2164): at org.apache.harmony.luni.internal.net.www.protocol.http.HttpURLConnectionImpl.sendRequest(HttpURLConnectionImpl.java:1332)
W/System.err( 2164): at org.apache.harmony.luni.internal.net.www.protocol.http.HttpURLConnectionImpl.doRequestInternal(HttpURLConnectionImpl.java:1656)
W/System.err( 2164): at org.apache.harmony.luni.internal.net.www.protocol.http.HttpURLConnectionImpl.doRequest(HttpURLConnectionImpl.java:1649)
W/System.err( 2164): at org.apache.harmony.luni.internal.net.www.protocol.http.HttpURLConnectionImpl.getInputStream(HttpURLConnectionImpl.java:1153)
W/System.err( 2164): at org.apache.harmony.luni.internal.net.www.protocol.https.HttpsURLConnectionImpl.getInputStream(HttpsURLConnectionImpl.java:253)
Does someone have an idea about what's wrong here?. Thanks!
The connection pool used by HttpURLConnection when it is keeping connections alive is broken such that it tries to use connections that have been closed by the server. By default Android sets KeepAlive on all connections.
System.setProperty("http.keepAlive", "false"); is a workaround that disables KeepAlive for all connections so then you avoid the bug in the connection pool.
conn.setRequestProperty("Connection","Keep-Alive"); turns KeepAlive on for this particular connection, essentially reversing what System.setProperty("http.keepAlive", "false"); does.
Also I always explicitly call connect() as it makes it clear where you are ending your connection setup. I'm not sure if calling this method is optional or not.
System.setProperty("http.keepAlive", "false");
HttpURLConnection conn = (HttpURLConnection) mURL.openConnection();
conn.setUseCaches(false);
conn.setRequestProperty("User-Agent", useragent);
conn.setConnectTimeout(30000);
conn.setDoOutput(true);
conn.setDoInput(true);
consumer.sign(conn);
conn.connect();
InputSource is = new InputSource(conn.getInputStream());
You dont need the System.setProperty("http.keepAlive", "false");
All you need is conn.setRequestProperty("connection", "close");
this fixes the issue but effectively kills keep alives and therefore potentially makes multiple connections slower (which is a shame). I was looking through the harmony bug tracker but could not really find anything.
#fonetik, do you know whether this is already raised with harmony? I mean not that it helps much since another http related luni defect is still unassigned after more than a month.
I solved the problem. Here I leave you the code, in case it might be helpful for someone. Basically I see a trend on Google for using HttpClient/HttpGet instead of HttpUrlConnection. So I tried with those classes, and everything worked:
final HttpClient client = new DefaultHttpClient();
final HttpGet conn = new HttpGet(mURL.toString());
OAuthConsumer consumer = mOAuthManager.getPostConsumer();
consumer.sign(conn);
HttpResponse response = client.execute(conn);
InputSource is = new InputSource(response.getEntity().getContent());
this bug had beed fixed in Android2.3 version,as we know System.setProperty("http.keepAlive", "false"); is not a very good solution ,because on mobile device ,create each connection is every time is high cost.
I believe your problem lies in the order of your code. Check those methods in the URLConnection JavaDocs - setRequestProperty should not be called after the connection is made on mUrl.openConnection(). It may be working the first time because the connection is made, then you are changing settings that are not affecting anything until the next time you try to make a connection. Try using the HttpURLConnection constructor instead so you can call connect() after you have set the properties.
When i am trying to open https connection it is working fine but second time it fails because i have set the system property value instead of HttpsURLConnection connection. I have got the java.io.IOException: Write error: I/O issue while opening the https connection second time. I have used following code in my applicaiton.
System.setProperty("http.proxyHost", proxy);
System.setProperty("http.proxyPort", port);
But when i changed the same to below it works fine.
javax.net.ssl.HttpsURLConnection ucon = (javax.net.ssl.HttpsURLConnection) urlWPF.openConnection(proxyserver);
ucon.setRequestProperty("http.proxyHost", proxy);
ucon.setRequestProperty("http.proxyPort", port);
If you set the system property, it will applicable for entire application. If you want reset the same, you can follow two ways. One is you have to take server refresh and second one is you have to change the HttpsURLConnection.setRequestProperty which is mentioned above where required.