I'm receiving java.io.EOFException's when using Spring REST template on Android.
The stacktrace cause reads like this:
Caused by: java.io.EOFException
at libcore.io.Streams.readAsciiLine(Streams.java:203)
at libcore.net.http.HttpEngine.readResponseHeaders(HttpEngine.java:560)
at libcore.net.http.HttpEngine.readResponse(HttpEngine.java:813)
at libcore.net.http.HttpURLConnectionImpl.getResponse(HttpURLConnectionImpl.java:274)
at libcore.net.http.HttpURLConnectionImpl.getResponseCode(HttpURLConnectionImpl.java:486)
at org.springframework.http.client.SimpleClientHttpResponse.getRawStatusCode(SimpleClientHttpResponse.java:49)
at org.springframework.http.client.SimpleClientHttpResponse.getStatusCode(SimpleClientHttpResponse.java:55)
at org.springframework.http.client.BufferingClientHttpResponseWrapper.getStatusCode(BufferingClientHttpResponseWrapper.java:47)
at com.company.util.LoggingClientHttpRequestInterceptor.intercept(LoggingClientHttpRequestInterceptor.java:33)
at org.springframework.http.client.InterceptingClientHttpRequest$RequestExecution.execute(InterceptingClientHttpRequest.java:81)
at com.company.api.interceptor.AuthTokenInterceptor.intercept(AuthTokenInterceptor.java:51)
at org.springframework.http.client.InterceptingClientHttpRequest$RequestExecution.execute(InterceptingClientHttpRequest.java:81)
at org.springframework.http.client.InterceptingClientHttpRequest.executeInternal(InterceptingClientHttpRequest.java:67)
at org.springframework.http.client.AbstractBufferingClientHttpRequest.executeInternal(AbstractBufferingClientHttpRequest.java:46)
at org.springframework.http.client.AbstractClientHttpRequest.execute(AbstractClientHttpRequest.java:63)
at org.springframework.web.client.RestTemplate.doExecute(RestTemplate.java:475)
... 14 more
Another similar stacktrace:
org.springframework.web.client.ResourceAccessException: I/O error: null; nested exception is java.io.EOFException
at org.springframework.web.client.RestTemplate.doExecute(RestTemplate.java:490)
at org.springframework.web.client.RestTemplate.execute(RestTemplate.java:438)
at org.springframework.web.client.RestTemplate.exchange(RestTemplate.java:414)
at com.company.api.ApiClient_.logLoginAttempt(ApiClient_.java:299)
at com.company.security.CompanyAuthenticationService$2.onCreateCall(CompanyAuthenticationService.java:206)
at com.company.api.SafeApiCall.doInBackground(SafeApiCall.java:49)
at com.company.api.SafeApiCall.doInBackground(SafeApiCall.java:22)
at android.os.AsyncTask$2.call(AsyncTask.java:287)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:305)
at java.util.concurrent.FutureTask.run(FutureTask.java:137)
at android.os.AsyncTask$SerialExecutor$1.run(AsyncTask.java:230)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1076)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:569)
at java.lang.Thread.run(Thread.java:856)
Caused by: java.io.EOFException
at libcore.io.Streams.readAsciiLine(Streams.java:203)
at libcore.net.http.HttpEngine.readResponseHeaders(HttpEngine.java:560)
at libcore.net.http.HttpEngine.readResponse(HttpEngine.java:813)
at libcore.net.http.HttpURLConnectionImpl.getResponse(HttpURLConnectionImpl.java:274)
at libcore.net.http.HttpURLConnectionImpl.getResponseCode(HttpURLConnectionImpl.java:486)
at org.springframework.http.client.SimpleClientHttpResponse.getRawStatusCode(SimpleClientHttpResponse.java:49)
at org.springframework.http.client.SimpleClientHttpResponse.getStatusCode(SimpleClientHttpResponse.java:55)
at org.springframework.http.client.BufferingClientHttpResponseWrapper.getStatusCode(BufferingClientHttpResponseWrapper.java:47)
at org.springframework.web.client.DefaultResponseErrorHandler.hasError(DefaultResponseErrorHandler.java:46)
at org.springframework.web.client.RestTemplate.doExecute(RestTemplate.java:476)
... 13 more
This is all happening on Android 4.1.2, installed on my Xoom tablet.
The problem appears and disappears. It's not triggered by long requests either. The server part is running on a machine within the local network. When I try to run the API Calls through curl, it works just fine.
AuthTokenInterceptor:
#Override
public ClientHttpResponse intercept(HttpRequest request, byte[] data, ClientHttpRequestExecution execution) throws IOException {
HttpHeaders headers = request.getHeaders();
if (!StringUtils.isEmpty(mAuthToken)) {
headers.add((mIsOAuth ? "Authorization" : "authToken"), (mIsOAuth ? "Bearer " : "") + mAuthToken);
}
return execution.execute(request, data);
}
LoggingClientHttpRequestInterceptor:
/** {#inheritDoc} */
#Override
public ClientHttpResponse intercept(HttpRequest httpRequest, byte[] bytes, ClientHttpRequestExecution clientHttpRequestExecution) throws IOException {
Log.d(TAG, "To : " + httpRequest.getURI());
Log.d(TAG, "Method : " + httpRequest.getMethod().name());
Log.d(TAG, "Data : " + new String(bytes));
for (Object key : httpRequest.getHeaders().keySet()) {
Log.d(TAG, "Header <" + key + ">: " + httpRequest.getHeaders().get(key));
}
final ClientHttpResponse response = clientHttpRequestExecution.execute(httpRequest, bytes);
if (response != null) {
Log.d(TAG, "Response: " + response.getStatusCode());
if (response.getBody() != null) {
Log.d(TAG, "Response: " + convertStreamToString(response.getBody()));
}
} else {
Log.d(TAG, "Response: " + response);
}
return response;
}
The Rest Template is configured like this:
final RestTemplate template = new RestTemplate(false);
template.getMessageConverters().add(new MappingJacksonHttpMessageConverter());
template.setRequestFactory(new BufferingClientHttpRequestFactory(template.getRequestFactory()));
ApiUtils.addAuthTokenHeaderToRestTemplate(template, mAuthToken, false);
ApiUtils.addRequestLoggingToRestTemplate(template);
The API call in question that crashed here is described in the Android annotations based interface:
#Post("/user/memberships")
#Accept(MediaType.APPLICATION_JSON)
CompanyApiResponse saveGroupMembership(UserGroupMembership membership) throws RestClientException;
Things I've tried:
Removed LoggingInterceptor
Called all API calls by CURL
Removed call BufferingClientHttpRequestFactory - Helped a little but the error still occurs.
Tested it on Android 2.3 - the error cannot be reproduced
I've been reading various forums posts, the EOF exception seems to appear if URLs are incorrect, which I double checked in this case.
Also of note, once the EOF Exception occurs, the call not even reaches the server side.
Where would be a good point to continue the search for a fix? Is this a Android 4.1 inconvenience?
While debugging this issue, I also found https://jira.springsource.org/browse/ANDROID-102 which prevented me from seeing the real error (EOF) before.
Update: Just found http://code.google.com/p/google-http-java-client/issues/detail?id=116 - it might be related.
The fix is also outlined in https://codereview.appspot.com/6225045/ - so it might've been merged for 4.1.
This one bit me as well, running Jelly Bean 4.2. After researching, it seems that it's happening because of a combination of Keep-Alive being set and using the standard J2SE HTTP Client, which I believe is HttpURLConnection.
There are 2 solutions that I can confirm are correct.
1) Switch off Keep-Alive.
For me, the solution given in Sebastian's answer, System.setProperty("http.keepAlive", "false"); didn't work. I had to use
HttpHeaders headers = new HttpHeaders();
headers.set("Connection", "Close");
and send those headers in an HttpEntity in the RestTemplate.
As mentioned, this solution could have an impact on performance
2) Change the HTTP Client.
In Spring for Android (tested on 1.0.1.RELEASE, but could be in earlier releases too) the default HTTP Client for a RestTemplate instance is determined by the version of Android on the device. API 9 or newer uses HttpURLConnection, older uses HTTPClient. To explicitly set the client to the old one, use
restTemplate.setRequestFactory(new HttpComponentsClientHttpRequestFactory());
More info can be found here: http://static.springsource.org/spring-android/docs/1.0.1.RELEASE/reference/htmlsingle/#d4e34
I'm not sure what impact this will have on performance, but I guess it's more performant than an app that doesn't work.
Anyway, hope that helps someone. I just wasted a week wild-goose-chasing this one down.
http://code.google.com/p/google-http-java-client/issues/detail?id=116 contains a workaround in the latest comment:
This is defenetly somehow connected with keepAlive connections.
When I use: System.setProperty("http.keepAlive", "false"); problems
disappears.
But from my understanding keep alive connections are greatly increase
performance so it is better not to disable them.
Im also awere that keep alive should be disabled for old versions, but
my device is Jelly Bean.
Once applied the error disappeared.
Seems it's not entirely related to Spring, but a JB problem.
Recently I faced this issue and will able to resolved this issue after setting headers with following piece of code :
headers.set("Accept-Language", "en-US,en;q=0.8");
RestTemplate restTemplate = new RestTemplate();
((SimpleClientHttpRequestFactory)restTemplate.getRequestFactory()).setOutputStreaming(false);
restTemplate.postForObject......
Related
I use okHttp do a POST request, it works when the fields String.length less than about 500.
code like this, a regular way to use okHttp:
Request.Builder builder = new Request.Builder();
builder.url(getHostname() + getUrl());
builder.post(new FormBody.Builder.add("key", "the large string").build())
Call call = mOkHttpClient.newCall(builder.build());
call.enqueue(new Callback() {
#Override
public void onFailure(Call call, IOException e) {
e.printStackTrace();
}
#Override
public void onResponse(Call call, Response response) throws IOException {
String output = response.body().string();
Log.d("okHttp response", output);
}
});
but when this params field length > 500, like 1k, not sure the exactly value, it throws java.net.SocketTimeoutException: timeout:
java.net.SocketTimeoutException: timeout
at okio.SocketAsyncTimeout.newTimeoutException(JvmOkio.kt:143)
at okio.AsyncTimeout.access$newTimeoutException(AsyncTimeout.kt:162)
at okio.AsyncTimeout$source$1.read(AsyncTimeout.kt:335)
at okio.RealBufferedSource.indexOf(RealBufferedSource.kt:427)
at okio.RealBufferedSource.readUtf8LineStrict(RealBufferedSource.kt:320)
at okhttp3.internal.http1.HeadersReader.readLine(HeadersReader.kt:29)
at okhttp3.internal.http1.Http1ExchangeCodec.readResponseHeaders(Http1ExchangeCodec.kt:178)
at okhttp3.internal.connection.Exchange.readResponseHeaders(Exchange.kt:106)
at okhttp3.internal.http.CallServerInterceptor.intercept(CallServerInterceptor.kt:79)
at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.kt:109)
at okhttp3.internal.connection.ConnectInterceptor.intercept(ConnectInterceptor.kt:34)
at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.kt:109)
at okhttp3.internal.cache.CacheInterceptor.intercept(CacheInterceptor.kt:95)
at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.kt:109)
at okhttp3.internal.http.BridgeInterceptor.intercept(BridgeInterceptor.kt:83)
at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.kt:109)
at okhttp3.internal.http.RetryAndFollowUpInterceptor.intercept(RetryAndFollowUpInterceptor.kt:76)
at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.kt:109)
at okhttp3.internal.connection.RealCall.getResponseWithInterceptorChain$okhttp(RealCall.kt:201)
at okhttp3.internal.connection.RealCall$AsyncCall.run(RealCall.kt:517)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1162)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:636)
at java.lang.Thread.run(Thread.java:764)
Caused by: java.net.SocketException: Socket closed
at java.net.SocketInputStream.read(SocketInputStream.java:203)
at java.net.SocketInputStream.read(SocketInputStream.java:139)
at okio.InputStreamSource.read(JvmOkio.kt:90)
at okio.AsyncTimeout$source$1.read(AsyncTimeout.kt:129)
... 20 more
whether I set timeout:
int TIMEOUT = 60;
mOkHttpClient = new OkHttpClient.Builder()
.connectTimeout(TIMEOUT, TimeUnit.SECONDS)
.readTimeout(TIMEOUT,TimeUnit.SECONDS)
.writeTimeout(TIMEOUT, TimeUnit.SECONDS)
.build();
It always throw SocketTimeoutException after the TIMEOUT I set. It's like it not even send data just wait for timeout.
I tried:
1.Extended TIMEOUT setting value, but it just keep posting and throw timeoutException until the end of time.
2.Tested in postman use the same large data, it uploaded and tooks 1s. The field's length is not too long, just about 8k.
3.Updated the okhttp version from 3.12.0 to the last 4.9.1 now.
But not solved this issue.
I continued to test the program and found something new:
This problem only appears on the Android Pad. When I test with my mobile phone, the same data does not timeout.
1.I tested on Huawei P30 pro(android 10) it works, and not works on Samsung Galaxy Tab active pro(android11) and Xiaomi PAD4(android8.1),Im not sure if this only occurs on pad, or it's just a coincidence.
2.I tested with my Samsung pad and set up a proxy with Charles. When I used the proxy to access the network and want to capture the content data, the program was works. If the proxy was cancelled, the problem reappeared.
The issue has been resolved. it is a network configuration problem. After some comparative tests, I tried hotspot, or cellular network, or change another WiFi. This POST API can succeed. Strangely, there are still differences between mobile phones and PAD under the same WiFi. However, I still don't know whether it is the problem of configuration of router's or pad's WiFi.
When using Gluon Maps on android the actual map doesn't get loaded. Instead the empty white space is shown, and I can see this in logs:
05-31 14:20:34.041 E/CachedOsmTileRetriever(18834): null
05-31 14:20:34.041 E/CachedOsmTileRetriever(18834): java.io.FileNotFoundException: http://tile.openstreetmap.org/16/33497/22228.png
05-31 14:20:34.041 E/CachedOsmTileRetriever(18834): at com.android.okhttp.internal.huc.HttpURLConnectionImpl.getInputStream(HttpURLConnectionImpl.java:251)
05-31 14:20:34.041 E/CachedOsmTileRetriever(18834): at com.gluonhq.impl.maps.tile.osm.CachedOsmTileRetriever$CacheThread.doCache(CachedOsmTileRetriever.java:189)
05-31 14:20:34.041 E/CachedOsmTileRetriever(18834): at com.gluonhq.impl.maps.tile.osm.CachedOsmTileRetriever$CacheThread.run(CachedOsmTileRetriever.java:157)
The code itself is trivial:
private void showMap(Double lat, Double lon) {
mapView = new MapView();
PoiLayer poiLayer = new PoiLayer();
MapPoint mapPoint = new MapPoint(lat, lon);
poiLayer.addPoint(mapPoint, new Circle(7, Color.RED));
mapView.setZoom(16);
mapView.addLayer(poiLayer);
mapView.flyTo(0.1, mapPoint, 0.1);
tabMap.setContent(mapView);
}
The same code works just fine on iOS: the map is loaded as expected.
Using
compile 'com.gluonhq:maps:1.0.2'
in build.gradle (same thing with 1.0.3)
Note, that if I enter the URL (from the exception mentioned above) in browser, I get redirected to https:
http://tile.openstreetmap.org/16/33497/22228.png
to
https://tile.openstreetmap.org/16/33497/22228.png
Any ideas why the exception on android?
It looks like something has changed recently from the OpenStreetMaps servers. Gluon Maps fails on desktop as well.
I've tested with http, and I get the reported 403 error:
java.io.IOException: Server returned HTTP response code: 403 for URL: http://tile.openstreetmap.org/5/19/10.png
at java.base/sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1913)
at java.base/sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1509)
at com.gluonhq.maps/com.gluonhq.impl.maps.tile.osm.CachedOsmTileRetriever$CacheThread.doCache(CachedOsmTileRetriever.java:190)
at com.gluonhq.maps/com.gluonhq.impl.maps.tile.osm.CachedOsmTileRetriever$CacheThread.run(CachedOsmTileRetriever.java:157)
and also with https, and now I get a different error:
java.io.IOException: Server returned HTTP response code: 429 for URL: https://tile.openstreetmap.org/6/37/22.png
at java.base/sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1913)
at java.base/sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1509)
at java.base/sun.net.www.protocol.https.HttpsURLConnectionImpl.getInputStream(HttpsURLConnectionImpl.java:245)
at com.gluonhq.maps/com.gluonhq.impl.maps.tile.osm.CachedOsmTileRetriever$CacheThread.doCache(CachedOsmTileRetriever.java:190)
at com.gluonhq.maps/com.gluonhq.impl.maps.tile.osm.CachedOsmTileRetriever$CacheThread.run(CachedOsmTileRetriever.java:157)
In both cases (http or https), the URL works fine on a browser, which might suggest the need of adding "User-agent" to the URLConnection request.
Initially this seemed to solve the issue, as there were no more http errors. However the tiles were blank.
Finally, the solution that fully worked for me, at least on desktop, was this one:
static {
System.setProperty("http.agent", "Gluon Mobile/1.0.3");
}
Since this is a system property, it can be added to the project that uses Gluon Maps, so it can be tested before a new release is done to solve this issue.
Android
On Android it looks like you need to set a valid agent.
Again, this is far from a solution, but it is a quick fix.
Add to your project, and deploy to your Android device:
static {
String userAgent = System.getProperty("http.agent");
System.out.println("HTTP.AGENT: " + userAgent);
System.setProperty("http.agent", userAgent);
}
With your device plugged, open a terminal, go to the Android SDK folder, and type:
cd platform-tools
adb logcat -v threadtime
And now launch the application and see the console for the output.
In my case, I see:
06-10 09:57:40.784 32630 32656 I System.out: HTTP.AGENT: Dalvik/2.1.0 (Linux; U; Android 9; Pixel XL Build/PQ3A.190505.001)
which means Android provides a valid agent, but somehow it has to be explicitly set.
Now the tiles are downloaded.
However, once this is working, I can remove this static block from my app, and install it again fresh. And the tiles are downloaded as well. It looks like it has to be done only once. Maybe the OSM servers include that agent/device/app bundle/IP in a whitelist, but this is just a pure speculation.
iOS
As mentioned, on iOS it works just fine, even if user.agent returns null.
EDIT
Finally, a proper solution can be this:
static {
String httpAgent = System.getProperty("http.agent");
if (httpAgent == null) {
httpAgent = "(" + System.getProperty("os.name") + " / " + System.getProperty("os.version") + " / " + System.getProperty("os.arch") + ")";
}
System.setProperty("http.agent", "Gluon Mobile/1.0.3 " + httpAgent);
}
I would like to know if there are known issues on Android with HttpUrlConnection and POST requests. We are experiencing intermittent EOFExceptions when making POST requests from an Android client. Retrying the same request will eventually work. Here is a sample stack trace:
java.io.EOFException
at libcore.io.Streams.readAsciiLine(Streams.java:203)
at libcore.net.http.HttpEngine.readResponseHeaders(HttpEngine.java:579)
at libcore.net.http.HttpEngine.readResponse(HttpEngine.java:827)
at libcore.net.http.HttpURLConnectionImpl.getResponse(HttpURLConnectionImpl.java:283)
at libcore.net.http.HttpURLConnectionImpl.getResponseCode(HttpURLConnectionImpl.java:497)
at libcore.net.http.HttpsURLConnectionImpl.getResponseCode(HttpsURLConnectionImpl.java:134)
There are many similar bug reports and posts to stack overflow but I cannot understand if there really is an issue and if so, what versions of Android are affected and what the proposed fix/work around is.
Here are some of the similar reports I am referring to:
Android HttpsUrlConnection eofexception
Android HttpURLConnection throwing EOFException
EOFException and FileNotFoundException in HttpURLConnection getInputStream()
https://code.google.com/p/google-http-java-client/issues/detail?id=213
https://code.google.com/p/android/issues/detail?id=29509
https://code.google.com/p/google-http-java-client/issues/detail?id=230
https://code.google.com/p/android/issues/detail?id=41576
Here is a potential Android framework fix
https://android.googlesource.com/platform/libcore/+/19aa40c81c48ff98ccc7272f2a3c41479b806376
I do know there was an issue with poisoned connections in the connection pool in pre-Froyo but these issues are occurring on new ICS+ devices exclusively. If there were a problem on later devices I would expect some kind of official Android documentation of the issue.
Our conclusion is that there is an issue in the Android platform. Our workaround was to catch the EOFException and retry the request N number of times. Below is the pseudo code:
private static final int MAX_RETRIES = 3;
private ResponseType fetchResult(RequestType request) {
return fetchResult(request, 0);
}
private ResponseType fetchResult(RequestType request, int reentryCount) {
try {
// attempt to execute request
} catch (EOFException e) {
if (reentryCount < MAX_RETRIES) {
fetchResult(request, reentryCount + 1);
}
}
// continue processing response
}
HttpURLConnection library internally maintains a pool of Connections. So, whenever a request is send, it first checks if there is an existing connection already present in the pool, based on which it decides to create a new one.
These connections are nothing but sockets, and this library by default does not closes these sockets. It may sometimes happen that a connection (socket) which is not currently being used and is present in the pool is no longer usable as the Server may choose to terminate the connection after some time. Now, since the connection even though is closed by the server, the library does not knows about it and assumes the connection/socket to be still connected. Thus it sends the new request using this stale connection and hence we get EOFException.
The best way to handle this is to check the Response Headers after each request you send. The server always sends a "Connection: Close" before terminating a connection (HTTP 1.1). So, you can use getHeaderField() and check for "Connection" field. Another thing to note is that server ONLY sends this connection field when it is about to terminate the connection. So, you need to code around this with the possibility of getting a "null" in the normal case (when server is not closing the connection)
This workaround tends to be reliable and performant:
static final int MAX_CONNECTIONS = 5;
T send(..., int failures) throws IOException {
HttpURLConnection connection = null;
try {
// initialize connection...
if (failures > 0 && failures <= MAX_CONNECTIONS) {
connection.setRequestProperty("Connection", "close");
}
// return response (T) from connection...
} catch (EOFException e) {
if (failures <= MAX_CONNECTIONS) {
disconnect(connection);
connection = null;
return send(..., failures + 1);
}
throw e;
} finally {
disconnect(connection);
}
}
void disconnect(HttpURLConnection connection) {
if (connection != null) {
connection.disconnect();
}
}
This implementation relies on the fact that the default number of connections that can be opened with a server is 5 (Froyo - KitKat). This means that up to 5 stale connections may exist, each of which will have to be closed.
After each failed attempt, the Connection:close request property will cause the underlying HTTP engine to close the socket when connection.disconnect() is called. By retrying up to 6 times (max connections + 1), we ensure that the last attempt will always be given a new socket.
The request may experience additional latency if no connections are alive, but that is certainly better than an EOFException. In that case, the final send attempt won't immediately close the freshly opened connection. That's the only practical optimization that can be made.
Instead of relying on the magic default value of 5, you may be able to configure the system property yourself. Keep in mind that this property is accessed by a static initializer block in KitKat's ConnectionPool.java, and it works like this in older Android versions too. As a result, the property may be used before you have a chance to set it.
static final int MAX_CONNECTIONS = 5;
static {
System.setProperty("http.maxConnections", String.valueOf(MAX_CONNECTIONS));
}
Yes. There is a problem in the Android platform, specifically, in Android libcore with version 4.1-4.3.
The problem is introduced in this commit: https://android.googlesource.com/platform/libcore/+/b2b02ac6cd42a69463fd172531aa1f9b9bb887a8
Android 4.4 switched http lib to "okhttp" which doesn't have this problem.
Problem explained as follow:
On android 4.1-4.3, when you are using URLConnection/HttpURLConnection to POST with "ChunkedStreamingMode" or "FixedLengthStreamingMode" set, URLConnection/HttpURLConnection will not do silent retry if the reused connection is stale. You should retry POST at most "http.maxConnections+1" times in your code, just as previous answers suggest.
I suspect it might be the server that is at fault here, and the HttpURLConnection is not as forgiving as other implementations. That was the cause of my EOFException. I suspect in my case this would not be intermittent (fixed it before testing the N retry workaround), so the answers above relate to other issues and be a correct solution in those cases.
My server was using python SimpleHTTPServer and I was wrongly assuming all I needed to do to indicate success was the following:
self.send_response(200)
That sends the initial response header line, a server and a date header, but leaves the stream in the state where you are able to send additional headers too. HTTP requires an additional new line after headers to indicate they are finished. It appears if this new line isn't present when you attempt to get the result body InputStream or response code etc with HttpURLConnection then it throws the EOFException (which is actually reasonable, thinking about it). Some HTTP clients did accept the short response and reported the success result code which lead to me perhaps unfairly pointing the finger at HttpURLConnection.
I changed my server to do this instead:
self.send_response(200)
self.send_header("Content-Length", "0")
self.end_headers()
No more EOFException with that code.
This worked for me.
public ResponseObject sendPOST(String urlPrefix, JSONObject payload) throws JSONException {
String line;
StringBuffer jsonString = new StringBuffer();
ResponseObject response = new ResponseObject();
try {
URL url = new URL(POST_URL);
HttpURLConnection connection = (HttpURLConnection) url.openConnection();
connection.setDoInput(true);
connection.setDoOutput(true);
connection.setReadTimeout(10000);
connection.setConnectTimeout(15000);
connection.setRequestMethod("POST");
connection.setRequestProperty("Accept", "application/json");
connection.setRequestProperty("Content-Type", "application/json; charset=UTF-8");
OutputStream os = connection.getOutputStream();
os.write(payload.toString().getBytes("UTF-8"));
os.close();
BufferedReader br = new BufferedReader(new InputStreamReader(connection.getInputStream()));
while ((line = br.readLine()) != null) {
jsonString.append(line);
}
response.setResponseMessage(connection.getResponseMessage());
response.setResponseReturnCode(connection.getResponseCode());
br.close();
connection.disconnect();
} catch (Exception e) {
Log.w("Exception ",e);
return response;
}
String json = jsonString.toString();
response.setResponseJsonString(json);
return response;
}
connection.addRequestProperty("Accept-Encoding", "gzip");
is the answer
for study we have to develop an Android game which is location based. Currently we use OSMDroid to show the map. There are resources (like wood, stone, ...) which the player has to collect. These resources are currently stored in our backend with hardcoded long/lat and will be added with setMarker onto the current map.
To provide this game globally, we want to set the resources dynamically based on the "real" world. So we need different layers from OSM (like forest, sea, ..) to set our resources automatically without asking our backend.
After some hours searching with google I found out that the Overpass API seems to help me implementing this functionality. But I can't find any tutorial for using Overpass API in Android. I tried some things but I don't get it... So I need your help, please give me an example or explanation how to implement this :/
This is my current code, but I don't think that this is correct..
URL url = new URL("http://overpass-api.de/api/interpreter");
HttpURLConnection urlConnection = (HttpURLConnection) url.openConnection();
urlConnection.setRequestMethod("GET");
urlConnection.setDoOutput(true);
urlConnection.connect();
InputStream inputStream = urlConnection.getInputStream();
inputStream.close();
Following exception will be thrown at InputStream inputStream = urlConnection.getInputStream();:
W/System.err(3958): java.io.FileNotFoundException: http://overpass-api.de/api/interpreter
W/System.err(3958): at libcore.net.http.HttpURLConnectionImpl.getInputStream(HttpURLConnectionImpl.java:177)
W/System.err(3958): at de.htw.berlin.games.based.location.gui.MapActivity$test.doInBackground(MapActivity.java:536)
W/System.err(3958): at de.htw.berlin.games.based.location.gui.MapActivity$test.doInBackground(MapActivity.java:1)
W/System.err(3958): at android.os.AsyncTask$2.call(AsyncTask.java:287)
W/System.err(3958): at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:305)
W/System.err(3958): at java.util.concurrent.FutureTask.run(FutureTask.java:137)
W/System.err(3958): at android.os.AsyncTask$SerialExecutor$1.run(AsyncTask.java:230)
W/System.err(3958): at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1076)
W/System.err(3958): at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:569)
W/System.err(3958): at java.lang.Thread.run(Thread.java:856)
Thanks for all helpful replies :)
This exception you're getting is thrown because an HTTP GET call to http://overpass-api.de/api/interpreter returns a 400 Bad Request response.
What you want to do is a POST request to http://overpass-api.de/api/interpreter. An example of form-data to pass to this API is :
data='<?xml version="1.0" encoding="UTF-8"?><osm-script><!--
This is an example Overpass query.
Try it out by pressing the Run button above!
You can find more examples with the Load tool.
-->
<query type="node">
<has-kv k="amenity" v="drinking_water"/>
<bbox-query s="41.88659196260802" w="12.488558292388916" n="41.89248629819397" e="12.51119613647461"/><!--this is auto-completed with the
current map view coordinates.-->
</query>
<print/></osm-script>'
To find out how the API is working you should check, using your browser, what HTTP query is made to the API when clicking on Run in the example I pointed out.
EDIT
You can find plenty of examples like this one that shows how to post data using HTTP in Android. You'll have to add data as a key and the XML query string as a value in the used value pair container, such as :
List<NameValuePair> nameValuePairs = new ArrayList<NameValuePair>(1);
nameValuePairs.add(new BasicNameValuePair("data", _The_XML_Query_String_));
Stick to the linked example for the rest and you hould be fine.
Check out https://github.com/zsoltk/overpasser. It's a Java library to ease working with the Overpass API.
You don't have to write the QL string yourself
It comes with a Retrofit adapter (so you can skip the networking part)
It has an Android sample showing how to put them all together over a Google Maps to get you started in no time
For anyone in future who needs to find a solution with overpass api, here is what I did very often: Overpass API can be addressed with a GET-Request. A GET-Request comes with a HTTP-protocoll and can be used in (I think) every programming language. You have to make a GET-request to the overpass-interpreter with all the queries in the url. In Android it would look like this:
String urlOverpass = "https://overpass-api.de/api/interpreter?data=[out:json][timeout:100];(node[shop=supermarket](52.402957,13.337429,52.420730,13.379530);way[shop=supermarket](52.402957,13.337429,52.420730,13.379530););out%20body;%3E;out%20skel%20qt;";
/* here you speak to the interpreter and you can insert whatever query you need. As an example look at overpass-turbo.eu*/
StringRequest request = new StringRequest(urlOverpass, new Response.Listener<String>() {
#Override
public void onResponse(String string) {
/* Here you can do whatever you like with the data which comes from the interpreter. The "string" is the response.*/
}
}, new Response.ErrorListener() {
#Override
public void onErrorResponse(VolleyError volleyError) {
/* Here you can explain what happens when an error is coming from the interpreter*/
}
});
RequestQueue rQueue = Volley.newRequestQueue(MainActivity.this);
rQueue.add(request);
Dont forget to implement the library: implementation 'com.android.volley:volley:1.1.1'
But there are multiple possibilities to fetch data from an api with a GET-request.
I've been through every related post I can find trying to get to the bottom of this and am no clearer - hoping someone can put me out of my misery…
I am trying to get Android 2.3 to POST over HTTPS via a Proxy. This code works perfectly on 2.2 through a proxy, and perfectly on both 2.2 and 2.3 using HTTPS when not going through a proxy, and in all cases (2.2 and 2.3) i can GET over HTTPS through a proxy. Its just 2.3 POST using HTTPS through a proxy that seems to be the issue. I get the dreaded "broken pipe" error. The error is thrown when I try and read the inputstream response from the connection - presumably because the TCP socket has been closed underneath my stream. I've tried everything I can think of, including using Connection and Proxy-connection headers (setting to both close and keep-alive) and setting big readTimeout numbers (30 seconds). From my relentless googling, I can see there are known issues with SSL on Android 2.3, but I can't seem to find anything that suggests why the POST might be an issue. Wireshark has yielded some results, but given this is SSL if just a little bit tricky to get to the issue.
Has anyone seem this. I'm using HttpsURLConnection as various posts suggest this is more stable that AndroidHttpClient. Here is my code…any help at all invaluable. Thanks
urlConnection.setSSLSocketFactory(factory);
urlConnection.setHostnameVerifier(new AllowAllHostnameVerifier() );
String dateText = "{\"loopParam\":\"" + String.valueOf(d.getHours()) + ":" + String.valueOf(d.getMinutes()) + ":" + String.valueOf(d.getSeconds()) + "\"}";
txtOutput.setText("Sending " + String.valueOf(dateText.length() ) + " bytes of JSON to /pulse/loop" );
urlConnection.addRequestProperty("Content-type", "application/json");
urlConnection.setDoOutput(true);
urlConnection.setRequestProperty("Proxy-connection", "Keep-Alive");
urlConnection.setRequestProperty("Connection", "Keep-Alive");
urlConnection.setDoInput(true);
urlConnection.setUseCaches(false);
urlConnection.setReadTimeout(30000);
urlConnection.setRequestMethod("POST");
DataOutputStream dataOut = new DataOutputStream(urlConnection.getOutputStream());
dataOut.writeBytes(dateText);
dataOut.flush();
BufferedReader bufIn = new BufferedReader(new InputStreamReader(urlConnection.getInputStream()));
String sResponse;
StringBuilder s = new StringBuilder();
//bufIn is null as error as closed urlcConnection
while ((sResponse = bufIn.readLine()) != null) {
s = s.append(sResponse);
}
Error details:
08-May-12 09:09:51 SsliferSnifferActivity Connecting through proxy INFO
08-May-12 09:09:54 SsliferSnifferActivity javax.net.ssl.SSLException: Write error: ssl=0x2d42b8: I/O error during system call, Broken pipe
at org.apache.harmony.xnet.provider.jsse.NativeCrypto.SSL_write(Native Method)
at org.apache.harmony.xnet.provider.jsse.OpenSSLSocketImpl$SSLOutputStream.write(OpenSSLSocketImpl.java:837)
at java.io.OutputStream.write(OutputStream.java:80)
at org.apache.harmony.luni.internal.net.www.protocol.http.HttpURLConnectionImpl.writeRequestHeaders(HttpURLConnectionImpl.java:799)
at org.apache.harmony.luni.internal.net.www.protocol.http.HttpURLConnectionImpl.retrieveResponse(HttpURLConnectionImpl.java:1028)
at org.apache.harmony.luni.internal.net.www.protocol.http.HttpURLConnectionImpl.getResponseCode(HttpURLConnectionImpl.java:726)
at org.apache.harmony.luni.internal.net.www.protocol.https.HttpsURLConnectionImpl.getResponseCode(HttpsURLConnectionImpl.java:110)
at uk.co.flurrished.sslifersniffer.SslifersnifferActivity.makeRequest(SslifersnifferActivity.java:236)
at uk.co.flurrished.sslifersniffer.SslifersnifferActivity.access$2(SslifersnifferActivity.java:148)
at uk.co.flurrished.sslifersniffer.SslifersnifferActivity$2.onClick(SslifersnifferActivity.java:76)
at android.view.View.performClick(View.java:2485)
at android.view.View$PerformClick.run(View.java:9080)
at android.os.Handler.handleCallback(Handler.java:587)
at android.os.Handler.dispatchMessage(Handler.java:92)
at android.os.Looper.loop(Looper.java:130)
at android.app.ActivityThread.main(ActivityThread.java:3822)
at java.lang.reflect.Method.invokeNative(Native Method)
at java.lang.reflect.Method.invoke(Method.java:507)
at com.android.internal.os.ZygoteInit$MethodAndArgsCaller.run(ZygoteInit.java:839)
at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:597)
at dalvik.system.NativeStart.main(Native Method)
EDIT : This looks like it is being caused by the server raising a 400 (Bad Request) and closing the pipe. What is it about ANdroid 2.3 that is adding extra content when routed through a proxy that causes the 400?
'Broken pipe' has exactly one meaning. You have written to a connection that has already been closed by the other end. Are you sure the peer is really speaking SSL?