Using Robolectric 2.3-SNAPSHOT, I want to test an object that'll execute a request in the background. In order to isolate it, I'm trying to mock the HttpResponse returned, without success after some hours invested.
I've created a project that anyone can clone. Simly run ./gradlew check https://github.com/Maragues/RobolectricDummyProject (git clone https://github.com/Maragues/RobolectricDummyProject.git)
I've tried
Robolectric.setDefaultHttpResponse(200, "my_mocked_word");
MockWebServer (https://code.google.com/p/mockwebserver/)
But the tests fail because they query the real URL
private static final String MOCKED_WORD = "MOCKED";
#Test
public void mockedRequestUsingMockServer() throws Exception {
mMockWebServer.enqueue(new MockResponse().setBody(MOCKED_WORD));
mMockWebServer.play();
Robolectric.getFakeHttpLayer().interceptHttpRequests(false);
Robolectric.getFakeHttpLayer().interceptResponseContent(false);
String result = request.loadDataFromNetwork();
assertEquals(MOCKED_WORD, result);
}
#Test
public void mockedRequestUsingRobolectric() throws Exception {
Robolectric.setDefaultHttpResponse(200, MOCKED_WORD);
String result = request.loadDataFromNetwork();
assertEquals(MOCKED_WORD, result);
}
The code executing the request
public String loadDataFromNetwork() throws Exception {
// With Uri.Builder class we can build our url is a safe manner
Uri.Builder uriBuilder = Uri.parse("http://robospice-sample.appspot.com/reverse").buildUpon();
uriBuilder.appendQueryParameter("word", word);
String url = uriBuilder.build().toString();
HttpURLConnection urlConnection = (HttpURLConnection) new URL(url)
.openConnection();
String result = IOUtils.toString(urlConnection.getInputStream());
urlConnection.disconnect();
return result;
}
Possibly related questions
Can't capture HTTP request with robolectric (I've tried that without success. Perhaps I'm missing something)
Anyone had success mocking HttpRequests with Robolectric? (I'm not using eclipse)
You're disabling Roboelectric's HTTP layer, so you're using the real HTTP layer. This means that there's no clever magic happening under the hood of your test: when you send an HTTP request, it's really going out onto the internet (as you are seeing).
MockWebServer doesn't stop this. It just sets up a web server locally, that your test can connect to.
So to resolve this problem, you need to stop attempting to connect to a real server, and instead, connect to the mock server. To do this, yo need to inject/set the URL in the request.
#Test
public void mockedRequestUsingMockServer() throws Exception {
mMockWebServer = new MockWebServer();
mMockWebServer.play();
mMockWebServer.enqueue(new MockResponse().setResponseCode(200).setBody(MOCKED_WORD));
request.myUrl = mMockWebServer.getUrl("/");
String result = request.loadDataFromNetwork();
assertEquals(MOCKED_WORD, result);
mMockWebServer.shutdown();
}
It turns out that Robolectric's FakeHttpLayer only works with Apache's HttpClient, which is highly discouraged on versions greater than Froyo. Extracted from Robolectric's Google Group
That being said, the usage of HttpUrlConnection will cause you trouble. I'd try to use Android's implementation of HttpClient where possible, since Robolectric intercepts all calls to that library and lets you set up canned responses to your HTTP calls. We're looking at doing the same for HttpUrlConnection, though it's not clear when that'll happen.
Apart from that, a unit test should not need to mock the HTTP layer. My approach was wrong from the beginning.
You can try this(ref:https://github.com/square/okhttp/tree/master/mockwebserver).
// Create a MockWebServer. These are lean enough that you can create a new
// instance for every unit test.
MockWebServer server = new MockWebServer();
// Schedule some responses.
server.enqueue(new MockResponse().setBody("it's all cool"));
// Start the server.
server.play();
// Ask the server for its URL. You'll need this to make HTTP requests.
//Http is my own http executor.
Http.Response response = http.get(server.getUrl("/"));
then, you can compare the response to server.enqueue(new MockResponse().setBody("it's all cool"));
MockWebServer is a part of okhttp https://github.com/square/okhttp/tree/master/mockwebserver. the URLConnectionImpl in android 4.4 have been changed from defaultHttpClient to Okhttp.
Related
I am using mockwebserver to mock request and response for my android app. I am testing a login feature which goes through a series of 4 service calls.
Get access token
Re-direct
Get user info (different base url)
Get some other stuff (original base url)
I am trying to mock the response of the redirected call. Here is my code:
#Test
public void testSuccessfulLogin() throws Exception {
// Post
server.enqueue(new MockResponse()
.setResponseCode(HTTP_OK)
.setBody(getStringFromFile(getInstrumentation().getContext(), "access_token.json")));
// Redirect
server.enqueue(new MockResponse().setResponseCode(HTTP_MOVED_TEMP));
// GET user info
server.enqueue(new MockResponse().setResponseCode(HTTP_OK).setBody(getStringFromFile(getInstrumentation().getContext(), "userinfo.json")));
// GET some other stuff
server.enqueue(new MockResponse().setResponseCode(HTTP_OK)
.setBody(getStringFromFile(getInstrumentation().getContext(), "sts.json")));
// Init call
loginWithoutWaiting(Data.serviceLoginUsername, Data.serviceLoginPassword);
// Debug (need to loop 4 times to get all 4 call paths)
RecordedRequest request = server.takeRequest();
request.getPath();
}
My test fails at the Redirect code. I cannot login. I have found some hints here but I do not fully understand what is going on, thus can't make it work at the moment.
It turned out to be quite easy. In the call that makes redirect, create a new mocked response with response code 302 and header with location url. The next call will use that location url.
case "/userinfo":
return new MockResponse().setResponseCode(HTTP_MOVED_TEMP).setHeader("Location", "/api-test.com/users");
case "/api-test.com/users":
return new MockResponse().setBody("{}")).setResponseCode(HTTP_OK);
I am using retrofit for http calls in my Android application and retrofit.client.UrlConnectionClient as the client while building the adapter.
RestAdapter.Builder builder = new RestAdapter.Builder()
.setEndpoint(url)
.setLogLevel(RestAdapter.LogLevel.FULL)
.setClient(
new Client.Provider() {
public Client get() {
return new UrlConnectionClient() {
#Override
protected HttpURLConnection openConnection(Request request)
throws IOException {
HttpURLConnection connection = super.openConnection(request);
connection.setConnectTimeout(connectionTimeout);
return connection;
}
I wanted to set the timeout so I have used UrlConnectionClient as my client. I could not find a way with other clients like OkHttp.
Question is : How can I cancel the ongoing request ?.
I have seen a similar post # Using Square's Retrofit Client, is it possible to cancel an in progress request? If so how? but my code would get really complex if I try to add my own executors and try to cancel the request using that. I am looking if there is slightly a better way with my existing code.
I also see that Retorofit V2.0 has plan for Retry and Cancel but not sure when that would be released..https://github.com/square/retrofit/issues/297
Need help !
In fact I also need a way to retry with the same code.
This has been available since 2.0.0-beta2 (https://github.com/square/retrofit/releases/tag/parent-2.0.0-beta2). I don't know if there is a doc that explains that but here is the link to API:
http://square.github.io/retrofit/2.x/retrofit/retrofit/Call.html#cancel--
'Call' API allows to do Retry as well by 'Clone'ing the request.
Spoiled by Python (e.g. requests.post(url, data={'p1':'v1','p2':'v2'}, headers={'H1': 'V1'})), I am looking for an equivalent to use on Android.
My code is already run on separate Threads, so I don't need AsyncWhatever.
HttpURLConnection is recommended for Android because it is all kinds of lightweight, but the equivalent of the Python request is... large.
I see multiple partial (e.g. pre-encoded params string) solutions, and several problems to work around. Rather than risk mistakes or overlooking something writing it myself, I ask:
Is such a wrapper available already, a request(Method, String, Map<String,String>, Map<String,String>), or similar?
I expect such a solution to need little code per call, and manage all weirdness (e.g. pre-Froyo keepAlive hack) itself.
I suppose I have the right solution for you. Had the same problem (spoiled by Node.js request) and didn't like the "interface" of HttpURLConnection.
You can find a tiny library without dependencies which wrappes HttpURLConnection in a way that not so common use cases can be implemented by using the wrapped HUC. It's called DavidWebb. There is also a link to alternative libraries in case you miss something.
A typical POST request with JSON payload and JSON response with some headers would look like this:
JSONObject postObj = new JSONObject();
postObj.put("p1", "v1");
postObj.put("p2", "v2");
Webb webb = Webb.create();
Response<JSONObject> response = webb
.post("http://www.example.com/app/myresource")
.header("x-date-header", new Date())
.header("x-other-header", "some-text")
.body(postObj)
.connectTimeout(3000)
.asJsonObject();
if (response.isSuccess()) {
JSONObject outcome = response.getBody();
// ...
} else {
System.out.println(response.getStatusCode());
System.out.println(response.getResponseMessage());
System.out.println(response.getErrorBody());
}
There are many ways to set default values for the Webb instance and use or overwrite them in the Request object.
The library is well tested, especially on Android, but it has no specific code or dependencies on Android API, so you have to manage keepAlive for Froyo by yourself.
I would like to know if there are known issues on Android with HttpUrlConnection and POST requests. We are experiencing intermittent EOFExceptions when making POST requests from an Android client. Retrying the same request will eventually work. Here is a sample stack trace:
java.io.EOFException
at libcore.io.Streams.readAsciiLine(Streams.java:203)
at libcore.net.http.HttpEngine.readResponseHeaders(HttpEngine.java:579)
at libcore.net.http.HttpEngine.readResponse(HttpEngine.java:827)
at libcore.net.http.HttpURLConnectionImpl.getResponse(HttpURLConnectionImpl.java:283)
at libcore.net.http.HttpURLConnectionImpl.getResponseCode(HttpURLConnectionImpl.java:497)
at libcore.net.http.HttpsURLConnectionImpl.getResponseCode(HttpsURLConnectionImpl.java:134)
There are many similar bug reports and posts to stack overflow but I cannot understand if there really is an issue and if so, what versions of Android are affected and what the proposed fix/work around is.
Here are some of the similar reports I am referring to:
Android HttpsUrlConnection eofexception
Android HttpURLConnection throwing EOFException
EOFException and FileNotFoundException in HttpURLConnection getInputStream()
https://code.google.com/p/google-http-java-client/issues/detail?id=213
https://code.google.com/p/android/issues/detail?id=29509
https://code.google.com/p/google-http-java-client/issues/detail?id=230
https://code.google.com/p/android/issues/detail?id=41576
Here is a potential Android framework fix
https://android.googlesource.com/platform/libcore/+/19aa40c81c48ff98ccc7272f2a3c41479b806376
I do know there was an issue with poisoned connections in the connection pool in pre-Froyo but these issues are occurring on new ICS+ devices exclusively. If there were a problem on later devices I would expect some kind of official Android documentation of the issue.
Our conclusion is that there is an issue in the Android platform. Our workaround was to catch the EOFException and retry the request N number of times. Below is the pseudo code:
private static final int MAX_RETRIES = 3;
private ResponseType fetchResult(RequestType request) {
return fetchResult(request, 0);
}
private ResponseType fetchResult(RequestType request, int reentryCount) {
try {
// attempt to execute request
} catch (EOFException e) {
if (reentryCount < MAX_RETRIES) {
fetchResult(request, reentryCount + 1);
}
}
// continue processing response
}
HttpURLConnection library internally maintains a pool of Connections. So, whenever a request is send, it first checks if there is an existing connection already present in the pool, based on which it decides to create a new one.
These connections are nothing but sockets, and this library by default does not closes these sockets. It may sometimes happen that a connection (socket) which is not currently being used and is present in the pool is no longer usable as the Server may choose to terminate the connection after some time. Now, since the connection even though is closed by the server, the library does not knows about it and assumes the connection/socket to be still connected. Thus it sends the new request using this stale connection and hence we get EOFException.
The best way to handle this is to check the Response Headers after each request you send. The server always sends a "Connection: Close" before terminating a connection (HTTP 1.1). So, you can use getHeaderField() and check for "Connection" field. Another thing to note is that server ONLY sends this connection field when it is about to terminate the connection. So, you need to code around this with the possibility of getting a "null" in the normal case (when server is not closing the connection)
This workaround tends to be reliable and performant:
static final int MAX_CONNECTIONS = 5;
T send(..., int failures) throws IOException {
HttpURLConnection connection = null;
try {
// initialize connection...
if (failures > 0 && failures <= MAX_CONNECTIONS) {
connection.setRequestProperty("Connection", "close");
}
// return response (T) from connection...
} catch (EOFException e) {
if (failures <= MAX_CONNECTIONS) {
disconnect(connection);
connection = null;
return send(..., failures + 1);
}
throw e;
} finally {
disconnect(connection);
}
}
void disconnect(HttpURLConnection connection) {
if (connection != null) {
connection.disconnect();
}
}
This implementation relies on the fact that the default number of connections that can be opened with a server is 5 (Froyo - KitKat). This means that up to 5 stale connections may exist, each of which will have to be closed.
After each failed attempt, the Connection:close request property will cause the underlying HTTP engine to close the socket when connection.disconnect() is called. By retrying up to 6 times (max connections + 1), we ensure that the last attempt will always be given a new socket.
The request may experience additional latency if no connections are alive, but that is certainly better than an EOFException. In that case, the final send attempt won't immediately close the freshly opened connection. That's the only practical optimization that can be made.
Instead of relying on the magic default value of 5, you may be able to configure the system property yourself. Keep in mind that this property is accessed by a static initializer block in KitKat's ConnectionPool.java, and it works like this in older Android versions too. As a result, the property may be used before you have a chance to set it.
static final int MAX_CONNECTIONS = 5;
static {
System.setProperty("http.maxConnections", String.valueOf(MAX_CONNECTIONS));
}
Yes. There is a problem in the Android platform, specifically, in Android libcore with version 4.1-4.3.
The problem is introduced in this commit: https://android.googlesource.com/platform/libcore/+/b2b02ac6cd42a69463fd172531aa1f9b9bb887a8
Android 4.4 switched http lib to "okhttp" which doesn't have this problem.
Problem explained as follow:
On android 4.1-4.3, when you are using URLConnection/HttpURLConnection to POST with "ChunkedStreamingMode" or "FixedLengthStreamingMode" set, URLConnection/HttpURLConnection will not do silent retry if the reused connection is stale. You should retry POST at most "http.maxConnections+1" times in your code, just as previous answers suggest.
I suspect it might be the server that is at fault here, and the HttpURLConnection is not as forgiving as other implementations. That was the cause of my EOFException. I suspect in my case this would not be intermittent (fixed it before testing the N retry workaround), so the answers above relate to other issues and be a correct solution in those cases.
My server was using python SimpleHTTPServer and I was wrongly assuming all I needed to do to indicate success was the following:
self.send_response(200)
That sends the initial response header line, a server and a date header, but leaves the stream in the state where you are able to send additional headers too. HTTP requires an additional new line after headers to indicate they are finished. It appears if this new line isn't present when you attempt to get the result body InputStream or response code etc with HttpURLConnection then it throws the EOFException (which is actually reasonable, thinking about it). Some HTTP clients did accept the short response and reported the success result code which lead to me perhaps unfairly pointing the finger at HttpURLConnection.
I changed my server to do this instead:
self.send_response(200)
self.send_header("Content-Length", "0")
self.end_headers()
No more EOFException with that code.
This worked for me.
public ResponseObject sendPOST(String urlPrefix, JSONObject payload) throws JSONException {
String line;
StringBuffer jsonString = new StringBuffer();
ResponseObject response = new ResponseObject();
try {
URL url = new URL(POST_URL);
HttpURLConnection connection = (HttpURLConnection) url.openConnection();
connection.setDoInput(true);
connection.setDoOutput(true);
connection.setReadTimeout(10000);
connection.setConnectTimeout(15000);
connection.setRequestMethod("POST");
connection.setRequestProperty("Accept", "application/json");
connection.setRequestProperty("Content-Type", "application/json; charset=UTF-8");
OutputStream os = connection.getOutputStream();
os.write(payload.toString().getBytes("UTF-8"));
os.close();
BufferedReader br = new BufferedReader(new InputStreamReader(connection.getInputStream()));
while ((line = br.readLine()) != null) {
jsonString.append(line);
}
response.setResponseMessage(connection.getResponseMessage());
response.setResponseReturnCode(connection.getResponseCode());
br.close();
connection.disconnect();
} catch (Exception e) {
Log.w("Exception ",e);
return response;
}
String json = jsonString.toString();
response.setResponseJsonString(json);
return response;
}
connection.addRequestProperty("Accept-Encoding", "gzip");
is the answer
I am currently playing with Spring Android Resttemplate to interact with a java-backed REST API. Actually, I am using android annotations to send http calls to this back-end service and I must say it rocks. Basically, Android annotations allows you to define an interface for the service calls and the http methods to be used for each api call available : it will generate all the boiler-plate code related to low-level stuff like marshalling/unmarshalling, calling the right http method according to the interface definition.
Now, I would like to set some headers to http requests : How can I achieve this knowing that I only have a reference to the Service interface defining all the calls ?
I can also have reference to the RestTemplate object but it seems there is now way of setting the headers.
any help would really be appreciated
thanks
The way I approached it is by creating an instance of ApiClient in the application class and set a custom REST template.
In my case I was using Jackson for JSON message conversion:
RestTemplate restTemplate = new RestTemplate(fac);
MappingJacksonHttpMessageConverter converter =
new MappingJacksonHttpMessageConverter();
converter.getObjectMapper().configure(Feature.UNWRAP_ROOT_VALUE, true);
restTemplate
.getMessageConverters()
.add(converter);
mClient.setRestTemplate(restTemplate);
My request factory fac then looks like this:
ClientHttpRequestFactory fac = new HttpComponentsClientHttpRequestFactory() {
#Override
protected HttpUriRequest createHttpRequest(HttpMethod httpMethod, URI uri) {
HttpUriRequest uriRequest = super.createHttpRequest(httpMethod, uri);
// Add request headers
uriRequest.addHeader(
"Content-Type",
MediaType.APPLICATION_JSON_VALUE);
return uriRequest;
}
#Override
public ClientHttpRequest createRequest(URI uri, HttpMethod httpMethod)
throws IOException {
if (Config.DEBUG_REQUESTS) {
Log.d(TAG, uri);
}
return super.createRequest(uri, httpMethod);
}
};
WARNING
Although this works on all Android devices in our office, I've recently discovered that headers don't appear to be added with all devices! I'm not sure why this is (or which devices specifically), but I'm looking in to it and will try to update this answer when I've found a resolution.