Develop Reference

Android: HttpUrlConnection object returns error 301 after connection

I'm trying to connect to a web API using a URL. However, I get a 301 error from the server (Moved Permanently), although the provided URL works very well with no errors when I try it in my browser.
Here is the code that builds the URL:
public Loader<List<Earthquake>> onCreateLoader(int i, Bundle bundle) {
SharedPreferences sharedPrefs = PreferenceManager.getDefaultSharedPreferences(this);
String minMagnitude = sharedPrefs.getString(
getString(R.string.settings_min_magnitude_key),
getString(R.string.settings_min_magnitude_default));
String orderBy = sharedPrefs.getString(
getString(R.string.settings_order_by_key),
getString(R.string.settings_order_by_default)
);
Uri baseUri = Uri.parse(USGS_REQUEST_URL);
Uri.Builder uriBuilder = baseUri.buildUpon();
uriBuilder.appendQueryParameter("format", "geojson");
uriBuilder.appendQueryParameter("limit", "10");
uriBuilder.appendQueryParameter("minmag", minMagnitude);
uriBuilder.appendQueryParameter("orderby", orderBy);
Log.i ("the uri is ", uriBuilder.toString());
return new EarthquakeLoader(this, uriBuilder.toString());
}
Here is the code that tries to connect to the resource represented by the URL:
private static String makeHttpRequest(URL url) throws IOException {
String jsonResponse = "";
// If the URL is null, then return early.
if (url == null) {
return jsonResponse;
}
Log.i("The received url is " , url +"");
HttpURLConnection urlConnection = null;
InputStream inputStream = null;
try {
urlConnection = (HttpURLConnection) url.openConnection();
urlConnection.setReadTimeout(10000 /* milliseconds */);
urlConnection.setConnectTimeout(15000 /* milliseconds */);
urlConnection.setRequestMethod("GET");
urlConnection.connect();
// If the request was successful (response code 200),
// then read the input stream and parse the response.
if (urlConnection.getResponseCode() == 200) {
inputStream = urlConnection.getInputStream();
jsonResponse = readFromStream(inputStream);
} else {
Log.e(LOG_TAG, "Error response code: " + urlConnection.getResponseCode()); //this log returns 301
}
} catch (IOException e) {
Log.e(LOG_TAG, "Problem retrieving the earthquake JSON results.", e);
} finally {
if (urlConnection != null) {
urlConnection.disconnect();
}
if (inputStream != null) {
// Closing the input stream could throw an IOException, which is why
// the makeHttpRequest(URL url) method signature specifies than an IOException
// could be thrown.
inputStream.close();
}
}
return jsonResponse;
}
I could know that the connection returns status code of 301 from the log provided in the case when the status code is not 200. I have also logged the generated URL, I copied it from the logcat and tried it in my browser and it worked well. Here is the built URL: http://earthquake.usgs.gov/fdsnws/event/1/query?format=geojson&limit=10&minmag=6&orderby=magnitude
I checked this question: Android HttpURLConnection receives HTTP 301 response code but it wasn't clear to me what is the solution for the problem.
Can you please help me identify and solve the problem?
UPDATE: As greenapps indicated in his comment, the connection is done through https. That comment identified the problem and helped me fix the code.
In my code, the string I used to build the basic URL, had the protocol value as http not https, it was:
private static final String USGS_REQUEST_URL =
"http://earthquake.usgs.gov/fdsnws/event/1/query";
After reading greenapps comment, I just changed the protocol part in the string to https, so it became:
private static final String USGS_REQUEST_URL =
"https://earthquake.usgs.gov/fdsnws/event/1/query";
That solved the problem.
Thanks.

If you click your http link here you will see that the browser shows a https page. You better use that url directly as there is redirection now.

This is because the address http to https transferred.
To avoid this, you need to convert the request address to https.

How to use Socket.IO with server demanding authentication

This is where I got the socketIO files from.
https://github.com/Gottox/socket.io-java-client/tree/master/src/io/socket
I am on the client side.
I know connecting works when the server does not need authentication.
But when it needs authentication (Username and password), I get a handshaking error message.
How do I get passed authentication?? Could it be a server side error? Would the server side of things change if authentication was added?
This is the function that throws an error...I did not write it.
This line is the one causing problems: InputStream stream = connection.getInputStream();
It says it is caused by: java.io.FileNotFoundException: url:80/socket.io/1/
private void handshake() {
URL url;
String response;
URLConnection connection;
try {
setState(STATE_HANDSHAKE);
url = new URL(IOConnection.this.url.toString() + SOCKET_IO_1);
connection = url.openConnection();
if (connection instanceof HttpsURLConnection) {
((HttpsURLConnection) connection)
.setSSLSocketFactory(sslContext.getSocketFactory());
}
connection.setConnectTimeout(connectTimeout);
connection.setReadTimeout(connectTimeout);
/* Setting the request headers */
for (Entry<Object, Object> entry : headers.entrySet()) {
connection.setRequestProperty((String) entry.getKey(),
(String) entry.getValue());
}
InputStream stream = connection.getInputStream();
Scanner in = new Scanner(stream);
response = in.nextLine();
String[] data = response.split(":");
sessionId = data[0];
heartbeatTimeout = Long.parseLong(data[1]) * 1000;
closingTimeout = Long.parseLong(data[2]) * 1000;
protocols = Arrays.asList(data[3].split(","));
} catch (Exception e) {
error(new SocketIOException("Error while handshaking", e));
}
}

Problem solved (sort of), here: Android developpement, Gottox socket.io-java-client: file not fount Exception /socket.io/1/
(try using an earlier version of socket.io - by first deleting socket.io folder from node_modules and then install an older version, e.g., 0.9.16, using this command: npm install socket.io#0.9.16)

How to check for unrestricted Internet access? (captive portal detection)

I need to reliably detect if a device has full internet access, i.e. that the user is not confined to a captive portal (also called walled garden), i.e. a limited subnet which forces users to submit their credentials on a form in order to get full access.
My app is automating the authentication process, and therefore it is important to know that full internet access is not available before starting the logon activity.
The question is not about how to check that the network interface is up and in a connected state. It is about making sure the device has unrestricted internet access as opposed to a sandboxed intranet segment.
All the approaches I have tried so far are failing, because connecting to any well-known host would not throw an exception but return a valid HTTP 200 response code because all requests are routed to the login page.
Here are all the approaches I tried but they all return true instead of false for the reasons explained above:
1:
InetAddress.getByName(host).isReachable(TIMEOUT_IN_MILLISECONDS);
isConnected = true; <exception not thrown>
2:
Socket socket = new Socket();
SocketAddress sockaddr = new InetSocketAddress(InetAddress.getByName(host), 80);
socket.connect(sockaddr, pingTimeout);
isConnected = socket.isConnected();
3:
URL url = new URL(hostUrl));
URLConnection urlConn = url.openConnection();
HttpURLConnection httpConn = (HttpURLConnection) urlConn;
httpConn.setAllowUserInteraction(false);
httpConn.setRequestMethod("GET");
httpConn.connect();
responseCode = httpConn.getResponseCode();
isConnected = responseCode == HttpURLConnection.HTTP_OK;
So, how do I make sure I connected to an actual host instead of the login redirection page? Obviously, I could check the actual response body from the 'ping' host I use but it does not look like a proper solution.

For reference, here is the 'official' method from the Android 4.0.1 AOSP code base:
WifiWatchdogStateMachine.isWalledGardenConnection(). I am including the code below just in case the link breaks in the future.
private static final String mWalledGardenUrl = "http://clients3.google.com/generate_204";
private static final int WALLED_GARDEN_SOCKET_TIMEOUT_MS = 10000;
private boolean isWalledGardenConnection() {
HttpURLConnection urlConnection = null;
try {
URL url = new URL(mWalledGardenUrl); // "http://clients3.google.com/generate_204"
urlConnection = (HttpURLConnection) url.openConnection();
urlConnection.setInstanceFollowRedirects(false);
urlConnection.setConnectTimeout(WALLED_GARDEN_SOCKET_TIMEOUT_MS);
urlConnection.setReadTimeout(WALLED_GARDEN_SOCKET_TIMEOUT_MS);
urlConnection.setUseCaches(false);
urlConnection.getInputStream();
// We got a valid response, but not from the real google
return urlConnection.getResponseCode() != 204;
} catch (IOException e) {
if (DBG) {
log("Walled garden check - probably not a portal: exception "
+ e);
}
return false;
} finally {
if (urlConnection != null) {
urlConnection.disconnect();
}
}
}
This approach relies on a specific URL, mWalledGardenUrl = "http://clients3.google.com/generate_204" always returning a 204 response code. This will work even if DNS has been interfered with since in that case a 200 code will be returned instead of the expected 204. I have seen some captive portals spoofing requests to this specific URL in order to prevent the Internet not accessible message on Android devices.
Google has a variation of this theme: fetching http://www.google.com/blank.html will return a 200 code with a zero-length response body. So if you get a non-empty body this would be another way to figure out that you are behind a walled garden.
Apple has its own URLs for detecting captive portals: when network is up IOS and MacOS devices would connect to an URL like http://www.apple.com/library/test/success.html, http://attwifi.apple.com/library/test/success.html, or http://captive.apple.com/hotspot-detect.html which must return an HTTP status code of 200 and a body containing Success.
NOTE:
This approach will not work in areas with regionally restricted Internet access such as China where the whole country is a walled garden, and where some Google/Apple services might be blocked. Some of these might not be blocked: http://www.google.cn/generate_204, http://g.cn/generate_204, http://gstatic.com/generate_204 or http://connectivitycheck.gstatic.com/generate_204 — yet these all belong to google so not guaranteed to work.

Another possible solution might be to connect via HTTPS and inspect the target certificate. Not sure if walled gardens actually serve the login page via HTTPS or just drop the connections. In either case, you should be able to see that your destination is not the one you expected.
Of course, you also have the overhead of TLS and certificate checks. Such is the price of authenticated connections, unfortunately.

I believe preventing redirection for your connection will work.
URL url = new URL(hostUrl));
HttpURLConnection httpConn = (HttpURLConnection)url.openConnection();
/* This line prevents redirects */
httpConn.setInstanceFollowRedirects( false );
httpConn.setAllowUserInteraction( false );
httpConn.setRequestMethod( "GET" );
httpConn.connect();
responseCode = httpConn.getResponseCode();
isConnected = responseCode == HttpURLConnection.HTTP_OK;
If that doesn't work, then I think the only way to do it is to check the body of the response.

This has been implemented on Android 4.2.2+ version - I find their approach fast and interesting :
CaptivePortalTracker.java detects walled garden as follows
- Try to connect to www.google.com/generate_204
- Check that the HTTP response is 204
If the check fails, we are in a walled garden.
private boolean isCaptivePortal(InetAddress server) {
HttpURLConnection urlConnection = null;
if (!mIsCaptivePortalCheckEnabled) return false;
mUrl = "http://" + server.getHostAddress() + "/generate_204";
if (DBG) log("Checking " + mUrl);
try {
URL url = new URL(mUrl);
urlConnection = (HttpURLConnection) url.openConnection();
urlConnection.setInstanceFollowRedirects(false);
urlConnection.setConnectTimeout(SOCKET_TIMEOUT_MS);
urlConnection.setReadTimeout(SOCKET_TIMEOUT_MS);
urlConnection.setUseCaches(false);
urlConnection.getInputStream();
// we got a valid response, but not from the real google
return urlConnection.getResponseCode() != 204;
} catch (IOException e) {
if (DBG) log("Probably not a portal: exception " + e);
return false;
} finally {
if (urlConnection != null) {
urlConnection.disconnect();
}
}
}

if you are already using retrofit you can do it by retrofit. just make a ping.html page and send an head request to it using retrofit and make sure your http client is configured like below: (followRedirects(false) part is the most important part)
private OkHttpClient getCheckInternetOkHttpClient() {
return new OkHttpClient.Builder()
.readTimeout(2L, TimeUnit.SECONDS)
.connectTimeout(2L, TimeUnit.SECONDS)
.followRedirects(false)
.build();
}
then build your retrofit like below:
private InternetCheckApi getCheckInternetRetrofitApi() {
return (new Retrofit.Builder())
.baseUrl("[base url of your ping.html page]")
.addConverterFactory(GsonConverterFactory.create(new Gson()))
.client(getCheckInternetOkHttpClient())
.build().create(InternetCheckApi.class);
}
your InternetCheckApi.class would be like:
public interface InternetCheckApi {
#Headers({"Content-Typel: application/json"})
#HEAD("ping.html")
Call<Void> checkInternetConnectivity();
}
then you can use it like below:
getCheckInternetOkHttpClient().checkInternetConnectivity().enqueue(new Callback<Void>() {
public void onResponse(Call<Void> call, Response<Void> response) {
if(response.code() == 200) {
//internet is available
} else {
//internet is not available
}
}
public void onFailure(Call<Void> call, Throwable t) {
//internet is not available
}
}
);
note that your internet check http client must be separate from your main http client.

This is best done here as in AOSP :
https://github.com/aosp-mirror/platform_frameworks_base/blob/6bebb8418ceecf44d2af40033870f3aabacfe36e/core/java/android/net/captiveportal/CaptivePortalProbeResult.java#L61
https://github.com/aosp-mirror/platform_frameworks_base/blob/e3a0f42e8e8678f6d90ddf104d485858fbb2e35b/services/core/java/com/android/server/connectivity/NetworkMonitor.java
private static final String GOOGLE_PING_URL = "http://google.com/generate_204";
private static final int SOCKET_TIMEOUT_MS = 10000;
public boolean isCaptivePortal () {
try {
URL url = new URL(GOOGLE_PING_URL);
urlConnection = (HttpURLConnection) url.openConnection();
urlConnection.setConnectTimeout(SOCKET_TIMEOUT_MS);
urlConnection.setReadTimeout(SOCKET_TIMEOUT_MS);
urlConnection.setUseCaches(false);
urlConnection.getInputStream();
return (urlConnection.getResponseCode() != 204)
&& (urlConnection.getResponseCode() >= 200)
&& (urlConnection.getResponseCode() <= 399);
} catch (Exception e) {
// for any exception throw an exception saying check was unsuccesful
} finally {
if (urlConnection != null) {
urlConnection.disconnect();
}
}
}
Please note this will probably not work on a proxy network and something more advanced as in the AOSP url needs to be done

Android: HttpURLConnection redirect - doInBackground

I'm using HttpURLConnection to do communication with a backend server and im doing so in an async task in the doInBackground method as you should.
Now I need to be able to follow 302 redirects, but I'm having some problems with this. The issue is that the new location usually will be on another host, however when doing the redirect request it seem not to change the URL to a new host hence I get a 404 error saying the specified path does not exits.
Now I know I could set HtppURLConnection.setFollowRedirect but I need to have more control over the redirects so they should not just be followed blindly. The Redirect behavour should be controlled by the object who called the asynctask (when an asynctask object is created you pass the object who creates it in a parameter called _callback).
Heres's my current code:
protected HttpResponse doInBackground(String... req) {
HttpURLConnection urlConnection = null;
try {
urlConnection = (HttpURLConnection) this._url.openConnection();
urlConnection.setConnectTimeout( (int) this._timeout*1000);
String body = req[0];
// set headers / write information to output stream if request is post
// create the response object
HttpResponse responseObject = null;
try
{
// get status, contenttype, charset...
InputStream in = null;
if (urlConnection.getResponseCode() != -1 && urlConnection.getResponseCode() < 300)
{
in = new BufferedInputStream(urlConnection.getInputStream(), 8192);
}
else
{
in = new BufferedInputStream(urlConnection.getErrorStream(), 8192);
}
responseObject = new HttpResponse(in, status, contentType, charset);
// if redirect
if (status == 302 && this._callback.onRedirect(responseObject) == true)
{
// recall
String url = urlConnection.getHeaderField("location");
Log.v("Async Task", "Redirect location: " + url);
this._url = null;
this._url = new URL(url);
urlConnection.disconnect();
urlConnection = null;
responseObject = this.doInBackground(req);
}
} catch (IOException e)
{
// TODO Auto-generated catch block
e.printStackTrace();
}
// return the response
return responseObject;
}
// catch some other exceptions
finally
{
if (urlConnection != null)
{
urlConnection.disconnect();
} }
}
And as said the problem is that the redirect request seem to change the path of the URL but not the host. The URL object itself seem to contain the right information so I have no idea why this is happening. (I'm getting HTML as response which is an 404 error page that includes the server name of the old server)
Thanks for any help!
Note: HttpResponse is just an object I created for holding the relevant information about the response.

This was caused by the fact that I sent the same headers and did not change the "host" header of the request which caused Apache to be confused it seems.

Wamp server to android physical device connection?

I have a wamp server. I have written my android client. If I run that app, the response is fine on an emulator...but the same code does not work on the real device, I mean I dont get a response.....
Here s the code...
public static final String SERVER_URL = "http://192.168.1.3/AndroidListServer/server.php?command=getAnimalList";
private static String executeHttpRequest(String data) {
String result = "";
try {
URL url = new URL(SERVER_URL);
URLConnection connection = url.openConnection();
/*
* We need to make sure we specify that we want to provide input and
* get output from this connection. We also want to disable caching,
* so that we get the most up-to-date result. And, we need to
* specify the correct content type for our data.
*/
connection.setDoInput(true);
connection.setDoOutput(true);
connection.setUseCaches(false);
connection.setRequestProperty("Content-Type", "application/x-www-form-urlencoded");
// Send the POST data
DataOutputStream dataOut = new DataOutputStream(connection.getOutputStream());
dataOut.writeBytes(data);
dataOut.flush();
dataOut.close();
// get the response from the server and store it in result
DataInputStream dataIn = new DataInputStream(connection.getInputStream());
String inputLine;
while ((inputLine = dataIn.readLine()) != null) {
result += inputLine;
}
dataIn.close();
} catch (IOException e) {
/*
* In case of an error, we're going to return a null String. This
* can be changed to a specific error message format if the client
* wants to do some error handling. For our simple app, we're just
* going to use the null to communicate a general error in
* retrieving the data.
*/
e.printStackTrace();
result = null;
}
return result;
}

Solved it guys....it was the problem with the firewall as Rajesh mentioned...I should do a thorough testing of all the possible parameters....but hey I'm learning :)