Develop Reference

HttpClient request to HTTPS host goes to default nameserver

I am using Apache httpClient library in my android project to send get/post requests to the server. My server is set up using apache namevirtualhost - there are multiple virtual hosts on the same server.
For those not familiar with how apache namevirtualhost works, the simple explanation is that there are multiple configurations defined in the server config, and apache uses Host request header to determine which configuration to use. When multiple hosts are defined, the first one is considered default and all requests with Host request header not explicitly matching one of the defined namevirtualhosts will be handled using the default configuration.
Now, the server I'm trying to connect to is not the default one. When my app runs and the request is made to the server, it does not go to my virtual host but instead is handled by the default one, resulting in the certificate name mismatch. (Note that I do have the correct certificates set up.)
Here's my code - copy/paste, except for the actual URL:
String targetUrl = getTargetUrl();
//this returns something like https://www.example.com/api/1/orders (without spaces, of course)
HttpClient client = new DefaultHttpClient();
HttpPost post = new HttpPost(targetUrl);
List<NameValuePair> data = new ArrayList<NameValuePair>(1);
data.add(new BasicNameValuePair("orders", json.toString()));
data.add(new BasicNameValuePair("username", username));
data.add(new BasicNameValuePair("password", password));
post.setEntity(new UrlEncodedFormEntity(data));
HttpResponse response = client.execute(post);
This results in an error message saying that certificate name doesn't match the requested url and shows the requested url and the certificate url - the certificate url is the default one in the apache config.
I added this code right before the line to execute the request:
Header[] hds = post.getAllHeaders();
for(Header h : hds) {
Log.d("PasteneOrers", h.getName() + ": " + h.getValue());
}
to see what headers are included. Interesting, Host header is not shown. I then added this code before executing the request:
URL url = new URL(targetUrl);
post.setHeader("Host", url.getHost());
Now the above debugging output correctly shows the Host header - but it doesn't help and the request is still going to the default one.
To verify that this is not a problem with the server misconfiguration, I copied the target URL and pasted it into the browser running on the same android emulator - this works correctly and I get the right results. Hence it's definitely something in my code - but what? At this point I'm stuck. Any suggestions would be appreciated.

In the interest of others having this problem, I couldn't resolve it with apache httpclient. I switched to using HttpUrlConnection and everything worked correctly.

Android app getting data from a third-party JSP & Servlet

I'm writing an Android app that should get data from a certain web application. That web app is based on Servlets and JSP, and it's not mine; it's a public library's service. What is the most elegant way of getting this data?
I tried writing my own Servlet to handle requests and responses, but I can't get it to work. Servlet forwarding cannot be done, due to different contexts, and redirection doesn't work either, since it's a POST method... I mean, sure, I can write my own form that access the library's servlet easily enough, but the result is a jsp page.. Can I turn that page into a string or something? Somehow I don't think I can.. I'm stuck.
Can I do this in some other way? With php or whatever? Or maybe get that jsp page on my web server, and then somehow extract data from it (with jQuery maybe?) and send it to Android? I really don't want to display that jsp page in a browser to my users, I would like to take that data and create my own objects with it..

Just send a HTTP request programmatically. You can use Android's builtin HttpClient API for this. Or, a bit more low level, the Java's java.net.URLConnection (see also Using java.net.URLConnection to fire and handle HTTP requests). Both are capable of sending GET/POST requests and retrieving the response back as an InputStream, byte[] or String.
At most simplest, you can perform a GET as follows:
InputStream responseBody = new URL("http://example.com").openStream();
// ...
A POST is easier to be performed with HttpClient:
List<NameValuePair> params = new ArrayList<NameValuePair>(2);
params.add(new BasicNameValuePair("name1", "value1"));
params.add(new BasicNameValuePair("name2", "value2"));
HttpPost post = new HttpPost("http://example.com");
post.setEntity(new UrlEncodedFormEntity(params));
HttpClient client = new DefaultHttpClient();
HttpResponse response = client.execute(post);
InputStream responseBody = response.getEntity().getContent();
// ...
If you need to parse the response as HTML (I'd however wonder if that "public library service" (is it really public?) doesn't really offer XML or JSON services which are way much easier to parse), Jsoup may be a life saver as to traversing and manipulating HTML the jQuery way. It also supports sending POST requests by the way, only not as fine grained as with HttpClient.

Submit a web form in android programatically

I am trying to log into a site and load a webpage programatically in android. Meaning, I have the password and login and need to submit a webform and get the response page. I tried the code here: Doing HTTP Post with Android
but I think I may be doing it wrong.
If this is the site I'm trying to access: http://goo.gl/eiBhP
and my code is
HttpClient httpclient = new DefaultHttpClient(httpParameters);
HttpPost httppost = new HttpPost(Constants.MAIN_URL);
List<namevaluepair> nameValuePairs = new ArrayList<namevaluepair>(2);
nameValuePairs.add(new BasicNameValuePair("username", "correctusername"));
nameValuePairs.add(new BasicNameValuePair("password", "correctpassword"));
httppost.setEntity(new UrlEncodedFormEntity(nameValuePairs));
HttpResponse response = httpClient.execute(httpost);
Then I should be able to use
BufferedReader br = new BufferedReader(new InputStreamReader(
entity.getContent()), 8096);
to get the response. The id of the login and pass on the site ate username and password. should I also somehow submit the button as a name value pair? I cant seem to get this to work, it just returns the login page. Please Help. I've tried reading over the other similar questions but I can't seem to get it to work.

Basically, you need to make sure that your code is submitting exactly the same information as the webpage.
As Selvin points out, there's a good chance that the website is using some form of tracking - be it in hidden input values, cookies or some other state-based data.
You need to look at the source of the login webpage and understand what it is doing when you submit login details - you don't necessarily need to know what all the values mean, but your code must submit the same POST data.
If the website is using state information, you won't be able to hard-code those input values in your code. You'll probably need to retrieve a new instance of the login webpage each time using a HTTP GET request and then parse the data to extract the relevant state data. Don't forget that they may also be using cookies which you may need to submit.
All in all, you probably need to do a lot more work to get it to a working state. Not trying to dissuade you (and I don't know what you're trying to achieve), but perhaps it's easier just to use the website!

rails Devise http authenticating mobile

I'm trying to authenticate an android client app to my server ruby on rails app which uses Devise gem. But I've tried http authentication, and post requests to authenticate, and the server just responds 200 for any given username/password.
I've already set up the config.http_authenticatable = true and the :database_authenticable at the user model...
I'll post my authenticate method so u guys can have a look on it...
public static boolean authenticate(User user, String verb) throws IOException, JSONException
{
DefaultHttpClient httpClient = new DefaultHttpClient();
HttpPost httpPost = new HttpPost(verb);
CredentialsProvider credProvider = new BasicCredentialsProvider();
credProvider.setCredentials(new AuthScope(AuthScope.ANY_HOST, AuthScope.ANY_PORT),
new UsernamePasswordCredentials(user.getMail(), user.getPassword()));
httpClient.setCredentialsProvider(credProvider);
List<NameValuePair> nameValuePairs = new ArrayList<NameValuePair>();
nameValuePairs.add(new BasicNameValuePair("email", user.getMail()));
nameValuePairs.add(new BasicNameValuePair("password", user.getPassword()));
httpPost.setEntity(new UrlEncodedFormEntity(nameValuePairs));
HttpResponse httpResponse = httpClient.execute(httpPost);
int statusCode = httpResponse.getStatusLine().getStatusCode();
//JSONObject resp = null;
if (statusCode < 200 || statusCode >= 300){
throw new IOException("Error");
}
return true;
}

If server is responding 200, it really sounds like server side configuration, so you should double-check your URLs are actually secured, using a desktop web browser and a tool like Fiddler so you can see everything. Pay particular attention to the Authentication headers, and the Status codes; at the least you should see a 401 from the server to start things off.
You can also turn on diagnostics for Apache HTTP on your device, and it will also dump headers and content to LOGCAT, so you can make sure everything is proceeding.
Check the WWW-Autnenticate header's contents, it will specify which schemes are accepted. The client side will re-request the URL, but it will put the Authorization header into its request.
In short, make sure your server side works outside of your application, in an environment that's easier to troubleshoot.
Client side, it looks like you are only activating BASIC authentication (everyone stop using it!), and your endpoint may only want DIGEST or NTLM or KERBEROS or any other authentication scheme than BASIC. Since it looks like you didn't set up for SSL, certainly use at least DIGEST or you have clear text issues!
Using form variables (for authentication) only works at the application level, and not the HTTP protocol level, which uses HTTP Headers (WWW-Autnenticate, Authorization) and Status codes (401, 403) for the authentication process. And again, if you aren't configuring your server (and client) for SSL-only, there will be clear text problems.

Authentication Error when using HttpPost with DefaultHttpClient on Android

I'm running into a strange problem using HttpClient. I am using a DefaultHttpClient() with HttpPost. I was using HttpGet with 100% success but now trying to switch to HttpPost as the REST API I'm using wants POST parameters rather than GET. (Only for some API calls though so I know that the GET calls were working fine so it's not a fault of the API).
Also, I tried using HttpPost on a simple php script I wrote that looks for a POST parameter 'var' and echoes it to screen, passing this parameters as follows worked fine:
List<NameValuePair> nameValuePairs = new ArrayList<NameValuePair>();
postMethod = new HttpPost("http://www.examplewebsite.com");
nameValuePairs.add(new BasicNameValuePair("var", "lol"));
try {
postMethod.setEntity(new UrlEncodedFormEntity(nameValuePairs));
response = httpClient.execute(postMethod, responseHandler);
Log.i("RESTMethod", response);
...
The problem is that when I tried and do the same call to the API (but with the params changed to the API params obviously) I get the following error:
Authentication error: Unable to respond to any of these challenges: {}
The page I am requesting is an HTTPS page, could this be the problem?
But doing the same type of POST request to a raw HTTP page on the API gives the same error, unless I comment out the StringEntity part and then it runs (but returns xml and I want to pass a parameter to request the data in JSON).
This seems like a really strange problem (the non-https part) but couldn't really find any help on this problem so sorry if the answer is out there.
Any ideas?
Thanks in advance,
Infinitifzz
EDIT: Okay I'm getting nowhere so I thought if I directed you to the API it might shed some light, it's the 8Tracks API and as you can see you need to pass a dev key (api_key) for all requests and I the part I'm stuck on is using https to log a user in with: http://www.8tracks.com/sessions.xml" part.
Hope this helps somehow because I am at a dead end.
Thanks,
Infinitifizz

Authentication error: Unable to
respond to any of these challenges: {}
This error message means that the server responded with 401 (Unauthorized) status code but failed to provide a single auth challenge (WWW-Authenticate header) thus making it impossible for HttpClient to automatically recover from the authentication failure.
Most likely application expects some soft of credentials in the HTML form enclosed in the HTTP POST request.

Don't you have to declare the port and protocol? I'm just swagging this code so please don't be upset if it doesn't immediatley compile correctly. Also, I usually supply a UsernamePasswordCredentials to my setCredentials() but I imagine it's the same.
HttpHost host = new HttpHost("www.foo.com", 443, "https");
// assemble your GET or POST
client.getCredentialsProvider().setCredentials(new AuthScope(host.getHostName(), host.getPort()));
HttpResponse response = client.execute(host, [HttpPost or HttpGet]);
More info about setCredentials here.

Here's how I ended up with similar problem:
DefaultHttpClient client = new DefaultHttpClient();
client.getCredentialsProvider().setCredentials(AuthScope.ANY,
new UsernamePasswordCredentials(username, password));
Thanks to Ryan for right direction.

Not specifying a Callback URL for my Twitter App resulted in the same error for me:
Authentication error: Unable to respond to any of these challenges: {oauth=WWW-Authenticate: OAuth realm="https://api.twitter.com"}
Setting a callback URL on Twitter fixed the problem