I am working on an app which shall log in to a web site (via http://......?password=xyz).
I use DefaultHttpClient for this.
Along with the GET response, the website sends a cookie, which I want to store for further POST requests.
My problem is that client.getCookieStore().getCookies() always receives an empty list of cookies.
If I open http://www.google.com (insted of my intended website), I receive the cookies properly, but the website I am working with, seems to send the cookie in some other way (it's a MailMan mailing list moderating page)
I can see the respective cookie in Firefox cookie manager, but not in Firebug network/cookie panel (why?). InternetExplorer HttpWatchProfessional however shows the cookie when recording the traffic....
There is some small difference, I observed between the cookies www.google.com sent and my target website: In HttpWatchProfessional, those cookies from google are marked as "Direction: sent", while the cookie from my website are marked as "Direction: Received".
(how can the google cookies be sent, while I cleared browser/cookie cache just before?)
Can someone explain the difference to me?
My code is the following:
DefaultHttpClient client = new DefaultHttpClient();
HttpGet httpGet = new HttpGet(url);
HttpResponse execute = client.execute(httpGet);
List<Cookie> cookies = client.getCookieStore().getCookies();
After further investigation, I found out that the cookie was received, but actually rejected by the httpclient, due to a path the cookie, which differed to that from the called URL.
I found the solution at:
https://stackoverflow.com/a/8280340/1083345
Related
How can I disable/ignore default cookie handling of httpclient. I want to do it manually. I want to set a pre-defined cookie header for all http requests.
The latest httpclient (4.5.1) has a method called "disableCookieManagement", and it appears this just disables the internal cookie management, not the ability to send or receive cookies, and is working for me-
http://hc.apache.org/httpcomponents-client-ga/httpclient/apidocs/org/apache/http/impl/client/HttpClientBuilder.html#disableCookieManagement()
Set httpclient.setCookieStore(cookieStore); before you execute your HttpClient
Force to setup the HttpContext before executing the request:
private HttpClientContext httpContext = new HttpClientContext();
httpContext.setCookieStore(new BasicCookieStore());
The setCookieStore will create an empty cookie store to replace the default cookie store of http connection.
After that we can execute the http method:
org.apache.http.client.HttpClient.execute(HttpUriRequest, HttpContext)
Also we can reuse the cookie store (hold the BasicCookieStore) to keep the connection alive.
I could not find a way to disable/ignore cookie handling by DefaultHttpClient in Andoroid(I should have explored more into Android source code but have a time limitation). But I resolved it by removing all cookies before doing httpClient.execute() like this -
((AbstractHttpClient) myDefaultHttpClient).getCookieStore().clear();
This removes all the cookies stored by the defaultHttpClient and then you can manually handle(add/delete) cookies using -
myHttpPost.setHeader("Cookie", myCookie);
Hope it helps.
I'm a junior developer who has been working on a RSS Reader.
I'm trying to download a webpage from my app for offline viewing but I am having a few issues.
When I try to download an asp page I don't seem to get the right content, but instead a html page with asp form widgets.
Can anyone help me with understanding what's going on and how I could possibly download the content of the page?
I should also mention the webpage is a sharepoint webpage using https ssl authentication, using httpclient as my means to connect and download the webpage.
To communicate with ASP you usually need to send __VIEWSTATE and _EVENTVALIDATION tokens in your HttpPost and other requests. You can get those once by calling HttpGet on the basic page and using Patten with regexes or a simple str.contains("_VIEWSTATE") and strip it out of the HTML and send with every request.
If you're not doing any POSTs, just basic GETs, then make sure you're setting the headers appropriately, as so:
HttpGet req = new HttpGet("YOUR SITE'S URL");
req.setHeader("Content-Type", "application/x-www-form-urlencoded");
req.setHeader("Host", "YOUR SITE'S ROOT PAGE");
req.setHeader("User-Agent", "Mozilla/5.0 ...");
req.setHeader("Accept-Encoding", "gzip,deflate,sdch");
req.setHeader("Accept", "text/html,application/xhtml+xml,application/xml");
req.setHeader("Accept-Language", "en-us,en");
req.setHeader("Accept-Charset", "ISO-8859-1,utf-8");
HttpResponse resp = client.execute(req, localContext);
Don't forget about the possible session cookie you can store in httpcontext and also pass in with every execute as seen above:
CookieStore cookieStore = new BasicCookieStore();
HttpContext localContext = new BasicHttpContext();
localContext.setAttribute(ClientContext.COOKIE_STORE, cookieStore);
The best way to go about it in theory is to download Fiddler, run the site in Chrome, see what's going on and emulate actual browser requests in your app: http://www.fiddler2.com/fiddler2/
I'm writing an Android app that should get data from a certain web application. That web app is based on Servlets and JSP, and it's not mine; it's a public library's service. What is the most elegant way of getting this data?
I tried writing my own Servlet to handle requests and responses, but I can't get it to work. Servlet forwarding cannot be done, due to different contexts, and redirection doesn't work either, since it's a POST method... I mean, sure, I can write my own form that access the library's servlet easily enough, but the result is a jsp page.. Can I turn that page into a string or something? Somehow I don't think I can.. I'm stuck.
Can I do this in some other way? With php or whatever? Or maybe get that jsp page on my web server, and then somehow extract data from it (with jQuery maybe?) and send it to Android? I really don't want to display that jsp page in a browser to my users, I would like to take that data and create my own objects with it..
Just send a HTTP request programmatically. You can use Android's builtin HttpClient API for this. Or, a bit more low level, the Java's java.net.URLConnection (see also Using java.net.URLConnection to fire and handle HTTP requests). Both are capable of sending GET/POST requests and retrieving the response back as an InputStream, byte[] or String.
At most simplest, you can perform a GET as follows:
InputStream responseBody = new URL("http://example.com").openStream();
// ...
A POST is easier to be performed with HttpClient:
List<NameValuePair> params = new ArrayList<NameValuePair>(2);
params.add(new BasicNameValuePair("name1", "value1"));
params.add(new BasicNameValuePair("name2", "value2"));
HttpPost post = new HttpPost("http://example.com");
post.setEntity(new UrlEncodedFormEntity(params));
HttpClient client = new DefaultHttpClient();
HttpResponse response = client.execute(post);
InputStream responseBody = response.getEntity().getContent();
// ...
If you need to parse the response as HTML (I'd however wonder if that "public library service" (is it really public?) doesn't really offer XML or JSON services which are way much easier to parse), Jsoup may be a life saver as to traversing and manipulating HTML the jQuery way. It also supports sending POST requests by the way, only not as fine grained as with HttpClient.
So basically i need my android app to connect to a web service using a url as such
"http://username:password#0.0.0.0" aka basic authentication.
obviously the username and password are checked by the web app before allowing access and otherwise doesn't allow the request.
my issue is that all the methods i try always say unauthorised (response code 401) regardless of what combination of classes and methods ive used to try and connect to the the url.
The web app in question is designed to return things only is un/pw clears otherwise it returns nothing, the web app and un/pw etc have all be checked and cleared.
so does anyone no the correct way to send a request to a url like that and have it work correctly?
android api8 btw
UPDATE
Turns out my issue is due to the web app using NTLM windows authentication which is not supported directly by androids/apache http library, investigating appropriate workarounds now
Here's some code form a really old project of mine. I used basic auth for some web service, and this worked at the time. I'm not sure if there are updated api's since then (this was Android 1.6), but it should still work.
HttpGet request = new HttpGet();
request.setURI(new URI(url));
UsernamePasswordCredentials credentials =
new UsernamePasswordCredentials(authUser, authPass);
BasicScheme scheme = new BasicScheme();
Header authorizationHeader = scheme.authenticate(credentials, request);
request.addHeader(authorizationHeader);
Basically, Basic HTTP auth is a simple hash of the user and password. The browser allows you to stuff these values in the url, but it actually does the work of adding the basic auth header to your request.
I'm running into a strange problem using HttpClient. I am using a DefaultHttpClient() with HttpPost. I was using HttpGet with 100% success but now trying to switch to HttpPost as the REST API I'm using wants POST parameters rather than GET. (Only for some API calls though so I know that the GET calls were working fine so it's not a fault of the API).
Also, I tried using HttpPost on a simple php script I wrote that looks for a POST parameter 'var' and echoes it to screen, passing this parameters as follows worked fine:
List<NameValuePair> nameValuePairs = new ArrayList<NameValuePair>();
postMethod = new HttpPost("http://www.examplewebsite.com");
nameValuePairs.add(new BasicNameValuePair("var", "lol"));
try {
postMethod.setEntity(new UrlEncodedFormEntity(nameValuePairs));
response = httpClient.execute(postMethod, responseHandler);
Log.i("RESTMethod", response);
...
The problem is that when I tried and do the same call to the API (but with the params changed to the API params obviously) I get the following error:
Authentication error: Unable to respond to any of these challenges: {}
The page I am requesting is an HTTPS page, could this be the problem?
But doing the same type of POST request to a raw HTTP page on the API gives the same error, unless I comment out the StringEntity part and then it runs (but returns xml and I want to pass a parameter to request the data in JSON).
This seems like a really strange problem (the non-https part) but couldn't really find any help on this problem so sorry if the answer is out there.
Any ideas?
Thanks in advance,
Infinitifzz
EDIT: Okay I'm getting nowhere so I thought if I directed you to the API it might shed some light, it's the 8Tracks API and as you can see you need to pass a dev key (api_key) for all requests and I the part I'm stuck on is using https to log a user in with: http://www.8tracks.com/sessions.xml" part.
Hope this helps somehow because I am at a dead end.
Thanks,
Infinitifizz
Authentication error: Unable to
respond to any of these challenges: {}
This error message means that the server responded with 401 (Unauthorized) status code but failed to provide a single auth challenge (WWW-Authenticate header) thus making it impossible for HttpClient to automatically recover from the authentication failure.
Most likely application expects some soft of credentials in the HTML form enclosed in the HTTP POST request.
Don't you have to declare the port and protocol? I'm just swagging this code so please don't be upset if it doesn't immediatley compile correctly. Also, I usually supply a UsernamePasswordCredentials to my setCredentials() but I imagine it's the same.
HttpHost host = new HttpHost("www.foo.com", 443, "https");
// assemble your GET or POST
client.getCredentialsProvider().setCredentials(new AuthScope(host.getHostName(), host.getPort()));
HttpResponse response = client.execute(host, [HttpPost or HttpGet]);
More info about setCredentials here.
Here's how I ended up with similar problem:
DefaultHttpClient client = new DefaultHttpClient();
client.getCredentialsProvider().setCredentials(AuthScope.ANY,
new UsernamePasswordCredentials(username, password));
Thanks to Ryan for right direction.
Not specifying a Callback URL for my Twitter App resulted in the same error for me:
Authentication error: Unable to respond to any of these challenges: {oauth=WWW-Authenticate: OAuth realm="https://api.twitter.com"}
Setting a callback URL on Twitter fixed the problem