I'm having a strange problem here. Here's the code, I'm using to fetch a url content:
URL u = new URL(url);
InputStream is = new BufferedInputStream(u.openStream());
I've got two urls, I want to fetch with this code. Both contain xml data. To be specific, the first one is http://www.berlingske.dk/unwire/latest/news_article/2/10, the second one is http://www.bt.dk/mecommobile/latest/news_article/1368/10?output_type=xml. The first one gets fetched correctly, the second one does not. I added some logging, and found out, that for the second url some weird html page gets fetched, instead of the expected xml. How can that be even possible?
I think you're talking about URL redirects, which was a problem I was having. Try the following code:
URL url = new URL(url);
HttpURLConnection ucon = (HttpURLConnection) url.openConnection();
ucon.setInstanceFollowRedirects(false);
URL secondURL = new URL(ucon.getHeaderField("Location"));
URLConnection conn = secondURL.openConnection();
InputStream is = new BufferedInputStream(conn.openStream());
The "magic" here happens in these 2 steps:
ucon.setInstanceFollowRedirects(false);
URL secondURL = new URL(ucon.getHeaderField("Location"));
By default InstanceFollowRedirects are set to true, but you want to set it to false to capture the second url. To be able to get that second url from the "weird html page", you need to get the header field called "Location".
Unless i misunderstood your problem, I hope this helps!
Related
Why is the following code
URL url = new URL(urlStr);
Log.d(TAG, "Opening URL " + url.toString());
HttpURLConnection urlConnection = (HttpURLConnection) url.openConnection();
urlConnection.setRequestMethod("GET");
urlConnection.setDoInput(true);
urlConnection.setDoOutput(true);
urlConnection.connect();
String response = streamToString(urlConnection.getInputStream());
always giving me the following Exception at urlConnection.getInputStream()?
W/System.err(16253): java.io.FileNotFoundException: https://api.instagram.com/v1/media/search/access_token=1559619173.3c922fe.4fd71e26225a42a0a03fdd90ef8679a6?lat=48.858318956&lng=2.294427258&distance=500
W/System.err(16253): at com.android.okhttp.internal.http.HttpURLConnectionImpl.getInputStream(HttpURLConnectionImpl.java:186)
W/System.err(16253): at com.android.okhttp.internal.http.HttpsURLConnectionImpl.getInputStream(HttpsURLConnectionImpl.java:246)
W/System.err(16253): at br.com.dina.oauth.instagram.InstagramApp$5.run(InstagramApp.java:192)
You should remove urlConnection.setDoOutput(true); line because setDoOutput method force POST but instagram api work GET method.
You can also check this issue.
https://github.com/thiagolocatelli/android-instagram-oauth/issues/2
The code throwing the exception can be found for example in the okhttp github (Android uses that internally). The lines responsible are:
if (getResponseCode() >= HTTP_BAD_REQUEST) {
throw new FileNotFoundException(url.toString());
}
That simply means that the url you've provided did not result in a successful response. Throwing a *File*NotFoundException is behavior okhttp just copies from other implementations. Why someone chose this particular exception is beyond me.
If you simply put the url in the error message into a browser you'll see that you get a "Not found" page.
Explanation is simple: You're building the URL wrong. Instead of
search/access_token=...?lat=...
it needs to be
search?access_token=...&lat=...
access_token is a url parameter, not part of the path so it needs to get the ? as separator while lat becomes the second parameter which means ? needs to turn into &.
I am trying to download files from a server in android and show progress dialog using code very similar to the answer provided in this thread but i am not able to get content length in HttpURLConnection's getContentLength() method. Content length for all files is -1.
For the same file, i get correct content length in iOS app with NSHTTPURLResponse's expectedContentLength method.
Is there some basic difference in the way these methods fetch the content length for an http connection/response?
EDIT 1:
Tried following few things as suggested some answers and comments.
Set Accept-Encoding header to identity
Fetching the content length as string (from header field Content-Length) and then converting it to long
Tried conn.getContent().toString().length() instead of getContentLength()
None of these worked for me yet.
What baffles me most is i get the content length in iOS but not on android.
EDIT 2:
Heres my iOS and Android code for comparison -
iOS:
NSURLRequest *request = [NSURLRequest requestWithURL:self.url cachePolicy:NSURLRequestUseProtocolCachePolicy timeoutInterval:1200.0];
NSURLConnection *connection = [[NSURLConnection alloc] initWithRequest:request delegate:self startImmediately:NO];
[connection scheduleInRunLoop:[NSRunLoop currentRunLoop] forMode:NSDefaultRunLoopMode];
[connection start];
Android:
URL url = new URL(downloadUrlString);
connection = (HttpURLConnection) url.openConnection();
connection .setRequestProperty("Accept-Encoding", "identity");
connection.connect();
The only difference i can see is caching.
So i added following line in android code as well but nothing changed.
connection.setUseCaches(true);
try this:
HttpURLConnection conn = (HttpURLConnection)url.openConnection();
conn .setRequestProperty("Accept-Encoding", "identity");
conn.connect();
if you just need to content length, can you try
conn.getContent().toString().length()
where conn is the HttpURLConnection object
Root cause for this problem turned out to be Cookies.
I am using a web view in one of the activities in my application. Some cookies are stored by the web view. All other REST api and file download requests work without those cookies however for a particular type of requests, the cookies are necessary.
Apparently, android web view and the connection requests do not share cookies out of the box like iOS. As a result i had to make changes in my code to make sure that the HttpUrlConnection uses WebKit's cookie store. I did it using method described in the accepted answer for this question.
Has anyone released code to show the full HTTP request/response headers, any intermediate redirects, and any cookie data for the Android HttpURLConnection? This would be similar to Firefox Web Console
I roughly know how to write this myself, but 1) it's a non-trivial amount of code 2) it's tricky to get this kind of code to work in all instances. So i'm interested in finding a readymade solution. I know how to tcpdump the emulator, but I'm searching for code to print this information into the Android Log class for really quick runtime debugging.
for header fields
URL url = new URL(str_url);
HttpURLConnection conection = (HttpURLConnection) url.openConnection();
conection.setConnectTimeout(TIMEOUT_SOCKET);
conection.setReadTimeout(TIMEOUT_CONNECTION);
conection.addRequestProperty("Accept-Encoding", "gzip");
RedirectLocations locations = new RedirectLocations();
// here u get all header fields and properties write it in logs
conection.getHeaderFields();
conection.getRequestProperties();
// conection.getOutputStream().write(buffer);
// download the file
InputStream is = conection.getInputStream();
// This is file path were a; quiz data will get saved.
// String file_path = context.getDir(folder,Activity.MODE_PRIVATE).getAbsolutePath();
return unzip(is,save_file_path);
for redirects
link
after u get response, again u ve to look for header fields
I am trying to develop an application where I can get the HTML source of any web page. I am able to get the code but when I am trying the same code using some facebook profile links, it gives me an empty string.
I am using HttpURLConnection. I am trying the code as follows:
URL url = new URL(urlPassed);
HttpURLConnection urlc = (HttpURLConnection) url.openConnection();
BufferedInputStream buffer = new BufferedInputStream(urlc.getInputStream());
and I am trying to read that buffer in a loop as buffer.read(by) where "by" is a byte array of size 7024. However, it works fine for all other web pages except facebook pages.
Any reason for this? Any idea to solve this?
I'm downloading a file in my android app and I want to display a progressbar. I need to know the total file size by using this method : here.
The problem is that I always get -1 despite the fact that it's set on the server and I can see it from my computer.
This is the code :
URL url = new URL(fileUrl);
URLConnection ucon = url.openConnection();
ucon.connect();
int length = ucon.getContentLength();
What is the problem ?
This is not related to Android, most probably it was not sent from the server..please use any sniffing tool to make sure the content length header is there.