I'm using Android WebView to download files from the URL entered by the user (including protected by Cloudflare), it looks something like this:
private final DownloadListener downloadListener = new DownloadListener() {
#Override
public void onDownloadStart(String url,
String userAgent,
String contentDisposition,
String mimeType,
long contentLength)
{
while (redirectionCount++ < MAX_REDIRECTS) {
HttpURLConnection conn = new URL(url).openConnection();
conn.setInstanceFollowRedirects(false);
conn.setConnectTimeout(...);
conn.setReadTimeout(...);
conn.setRequestHeader("User-Agent", WebSettings.getDefaultUserAgent(context));
conn.setRequestProperty("Accept-Encoding", "identity");
// Get the cookies for the current URL.
String cookiesString = CookieManager.getInstance().getCookie(url);
conn.setRequestProperty("Cookie", cookiesString);
...
}
}
};
I get the url from the WebView's DownloadListener, open the connection and pass the cookies obtained from WebView to the request header (cf_clearance=***; cf_chl_2=***; cf_chl_prog=***; cf_chl_rc_ni=***) and also WebView user-agent. JS and cookies are enabled in the WebView.
But the problem is that HttpURLConnection keeps returning HTTP 503 even if I pass the Cloudflare cookies, although the WebView seems to pass the Cloudflare checking. Maybe it requires something other than the cookies themselves?
Related
My current code for intercepting a request in webview is
#Override
public WebResourceResponse shouldInterceptRequest(WebView view,
String url) {
String ext = MimeTypeMap.getFileExtensionFromUrl(url);
String mime = MimeTypeMap.getSingleton().getMimeTypeFromExtension(ext);
if (mime == null) {
return super.shouldInterceptRequest(view, url);
} else {
HttpURLConnection conn = (HttpURLConnection) new URL(
url).openConnection();
conn.setRequestProperty("User-Agent", userAgent);
return new WebResourceResponse(mime, "UTF-8",
conn.getInputStream());
}
}
I got this code from
The best way to intercept a WebView request in Android.
However, whenever I try to perform authentication, let's say I am loading facebook in my webview.
mWebView.loadUrl("https://www.facebook.com/");
Nothing is happening, what I noticed is that, the request headers are incomplete and also the response. Also, there are no cookies in the Sources. (I saw this when I remotely debugged the webview through Chrome).
Please correct me if I'm wrong, but I think that the incomplete headers and missing cookies is what causing the login request to fail.
Is there a way where I can modify the request and set its headers? Also for the response, should I do it too? And finally, how will I be able to have the cookies.
This question hasn't been answered for 6 months, so I don't know whether you will still need this, but maybe someone else has a similar question.
request headers are incomplete
When using HttpURLConnection you will be responsible to set any request headers, you might need, but it is as simple as setting the User-Agent, which you already did: conn.setRequestHeader(header, value) or if you want to add and not overwrite a header value: conn.addRequestHeader(header, value)
Alternatively, you could use okhttp, a HTTP client, which should add default values for headers, that are commonly expected.
there are no cookies in the Sources
When intercepting the request, you will also be in charge for handling cookies. You could store the cookie manually, by parsing the headers from the response e.g.
public WebResourceResponse shouldInterceptRequest(WebView view,
String url) {
// do your stuff
conn.connect(); // required to tell that the connection should be established
String cookie = getCookieFromConnection(conn);
// do more stuff and return WebResourceResponse
}
/**
* iterates all headers, and finds "cookie" headers
* (there could be more than one)
* #return cookie (concatenated value of all the found cookies)
* or null if no cookie has been found
*/
private String getCookieFromConnection(HttpURLConnection connection) {
String cookie = "";
Map<String, List<String>> header = connection.getHeaderFields();
List<String> cookies = header.get(COOKIE_HEADER);
if (cookies != null) {
for (String c : cookies) {
if (c != null) cookie += c + ";";
}
}
if (cookie.isEmpty()) return null;
return cookie;
}
or you could use a CookieManager, which would handle everything for you:
cookieManager = new CookieManager();
CookieHandler.setDefault(cookieManager);
cookieManager.setCookiePolicy(CookiePolicy.ACCEPT_ALL);
You could will also need to handle your cookies, when using okhttp, but again you could use the CookieManager as stated above. See this docu for more details, or this stackoverflow question.
Please correct me if I'm wrong, but I think that the incomplete headers and missing cookies is what causing the login request to fail.
There is another problem, when intercepting requests in a WebView: it somehow stops loading and evaluating javascript. I found this blog by Artem Zinnatullin online, who describes this behavior, and I experienced the same behavior.
If anyone would has a solution for this, I would be very happy.
I am trying to implement a payment gateway in android, and the payment processor requires sending some parameters when the "Pay" button is clicked. The sample link is:
https://vpay.com/?p=linkToken&v_merchant_id=qa331322179752&merchant_ref=234-567-890&memo=Bulk+order+from+McAckney+Web+Shop&total=13000&
notify_url=http%3A%2F%2Fwww.example.com%2Fnotification.php&
success_url=http%3A%2F%2Fwww.example.com%2Fthank_you.html&fail_url=http%3A%2F%2Fwww.example.com%2Ffailed.html
Now if the parameters are inputted correctly, the link returns another link in the format: https://vpay.com/pay/bnlink/xxxxxxxx-x0 which when visited brings up VPay payment page that can be used for payment based on the parameters supplied.
The payment processor should have employed an automatic redirection when the new link is generated, instead, it just displays the new link and stays there. Is there a way to get this new "RETURNED" link and then visit it so users can input the payment info.
Thank you!
Following method of Webview will work if you are using your own webview in activity or fragment for the payment stuff. below is the piece of code which will help:
webview.setWebViewClient(new WebViewClient() {
public boolean shouldOverrideUrlLoading(WebView view, String url) {
Log.i(TAG, "Processing webview url click...");
view.loadUrl(url);
Log.v(TAG,"html content of url:"+ getHtml(String url) );
return true;
}
public void onPageFinished(WebView view, String url) {
Log.i(TAG, "Finished loading URL: " +url);
}
});
in shouldOverrideUrlLoading method you will add your condition according to the url whether to load this url in webview or not.
And to get html content of the url use below Method:
public String getHtml(String url) {
HttpClient vClient = new DefaultHttpClient();
HttpGet vGet = new HttpGet(url);
String response = "";
try {
ResponseHandler<String> vHandler = new BasicResponseHandler();
response = vClient.execute(vGet, vHandler);
} catch (Exception e) {
e.printStackTrace();
}
return response;
}
I'm using DownloadManager to download video files from a url.
The problem is if I use the default folder to download the file I can not see the video in the galery.
Also, If I try to use this method:
request.setDestinationInExternalPublicDir(Environment.DIRECTORY_DOWNLOADS, 'filename');
I need to know the file name before download which in this case, I don't.
And also, I don't have the name of the file in the url.
How can I do to get the file name from the headers and pass the name to the method setDestinationInExternalPublicDir ?
Other alternatives?
In case anyone want an implementation of doing a HEAD request to get the filename:
class GetFileName extends AsyncTask<String, Integer, String>
{
protected String doInBackground(String... urls)
{
URL url;
String filename = null;
try {
url = new URL(urls[0]);
String cookie = CookieManager.getInstance().getCookie(urls[0]);
HttpURLConnection con = (HttpURLConnection) url.openConnection();
con.setRequestProperty("Cookie", cookie);
con.setRequestMethod("HEAD");
con.setInstanceFollowRedirects(false);
con.connect();
String content = con.getHeaderField("Content-Disposition");
String contentSplit[] = content.split("filename=");
filename = contentSplit[1].replace("filename=", "").replace("\"", "").trim();
} catch (MalformedURLException e1) {
e1.printStackTrace();
} catch (IOException e) {
}
return filename;
}
#Override
protected void onPreExecute() {
super.onPreExecute();
}
#Override
protected void onPostExecute(String result) {
}
}
Small tip, there is a nice helper method in Android
URLUtil.guessFileName(url, contentDisposition, contentType);
So after completing the call to the server, getting the contenttype and contentDisposition from the Headers this will try and find the filename from the information.
I got into the same problem.I used #rodeleon answer but there was no Content-Disposition in response header. Then I analyzed url header from Chrome dev tools and got 'Location' in response header which contained filename at the end, it was like "b/Ol/fire_mp3_24825.mp3". so instead of using
String content = con.getHeaderField("Content-Disposition")
I used
String content = con.getHeaderField("Location")
and at the end in onPostExecute
protected void onPostExecute(String result) {
super.onPostExecute(result);
String fileName = result.substring(result.lastIndexOf("/") + 1);
// use result as file name
Log.d("MainActivity", "onPostExecute: " + fileName);
}
Method URLUtil.guessFileName() (while not yet knowing about the Content-Disposition) and also the meanwhile deprecated class AsyncTask are both kinda problematic approaches these days. To properly enqueue the download with the DownloadManager:
One first has to disable "following redirects" in whatever HTTP client (that's the clue).
Then the first response will be HTTP 302 (with a Location header), instead of HTTP 200.
When fetching that, one will get HTTP 200 with a Content-Disposition header (filename).
Only then one can enqueue with DownloadManager (whilst knowing the filename already).
Here's an example of mine: RepositoryFragment (a GitHub Client).
request.setDestinationInExternalPublicDir(Environment.DIRECTORY_MOVIES, uri.getLastPathSegment());
I track my visitors' usernames and I want to pass the username data from webpage to my activity method. What is the proper way to pass this data ?
(Putting username data inside HTML page's hidden div and parsing webpage is my last option, I don't prefer it)
I use Webview in my Android application. I load URLs like this:
public boolean shouldOverrideUrlLoading(WebView view, String url) {
...
String url = "http://example.com";
webView.postUrl(url, EncodingUtils.getBytes(myPostData, "base64"));
...
}
There is some custom headers inside my page response. Response headers have this:
header('User name: michael');
If page is successfully loaded, I need to get that headers from the response and use it inside my Google Analytics as custom dimension.
EasyTracker easyTracker = EasyTracker.getInstance();
easyTracker.send(MapBuilder
.createAppView("Home screen")
.set(Fields.customDimension(1), "michael");
.build()
);
My problem is what is the proper way for this ? Is it possible to get it from onPageFinished and set this header inside Analytics code or is there a better way ? I'm open to any method/solution that is compatible with this structure.
Note that I override these methods for my activity works:
WebViewClient::onPageStarted(WebView view, String url, Bitmap favicon)
WebViewClient::onPageFinished(WebView view, String url)
WebViewClient::shouldOverrideUrlLoading(WebView view, String url)
WebViewClient::onLoadResource(WebView view, String url)
WebViewClient::onReceivedError(WebView view, int errorCode, String description, String failingUrl)
WebChromeClient::onProgressChanged(WebView view, int newProgress)
Edit:
There is a similar answer here which is 3 years old: https://stackoverflow.com/a/3134609/429938
But It offers to catch headers in shouldOverrideUrlLoading. Is it not possible to catch it in onPageFinished ? Or how can I use it with Webview.postUrl()
Edit2:
Another solution is to use addJavascriptInterface and set user name from the javascript code and set google analytics code inside activity. But this is a Android specific solution. I would prefer a common solution that can be used in IOS etc.
the most cross platform solution is a javascript bridge ( addJavascriptInterface ). This is what I would recommend you.
The native part can be implemented on iOS too. And the HTML part stay the same.
If not, you need to implement all the logic on the native side using one of the events, like onPageFinished. If the header are not accessible from here, do the request yourself with HTTPURLConnection then loadDataWithBaseURL.
I'm using following code to konw when page is loaded:
webview.setWebChromeClient(new WebChromeClient() {
public void onProgressChanged(WebView view, int progress)
{
MyActivity.setTitle("Loading...");
MyActivity.setProgress(progress * 100);
if (progress == 100) {
//page is successfully loaded and you can get our header.
}
}
});
Try this one: here I'm getting "User name" value from header
HttpURLConnection conn = null;
final HostnameVerifier DO_NOT_VERIFY = new HostnameVerifier() {
public boolean verify(String hostname, SSLSession session) {
return true;
}
};
try {
URL url = new URL(YOUR_URL);
if (url.getProtocol().toLowerCase().equals("https")) {
trustAllHosts();
HttpsURLConnection https = (HttpsURLConnection) url.openConnection();
https.setHostnameVerifier(DO_NOT_VERIFY);
conn = https;
} else {
conn = (HttpURLConnection) url.openConnection();
}
conn.setInstanceFollowRedirects(false);
conn.connect();
String location = conn.getHeaderField( "User name" );
I want to make an http post request using webview.
webView.setWebViewClient(new WebViewClient(){
public void onPageStarted(WebView view, String url,
Bitmap favicon) {
super.onPageStarted(view, url, favicon);
}
public boolean shouldOverrideUrlLoading(WebView view,
String url) {
webView.postUrl(Base_Url, postData.getBytes());
return true;
}
});
The above code snippet loads the webpage. I want to access the response of this request.
How can i obtain the response of an http post request using webview?
Thanks in Advance
First add the support of the http library to your gradle file:
To be able to use
useLibrary 'org.apache.http.legacy'
After this you can use the following code to perform post request in your webview:
public void postUrl (String url, byte[] postData)
String postData = "submit=1&id=236";
webview.postUrl("http://www.belencruzz.com/exampleURL",EncodingUtils.getBytes(postData, "BASE64"));
http://belencruz.com/2012/12/do-post-request-on-a-webview-in-android/
The WebView does not let you access the content of the HTTP response.
You have to use HttpClient for that, and then forward the content to the view by using the function loadDataWithBaseUrl and specifying the base url so that the user can use the webview to continue navigating in the website.
Example:
// Executing POST request
HttpClient httpclient = new DefaultHttpClient();
HttpPost httppost = new HttpPost(url);
httppost.setEntity(postContent);
HttpResponse response = httpclient.execute(httppost);
// Get the response content
String line = "";
StringBuilder contentBuilder = new StringBuilder();
BufferedReader rd = new BufferedReader(new InputStreamReader(response.getEntity().getContent()));
while ((line = rd.readLine()) != null) {
contentBuilder.append(line);
}
String content = contentBuilder.toString();
// Do whatever you want with the content
// Show the web page
webView.loadDataWithBaseURL(url, content, "text/html", "UTF-8", null);