Develop Reference

OAuth2 authorization to 'My Anime List' not working

I am trying to authenticate a "My Anime List" user using Oauth2 (following this guide) for my Android application.
Step 1: getting the authorization token
Here, I am using a WebView to prompt the user for its username and password. This step seems to work as far as I can see.
private static final String REDIRECT_URL = "http://localhost/oauth";
private static final String CLIENT_ID = "9c..."; // omitted
private static final String OAUTH_BASE_URL = "https://myanimelist.net/v1/oauth2/";
protected void onCreate(Bundle savedInstanceState) {
super.onCreate(savedInstanceState);
setContentView(R.layout.activity_login);
/*
* Before you can authenticate a user, your client needs to generate a Code Verifier and a
* Code Challenge. A Code Verifier is a high-entropy, cryptographic, random string
* containing only the characters [A-Z] / [a-z] / [0-9] / "-" / "." / "_" / "~".
* The length of the string must be between 43 and 128 characters.
*
* MAL only allows the plain transformation for the Code Challenge.
* In other words, it means that you have to set the Code Challenge equal to the
* Code Verifier.
*/
String codeChallenge = PKCEGenerator.generateVerifier(128);
webview = findViewById(R.id.login_webview);
webview.getSettings().setJavaScriptEnabled(true);
webview.setWebViewClient(new WebViewClient(){
public boolean shouldOverrideUrlLoading(WebView view, WebResourceRequest request){
Log.d(TAG, "Redirecting to: " + request.getUrl());
Uri url = request.getUrl();
if(url.toString().contains(REDIRECT_URL)){
String authorizationCode = url.getQueryParameter("code");
Log.d(TAG, "Received authorization code: " + authorizationCode);
webview.setVisibility(View.GONE);
getUserAccessToken(authorizationCode, codeChallenge);
}
return false;
}
});
authenticateMAL(codeChallenge);
}
private void authenticateMAL(String codeChallenge) {
Log.d(TAG, "Code challenge (" + codeChallenge.length() + "): " + codeChallenge);
String loginUrl = OAUTH_BASE_URL + "authorize" +
"?response_type=code" +
"&redirect_uri=" + REDIRECT_URL +
"&client_id=" + CLIENT_ID +
"&code_challenge=" + codeChallenge;
Log.d(TAG, "Login url: " + loginUrl);
webview.loadUrl(loginUrl);
}
As far as I can see, this works well. I am getting the authorizationCode as expected.
Step 2: Getting the user access token & refresh token
Here, I am using Mal4J for the next authentication step:
private void getUserAccessToken(String authorizationCode, String codeChallenge) {
Single.fromCallable(() -> {
MyAnimeListAuthenticator authenticator = new MyAnimeListAuthenticator(
CLIENT_ID, null, authorizationCode, codeChallenge);
return authenticator.getAccessToken();
})
.subscribeOn(Schedulers.io())
.doOnError(throwable -> {
Log.e(TAG, "Error while retrieving token!", throwable);
})
.onErrorComplete()
.subscribe(token -> {
Log.d(TAG, "--> access token: " + token.getToken());
Log.d(TAG, "--> refresh token: " + token.getRefreshToken());
});
}
Unfortunately, this results in the following error:
E/LoginActivity: Error while retrieving token!
com.kttdevelopment.mal4j.HttpException: Server returned code 400 from 'https://myanimelist.net/v1/oauth2/token':
at com.kttdevelopment.mal4j.MyAnimeListAuthenticator.parseToken(MyAnimeListAuthenticator.java:505)
at com.kttdevelopment.mal4j.MyAnimeListAuthenticator.<init>(MyAnimeListAuthenticator.java:139)
at florian.baierl.daily_anime_news.ui.LoginActivity.lambda$getUserAccessToken$0(LoginActivity.java:99)
at florian.baierl.daily_anime_news.ui.-$$Lambda$LoginActivity$-bBBIb9OKRzdaFNsFkQdJSeVW74.call(Unknown Source:4)
at io.reactivex.rxjava3.internal.operators.single.SingleFromCallable.subscribeActual(SingleFromCallable.java:43)
at io.reactivex.rxjava3.core.Single.subscribe(Single.java:4813)
at io.reactivex.rxjava3.internal.operators.single.SingleSubscribeOn$SubscribeOnObserver.run(SingleSubscribeOn.java:89)
at io.reactivex.rxjava3.core.Scheduler$DisposeTask.run(Scheduler.java:614)
at io.reactivex.rxjava3.internal.schedulers.ScheduledRunnable.run(ScheduledRunnable.java:65)
at io.reactivex.rxjava3.internal.schedulers.ScheduledRunnable.call(ScheduledRunnable.java:56)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:301)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1167)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:641)
at java.lang.Thread.run(Thread.java:923)
Any ideas as to why that may happen? Am I missing some Android specific stuff for Oauth2? As far as I can see, I am correctly retrieving the auth code from step 1. After that, my code seems very straight-forward, so I fail to see where the error could be. Any hints are greatly appreciated!
Edit:
This is how the request looks like (from the android studio profile view):
and here is the reply:
Edit 2:
Hard coding the code challenge/verifier to 128 times 'A' (AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA) doesn't change the behavior either:

When you include the redirect_uri in the authorization request you also need to include it in the /token request. Maybe it't that.

scheme call from JS (web browser) to android activity doesn't update the intent with the scheme data in the url string

I am launching my app from the browser and if the app is not already loaded/running then it launches the app through onCreate and collects the intent.data and does the right thing. If the app is already running then launching the app from the browser bypasses the onCreate and goes right to onResume which I "believe" is expected behavior. But the intent.data associated with the launch from the browser is null. I was hoping that the intent data would be overwritten with the new intent data from scheme url string so that I can relaunch the webview inside the app with the new parameters. Is there some way that I can push this new data into the app when it is launched (i.e., resumed) by the browser?
// JS Browser code with arguments to be passed to the app.
var r = Math.random();
window.open("myScheme://caca.com/?mode=driver" + "&ec=" + ec + "&uh=" + $("#userHandle").val() + "&at=0" + "&random=" + r);
// Xamarin
onResume(...) {
...
try
{
// currentActivity.Intent.Data != null if app is not already running
// is null if already running
Android.Net.Uri data = currentActivity.Intent.Data;
string scheme = data.Scheme;
DebugToast("scheme " + scheme, ToastLength.Long);
if (scheme == "myScheme")
{
// force the cookies so the web app will come up as we specify
SetStorageValue("browserLaunched", "true");
SetStorageValue("mode", data.GetQueryParameter("mode"));
SetStorageValue("ec", data.GetQueryParameter("ec"));
SetStorageValue("uh", data.GetQueryParameter("uh"));
SetStorageValue("authToken", data.GetQueryParameter("at"));
SetStorageValue("at", data.GetQueryParameter("at"));
DebugToast("parms " + data.GetQueryParameter("mode") + " " + data.GetQueryParameter("ec") + " " + data.GetQueryParameter("uh") + " " + data.GetQueryParameter("at"), ToastLength.Long);
// restart the web view with arguments above
...
}
}
catch
{
....
}
}

Homer Simpson D'oh!. Needed to add 'onNewIntent()' to activity which receives the "um er dah" new Intent with the appropriate data attached.

Developing a Web Monitor in Android

I would like to monitor/filter the websites that an user opens in Android.
I know how to retrieve the last visited URL (in Android default browser) using a ContentObserver on the browser history...
private static class BrowserObserver extends ContentObserver {
private static String lastVisitedURL = "";
private static String lastVisitedWebsite = "";
//Query values:
final String[] projection = new String[] { Browser.BookmarkColumns.URL }; // URLs
final String selection = Browser.BookmarkColumns.BOOKMARK + " = 0"; // history item
final String sortOrder = Browser.BookmarkColumns.DATE; // the date the item was last visited
public BrowserObserver(Handler handler) {
super(handler);
}
#Override
public void onChange(boolean selfChange) {
onChange(selfChange, null);
}
#Override
public void onChange(boolean selfChange, Uri uri) {
super.onChange(selfChange);
//Retrieve all the visited URLs:
final Cursor cursor = getContentResolver().query(Browser.BOOKMARKS_URI, projection, selection, null, sortOrder);
//Retrieve the last URL:
cursor.moveToLast();
final String url = cursor.getString(cursor.getColumnIndex(projection[0]));
//Close the cursor:
cursor.close();
if ( !url.equals(lastVisitedURL) ) { // to avoid information retrieval and/or refreshing...
lastVisitedURL = url;
//Debug:
Log.d(TAG, "URL Visited: " + url + "\n");
}
}
}
To register the ContentObserver I use:
browserObserver = new BrowserObserver(new Handler());
getContentResolver().registerContentObserver(Browser.BOOKMARKS_URI, true, browserObserver);
And to unregister it:
getContentResolver().unregisterContentObserver(browserObserver);
This works. However, in this way, I can analyze the URLs only after the browser has loaded them.
Now, is there a way to retrieve the URLs before the browser actually loads them in Android?

A solution which could help to create a Web Monitor, is to creation your own VPN service, so that you monitor all the device traffic. A good example of this is the project NetGuard.
https://github.com/M66B/NetGuard
Note that in some devices, the system will not pass through the VPN some applications (ex, in Samsung devices, the Samsung Web Browser is not forwarded through the system VPN, checked in S5 with Android 6.0).
Also your application should request the permission to be used as a VPN Service, but once the user gives this permission, it can monitor and filter most of the device network traffic.

How to capture http post data from Android WebView [duplicate]

I've started to write an app which provides the user with an HTML form via a WebView. As the form is not under my control, the data filled in may be sent as either GET or POST request. My app is required to capture the transported form data, that is, get a hold on what was entered into the form fields.
Using an adequate callback from WebViewClient such as onPageLoaded(), it is easy to capture form data from a GET request. However, I cannot find any appropriate method to allow the same for POSTed data, i.e., be able to access the HTTP POST message body containing the form data. Am I missing a relevant callback here or is there simply no way to accomplish the specified goal with the given API (even the latest level 8)?
Assuming it wasn't possible, I considered overriding and extending parts of android.webkit in order to introduce a new callback hook that is passed the POST body somehow. That way, my app could be shipped with a customized browser/WebViewClient that fulfills the desired feature. However, I couldn't find any good spot to start with in the code and would be glad for any hints in this regards (in case the approach looks promising at all).
Thanks in advance!

As indicated in my own comment to the original question, the JavaScript injection approach works. Basically, what you need to do is add some piece of JavaScript code to the DOM onsubmit event, have it parse the form's fields, and return the result back to a Java-registered function.
Code example:
public class MyBrowser extends Activity {
private final String jsInjectCode =
"function parseForm(event) {" +
" var form = this;" +
" // make sure form points to the surrounding form object if a custom button was used" +
" if (this.tagName.toLowerCase() != 'form')" +
" form = this.form;" +
" var data = '';" +
" if (!form.method) form.method = 'get';" +
" data += 'method=' + form.method;" +
" data += '&action=' + form.action;" +
" var inputs = document.forms[0].getElementsByTagName('input');" +
" for (var i = 0; i < inputs.length; i++) {" +
" var field = inputs[i];" +
" if (field.type != 'submit' && field.type != 'reset' && field.type != 'button')" +
" data += '&' + field.name + '=' + field.value;" +
" }" +
" HTMLOUT.processFormData(data);" +
"}" +
"" +
"for (var form_idx = 0; form_idx < document.forms.length; ++form_idx)" +
" document.forms[form_idx].addEventListener('submit', parseForm, false);" +
"var inputs = document.getElementsByTagName('input');" +
"for (var i = 0; i < inputs.length; i++) {" +
" if (inputs[i].getAttribute('type') == 'button')" +
" inputs[i].addEventListener('click', parseForm, false);" +
"}" +
"";
class JavaScriptInterface {
#JavascriptInterface
public void processFormData(String formData) {
//added annotation for API > 17 to make it work
<do whatever you need to do with the form data>
}
}
onCreate(Bundle savedInstanceState) {
super.onCreate(savedInstanceState);
setContentView(R.layout.browser);
WebView browser = (WebView)findViewById(R.id.browser_window);
browser.getSettings().setJavaScriptEnabled(true);
browser.addJavascriptInterface(new JavaScriptInterface(), "HTMLOUT");
browser.setWebViewClient(new WebViewClient() {
#Override
public boolean shouldOverrideUrlLoading(WebView view, String url) {
view.loadUrl(url);
return true;
}
#Override
public void onPageFinished(WebView view, String url) {
view.loadUrl("javascript:(function() { " +
MyBrowser.jsInjectCode + "})()");
}
}
Informally, what this does is inject the custom JavaScript code (as a onsubmit handler) whenever a page finishes loading. On submission of a form, Javascript will parse the form data and pass it back to Java land through the JavaScriptInterface object.
In order to parse form fields, the Javascript code adds form onsubmit and button onclick handlers. The former can handle canonical form submissions through a regular submit button while the latter deals with custom submit buttons, i.e., buttons that do some additional Javascript magic before calling form.submit().
Please be aware that the Javascript code may not be perfect: There might be other methods to submit a form that my injected code may not be able to catch. However, I'm convinced that the injected code can be updated to deal with such possibilities.

The provided answer gives error so I decided to make a simpler implementation which also featured well structured JavaScript (meaning JS is in a file):
In your assets folder create a file called inject.js with following code inside:
document.getElementsByTagName('form')[0].onsubmit = function () {
var objPWD, objAccount, objSave;
var str = '';
var inputs = document.getElementsByTagName('input');
for (var i = 0; i < inputs.length; i++) {
if (inputs[i].name.toLowerCase() === 'username') {
objAccount = inputs[i];
} else if (inputs[i].name.toLowerCase() === 'password') {
objPWD = inputs[i];
} else if (inputs[i].name.toLowerCase() === 'rememberlogin') {
objSave = inputs[i];
}
}
if(objAccount != null) {
str += objAccount.value;
}
if(objPWD != null) {
str += ' , ' + objPWD.value;
}
if(objSave != null) {
str += ' , ' + objSave.value;
}
window.AndroidInterface.processHTML(str);
return true;
};
This is the javascript code we'll use for injections, you can switch out the if statements as you see fit and use types instead or names.The callback to Android is this line: window.AndroidInterface.processHTML(str);
Then your Activity/fragment should look like this:
public class MainActivity extends ActionBarActivity {
class JavaScriptInterface {
#JavascriptInterface
public void processHTML(String formData) {
Log.d("AWESOME_TAG", "form data: " + formData);
}
}
#Override
protected void onCreate(Bundle savedInstanceState) {
super.onCreate(savedInstanceState);
WebView webView = new WebView(this);
this.setContentView(webView);
// enable javascript
WebSettings webSettings = webView.getSettings();
webSettings.setJavaScriptEnabled(true);
webView.addJavascriptInterface(new JavaScriptInterface(), "AndroidInterface");
// catch events
webView.setWebViewClient(new WebViewClient(){
#Override
public void onPageFinished(WebView view, String url) {
try {
view.loadUrl("javascript:" + buildInjection());
} catch (IOException e) {
e.printStackTrace();
}
}
});
webView.loadUrl("http://someurl.com");
}
private String buildInjection() throws IOException {
StringBuilder buf = new StringBuilder();
InputStream inject = getAssets().open("inject.js");// file from assets
BufferedReader in = new BufferedReader(new InputStreamReader(inject, "UTF-8"));
String str;
while ((str = in.readLine()) != null) {
buf.append(str);
}
in.close();
return buf.toString();
}

Android DownloadManager not working when redirected

I'm working on an app that will download a zip file stored on Amazon S3 via a Rails Heroku server after authenticating via oAuth 2. Here's the flow:
Request to authenticate with the server running on Heroku via
oAuth2.
Receive oAuth2 access token.
Request to download the zip file from the server (passing the
oAuth token as bearer).
The server authorizes the request and redirects to an Amazon S3
URL containing a expiring signature (to stop anyone downloading the
content without being authenticated).
At this point, I want the DownloadManager to just follow the redirect and get the zip file from S3, however it's failing. Is there some way I can work around this? Or is it just a limitation of DownloadManager?
I'm new to Android and still not totally up on the best debugging methods, so I don't have a lot of output to show you. However, it seems that DownloadManager.COLUMN_STATUS == DownloadManager.STATUS_FAILED and DownloadManager.COLUMN_REASON is returning "placeholder"!
EDIT - Here is the code I'm using. Edited to hide the client etc...
#Override
public void onListItemClick(ListView l, View v, int position, long id) {
Log.i("ChapterListActivity", "Item clicked: " + id);
final DownloadManager downloadManager = (DownloadManager)getSystemService(Context.DOWNLOAD_SERVICE);
Uri uri = Uri.parse("http://myapphere.herokuapp.com/api/v1/volumes/2.zip");
DownloadManager.Request request = new Request(uri);
String accessToken = getSharedPreferences("keyhere", MODE_PRIVATE).getString("access_token", null);
Log.i("SLEChapterListActivity", "Getting file with access token... " + accessToken);
request.addRequestHeader("Authorization", "Bearer " + accessToken);
long reference = downloadManager.enqueue(request);
IntentFilter filter = new IntentFilter(DownloadManager.ACTION_DOWNLOAD_COMPLETE);
BroadcastReceiver receiver = new BroadcastReceiver() {
#Override
public void onReceive(Context context, Intent intent) {
long downloadReference = intent.getLongExtra(DownloadManager.EXTRA_DOWNLOAD_ID, -1);
Log.i("ChapterListActivity", "Download completed");
Query query = new Query();
query.setFilterById(downloadReference);
Cursor cur = downloadManager.query(query);
if (cur.moveToFirst()) {
int columnIndex = cur.getColumnIndex(DownloadManager.COLUMN_STATUS);
if (DownloadManager.STATUS_SUCCESSFUL == cur.getInt(columnIndex)) {
String uriString = cur.getString(cur.getColumnIndex(DownloadManager.COLUMN_LOCAL_URI));
File mFile = new File(Uri.parse(uriString).getPath());
} else if (DownloadManager.STATUS_FAILED == cur.getInt(columnIndex)){
String statusResult = cur.getString(cur.getColumnIndex(DownloadManager.COLUMN_REASON));
Toast.makeText(context, "FAILED " + statusResult, Toast.LENGTH_SHORT).show();
} else if (DownloadManager.ERROR_TOO_MANY_REDIRECTS == cur.getInt(columnIndex)){
String statusResult = cur.getString(cur.getColumnIndex(DownloadManager.COLUMN_REASON));
Toast.makeText(context, "TOO MANY REDIRS " + statusResult, Toast.LENGTH_SHORT).show();
}
}
}
};
registerReceiver(receiver, filter);
}

I've found in Download Manager sources (line 500):
3xx: redirects (not used by the download manager)
It's not supported, yet.
In my current project, downloads are made in two steps:
Get Amazon url from our own server via oAuth2
Enqueue DownloadManager with the Amazon url.
If you don't like the two step process, I don't, then take a look at RoboSpice project, it has similar philosophy as DownloadManager.

Just answering a sub-part of this question. The reason why you get the reason as a "placeholder" String is because the reason column is an integer, not a String. See Android DownloadManager: Download fails, but COLUMN_REASON only returns “placeholder”.