Develop Reference

Streaming upload of a base64 image using retrofit

I have an upstream server that accepts image submissions using rest. The submitted image is part of a JSON payload similar to this one
{
"name": "Blah.jpg",
"uploader": "user1",
"image": "<base64.....>"
}
Using this strategy works for small images but generates Out of Memory errors on larger images.
Is it possible to stream the base64 component of the image? Pass in something like an iterator that will be used to read chunks of the image, base64 them and send them directly to the network?

Not with Gson or Moshi. Both of these libraries require strings to be in memory before emitting them to a stream.

I solved this with the following, in a class that extends okhttp3.RequestBody:
private void writeFile(File file, BufferedSink sink) throws IOException {
byte buf[] = new byte[3000];
try (FileInputStream fin = new FileInputStream(file)) {
while (fin.read(buf) >= 0) {
byte encoded[] = Base64.encodeBase64(buf);
sink.write(encoded);
}
}
}
It uses Android's android.util.Base64 Apache Commons' org.apache.commons.codec.binary.Base64 to encode a buffered chunk of data.
I ended up writing the other json fields separately, with enough granularity that I could insert the file record exactly where I needed to.
EDIT:
As you can see in my edits above, I ended up switching to Apache commons-codec, via compile 'commons-codec:commons-codec:1.5' in my build.gradle file.
I didn't have time to investigate why the Android SDK solution didn't work. I tried their Base64.encode(buf, Base64.NO_WRAP) as suggested elsewhere - supposedly equivalent to Apache Commons' encodeBase64(byte[]) - but this did not work, hence the switch.
The problem could have been on our backend, so don't rule out Android SDK's solution based on my post alone - I just wanted to add this note so readers can see the code snippet that actually worked for me in the end.

Android Webview POST request

I'm trying to do a post request with a WebView on Android.
After searching for days and trying dozens of things i couldn't get it work. In SWIFT it's just a few lines of code so i thought there must also be a simple way to do a post request for a webview on android.
As (for 2016) EncodingUtils and HTTPClient are deprecated this are my current approaches:
String url = "http://example.com/php.php";
String postData = null;
postData = "param1=" + URLEncoder.encode("1234567890", "UTF-8");
webcontent.postUrl(url,postData.getBytes());
//or
webcontent.postUrl(url, Base64.encode(postData.getBytes(), Base64.DEFAULT));
Both just result in a blank screen. There is just one parameter to be sent and a string containing html from the server should be received.
In addition, the php on the server returns a html-string with colored background irrespective of any input, but even this isn't displayed so maybe the whole request never reaches the server?
Thanks in advance!

In Android you do not use webView to access the content of the HTTP response. You'll need to use HttpClient for that purpose!
See this nice tutorial which explains the fundamentals! Also see this video if you find it hard!
Hope it helps!

Missing parameter access_token on OAuth2 request

I'm using the Apache Amber libraries to try to retrieve an OAuth2 access token from a Web site under my control. My client code is running under Android.
My code is patterned on the example at:
https://cwiki.apache.org/confluence/display/AMBER/OAuth+2.0+Client+Quickstart
In the first step, I'm able to retrieve a "code" by submitting a GET request using a WebView browser:
OAuthClientRequest request = OAuthClientRequest
.authorizationLocation(AUTHORIZE_URL)
.setClientId(CLIENT_ID)
.setRedirectURI(REDIR_URL)
.setResponseType(CODE_RESPONSE)
.buildQueryMessage();
webview.loadUrl(request.getLocationUri());
I use a WebViewClient callback to capture the redirect URL with the "code" parameter. So far, so good.
Using that code, I try to retrieve my access token:
OAuthClient oAuthClient = new OAuthClient(new URLConnectionClient());
OAuthClientRequest request = OAuthClientRequest
.tokenLocation(ACCESS_TOKEN_URL)
.setGrantType(GrantType.AUTHORIZATION_CODE)
.setClientId(CLIENT_ID)
.setClientSecret(CLIENT_SECRET)
.setRedirectURI(REDIR_URL)
.setCode(code)
.buildBodyMessage();
GitHubTokenResponse oAuthResponse =
oAuthClient.accessToken(request, GitHubTokenResponse.class);
Each time I run my code, I get an OAuthProblemException, where the message is that I have an invalid request due to a missing parameter, access_token.
Another StackOverflow post mentions this exception from a similar OAuth2 request, which in that case was caused by having different redirect URIs across OAuth requests. But I've made sure my redirect URIs are the same by using a named constant. Here's the link to that post:
OAuthProblem, missing parameter access_token
Now, I can print out the code returned by the first request, and paste it into a curl command run from my desktop machine:
curl -d "code=...&client_id=...&client_secret=...&grant_type=...&redirect_uri=..." http://my_website.com
and I get a nice JSON response from my site with an access_token.
Why does the call from Java fail, where my hand-rolled command line succeeds?

I had the same problem implementing the client and the server, the problem is about one mistake in the Client Example in the Apache Amber (Oltu) project:
First you have the Auth code request (which work):
OAuthClientRequest request = OAuthClientRequest
.authorizationLocation(AUTHORIZE_URL)
.setClientId(CLIENT_ID)
.setRedirectURI(REDIR_URL)
.setResponseType(CODE_RESPONSE)
.**buildQueryMessage**();
And second the request about the Access Token (which don't work):
OAuthClientRequest request = OAuthClientRequest
.tokenLocation(ACCESS_TOKEN_URL)
.setGrantType(GrantType.AUTHORIZATION_CODE)
.setClientId(CLIENT_ID)
.setClientSecret(CLIENT_SECRET)
.setRedirectURI(REDIR_URL)
.setCode(code)
.**buildBodyMessage**();
The mistake is about the buildBodyMessage() in the second request. Change it by buildQueryMessage().

Solved in my case.
Amber/Oltu "Missing parameter access_token" error may mean that GitHubTokenResponse or OAuthJSONAccessTokenResponse are unabled to translate response body for any reason. In my case (with Google+ oAuth2 authentication), the response body, is not parsed properly to the inner parameters map.
For example:
GitHubTokenResponse
parameters = OAuthUtils.decodeForm(body);
Parse a form-urlencoded result body
... and OAuthJSONAccessTokenResponse has the next parse function
parameters = JSONUtils.parseJSON(body);
This JSONUtils.parseJSON is a custom JSON parser that not allow for me JSON response body from GOOGLE+ and throws an JSONError (console not logged),
Each error throwed parsing this parameters, are not console visible, and then always is throwed doomed "Missing parameter: access_token" or another "missing parameter" error.
If you write your Custom OAuthAccessTokenResponse, you can see response body, and write a parser that works with your response.

This is what I encountered and what I did to get it working:
I quickly put together a similar example described in:
https://cwiki.apache.org/confluence/display/OLTU/OAuth+2.0+Client+Quickstart
and:
svn.apache.org/repos/asf/oltu/trunk/oauth-2.0/client/src/test/java/org/apache/oltu/oauth2/client/OAuthClientTest.java
This was my command to execute it:
java -cp .:./org.apache.oltu.oauth2.client-1.0.1-20150221.171248-36.jar OAuthClientTest
I also ended up with the above mentioned error where the access_token was expected. I ended up debugging in intellij and traced an anomaly with the if condition which checks that the string begins with the "{" character.
In doing so, I also added the following jar to my classpath so that I may debug the trace a little deeper.
./java-json.jar
(downloaded from http://www.java2s.com/Code/Jar/j/Downloadjavajsonjar.htm)
During the next debug session, the code actually started working. My mate and I eventually found the root cause was due to the JSON jar not being included.
This is the command which works:
java -cp .:./java-json.jar:./org.apache.oltu.oauth2.client-1.0.1-20150221.171248-36.jar OAuthClientTest

I was having the same problem when trying to get the access token from fitbit OAuth2. buildBodyMessage() and buildQueryMessage() were both giving me missing parameter, access_token.
I believe this is something to do with the apache oauth2 client library. I ended up making simple post requests using spring's RestTemplate and it's working fine.
HttpHeaders headers = new HttpHeaders();
headers.setContentType(MediaType.APPLICATION_FORM_URLENCODED);
headers.set("Authorization", "Basic " + "MjI5TkRZOjAwNDBhNDBkMjRmZTA0OTJhNTE5NzU5NmQ1N2ZmZGEw");
MultiValueMap<String, String> map = new LinkedMultiValueMap<>();
map.add("client_id", FITBIT_CLIENT_ID);
map.add("grant_type", "authorization_code");
map.add("redirect_uri", Constants.RESTFUL_PATH + "/fitbit/fitbitredirect");
map.add("code", code);
HttpEntity<MultiValueMap<String, String>> request = new HttpEntity<>(map, headers);
RestTemplate restTemplate = new RestTemplate();
ResponseEntity<String> response = restTemplate.postForEntity(FITBIT_TOKEN_URI, request, String.class);
log.debug("response.body: " + response.getBody());

Mystery URL decoding of certain characters - Android URI / Google App Engine

I am having a curious problem that perhaps someone has insight into. I encode a query string into a URL on Android using the following code:
request = REQUEST_BASE + "?action=loadauthor&author=" + URLEncoder.encode(author, "UTF-8");
I then add a few other parameters to the string and create a URI like this:
uri = new URI(request);
At a certain point, I pull out the query string to make a checksum:
uri.getRawQuery().getBytes();
Then I send it on its way with:
HttpGet get = new HttpGet(uri);
On the Appengine server, I then retrieve the string and try to match the checksum:
String query = req.getQueryString();
Normally, this works fine. However, there are a few characters that seem to get unencoded on the way to the server. For example,
action=loadauthor&author=Charles+Alexander+%28Ohiyesa%29+Eastman&timestamp=1343261225838&user=1479845600
shows up in the server logs (and in the GAE app) as:
action=loadauthor&author=Charles+Alexander+(Ohiyesa)+Eastman&timestamp=1343261226837&user=1479845600
This only happens to a few characters (like parentheses). Other characters remain encoded all the way through. Does anyone have a thought about what I might be doing wrong? Any feedback is appreciated.

I never did find a solution for this problem. I worked around it by unencoding certain characters on the client before sending things to the server:
request = request.replace("%28", "(");
request = request.replace("%29", ")");
request = request.replace("%27", "'");
If anyone has a better solution, I am sure that I (and others) would be interested!

URLEncoder does not encode parentheses and certain other characters, as they are supposed to be "safe" for most servers. See URLEncoder. You will have to replace these yourself if necessary.
Example:
URI uri = new URI(request.replace("(","%28"));
If a lot of replacements are needed, you can try request.replaceAll(String regularExpression, String replacement). This, of course, requires knowledge of regular expressions.

Setting a request body in Android's DownloadManager?

For the next step of my application, I need to add download functionality. The user chooses what they want to download and could select anything from 1 file to thousands of them if they could be bothered to select that many.
I want to use Android's built in DownloadManager to provide this downloading functionality, but unfortunately I cannot see how I could implement it for my scenario.
In order for the target server to authorize the download, I need to send some JSON along in the request body. Like this, if I was doing it manually:
DataOutputStream output = new DataOutputStream(connection.getOutputStream());
output.writeBytes(rawData);
output.flush();
Where rawData is the JSON string. The request body is always set to POST.
I can't seem to find any way to add this JSON string to the DownloadManager, and until I can do that, the server will always reject the download.
The only other solution that I can think of, which I desperately want to avoid, is writing a PHP script on my server to take some GET parameters, generate the JSON and then redirect the request.
Does anybody know of a way that I can send my JSON data along with the DownloadManager? Each file that I'm downloading needs its own, unique, JSON string.

you cannot do that in Android's Download manager, see Download Manager Issue,
I had a similar requirement and I ended up Using HttpClient (Xamarin).
Sample Code-
using(var httpClient = new HttpClient()) {
using(var request = new HttpRequestMessage(new HttpMethod("POST"), URL)) {
request.Headers.TryAddWithoutValidation("User-Agent", userAgent);
request.Headers.TryAddWithoutValidation("Accept", "text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8");
request.Headers.TryAddWithoutValidation("Accept-Language", "en-US,en;q=0.5");
request.Headers.TryAddWithoutValidation("Connection", "keep-alive");
request.Headers.TryAddWithoutValidation("Referer",RefererURL);
request.Content = new StringContent("YOUR_REQUEST_BODY_HERE");
request.Content.Headers.ContentType = MediaTypeHeaderValue.Parse("application/x-www-form-urlencoded");
var responser = await httpClient.SendAsync(request);
return responser;
}