I'm trying to upload an image (in streams) on the s3 presignedURL. However, it's giving me 403. The same file when uploaded via POSTMAN works, but not when done via retrofit. What am I missing?
The following is my code:-
val stream = ByteArrayOutputStream()
bitmap.compress(Bitmap.CompressFormat.JPEG, 100, stream)//bitmap is the bitmap of the image
val byteArray = stream.toByteArray()
val encodedImage = Base64.encodeToString(byteArray, Base64.DEFAULT)
val requestBody : RequestBody = encodedImage.toRequestBody()// for base64 format
val url = 'https://s3.presigned_url.com'
val baseUrl = url.split(".com").toTypedArray()[0] + ".com"
var queryUrl: String? = url.split(".com").toTypedArray()[1]
val client = OkHttpClient.Builder().build()
val retrofit_image_upload: Retrofit = Retrofit.Builder()
.baseUrl(baseUrl)
.addConverterFactory(GsonConverterFactory.create())
.client(client)
.build()
var request1 = retrofit_image_upload.create(Api::class.java)
try {
request1.upload(
url,
fileSize,//size of the image
requestBody,
).enqueue(
object : retrofit2.Callback<Void> {
override fun onResponse(call: Call<Void>, response: Response<Void>) {
println(response.code().toString())
}
override fun onFailure(call: Call<Void>, t: Throwable) {
println(t.toString())
}
}
)
} catch (e: IOException) {
e.printStackTrace()
}
And here is the upload method:-
#PUT
fun upload(
#Url url: String,
#Header("content-length") contentlength: Int,
#Body image: RequestBody,
): Call<Void>
Also converted the request body into streams but the result is the same.
I referred below articles but still cannot resolve it.
Android HttpURLConnection PUT to Amazon AWS S3 403 error
PUT upload file to AWS S3 presigned url Retrofit2 Android
https://gutier.io/post/android-upload-file-to-aws-s3-bucket-with-retrofit2/
Can I use this signer https://github.com/babbel/okhttp-aws-signer for upload file to s3?
I use this https://bytes.babbel.com/en/articles/2019-01-03-okhttp-aws-signer.html reference when build my sample app:
class AwsSigingInterceptor(private val signer: OkHttpAwsV4Signer) : Interceptor {
private val dateFormat: ThreadLocal<SimpleDateFormat>
init {
dateFormat = object : ThreadLocal<SimpleDateFormat>() {
override fun initialValue(): SimpleDateFormat {
val localFormat = SimpleDateFormat("yyyyMMdd'T'HHmmss'Z'", Locale.US)
localFormat.timeZone = TimeZone.getTimeZone("UTC")
return localFormat
}
}
}
override fun intercept(chain: Chain): Response =
chain.run {
val request = request()
val newRequest = request.newBuilder()
.addHeader("x-amz-date", dateFormat.get().format(clock.now()))
.addHeader("host", request.url().host())
.build()
val signed = signer.sign(newRequest, "<accessKeyId>", "<secretAccessKey>")
proceed(signed)
}
}
but I didn't see any x-amz-content-sha256 there.. only additional Authorization header added. When I try the debug result header in Advanced Rest Client or Postman, it says "Missing required header for this request: x-amz-content-sha256"
I'm using PRDownloader. I want to download a file from my Google Drive, hence I need to supply Authorization: Bearer <ACCESS_TOKEN> .. how can I do that with PRDownloader? I don't see any way to do this from the website, but perhaps there's some unofficial way?
I stumbled upon the same problem and ended up using Fetch instead. A working example looks then like this. The important line can be found in the download method:
request.addHeader("Authorization", "Bearer $accessToken")
Here is the complete DummyActivity:
class MainActivity : AppCompatActivity() {
private lateinit var fetch: Fetch
override fun onCreate(savedInstanceState: Bundle?) {
super.onCreate(savedInstanceState)
setContentView(R.layout.activity_main)
Timber.plant(Timber.DebugTree())
val okHttpClient = OkHttpClient.Builder().build()
val fetchConfiguration =
FetchConfiguration.Builder(this)
.setDownloadConcurrentLimit(10)
.setProgressReportingInterval(100)
.setHttpDownloader(OkHttpDownloader(okHttpClient))
.build()
fetch = getInstance(fetchConfiguration)
button.setOnClickListener { download() }
}
private fun download() {
val url = "https://yourdomain.com/download/path.zip"
val dirPath = getExternalFilesDir(null)?.path
val fileName = "ResultFileName.zip"
val accessToken = "your_access_token"
val request = Request(url, Uri.fromFile(File(dirPath, fileName)))
request.priority = Priority.HIGH
request.networkType = NetworkType.ALL
request.addHeader("Authorization", "Bearer $accessToken")
fetch.enqueue(
request,
Func { updatedRequest -> Timber.d("request updated: ${updatedRequest.id})") },
Func { error: Error? -> Timber.d("Error: ${error?.name}")}
)
}
}
For anyone else stumbling into this. You can try:
PRDownloader.download(url, dir.absolutePath, filename)
.setHeader("accept-language", headers.getString("accept-language"))
.setHeader("Authorization", headers.getString("Authorization"))
.build()
I want to do a login validation using POST method and to get some information using GET method.
I've URL, server Username and Password already of my previous project.
For Android, Volley is a good place to get started. For all platforms, you might also want to check out ktor client or http4k which are both good libraries.
However, you can also use standard Java libraries like java.net.HttpURLConnection
which is part of the Java SDK:
fun sendGet() {
val url = URL("http://www.google.com/")
with(url.openConnection() as HttpURLConnection) {
requestMethod = "GET" // optional default is GET
println("\nSent 'GET' request to URL : $url; Response Code : $responseCode")
inputStream.bufferedReader().use {
it.lines().forEach { line ->
println(line)
}
}
}
}
Or simpler:
URL("https://google.com").readText()
Send HTTP POST/GET request with parameters using HttpURLConnection :
POST with Parameters:
fun sendPostRequest(userName:String, password:String) {
var reqParam = URLEncoder.encode("username", "UTF-8") + "=" + URLEncoder.encode(userName, "UTF-8")
reqParam += "&" + URLEncoder.encode("password", "UTF-8") + "=" + URLEncoder.encode(password, "UTF-8")
val mURL = URL("<Your API Link>")
with(mURL.openConnection() as HttpURLConnection) {
// optional default is GET
requestMethod = "POST"
val wr = OutputStreamWriter(getOutputStream());
wr.write(reqParam);
wr.flush();
println("URL : $url")
println("Response Code : $responseCode")
BufferedReader(InputStreamReader(inputStream)).use {
val response = StringBuffer()
var inputLine = it.readLine()
while (inputLine != null) {
response.append(inputLine)
inputLine = it.readLine()
}
println("Response : $response")
}
}
}
GET with Parameters:
fun sendGetRequest(userName:String, password:String) {
var reqParam = URLEncoder.encode("username", "UTF-8") + "=" + URLEncoder.encode(userName, "UTF-8")
reqParam += "&" + URLEncoder.encode("password", "UTF-8") + "=" + URLEncoder.encode(password, "UTF-8")
val mURL = URL("<Yout API Link>?"+reqParam)
with(mURL.openConnection() as HttpURLConnection) {
// optional default is GET
requestMethod = "GET"
println("URL : $url")
println("Response Code : $responseCode")
BufferedReader(InputStreamReader(inputStream)).use {
val response = StringBuffer()
var inputLine = it.readLine()
while (inputLine != null) {
response.append(inputLine)
inputLine = it.readLine()
}
it.close()
println("Response : $response")
}
}
}
Using only the standard library with minimal code!
thread {
val json = try {
URL(url).readText()
} catch (e: Exception) {
return#thread
}
runOnUiThread { displayOrWhatever(json) }
}
This starts a GET request on a new thread, leaving the UI thread to respond to user input. However, we can only modify UI elements from the main/UI thread, so we actually need a runOnUiThread block to show the result to our user. This enqueues our display code to be run on the UI thread soon.
The try/catch is there so your app won't crash if you make a request with your phone's internet off. Add your own error handling (e.g. showing a Toast) as you please.
.readText() is not part of the java.net.URL class but a Kotlin extension method, Kotlin "glues" this method onto URL. This is enough for plain GET requests, but for more control and POST requests you need something like the Fuel library.
Have a look at Fuel library, a sample GET request
"https://httpbin.org/get"
.httpGet()
.responseString { request, response, result ->
when (result) {
is Result.Failure -> {
val ex = result.getException()
}
is Result.Success -> {
val data = result.get()
}
}
}
// You can also use Fuel.get("https://httpbin.org/get").responseString { ... }
// You can also use FuelManager.instance.get("...").responseString { ... }
A sample POST request
Fuel.post("https://httpbin.org/post")
.jsonBody("{ \"foo\" : \"bar\" }")
.also { println(it) }
.response { result -> }
Their documentation can be found here
I think using okhttp is the easiest solution. Here you can see an example for POST method, sending a json, and with auth.
val url = "https://example.com/endpoint"
val client = OkHttpClient()
val JSON = MediaType.get("application/json; charset=utf-8")
val body = RequestBody.create(JSON, "{\"data\":\"$data\"}")
val request = Request.Builder()
.addHeader("Authorization", "Bearer $token")
.url(url)
.post(body)
.build()
val response = client . newCall (request).execute()
println(response.request())
println(response.body()!!.string())
Remember to add this dependency to your project https://mvnrepository.com/artifact/com.squareup.okhttp3/okhttp
UPDATE: July 7th, 2019
I'm gonna give two examples using latest Kotlin (1.3.41), OkHttp (4.0.0) and Jackson (2.9.9).
UPDATE: January 25th, 2021
Everything is okay with the most updated versions.
<!-- https://mvnrepository.com/artifact/com.fasterxml.jackson.module/jackson-module-kotlin -->
<dependency>
<groupId>com.fasterxml.jackson.module</groupId>
<artifactId>jackson-module-kotlin</artifactId>
<version>2.12.1</version>
</dependency>
<!-- https://mvnrepository.com/artifact/com.squareup.okhttp3/okhttp -->
<dependency>
<groupId>com.squareup.okhttp3</groupId>
<artifactId>okhttp</artifactId>
<version>4.9.0</version>
</dependency>
Get Method
fun get() {
val client = OkHttpClient()
val url = URL("https://reqres.in/api/users?page=2")
val request = Request.Builder()
.url(url)
.get()
.build()
val response = client.newCall(request).execute()
val responseBody = response.body!!.string()
//Response
println("Response Body: " + responseBody)
//we could use jackson if we got a JSON
val mapperAll = ObjectMapper()
val objData = mapperAll.readTree(responseBody)
objData.get("data").forEachIndexed { index, jsonNode ->
println("$index $jsonNode")
}
}
POST Method
fun post() {
val client = OkHttpClient()
val url = URL("https://reqres.in/api/users")
//just a string
var jsonString = "{\"name\": \"Rolando\", \"job\": \"Fakeador\"}"
//or using jackson
val mapperAll = ObjectMapper()
val jacksonObj = mapperAll.createObjectNode()
jacksonObj.put("name", "Rolando")
jacksonObj.put("job", "Fakeador")
val jacksonString = jacksonObj.toString()
val mediaType = "application/json; charset=utf-8".toMediaType()
val body = jacksonString.toRequestBody(mediaType)
val request = Request.Builder()
.url(url)
.post(body)
.build()
val response = client.newCall(request).execute()
val responseBody = response.body!!.string()
//Response
println("Response Body: " + responseBody)
//we could use jackson if we got a JSON
val objData = mapperAll.readTree(responseBody)
println("My name is " + objData.get("name").textValue() + ", and I'm a " + objData.get("job").textValue() + ".")
}
Maybe the simplest GET
For everybody stuck with NetworkOnMainThreadException for the other solutions: use AsyncTask or, even shorter, (yet still experimental) Coroutines:
launch {
val jsonStr = URL("url").readText()
}
If you need to test with plain http don't forget to add to your manifest:
android:usesCleartextTraffic="true"
For the experimental Coroutines you have to add to build.gradle as of 10/10/2018:
kotlin {
experimental {
coroutines 'enable'
}
}
dependencies {
implementation "org.jetbrains.kotlinx:kotlinx-coroutines-core:0.24.0"
implementation "org.jetbrains.kotlinx:kotlinx-coroutines-android:0.24.0"
...
If you are using Kotlin, you might as well keep your code as succinct as possible. The run method turns the receiver into this and returns the value of the block.
this as HttpURLConnection creates a smart cast. bufferedReader().readText() avoids a bunch of boilerplate code.
return URL(url).run {
openConnection().run {
this as HttpURLConnection
inputStream.bufferedReader().readText()
}
}
You can also wrap this into an extension function.
fun URL.getText(): String {
return openConnection().run {
this as HttpURLConnection
inputStream.bufferedReader().readText()
}
}
And call it like this
return URL(url).getText()
Finally, if you are super lazy, you can extend the String class instead.
fun String.getUrlText(): String {
return URL(this).run {
openConnection().run {
this as HttpURLConnection
inputStream.bufferedReader().readText()
}
}
}
And call it like this
return "http://somewhere.com".getUrlText()
You can use kohttp library. It is a Kotlin DSL HTTP client. It supports the features of square.okhttp and provides a clear DSL for them. KoHttp async calls are powered by coroutines.
httpGet extension function
val response: Response = "https://google.com/search?q=iphone".httpGet()
you can also use async call with coroutines
val response: Deferred<Response> = "https://google.com/search?q=iphone".asyncHttpGet()
or DSL function for more complex requests
val response: Response = httpGet {
host = "google.com"
path = "/search"
param {
"q" to "iphone"
"safe" to "off"
}
}
You can find more details in docs
To get it with gradle use
implementation 'io.github.rybalkinsd:kohttp:0.12.0'
Without adding additional dependencies, this works. You don't need Volley for this. This works using the current version of Kotlin as of Dec 2018: Kotlin 1.3.10
If using Android Studio, you'll need to add this declaration in your AndroidManifest.xml:
<uses-permission android:name="android.permission.INTERNET" />
<uses-permission android:name="android.permission.ACCESS_NETWORK_STATE" />
You should manually declare imports here. The auto-import tool caused me many conflicts.:
import android.os.AsyncTask
import java.io.BufferedReader
import java.io.InputStreamReader
import java.io.OutputStream
import java.io.OutputStreamWriter
import java.net.URL
import java.net.URLEncoder
import javax.net.ssl.HttpsURLConnection
You can't perform network requests on a background thread. You must subclass AsyncTask.
To call the method:
NetworkTask().execute(requestURL, queryString)
Declaration:
private class NetworkTask : AsyncTask<String, Int, Long>() {
override fun doInBackground(vararg parts: String): Long? {
val requestURL = parts.first()
val queryString = parts.last()
// Set up request
val connection: HttpsURLConnection = URL(requestURL).openConnection() as HttpsURLConnection
// Default is GET so you must override this for post
connection.requestMethod = "POST"
// To send a post body, output must be true
connection.doOutput = true
// Create the stream
val outputStream: OutputStream = connection.outputStream
// Create a writer container to pass the output over the stream
val outputWriter = OutputStreamWriter(outputStream)
// Add the string to the writer container
outputWriter.write(queryString)
// Send the data
outputWriter.flush()
// Create an input stream to read the response
val inputStream = BufferedReader(InputStreamReader(connection.inputStream)).use {
// Container for input stream data
val response = StringBuffer()
var inputLine = it.readLine()
// Add each line to the response container
while (inputLine != null) {
response.append(inputLine)
inputLine = it.readLine()
}
it.close()
// TODO: Add main thread callback to parse response
println(">>>> Response: $response")
}
connection.disconnect()
return 0
}
protected fun onProgressUpdate(vararg progress: Int) {
}
override fun onPostExecute(result: Long?) {
}
}
GET and POST using OkHttp
private const val CONNECT_TIMEOUT = 15L
private const val READ_TIMEOUT = 15L
private const val WRITE_TIMEOUT = 15L
private fun performPostOperation(urlString: String, jsonString: String, token: String): String? {
return try {
val client = OkHttpClient.Builder()
.connectTimeout(CONNECT_TIMEOUT, TimeUnit.SECONDS)
.writeTimeout(WRITE_TIMEOUT, TimeUnit.SECONDS)
.readTimeout(READ_TIMEOUT, TimeUnit.SECONDS)
.build()
val body = jsonString.toRequestBody("application/json; charset=utf-8".toMediaTypeOrNull())
val request = Request.Builder()
.url(URL(urlString))
.header("Authorization", token)
.post(body)
.build()
val response = client.newCall(request).execute()
response.body?.string()
}
catch (e: IOException) {
e.printStackTrace()
null
}
}
private fun performGetOperation(urlString: String, token: String): String? {
return try {
val client = OkHttpClient.Builder()
.connectTimeout(CONNECT_TIMEOUT, TimeUnit.SECONDS)
.writeTimeout(WRITE_TIMEOUT, TimeUnit.SECONDS)
.readTimeout(READ_TIMEOUT, TimeUnit.SECONDS)
.build()
val request = Request.Builder()
.url(URL(urlString))
.header("Authorization", token)
.get()
.build()
val response = client.newCall(request).execute()
response.body?.string()
}
catch (e: IOException) {
e.printStackTrace()
null
}
}
Object serialization and deserialization
#Throws(JsonProcessingException::class)
fun objectToJson(obj: Any): String {
return ObjectMapper().writeValueAsString(obj)
}
#Throws(IOException::class)
fun jsonToAgentObject(json: String?): MyObject? {
return if (json == null) { null } else {
ObjectMapper().readValue<MyObject>(json, MyObject::class.java)
}
}
Dependencies
Put the following lines in your gradle (app) file. Jackson is optional. You can use it for object serialization and deserialization.
implementation 'com.squareup.okhttp3:okhttp:4.3.1'
implementation 'com.fasterxml.jackson.core:jackson-core:2.9.8'
implementation 'com.fasterxml.jackson.core:jackson-annotations:2.9.8'
implementation 'com.fasterxml.jackson.core:jackson-databind:2.9.8'
You can use this library Fuel Library as well, which makes it further easier.
val map = mutableMapOf<String, String>()
map.put("id","629eeb9da9d8f50016e1af96")
val httpAsync = url
.httpPost()
.jsonBody(
Gson().toJson(map) // for json string
)
.responseString { request, response, result -> //do something with the response }
I am uploading a dynamic number of files in a single multipart request using Retrofit2. My retrofit interface looks like this -
public interface FileUploadService {
#Multipart
#POST("upload")
Call<ResponseBody> uploadMultipleFilesDynamic(
#Part List<MultipartBody.Part> files);
}
Now I want to track progress for this multi-file upload. This solution explains how to get progress while uploading a single file in a multipart request by extending RequestBody. Though I can't seem to comprehend how to apply this for my multiple files request. One solution I could think of was to create ProgressRequestBody by extending OkHTTP MultipartBody class instead of RequestBody but OkHTTP3 implements MultipartBody as a final class making it impossible to extend. Can anyone point me in the right direction as it's a huge blocker for me to not be able to show the progress to the user for files upload. Or are there any work arounds that I can implement to achieve this functionality?
I've followed this blogpost: https://medium.com/#PaulinaSadowska/display-progress-of-multipart-request-with-retrofit-and-rxjava-23a4a779e6ba and then made the following adjustments to display the total progression instead of progression of separate files:
private fun prepareFileParts(reportAttachments: MutableList<ReportAttachment>, emitter: FlowableEmitter<Double>): List<MultipartBody.Part> {
val multiPartBodyList = mutableListOf<MultipartBody.Part>()
var offset = 0L
var totalLength = 0L
// calculate the total length of all files
for (attachment in reportAttachments) {
val file = File(attachment.filePath)
totalLength += file.length()
}
// create requestbody for each file and calculate the progression offset
for (attachment in reportAttachments) {
val file = File(attachment.filePath)
val mimeType = attachment.mimeType
val countingRequestBody = createCountingRequestBody(file, mimeType, emitter, offset, totalLength)
offset += file.length()
val multipartBody = MultipartBody.Part.createFormData("file", file.name, countingRequestBody)
multiPartBodyList.add(multipartBody)
}
return multiPartBodyList
}
private fun createCountingRequestBody(file: File, mimeType: String, emitter: FlowableEmitter<Double>, offset: Long, totalLength: Long): RequestBody {
val requestBody = RequestBody.create(MediaType.parse(mimeType), file)
return CountingRequestBody(requestBody, object : CountingRequestBody.Listener {
override fun onRequestProgress(bytesWritten: Long, contentLength: Long) {
val progress: Double = 1.0 * (offset + bytesWritten) / totalLength
emitter.onNext(progress)
}
})
}
If you want, you can also create an interceptor and add it to your OkHttpClient. This would track all outgoing API calls by default. It would look something like this:
class UploadProgressInterceptor(private val progressListener: CountingRequestBody.Listener) : Interceptor {
override fun intercept(chain: Interceptor.Chain): Response {
val originalRequest = chain.request()
if (originalRequest.body() == null) {
return chain.proceed(originalRequest)
}
val requestBody = originalRequest.body()
requestBody?.let {
val progressRequest = originalRequest.newBuilder()
.method(originalRequest.method(), CountingRequestBody(it, progressListener))
.build()
return chain.proceed(progressRequest)
}
return chain.proceed(originalRequest)
}