I have a progreesBar for uploading with retrofit and I implementation that with some of examples.
my problem is 'WriteTo' function in my custom requestBody class.
This function send progress value for use in my progressBar but this function is called twice. I used debugger and I think some interceptors call WriteTo function.
Let me explain the problem more clearly,When I click Upload button, The number of progress bar reaches one hundred and then it starts again from zero.
Some of the things I did:
I removed HttpLoggingInterceptor.
I used a boolean variable for check that 'writeTo' don't post anything the first time
I don't have any extra interceptors
Also I read this topics:
Retrofit 2 RequestBody writeTo() method called twice
using Retrofit2/okhttp3 upload file,the upload action always performs twice,one fast ,and other slow
Interceptor Problem
My codes:
ProgressRequestBody class
class ProgressRequestBody : RequestBody() {
var mutableLiveData = MutableLiveData<Int>()
lateinit var mFile: File
lateinit var contentType: String
companion object {
private const val DEFAULT_BUFFER_SIZE = 2048
}
override fun contentType(): MediaType? {
return "$contentType/*".toMediaTypeOrNull()
}
#Throws(IOException::class)
override fun contentLength(): Long {
return mFile.length()
}
#Throws(IOException::class)
override fun writeTo(sink: BufferedSink) {
val fileLength = mFile.length()
val buffer = ByteArray(DEFAULT_BUFFER_SIZE)
val `in` = FileInputStream(mFile)
var uploaded: Long = 0
`in`.use { `in` ->
var read: Int
while (`in`.read(buffer).also { read = it } != -1) {
val percentage = (100 * uploaded / fileLength).toInt()
mutableLiveData.postValue(percentage)
uploaded += read.toLong()
sink.write(buffer, 0, read)
}
}
}
}
private fun upload(file: File, fileType: FileType) {
val fileBody = ProgressRequestBody()
fileBody.mFile = file
fileBody.contentType = file.name
uploadImageJob = viewModelScope.launch(Dispatchers.IO) {
val body = MultipartBody.Part.createFormData("File", file.name, fileBody)
fileUploadRepo.upload(body).catch {
// ...
}.collect {
when (it) {
// ...
}
}
}
}
In my fragment I use liveData for collect progressBar progress value.
I have the request which works well in postman:
and I'm trying to make it with Retrofit. In general file sizes will be >500MB that. I did such uploading method:
fun uploadFile(file:File) {
val client = OkHttpClient().newBuilder()
.build()
val mediaType: MediaType? = "text/plain".toMediaTypeOrNull()
val body: RequestBody = MultipartBody.Builder().setType(MultipartBody.FORM)
.addFormDataPart(
"data", file.name,
file.asRequestBody()
)
.build()
val request: Request = Request.Builder()
.url("https://..../upload.php")
.method("POST", body)
.build()
val response: okhttp3.Response = client.newCall(request).execute()
println(response.message)
}
but I need to have file for uploading. I can create temporary file with such way:
val path = requireContext().cacheDir
val file = File.createTempFile(
name ?: "",
fileUri.lastPathSegment,
path
)
val os = FileOutputStream(file)
os.write(string)
os.close()
but I usually receive outOfMemoryException. I also added to the AndroidManifest.xml heap param:
android:largeHeap="true"
but it didn't help me at all during temp file creating. I don't know how postman uploads files, but in general I managed to upload with his help file with size about 600Mb. I can also cut selected file with chunks:
val data = result.data
data?.let {
val fileUri = data.data
var name: String? = null
var size: Long? = null
fileUri.let { returnUri ->
contentResolver?.query(returnUri!!, null, null, null, null)
}?.use { cursor ->
val nameIndex = cursor.getColumnIndex(OpenableColumns.DISPLAY_NAME)
val sizeIndex = cursor.getColumnIndex(OpenableColumns.SIZE)
cursor.moveToFirst()
name = cursor.getString(nameIndex)
size = cursor.getLong(sizeIndex)
}
val inputStream: InputStream? = fileUri?.let { it1 ->
contentResolver.openInputStream(
it1
)
}
val fileData = inputStream?.readBytes()
val mimeType = fileUri.let { returnUri ->
returnUri.let { retUri ->
if (retUri != null) {
contentResolver.getType(retUri)
}
}
}
fileData?.let {
val MAX_SUB_SIZE = 4194304 // 4*1024*1024 == 4MB
var start = 0 // From 0
var end = MAX_SUB_SIZE // To MAX_SUB_SIZE bytes
var subData: ByteArray // 4MB Sized Array
val max = fileData.size
if (max > 0) {
while (end < max) {
subData = fileData.copyOfRange(start, end)
start = end
end += MAX_SUB_SIZE
if (end >= max) {
end = max
}
println("file handling" + subData.size)
}
end-- // To avoid a padded zero
subData = fileData.copyOfRange(start, end)
println("file handling" + subData.size)
}
}
}
all actions will be made in:
private val filesReceiver =
registerForActivityResult(ActivityResultContracts.StartActivityForResult()) { result ->
if (result.resultCode == Activity.RESULT_OK) {
}
}
so I won't have any file path in normal way. Anyway I think I did something wrong.
UPDATE
right now I have such file uploading from inputStream:
private fun doSomeNetworkStuff(file:InputStream, name:String) {
GlobalScope.launch(Dispatchers.IO) {
val client = OkHttpClient()
.newBuilder()
.protocols(listOf(Protocol.HTTP_1_1))
.connectTimeout(10, TimeUnit.MINUTES)
.readTimeout(10, TimeUnit.MINUTES)
.build()
val mediaType: MediaType? = "text/plain".toMediaTypeOrNull()
val body: RequestBody = MultipartBody.Builder().setType(MultipartBody.FORM)
.addFormDataPart(
"data", name,
file.readBytes().toRequestBody(mediaType)
)
.build()
val request: Request = Request.Builder()
.url("https://.../upload.php")
.method("POST", body)
.build()
val response: Response = client.newCall(request).execute()
println(response.body)
}
}
and receive such error:
java.lang.OutOfMemoryError: Failed to allocate a 173410912 byte allocation with 25165824 free bytes and 89MB until OOM, max allowed footprint 199761800, growth limit 268435456
but I can upload with this code file with size about 90mb
The retrofit multipart stuff has a member that takes an Uri for a request body.
You try to use the one for a File instance.
Have you set log in loggingInterceptor or restadapter ?
if yes then try to set it NONE.
I want to get a body string from the object instance of okhttp3.RequestBody.
Can anyone help?
A RequestBody's content can be written out using the following method:
writeTo(okio.BufferedSink sink)
Okio also has a Buffer type which is both a BufferedSink (meaning you can write to it) and a BufferedSource (meaning you can read from it).
Thus, we can write the body to a Buffer and then read it back as a string.
val body = ...
val buffer = Buffer()
body.writeTo(buffer)
val bodyString = buffer.readUtf8()
(to complement #Jake's nice answer with a full example)
Here you can see how you could add a header with the signature of the request body:
import okhttp3.Interceptor
import okhttp3.Request
import okhttp3.Response
import okio.Buffer
class MyInterceptor(
private val signer: Sha256Signer
) : Interceptor {
override fun intercept(chain: Interceptor.Chain): Response {
return chain.proceed(
chain.request().newBuilder()
.addContentTypeHeader()
.addUserAgentHeader()
.addBunqHeaders(chain.request())
.build()
)
}
// ...
private fun Request.Builder.addSignatureHeader(request: Request) =
header("X-Client-Signature", signer.sign(request.getBodyAsString()))
private fun Request.getBodyAsString(): String {
val requestCopy = this.newBuilder().build()
val buffer = Buffer()
requestCopy.body?.writeTo(buffer)
return buffer.readUtf8()
}
}
I am getting an SSL exception when I try to upload a file to Amazon S3's pre-signed URL with OkHttp 3.9.1: SSLException: Write error: ssl=0xa0b73280: I/O error during system call, Connection reset by peer
It is the same problem as in another SO question but in my case it fails always. I upload just files over 1MiB in size, I have not tried small files.
As I mentioned in my answer in that question, switching to Java's HttpURLConnection fixed the problem and the upload works perfectly.
Here is my RequestBody implementation (in Kotlin) to upload a file from Android's Uri and I do use OkHttp's .put() method:
class UriRequestBody(private val file: Uri, private val contentResolver: ContentResolver, private val mediaType: MediaType = MediaType.parse("application/octet-stream")!!): RequestBody() {
override fun contentLength(): Long = -1L
override fun contentType(): MediaType? = mediaType
override fun writeTo(sink: BufferedSink) {
Okio.source((contentResolver.openInputStream(file))).use {
sink.writeAll(it)
}
}
}
and here is my HttpURLConnection implementation:
private fun uploadFileRaw(file: Uri, uploadUrl: String, contentResolver: ContentResolver) : Int {
val url = URL(uploadUrl)
val connection = url.openConnection() as HttpURLConnection
connection.doOutput = true
connection.requestMethod = "PUT"
val out = connection.outputStream
contentResolver.openInputStream(file).use {
it.copyTo(out)
}
out.close()
return connection.responseCode
}
What is OkHttp doing differently so it can lead to this SSL exception?
EDIT:
Here is the OkHttp code to upload the file (using the default application/octet-stream mime type):
val s3UploadClient = OkHttpClient().newBuilder()
.connectTimeout(30_000L, TimeUnit.MILLISECONDS)
.readTimeout(30_000L, TimeUnit.MILLISECONDS)
.writeTimeout(60_000L, TimeUnit.MILLISECONDS)
.retryOnConnectionFailure(true)
.build()
val body: RequestBody = UriRequestBody(file, contentResolver)
val request = Request.Builder()
.url(uploadUrl)
.put(body)
.build()
s3UploadClient.newCall(request).execute()
And this is the JavaScript server code that generates the pre-signed upload URL:
const s3 = new aws.S3({
region: 'us-west-2',
signatureVersion: 'v4'
});
const signedUrlExpireSeconds = 60 * 5;
const signedUrl = s3.getSignedUrl('putObject', {
Bucket: config.bucket.name,
Key: `${fileName}`,
Expires: signedUrlExpireSeconds
});
This seems to work with retrofit library:
fun uploadImage(imagePath: String, directUrl: String): Boolean {
Log.d(TAG, "Image: ${imagePath}, Url: $directUrl")
val request = Request.Builder()
.url(directUrl)
.put(RequestBody.create(null, File(imagePath)))
.build()
val response = WebClient.getOkHttpClient().newCall(request).execute()
return response.isSuccessful
}
I want to do a login validation using POST method and to get some information using GET method.
I've URL, server Username and Password already of my previous project.
For Android, Volley is a good place to get started. For all platforms, you might also want to check out ktor client or http4k which are both good libraries.
However, you can also use standard Java libraries like java.net.HttpURLConnection
which is part of the Java SDK:
fun sendGet() {
val url = URL("http://www.google.com/")
with(url.openConnection() as HttpURLConnection) {
requestMethod = "GET" // optional default is GET
println("\nSent 'GET' request to URL : $url; Response Code : $responseCode")
inputStream.bufferedReader().use {
it.lines().forEach { line ->
println(line)
}
}
}
}
Or simpler:
URL("https://google.com").readText()
Send HTTP POST/GET request with parameters using HttpURLConnection :
POST with Parameters:
fun sendPostRequest(userName:String, password:String) {
var reqParam = URLEncoder.encode("username", "UTF-8") + "=" + URLEncoder.encode(userName, "UTF-8")
reqParam += "&" + URLEncoder.encode("password", "UTF-8") + "=" + URLEncoder.encode(password, "UTF-8")
val mURL = URL("<Your API Link>")
with(mURL.openConnection() as HttpURLConnection) {
// optional default is GET
requestMethod = "POST"
val wr = OutputStreamWriter(getOutputStream());
wr.write(reqParam);
wr.flush();
println("URL : $url")
println("Response Code : $responseCode")
BufferedReader(InputStreamReader(inputStream)).use {
val response = StringBuffer()
var inputLine = it.readLine()
while (inputLine != null) {
response.append(inputLine)
inputLine = it.readLine()
}
println("Response : $response")
}
}
}
GET with Parameters:
fun sendGetRequest(userName:String, password:String) {
var reqParam = URLEncoder.encode("username", "UTF-8") + "=" + URLEncoder.encode(userName, "UTF-8")
reqParam += "&" + URLEncoder.encode("password", "UTF-8") + "=" + URLEncoder.encode(password, "UTF-8")
val mURL = URL("<Yout API Link>?"+reqParam)
with(mURL.openConnection() as HttpURLConnection) {
// optional default is GET
requestMethod = "GET"
println("URL : $url")
println("Response Code : $responseCode")
BufferedReader(InputStreamReader(inputStream)).use {
val response = StringBuffer()
var inputLine = it.readLine()
while (inputLine != null) {
response.append(inputLine)
inputLine = it.readLine()
}
it.close()
println("Response : $response")
}
}
}
Using only the standard library with minimal code!
thread {
val json = try {
URL(url).readText()
} catch (e: Exception) {
return#thread
}
runOnUiThread { displayOrWhatever(json) }
}
This starts a GET request on a new thread, leaving the UI thread to respond to user input. However, we can only modify UI elements from the main/UI thread, so we actually need a runOnUiThread block to show the result to our user. This enqueues our display code to be run on the UI thread soon.
The try/catch is there so your app won't crash if you make a request with your phone's internet off. Add your own error handling (e.g. showing a Toast) as you please.
.readText() is not part of the java.net.URL class but a Kotlin extension method, Kotlin "glues" this method onto URL. This is enough for plain GET requests, but for more control and POST requests you need something like the Fuel library.
Have a look at Fuel library, a sample GET request
"https://httpbin.org/get"
.httpGet()
.responseString { request, response, result ->
when (result) {
is Result.Failure -> {
val ex = result.getException()
}
is Result.Success -> {
val data = result.get()
}
}
}
// You can also use Fuel.get("https://httpbin.org/get").responseString { ... }
// You can also use FuelManager.instance.get("...").responseString { ... }
A sample POST request
Fuel.post("https://httpbin.org/post")
.jsonBody("{ \"foo\" : \"bar\" }")
.also { println(it) }
.response { result -> }
Their documentation can be found here
I think using okhttp is the easiest solution. Here you can see an example for POST method, sending a json, and with auth.
val url = "https://example.com/endpoint"
val client = OkHttpClient()
val JSON = MediaType.get("application/json; charset=utf-8")
val body = RequestBody.create(JSON, "{\"data\":\"$data\"}")
val request = Request.Builder()
.addHeader("Authorization", "Bearer $token")
.url(url)
.post(body)
.build()
val response = client . newCall (request).execute()
println(response.request())
println(response.body()!!.string())
Remember to add this dependency to your project https://mvnrepository.com/artifact/com.squareup.okhttp3/okhttp
UPDATE: July 7th, 2019
I'm gonna give two examples using latest Kotlin (1.3.41), OkHttp (4.0.0) and Jackson (2.9.9).
UPDATE: January 25th, 2021
Everything is okay with the most updated versions.
<!-- https://mvnrepository.com/artifact/com.fasterxml.jackson.module/jackson-module-kotlin -->
<dependency>
<groupId>com.fasterxml.jackson.module</groupId>
<artifactId>jackson-module-kotlin</artifactId>
<version>2.12.1</version>
</dependency>
<!-- https://mvnrepository.com/artifact/com.squareup.okhttp3/okhttp -->
<dependency>
<groupId>com.squareup.okhttp3</groupId>
<artifactId>okhttp</artifactId>
<version>4.9.0</version>
</dependency>
Get Method
fun get() {
val client = OkHttpClient()
val url = URL("https://reqres.in/api/users?page=2")
val request = Request.Builder()
.url(url)
.get()
.build()
val response = client.newCall(request).execute()
val responseBody = response.body!!.string()
//Response
println("Response Body: " + responseBody)
//we could use jackson if we got a JSON
val mapperAll = ObjectMapper()
val objData = mapperAll.readTree(responseBody)
objData.get("data").forEachIndexed { index, jsonNode ->
println("$index $jsonNode")
}
}
POST Method
fun post() {
val client = OkHttpClient()
val url = URL("https://reqres.in/api/users")
//just a string
var jsonString = "{\"name\": \"Rolando\", \"job\": \"Fakeador\"}"
//or using jackson
val mapperAll = ObjectMapper()
val jacksonObj = mapperAll.createObjectNode()
jacksonObj.put("name", "Rolando")
jacksonObj.put("job", "Fakeador")
val jacksonString = jacksonObj.toString()
val mediaType = "application/json; charset=utf-8".toMediaType()
val body = jacksonString.toRequestBody(mediaType)
val request = Request.Builder()
.url(url)
.post(body)
.build()
val response = client.newCall(request).execute()
val responseBody = response.body!!.string()
//Response
println("Response Body: " + responseBody)
//we could use jackson if we got a JSON
val objData = mapperAll.readTree(responseBody)
println("My name is " + objData.get("name").textValue() + ", and I'm a " + objData.get("job").textValue() + ".")
}
Maybe the simplest GET
For everybody stuck with NetworkOnMainThreadException for the other solutions: use AsyncTask or, even shorter, (yet still experimental) Coroutines:
launch {
val jsonStr = URL("url").readText()
}
If you need to test with plain http don't forget to add to your manifest:
android:usesCleartextTraffic="true"
For the experimental Coroutines you have to add to build.gradle as of 10/10/2018:
kotlin {
experimental {
coroutines 'enable'
}
}
dependencies {
implementation "org.jetbrains.kotlinx:kotlinx-coroutines-core:0.24.0"
implementation "org.jetbrains.kotlinx:kotlinx-coroutines-android:0.24.0"
...
If you are using Kotlin, you might as well keep your code as succinct as possible. The run method turns the receiver into this and returns the value of the block.
this as HttpURLConnection creates a smart cast. bufferedReader().readText() avoids a bunch of boilerplate code.
return URL(url).run {
openConnection().run {
this as HttpURLConnection
inputStream.bufferedReader().readText()
}
}
You can also wrap this into an extension function.
fun URL.getText(): String {
return openConnection().run {
this as HttpURLConnection
inputStream.bufferedReader().readText()
}
}
And call it like this
return URL(url).getText()
Finally, if you are super lazy, you can extend the String class instead.
fun String.getUrlText(): String {
return URL(this).run {
openConnection().run {
this as HttpURLConnection
inputStream.bufferedReader().readText()
}
}
}
And call it like this
return "http://somewhere.com".getUrlText()
You can use kohttp library. It is a Kotlin DSL HTTP client. It supports the features of square.okhttp and provides a clear DSL for them. KoHttp async calls are powered by coroutines.
httpGet extension function
val response: Response = "https://google.com/search?q=iphone".httpGet()
you can also use async call with coroutines
val response: Deferred<Response> = "https://google.com/search?q=iphone".asyncHttpGet()
or DSL function for more complex requests
val response: Response = httpGet {
host = "google.com"
path = "/search"
param {
"q" to "iphone"
"safe" to "off"
}
}
You can find more details in docs
To get it with gradle use
implementation 'io.github.rybalkinsd:kohttp:0.12.0'
Without adding additional dependencies, this works. You don't need Volley for this. This works using the current version of Kotlin as of Dec 2018: Kotlin 1.3.10
If using Android Studio, you'll need to add this declaration in your AndroidManifest.xml:
<uses-permission android:name="android.permission.INTERNET" />
<uses-permission android:name="android.permission.ACCESS_NETWORK_STATE" />
You should manually declare imports here. The auto-import tool caused me many conflicts.:
import android.os.AsyncTask
import java.io.BufferedReader
import java.io.InputStreamReader
import java.io.OutputStream
import java.io.OutputStreamWriter
import java.net.URL
import java.net.URLEncoder
import javax.net.ssl.HttpsURLConnection
You can't perform network requests on a background thread. You must subclass AsyncTask.
To call the method:
NetworkTask().execute(requestURL, queryString)
Declaration:
private class NetworkTask : AsyncTask<String, Int, Long>() {
override fun doInBackground(vararg parts: String): Long? {
val requestURL = parts.first()
val queryString = parts.last()
// Set up request
val connection: HttpsURLConnection = URL(requestURL).openConnection() as HttpsURLConnection
// Default is GET so you must override this for post
connection.requestMethod = "POST"
// To send a post body, output must be true
connection.doOutput = true
// Create the stream
val outputStream: OutputStream = connection.outputStream
// Create a writer container to pass the output over the stream
val outputWriter = OutputStreamWriter(outputStream)
// Add the string to the writer container
outputWriter.write(queryString)
// Send the data
outputWriter.flush()
// Create an input stream to read the response
val inputStream = BufferedReader(InputStreamReader(connection.inputStream)).use {
// Container for input stream data
val response = StringBuffer()
var inputLine = it.readLine()
// Add each line to the response container
while (inputLine != null) {
response.append(inputLine)
inputLine = it.readLine()
}
it.close()
// TODO: Add main thread callback to parse response
println(">>>> Response: $response")
}
connection.disconnect()
return 0
}
protected fun onProgressUpdate(vararg progress: Int) {
}
override fun onPostExecute(result: Long?) {
}
}
GET and POST using OkHttp
private const val CONNECT_TIMEOUT = 15L
private const val READ_TIMEOUT = 15L
private const val WRITE_TIMEOUT = 15L
private fun performPostOperation(urlString: String, jsonString: String, token: String): String? {
return try {
val client = OkHttpClient.Builder()
.connectTimeout(CONNECT_TIMEOUT, TimeUnit.SECONDS)
.writeTimeout(WRITE_TIMEOUT, TimeUnit.SECONDS)
.readTimeout(READ_TIMEOUT, TimeUnit.SECONDS)
.build()
val body = jsonString.toRequestBody("application/json; charset=utf-8".toMediaTypeOrNull())
val request = Request.Builder()
.url(URL(urlString))
.header("Authorization", token)
.post(body)
.build()
val response = client.newCall(request).execute()
response.body?.string()
}
catch (e: IOException) {
e.printStackTrace()
null
}
}
private fun performGetOperation(urlString: String, token: String): String? {
return try {
val client = OkHttpClient.Builder()
.connectTimeout(CONNECT_TIMEOUT, TimeUnit.SECONDS)
.writeTimeout(WRITE_TIMEOUT, TimeUnit.SECONDS)
.readTimeout(READ_TIMEOUT, TimeUnit.SECONDS)
.build()
val request = Request.Builder()
.url(URL(urlString))
.header("Authorization", token)
.get()
.build()
val response = client.newCall(request).execute()
response.body?.string()
}
catch (e: IOException) {
e.printStackTrace()
null
}
}
Object serialization and deserialization
#Throws(JsonProcessingException::class)
fun objectToJson(obj: Any): String {
return ObjectMapper().writeValueAsString(obj)
}
#Throws(IOException::class)
fun jsonToAgentObject(json: String?): MyObject? {
return if (json == null) { null } else {
ObjectMapper().readValue<MyObject>(json, MyObject::class.java)
}
}
Dependencies
Put the following lines in your gradle (app) file. Jackson is optional. You can use it for object serialization and deserialization.
implementation 'com.squareup.okhttp3:okhttp:4.3.1'
implementation 'com.fasterxml.jackson.core:jackson-core:2.9.8'
implementation 'com.fasterxml.jackson.core:jackson-annotations:2.9.8'
implementation 'com.fasterxml.jackson.core:jackson-databind:2.9.8'
You can use this library Fuel Library as well, which makes it further easier.
val map = mutableMapOf<String, String>()
map.put("id","629eeb9da9d8f50016e1af96")
val httpAsync = url
.httpPost()
.jsonBody(
Gson().toJson(map) // for json string
)
.responseString { request, response, result -> //do something with the response }