Send large data OkHttp3 outofmemory - android

I want to send a PDF image to the server in Android.
The server spec needs I should use Base64 encoding.
So I should convert the PDF Image file to Base64 String.
Below is the HTTP POST request:
<?xml version="1.0" encoding="UTF-8"?>
<request>
<auth>
<!-- auth info -->
</auth>
<imagefile>
<filename>Test Attachment</filename>
<type>pdf</type>
<data>"HERE IS BASE64 String"</data>
</imagefile>
</request>
And I used below code, it is just Http post request code.
fun post(url: String, xml: String) = Observable.defer {
val request = Request.Builder()
.url(url)
.post(RequestBody.create(contentType, xml))
.build()
okHttpClient.newCall(request).execute()
}
In case of the small pdf file, it's ok. (Works fine)
But when I used the large pdf file(over 20MB), out of memory occurs.
So I tried another way like below.
fun post(url: String, xmlFile: File) = Observable.defer {
val request = Request.Builder()
.url(url)
.post(RequestBody.create(contentType, xmlFile))
.build()
okHttpClient.newCall(request).execute()
}
The difference is that I don't use String, but File.
I created a file using the Base64 encoded text with XML syntax.
But it is failed too.
How can I send the large data in Android without OOM?

You're going to need to find a streaming base64 encoder and use that to write a RequestBody that streams to the server. There is no such encoder built-in to OkHttp or Okio but you can use the one in Android itself. Use wrap on the OutputStream that you'll get in the writeTo method of your custom RequestBody.
https://developer.android.com/reference/java/util/Base64.Encoder

Related

Put File as Binary with Ktor HttpClient in Kotlin Multiplatform Project

I have a multiplatform project in which api code is shared between iOS and Android.
There is "put" api to upload local audio file as Binary.
Ive created httpclient as follows
val client = HttpClient {
defaultRequest {
url {
protocol = ServiceConfiguration.protocol
host = ServiceConfiguration.baseUrl
port = ServiceConfiguration.port
}
contentType(ContentType.Application.Json)
}
install(JsonFeature) {
val json = kotlinx.serialization.json.Json {
ignoreUnknownKeys = true
isLenient = true
}
serializer = KotlinxSerializer(json)
}
}
To put the object into api, I am doing as follows
val response = ServiceRequest.client.put<String>(
body = File(path).readBytes()
)
It works fine and uploads the byte array to backend. But instead of byte array I want to upload the file as plain binary.
To make it more clear, In Postman mac app we can upload file as binary . I need to do similar thing.
When I checked in Ktor, it shows only multi-part form data can be submitted as binary. But In my case it is Put request.
Please help.
Looks like there is no straightforward way to put file as binary with Ktor.
I had to go for platform Dependent approaches like OkHttpClient for Android and URLSession for iOS.

How to download files from Dropbox using Retrofit 2

I'm trying to download audio and video files from the Dropbox API into my Android app using the Retrofit 2 library but I'm getting a JSON error because I'm not mapping the response correctly, but since the response is apparently the actual file I'm not sure how I can handle it.
Here is my code:
Media class
data class Media(
#Expose val id: String,
#Expose #SerializedName("name") val title: String,
var position: Int,
var uri: String
)
API interface
#POST("2/files/download")
fun download(#Header("Dropbox-API-Arg") pathJson: MediaRequest): Call<Media>
I added an interceptor to my client that adds the authorisation header to all calls and it's working because calls to other endpoints are receiving their responses properly.
MediaRequest class
data class MediaRequest(val path: String)
This class, when sent on the request, is serialised to JSON just as expected by the Dropbox API documentation, like the example below:
{ "path": "id:abcd1234" }
API call
val request = MediaRequest(media.id)
downloadApi.download(request).enqueue(object : Callback<Media> {
override fun onResponse(call: Call<Media>, response: Response<Media>) {
if (response.isSuccessful) {
// handle response
} else {
onError()
}
}
override fun onFailure(call: Call<Media>, t: Throwable) {
onError(t)
}
})
I also checked the exact same call with Postman and there are no problems with the call itself because I'm getting the response, but from what I saw in the Postman response, I should expect to receive the file itself as a response because when I tried downloading an MP3 audio file the response was an encoded string beginning with ID3, which means it's an MP3 file, as the screenshot below shows:
I already know that mapping the response with that Media class I created is not the right approach but I don't know how I can handle that response correctly in order to have the downloaded file, so any help will be much appreciated.
PS: I don't want to use the Dropbox SDK because I feel it would be quite an overkill since that's the only reason I'm calling their API.
The way I solved my problem was by mapping the response as a ResponseBody object and then calling response.body().byteStream() to get the contents as a byte stream and then decode them into a proper file.
Source: https://gldraphael.com/blog/downloading-a-file-using-retrofit/

Retrofit 1, sending multipart form data with custom key for value

I am using an API that is out of my control, as well having joined a development team recently that is using Retrofit1.
I am unable to send the request to the server in the format required, as the server requires multipart form data Body in the following format:
Uniqueidentifier:FileName.jpg:ReroutServerIP:Base64EncodedDocString.
I have tried many different techniques in order to accomplish this task but I cannot find any working method to do this. The server tells me that the message format not supported. Here is my current code (with the url stripped out). Please could someone assist?
#POST("URL")
public Response post_SendData(#Header("Content-Type") String ContentType, #Body String body);
In order to achieve the desired result, I can use postman with no headers and post a file from my system using the form-data post method. In the working postman post, the Key is the formatted string mentioned above and the value is a file selected from my desktop. Please see below for postman (edited to remove urls).
Postman
Thanks a lot guys.
If you want to send a file to the server :
public void uploadPictureRetrofit(File file, Callback<YourObject> response) {
// this will build full path of API url where we want to send data.
RestAdapter restAdapter = new RestAdapter
.Builder()
.setEndpoint(YOUR_BASE_URL)
.setConverter(new SimpleXMLConverter()) // if the response is an xml
.setLogLevel(RestAdapter.LogLevel.FULL)
.build();
// SubmitAPI is name of our interface which will send data to server.
SendMediaApiInterface api = restAdapter.
create(SendMediaApiInterface.class);
TypedFile typedFile = new TypedFile(MULTIPART_FORM_DATA, file);
api.sendImage(typedFile, response);
}
And this is the interface SendMediaApiInterface :
public interface SendMediaApiInterface {
#Multipart
#POST("url")
void sendImage(#Part("here_is_the_attribute_name_in_webservice") TypedFile attachments, Callback<YourObject> response);}

Retrofit 2 RequestBody Content Length Greater Than File Size

I have an Api which requires me to send up the size of a file in bytes before I then send up the actual file. When I call file.length() on the file to send up in the first Api call, it returns 1996.
When I then package the file up into a RequestBody the contentLength() becomes 2556. The server then rejects this, saying that these sizes must match.
Here is the code for creating the RequestBody.
val requestBody = RequestBody.create(
MediaType.parse("image/jpeg"),
avatarFile)
What is being added in the RequestBody to increase its content length?
Should I just send up the size of the RequestBody on the first call to get around this problem?
EDIT
Here is where I call the API the first time to send the size initially:
return authService.updateAvatar(
AvatarMetadataRequest(
size = avatarFile.length().toInt(),
crc = profileImageProvider.getFile()!!.checksum()!!.toInt()))
And then this is where I call it the second time when the file actually gets uploaded:
val avatarFile = profileImageProvider.getFile()
val requestBody = RequestBody.create(
MediaType.parse("image/jpeg"),
avatarFile)
return authService.uploadAvatar(
id.split("/").last(),
MultipartBody.Part.createFormData("avatar",
profileImageProvider.getFileName(),
requestBody))
It turns out the contentLength() of the requestBody is correct. So the Content-Length must be being modified or increased when the MultipartBody.Part.createFormData is called.
This seems to be a mistake on the part of the avatar upload API. The avatar is being uploaded as MultipartData and so will never equal the raw size of the file. This is because a multipart request will have additional data besides the file.
Here is an example multipart request:
--------------------------0b880724ca8aacd6
Content-Disposition: form-data; name="file1"; filename="test.txt"
Content-Type: text/plain
test test test
--------------------------0b880724ca8aacd6--
As you can see, the content of the file is just test test test. Everything else is additional data added by the multipart request.
The raw size on disk is 15 bytes, while the Content-Length of the request is 202, due to all the extra data.
So the avatar upload API should not just be checking the Content-Length of the entire request, but rather extracting the specific part relating to the file and just checking the size of the data contained in that part.
If you have no control over the behaviour of the avatar upload API, then you can attempt to calculate the total Content-Length by figuring out the total size of the overhead added by the multipart request and adding that to your file size.

Retrofit 2 Multipart requests

I'm migrating my existing codebase to Retrofit 2, but having some trouble understanding the new syntax for Multipart requests. I'm also using Kotlin, although apart from a few syntax changes I think it shouldn't matter for this particular question.
Here's what I have right now:
val audioDuration = RequestBody.create(null, audioDuration.toString())
val file = RequestBody.create(MediaType.parse("audio/mp4"),
File(context.filesDir, filename).absoluteFile)
sendAudioChunk(audioDuration, file).enqueue(callback)
And here's the definition of the API:
#Multipart
#POST("path_to_request")
fun sendAudioChunk(#Part("duration") audioDuration: RequestBody,
#Part("audio") audioBlob: RequestBody) : Call<ResponseObject>
On Retrofit 1.9 I used TypedString and TypedFile for the request parameters, and now it seems one need to use RequestBody from OkHttp but I must be missing something since the request does not execute correctly.
I eventually figured it out. My web-service expects a filename for file uploads. This seems to be a work in progress support in the new Retrofit 2, but it is possible to circumvent the problem by adding it to the named parameter definition.
More details here : https://github.com/square/retrofit/issues/1140
One thing that is different is that TypedString would have a Content-Type of "text/plain; charset=UTF-8", where you are not setting a Context-Type at all for your audioDuration parameter. Try setting it to text/plain to get the same behavior as TypedString (charset will be set to utf-8 by default).
val audioDuration = RequestBody.create(MediaType.parse("text/plain"), audioDuration.toString())
If that doesn't work, you'll need to provide more information about how the "request does not execute correctly". A working request that you are trying to replicate would also be helpful.

Categories

Resources