Upload multiple images(nearly 100) from Android to Amazon S3? - android

I am trying to upload multiple image to amazon s3 bucket. The size of the images which i am trying to upload is nearly 300KB each. I am using loop to upload the images. But it's taking more time compared to ios. I am using the below code to upload the images to S3.
val uploadObserver = transferUtility!!.upload(bucketname,
, "img_$timeStamp.jpg", File(fileUri.path!!),
md, CannedAccessControlList.PublicRead)
uploadObserver.setTransferListener(object : TransferListener {
override fun onStateChanged(id: Int, state: TransferState) {
if (TransferState.COMPLETED == state) {
} else if (TransferState.FAILED == state) {
}
}
override fun onProgressChanged(id: Int, bytesCurrent: Long, bytesTotal: Long) {
}
override fun onError(id: Int, ex: Exception) {
}
})
}
Please help me, how to increase the speed of upload.

Using RxJava and RxAndroid you can do multiple async task at a time. zip operate binds all task into one task. Sample code for your case is as following:
fun uploadTask(bucketname: String, timeStamp: Long, md: Any, uriList: List<Uri>) {
val singles = mutableListOf<Single<String>>()
uriList.forEach {
singles.add(upload(bucketname, timeStamp, it, md).subscribeOn(Schedulers.io()))
}
val disposable = Single.zip(singles) {
// If all request emits success response, then it will bind all emitted
// object (in this example String) into an Array and you will get callback here.
// You can change your final response here. In this can I am appending all strings
// to a string.
var finalString = ""
it.forEach { responseString ->
finalString += responseString
}
finalString
}
.subscribe({
// you will get final response here.
}, {
// If any one request failed and emits error all task will be stopped and error
// will be thrown here.
it.printStackTrace()
})
}
And upload() method can be as following:
fun upload(bucketname: String, timeStamp: Long, fileUri: Uri, md: Any): Single<String> {
return Single.create<String> { emitter -> // You can change String to anything you want to emmit
val uploadObserver = transferUtility!!.upload(bucketname,
, "img_$timeStamp.jpg", File(fileUri.path!!),
md, CannedAccessControlList.PublicRead)
uploadObserver.setTransferListener(object : TransferListener {
override fun onStateChanged(id: Int, state: TransferState) {
if (TransferState.COMPLETED == state) {
emitter.onSuccess("COMPLETED") // Emmit your object here
} else if (TransferState.FAILED == state) {
emitter.onSuccess("FAILED")
}
}
override fun onProgressChanged(id: Int, bytesCurrent: Long, bytesTotal: Long) {
}
override fun onError(id: Int, ex: Exception) {
emitter.onError(ex)
}
})
}
}
Just keep in mind, if any upload request is failed, all other task will be stopped. If you want to continue you have to use onErrorResumeNext() operator in that case.

Related

How can I query a random date from this api?

I am using the NASA api to fetch images of Mars. I am using retrofit to fetch the data. The Mars API states a query can be done by "sol"...or in other words, day rotation on mars. I am able to get the data when I call it by the specific sol, but I want the user to be shown a random date every time they enter the app. What is the correct way to implement this?
Example JSON
MarsPhotoApi.kt
interface MarsPhotoApi {
#GET("/mars-photos/api/v1/rovers/curiosity/photos")
suspend fun getCuriosityData(
#Query("sol")
solQuery: Int,
#Query("rover_id")
roverQuery: Int,
#Query("camera")
camera: String,
#Query("api_Key")
apiKey: String = API_KEY
): Response<MarsPhotos>
}
MarsPhotoViewModel.kt
class MarsPhotoViewModel(
val marsPhotoRepository: MarsPhotoRepository
): ViewModel() {
val marsPhotos: MutableLiveData<Resource<MarsPhotos>> = MutableLiveData()
init {
getCuriosityPhotos(1, 5, "NAVCAM")
}
fun getCuriosityPhotos(solQuery: Int, roverQuery: Int, camera: String) = viewModelScope.launch {
marsPhotos.postValue(Resource.Loading())
val response = marsPhotoRepository.getCuriosityPhotos(solQuery, roverQuery, camera)
marsPhotos.postValue(handlePhotosResponse(response))
}
private fun handlePhotosResponse(response: Response<MarsPhotos> ) : Resource<MarsPhotos> {
if(response.isSuccessful) {
response.body()?.let { resultResponse ->
return Resource.Success(resultResponse)
}
}
return Resource.Error(response.message())
}
}
I suppose you could just use a random Int for the sol value
Random.nextInt(1, 2000)

Launch instances of same coroutine one after the other

I have a scenario where I have multiple URLs to download PDFs. I need to make a network call for each url, process the pdf, send it to a printer, wait for the printer to finish and then continue with the next url. Im trying to get my head around coroutines but I cant understand how to apply it here.
Here I'm making a call for each url in an array and making a call, in the callback I call the method "printImage" where I send the pdf to a printer and I can listen for it to finish, how could I implement coroutines here? In this case the method "printImage" should be a coroutine
fun printLabels(labels: HashMap<Long, String>) {
for (item in labels.values) {
val call = labelService.getPDF(item)
call.enqueue(
object : retrofit2.Callback<ResponseBody> {
override fun onResponse(call: Call<ResponseBody>, response: Response<ResponseBody>) {
val outputDir = cacheDir
val outputFile = File.createTempFile("prefix", "extension", outputDir)
Files.asByteSink(outputFile).write(response.body()!!.bytes())
val bitmap = getBitmap(outputFile)!!
printImage(bitmap)
}
override fun onFailure(call: Call<ResponseBody>, t: Throwable) {
Timber.e(t.localizedMessage)
}
}
)
}
}
fun printImage(bm: Bitmap) {
Thread(Runnable {
printerRepository.currentPrinter.value?.let { printer ->
if (printer.startCommunication()) {
val result = printer.printImage(bm)
if (result.errorCode != PrinterInfo.ErrorCode.ERROR_NONE) {
Timber.d("ERROR - " + result.errorCode)
} else {
}
printer.endCommunication()
} else {
printerRepository.printerTestPassed.postValue(false)
}
}
}).start()
}

How to make two requests in one request with rxJava2

I am accessing the server in my Android app. I want to get a list of my friends and a list of friend requests in different queries. They have to come at the same time. Then I want to show this data on the screen.
I tried to get data from two queries at using flatMap.
interactor.getColleagues() and interactor.getTest() returns the data type Observable<List<Colleagues>>
private fun loadColleaguesEmployer() {
if (disposable?.isDisposed == true) disposable?.dispose()
//запрос на список друзей
interactor.getColleagues(view.getIdUser() ?: preferences.userId)
.subscribeOn(Schedulers.io())
.flatMap {
interactor.getTest().subscribeOn(Schedulers.io())
.doOnNext {
result-> view.showTest(mapper.map(result))
}
}
.observeOn(AndroidSchedulers.mainThread())
.subscribeBy(
onNext = { result1 ->
//Обработка списка коллег работодателей
view.showColleagues(mapper.map(result1.filter { data -> data.typeFriend == "Работодатель" }))
},
onError = { it.printStackTrace() }
)
}
I want to get and process data from different queries at the same time.
Combining observable results of multiple async http requests with rxjava's Observable.zip.
public class Statistics {
public static void main(String[] args) {
List<Observable<ObservableHttpResponse>> observableRequests = Arrays.asList(
Http.getAsync("http://localhost:3001/stream"),
Http.getAsync("http://localhost:3002/stream"),
Http.getAsync("http://localhost:3003/stream"),
Http.getAsync("http://localhost:3004/stream"));
List<Observable<Stats>> observableStats = observableRequests.stream()
.map(observableRequest ->
observableRequest.flatMap(response ->
response.getContent()
.map(new EventStreamJsonMapper<>(Stats.class))))
.collect(toList());
Observable<List<Stats>> joinedObservables = Observable.zip(
observableStats.get(0),
observableStats.get(1),
observableStats.get(2),
observableStats.get(3),
Arrays::asList);
// This does not work, as FuncN accepts (Object...) https://github.com/Netflix/RxJava/blob/master/rxjava-core/src/main/java/rx/functions/FuncN.java#L19
// Observable<List<Stats>> joinedObservables = Observable.zip(observableStats, Arrays::asList);
joinedObservables
.take(10)
.subscribe(
(List<Stats> statslist) -> {
System.out.println(statslist);
double average = statslist.stream()
.mapToInt(stats -> stats.ongoingRequests)
.average()
.getAsDouble();
System.out.println("avg: " + average);
},
System.err::println,
Http::shutdown);
}
}
you can do it by simple operation zip like
private fun callRxJava() {
RetrofitBase.getClient(context).create(Services::class.java).getApiName()
.subscribeOn(Schedulers.single())
.observeOn(AndroidSchedulers.mainThread())
getObservable()
.flatMap(object : io.reactivex.functions.Function<List<User>, Observable<User>> {
override fun apply(t: List<User>): Observable<User> {
return Observable.fromIterable(t); // returning user one by one from usersList.
} // flatMap - to return users one by one
})
.subscribe(object : Observer<User> {
override fun onSubscribe(d: Disposable) {
showProgressbar()
}
override fun onNext(t: User) {
userList.add(t)
hideProgressBar()
}
override fun onError(e: Throwable) {
Log.e("Error---", e.message)
hideProgressBar()
}
override fun onComplete() {
userAdapter.notifyDataSetChanged()
}
})
}
this function combines your response from 2 queries
private fun getObservable(): Observable<List<User>> {
return Observable.zip(
getCricketFansObservable(),
getFootlaballFansObservable(),
object : BiFunction<List<User>, List<User>, List<User>> {
override fun apply(t1: List<User>, t2: List<User>): List<User> {
val userList = ArrayList<User>()
userList.addAll(t1)
userList.addAll(t2)
return userList
}
})
}
here is example of first observable
fun getCricketFansObservable(): Observable<List<User>> {
return RetrofitBase.getClient(context).create(Services::class.java).getCricketers().subscribeOn(Schedulers.io())
}
If both observables return the same data type and you don't mind mixing of both sources data - consider using Observable.merge()
For example:
Observable.merge(interactor.getColleagues(), interactor.getTest())
.subscribeOn(Schedulers.io())
.subscribe(
(n) -> {/*do on next*/ },
(e) -> { /*do on error*/ });
Note, that .merge() operator doesn't care about emissions order.
Zip combine the emissions of multiple Observables together via a
specified function
You can use Zip (rx Java) http://reactivex.io/documentation/operators/zip.html, some sudo code will be like this -
val firstApiObserver = apIs.hitFirstApiFunction(//api parameters)
val secondApiObserver = apIs.hitSecondApiFunction(//api parameters)
val zip: Single<SubscriptionsZipper>//SubscriptionsZipper is the main model which contains first& second api response model ,
zip = Single.zip(firstApiObserver, secondApiObserver, BiFunction { firstApiResponseModel,secondApiResponseModel -> SubscriptionsZipper(firstApiResponseModelObjectInstance, secondApiResponseModelObjectInstance) })
zip.observeOn(AndroidSchedulers.mainThread())
.subscribeOn(Schedulers.io())
.subscribe(object : SingleObserver<SubscriptionsZipper> {
override fun onSubscribe(d: Disposable) {
compositeDisposable.add(d)
}
override fun onSuccess(subscriptionsZipper: SubscriptionsZipper) {
Utils.hideProgressDialog()
//here you will get both api response together
}
override fun onError(e: Throwable) {
Utils.hideProgressDialog()
}
})
Hope it helps you .

How to perform call sequence to a REST API in Android App?

I'm having a hard time making a call to my api. I'm using Reactivex with kotlin and Flowables. My API returns a list of items if the date I passed by the "If-Modified_since" header is less than the last update.
If there is no update I get as an app return android app a 304 error.
I need to do the following procedure.
1-> I make a call to the api
2-> If the call is successful, save the list in Realm and return to the viewmodel
3-> If the error is 304, I perform a cache search (Realm) of the items
4-> If it is another error, I return the error normally for the ViewModel
Here is the code below, but I'm not sure if it's that way.
override fun getTickets(eventId: String): Flowable<List<Ticket>> {
return factory
.retrieveRemoteDataStore()
.getTickets(eventId)
.map {
saveTickets(it)
it
}.onErrorResumeNext { t: Throwable ->
if (t is HttpException && t.response().code() == 304) {
factory.retrieveCacheDataStore().getTickets(eventId)
} else
//Should return error
}
The question is, what is the best way to do this?
Thank you.
I'm going to assume, that you're using Retrofit. If that's the case, then you could wrap your getTickets call in Single<Response<SomeModel>>. This way, on first map you can check the errorcode, something among the lines of:
...getTickets(id)
.map{ response ->
when {
response.isSuccessful && response.body!=null -> {
saveTickets(it)
it
}
!response.isSuccessful && response.errorCode() == 304 -> {
factory.retrieveCacheDataStore().getTickets(eventId)
}
else -> throw IOException()
}
}
This could of course be made pretty using standard/extension functions but wanted to keep it simple for readability purposes.
Hope this helps!
Most of my comments are my explanations.
data class Ticket(val id:Int) {
companion object {
fun toListFrom(jsonObject: JSONObject): TICKETS {
/**do your parsing of data transformation here */
return emptyList()
}
}
}
typealias TICKETS = List<Ticket>
class ExampleViewModel(): ViewModel() {
private var error: BehaviorSubject<Throwable> = BehaviorSubject.create()
private var tickets: BehaviorSubject<TICKETS> = BehaviorSubject.create()
/**public interfaces that your activity or fragment talk to*/
fun error(): Observable<Throwable> = this.error
fun tickets(): Observable<TICKETS> = this.tickets
fun start() {
fetch("http://api.something.com/v1/tickets/")
.subscribeOn(Schedulers.io())
.onErrorResumeNext { t: Throwable ->
if (t.message == "304") {
get(3)
} else {
this.error.onNext(t)
/** this makes the chain completed gracefuly without executing flatMap or any other operations*/
Observable.empty()
}
}
.flatMap(this::insertToRealm)
.subscribe(this.tickets)
}
private fun insertToRealm(tickets: TICKETS) : Observable<TICKETS> {
/**any logic here is mainly to help you save into Realm**/
/** I think realm has the option to ignore items that are already in the db*/
return Observable.empty()
}
private fun get(id: Int): Observable<TICKETS> {
/**any logic here is mainly to help you fetch from your cache**/
return Observable.empty()
}
private fun fetch(apiRoute: String): Observable<TICKETS> {
/**
* boilerplate code
wether you're using Retrofit or Okhttp, that's the logic you
should try to have
* */
val status: Int = 0
val rawResponse = ""
val error: Throwable? = null
val jsonResponse = JSONObject(rawResponse)
return Observable.defer {
if (status == 200) {
Observable.just(Ticket.toListFrom(jsonResponse))
}
else if (status == 304) {
Observable.error<TICKETS>(Throwable("304"))
}
else {
Observable.error<TICKETS>(error)
}
}
}
override fun onCleared() {
super.onCleared()
this.error = BehaviorSubject.create()
this.tickets = BehaviorSubject.create()
}
}

Amazon s3 upload multiple files android

Incase anyone still looking for a solution i ended up using a loop on the code blew i did not find an official api to do multiple files upload.
-------
I have an ArrayList of ImageFiles, that I want to upload to Amazon s3. Their doc provides this code:
credentials = new BasicAWSCredentials(key, secret);
s3 = new AmazonS3Client(credentials);
transferUtility = new TransferUtility(s3, getContext());
observer.setTransferListener(new TransferListener() {
#Override
public void onStateChanged(int id, TransferState state) {
}
#Override
public void onProgressChanged(int id, long bytesCurrent, long bytesTotal) {
}
#Override
public void onError(int id, Exception ex) {
}
});
observer = transferUtility.upload("buket name", upload_file.getNew_name(),
new File(upload_file.getFile_path()));
But this code only takes one file. How can i upload multiple files at once? And if they do not allow this so user can make more requests , what the alternative to do this ??
I know I'm late to answer but it will help others who'll come here looking for an answer.
Currently, the only option to upload multiple files is to use a loop passing the files from list as individual file but here's what I found last night and implemented it. I've tested it many times so far.
Advantage of this method is it runs concurrently for every file and not wait for every file to upload or download first to operate on the next file.
It's what I found here but I've modified it a little bit for my use in Kotlin.
First, Create a class, I've named it MultiUploaderS3 -
import android.content.Context
import com.amazonaws.mobileconnectors.s3.transferutility.TransferListener
import com.amazonaws.mobileconnectors.s3.transferutility.TransferState
import com.amazonaws.mobileconnectors.s3.transferutility.TransferUtility
import io.reactivex.Completable
import io.reactivex.Observable
import io.reactivex.Single
import java.io.File
class MultiUploaderS3 {
private fun transferUtility(context: Context): Single<TransferUtility?>? {
return Single.create { emitter ->
emitter.onSuccess(
TransferUtility(s3ClientInitialization(context), context)
)
}
}
fun uploadMultiple(fileToKeyUploads: Map<File, String>, context: Context): Completable? {
return transferUtility(context)!!
.flatMapCompletable { transferUtility ->
Observable.fromIterable(fileToKeyUploads.entries)
.flatMapCompletable { entry ->
uploadSingle(
transferUtility,
entry.key,
entry.value
)
}
}
}
private fun uploadSingle(
transferUtility: TransferUtility,
aLocalFile: File?,
toRemoteKey: String?
): Completable? {
return Completable.create { emitter ->
transferUtility.upload(bucketName,toRemoteKey, aLocalFile)
.setTransferListener(object : TransferListener {
override fun onStateChanged(
id: Int,
state: TransferState
) {
if (TransferState.FAILED == state) {
emitter.onError(Exception("Transfer state was FAILED."))
} else if (TransferState.COMPLETED == state) {
emitter.onComplete()
}
}
override fun onProgressChanged(
id: Int,
bytesCurrent: Long,
bytesTotal: Long
) {
}
override fun onError(id: Int, exception: Exception) {
emitter.onError(exception)
}
})
}
}
}
I've created a function for returning S3Client which is as follows -
fun s3ClientInitialization(context: Context): AmazonS3 {
val cognitoCachingCredentialsProvider = CognitoCachingCredentialsProvider(
context,
your key,
region
)
return AmazonS3Client(
cognitoCachingCredentialsProvider,
Region.getRegion(Regions.YOUR_REGION)
)
}
Then, Call it as -
val map: Map<File, String> = yourArrayList.map {it to Your_Key_For_Each_File}.toMap()
MultiUploaderS3().uploadMultiple(map, this)
.subscribeOn(Schedulers.io())
.observeOn(Schedulers.io())
.subscribe {
runOnUiThread {
Toast(this#AddActivity, "Uploading completed", Toast.LENGTH_LONG).show()
}
}
I've shared the complete working code, you can change it as you need. You can use the above MultiUploaderS3 class in the same class as well which will obviously make things easier to access the TransferListener.
For Downloading, change the
transferUtility.upload(bucketName,toRemoteKey, aLocalFile) in uploadSingle()
to
transferUtility.download(bucketName, fromRemoteKey, aLocalFile)
and Call it as
val map: Map<File, String> = yourKeysArrayList.map {Your_FILE_For_Each_KEY to it}.toMap()
What this will do is it will pass a map of Local file path to Keys.
I have tried uploading 10 files in a single run many times and it takes around 4-5 seconds to upload all of them, it also depends on your internet connection though. I hope the downloading part will work as well. Ask me if there's something or check my Github for this.
You can create observable for this task
public Observable<AWSFile> upload(List<String> paths) {
List<Observable<AWSFile>> list = new ArrayList<>();
for (String path: paths) {
list.add(upload(path));
}
return Observable.concat(list);
}
public Observable<AWSFile> upload(String filePath) {
if (filePath == null) {
Log.d(TAG, "uploadWithTransferUtility: ");
return Observable.never();
}
return Observable.create(emitter -> {
File file = new File(filePath);
TransferObserver observer = awsUtil.getTransferUtility(context).upload(awsUtil.getAwsConstants().BUCKET_NAME, file);
observer.setTransferListener(new TransferListener() {
#Override
public void onStateChanged(int id, TransferState state) {
stateChanged(id, state);
emitter.onNext(new AWSFile(id,state,file.getName()));
}
#Override
public void onProgressChanged(int id, long bytesCurrent, long bytesTotal) {
progressChanged(id, bytesCurrent, bytesTotal);
}
#Override
public void onError(int id, Exception ex) {
error(id,ex);
emitter.onError(ex);
}
});
emitter.setCancellable(observer::cleanTransferListener);
});
}
This is just only snippet and you can use it to many files or only one.
EDIT:
AWSFile.java
public class AWSFile {
private int id;
private TransferState newState;
private String filename;
}
You can move the uploading code to a method and pass it the file path.
Then you can loop that method on file paths list. At least that's how I did it. Let me know if you need any further help.
#Lalit-Fauzdar answer and the mentioned reference is the way to go about it. I am sharing a slight upgrade to the implementation
class MultiUploaderS3Client(bucketName:String) {
var bucketName = bucketName
fun uploadMultiple(fileToKeyUploads: MutableMap<String,File>, transferUtility: TransferUtility): Completable? {
return transferUtility(transferUtility)
.flatMapCompletable { transferUtility ->
Observable.fromIterable(fileToKeyUploads.entries)
.flatMapCompletable { entry ->
uploadSingle(
transferUtility,
entry.value,
entry.key
)
}
}
}
private fun transferUtility(transferUtility: TransferUtility): Single<TransferUtility?> {
return Single.create { emitter ->
emitter.onSuccess(
transferUtility
)
}
}
private fun uploadSingle(
transferUtility: TransferUtility?,
aLocalFile: File?,
toRemoteKey: String?
): Completable? {
return Completable.create { emitter ->
transferUtility?.upload(bucketName,toRemoteKey, aLocalFile)
?.setTransferListener(object : TransferListener {
override fun onStateChanged(
id: Int,
state: TransferState
) {
if (TransferState.FAILED == state) {
emitter.onError(Exception("Transfer state was FAILED."))
} else if (TransferState.COMPLETED == state) {
emitter.onComplete()
}
}
override fun onProgressChanged(
id: Int,
bytesCurrent: Long,
bytesTotal: Long
) {
}
override fun onError(id: Int, exception: Exception) {
emitter.onError(exception)
}
})
}
}
}
val multiUploadHashMap = mutableMapOf<String,File>()
MultiUploaderS3Client(AWSConfig.misTicketFilesBucketName).uploadMultiple(multiUploadHashMap, transferUtility)
?.subscribeOn(Schedulers.io())
?.observeOn(Schedulers.io())
?.subscribe {
//Called when all files uploaded
Runnable { callback.onComplete(_Result.Success<String>("uploaded successfully")) }
handler.post(returnCallback)
}

Categories

Resources