I'm currently using Retrofit by Square for Android network communications. Is there a way to get its progress during a task to create a progress notification, something similar to that which Facebook uses when uploading an image?
Use Case would be to load an image hopefully of full image quality without compression or scaling.
I see how it is possible with an asynctask but that would defeat the purpose of using Retrofit. However that might be the route I would have to take.
This answer is for Retrofit 1. For solution compatible with Retrofit 2 see this answer.
I had the same problem and finally managed to do it. I was using spring lib before and what I show below kind worked for Spring but was inconsistent since I made a mistake on using it for the InputStream. I moved all my API's to use retrofit and upload was the last one on the list, I just override TypedFile writeTo() to update me on the bytes read to the OutputStream. Maybe this can be improved but as I said I made it when I was using Spring so I just reused it.
This is the code for upload and it's working for me on my app, if you want download feedback then you can use #Streaming and read the inputStream.
ProgressListener
public interface ProgressListener {
void transferred(long num);
}
CountingTypedFile
public class CountingTypedFile extends TypedFile {
private static final int BUFFER_SIZE = 4096;
private final ProgressListener listener;
public CountingTypedFile(String mimeType, File file, ProgressListener listener) {
super(mimeType, file);
this.listener = listener;
}
#Override public void writeTo(OutputStream out) throws IOException {
byte[] buffer = new byte[BUFFER_SIZE];
FileInputStream in = new FileInputStream(super.file());
long total = 0;
try {
int read;
while ((read = in.read(buffer)) != -1) {
total += read;
this.listener.transferred(total);
out.write(buffer, 0, read);
}
} finally {
in.close();
}
}
}
MyApiService
public interface MyApiService {
#Multipart
#POST("/files")
ApiResult uploadFile(#Part("file") TypedFile resource, #Query("path") String path);
}
SendFileTask
private class SendFileTask extends AsyncTask<String, Integer, ApiResult> {
private ProgressListener listener;
private String filePath;
private FileType fileType;
public SendFileTask(String filePath, FileType fileType) {
this.filePath = filePath;
this.fileType = fileType;
}
#Override
protected ApiResult doInBackground(String... params) {
File file = new File(filePath);
totalSize = file.length();
Logger.d("Upload FileSize[%d]", totalSize);
listener = new ProgressListener() {
#Override
public void transferred(long num) {
publishProgress((int) ((num / (float) totalSize) * 100));
}
};
String _fileType = FileType.VIDEO.equals(fileType) ? "video/mp4" : (FileType.IMAGE.equals(fileType) ? "image/jpeg" : "*/*");
return MyRestAdapter.getService().uploadFile(new CountingTypedFile(_fileType, file, listener), "/Mobile Uploads");
}
#Override
protected void onProgressUpdate(Integer... values) {
Logger.d(String.format("progress[%d]", values[0]));
//do something with values[0], its the percentage so you can easily do
//progressBar.setProgress(values[0]);
}
}
The CountingTypedFile is just a copy of TypedFile but including the ProgressListener.
If you want to get the max value in order to show it on a ProgressDialog, Notification, etc.
ProgressListener
public interface ProgressListener {
void transferred(long num, long max);
}
CountingTypedFile
public class CountingTypedFile extends TypedFile {
private static final int BUFFER_SIZE = 4096;
private final ProgressListener listener;
public CountingTypedFile(String mimeType, File file, ProgressListener listener) {
super(mimeType, file);
this.listener = listener;
}
#Override
public void writeTo(OutputStream out) throws IOException {
byte[] buffer = new byte[BUFFER_SIZE];
FileInputStream in = new FileInputStream(super.file());
long total = 0;
try {
int read;
while ((read = in.read(buffer)) != -1) {
total += read;
this.listener.transferred(total, super.file().length());
out.write(buffer, 0, read);
}
} finally {
in.close();
}
}
}
Related
So I'm trying to download a very large file through Retrofit2. Files can range up to 5-10 GB. I'm launching an asynchronous structure from an activity/fragment (I've tried both AsyncTask and IntentService) and streaming the file and writing the bytes to a file on the internal storage. I'm publishing progress of the filewrite after each buffer read.
Files up to 150 MB or so work fine, but when I try a 5 GB file the stream silently dies after about 1 GB. There are no logs or logcat that I can see, no exceptions are thrown just poof.
Does anyone have an idea on what's happening, or perhaps I wrote something wrong?
public interface IFileshareDownload {
#Streaming
#GET("File/Download/{guid}")
Call<ResponseBody> downloadFileByGuid(#Path("guid") String guid);
}
public class FileshareDownloadService extends IntentService {
private static final String TAG = "[FileDownloadService]";
private static final String PATH = "/ftp/";
private static final int FILE_CHUNK_SIZE = 2 * 1024 * 1024; //2MB buffer
private String mFilename;
private String mFileshareDirectory;
private String mAbsoluteFilePath;
private String mBaseUrl;
private String mGuid;
private Long mFileSize;
private Retrofit mRetrofit;
private IFileshareDownload mDownloader;
private ResultReceiver mReceiver;
public FileshareDownloadService() {
super("FileshareDownload");
}
#Override
protected void onHandleIntent(#Nullable Intent intent) {
this.mBaseUrl = intent.getStringExtra("baseUrl");
this.mGuid = intent.getStringExtra("guid");
this.mFilename = intent.getStringExtra("fileName");
this.mFileSize = intent.getLongExtra("fileSize", -1);
this.mFileshareDirectory = getApplicationContext().getFilesDir().getAbsolutePath() + PATH;
this.mAbsoluteFilePath = mFileshareDirectory + mFilename;
this.mRetrofit = new Retrofit.Builder()
.baseUrl(mBaseUrl)
.callbackExecutor(Executors.newSingleThreadExecutor())
.build();
this.mDownloader = mRetrofit.create(IFileshareDownload.class);
this.mReceiver = intent.getParcelableExtra("listener");
downloadFile();
}
public void downloadFile() {
Call<ResponseBody> call = mDownloader.downloadFileByGuid(mGuid);
try {
Response<ResponseBody> response = call.execute();
if(response.isSuccessful()) {
File file = new File(mAbsoluteFilePath);
file.createNewFile();
try(FileOutputStream fos = new FileOutputStream(file)) {
InputStream is = response.body().byteStream();
setUpdateProgress(SHOW_PROGRESS);
int count = 0;
long bytesRead = 0;
byte[] buffer = new byte[FILE_CHUNK_SIZE];
try {
while ((count = is.read(buffer)) != -1) {
fos.write(buffer, 0, count);
fos.flush();
bytesRead += count;
int progress = getPercent(bytesRead, mFileSize);
Log.d(TAG, String.format("Read %d out of %d bytes.", bytesRead, mFileSize));
setUpdateProgress(UPDATE_PROGRESS, progress);
}
} catch (Throwable t)
{
Log.e(TAG, "What happened?", t);
}
}
setUpdateProgress(HIDE_PROGRESS);
} else {
setUpdateProgress(HIDE_PROGRESS);
}
} catch (IOException e) {
e.printStackTrace();
}
}
}
Here is a great tutorial about downloading files. It probably mentions what you need:
https://futurestud.io/tutorials/retrofit-2-how-to-download-files-from-server
If it doesn't, take a look at the #Multipart annotation, as well as #Part. I'm not sure if you can use it with GET, but since you have no answers yet, I'll just post it here so you can take the shot if you want.
This is an example from a project I had, in which we create a multipart body to upload an image. I know you want a GET, but the example should still be relevant:
// Setting up the multipart file
File newAvatar = new File(getRealPathFromURI(avatarUri)); // the new avatar
RequestBody filePart = RequestBody.create(MediaType.parse(getActivity().getContentResolver().getType(avatarUri)), newAvatar);
And your request (a POST in this example) should be something like this:
#Multipart
#POST("/api/v1/me/account/upload-cover") // upload avatar
Call<ResponseChangeAvatar> sendChangeAvatarRequest(#Part MultipartBody.Part file, #Header("Authorization") String token);
The retrofit documentation (just search for multipart):
http://square.github.io/retrofit/
A tutorial, in which he creates a multipart body to upload a file to a server:
https://futurestud.io/tutorials/retrofit-2-how-to-upload-files-to-server
Hope this helps. Let me know if you found the solution.
I'm trying to create a progress dialog that updates the amount of file updated to server and display them in progress dialog, I followed this post( https://stackoverflow.com/a/33384551 ) the code:
public class ProgressRequestBody extends RequestBody {
private File mFile;
private String mPath;
private UploadCallbacks mListener;
private static final int DEFAULT_BUFFER_SIZE = 2048;
public interface UploadCallbacks {
void onProgressUpdate(int percentage);
void onError();
void onFinish();
}
public ProgressRequestBody(final File file, final UploadCallbacks listener) {
mFile = file;
mListener = listener;
}
#Override
public MediaType contentType() {
// i want to upload only images
return MediaType.parse("image/*");
}
#Override
public long contentLength() throws IOException {
return mFile.length();
}
#Override
public void writeTo(BufferedSink sink) throws IOException {
long fileLength = mFile.length();
byte[] buffer = new byte[DEFAULT_BUFFER_SIZE];
FileInputStream in = new FileInputStream(mFile);
long uploaded = 0;
try {
int read;
Handler handler = new Handler(Looper.getMainLooper());
while ((read = in.read(buffer)) != -1) {
// update progress on UI thread
handler.post(new ProgressUpdater(uploaded, fileLength));
uploaded += read;
sink.write(buffer, 0, read);
}
} finally {
in.close();
}
}
private class ProgressUpdater implements Runnable {
private long mUploaded;
private long mTotal;
public ProgressUpdater(long uploaded, long total) {
mUploaded = uploaded;
mTotal = total;
}
#Override
public void run() {
mListener.onProgressUpdate((int)(100 * mUploaded / mTotal));
}
}
}
In api interface:
#Multipart
#POST("/upload")
Call<JsonObject> uploadImage(#Part MultipartBody.Part file);
in myActiviy:
class MyActivity extends AppCompatActivity implements ProgressRequestBody.UploadCallbacks {
ProgressBar progressBar;
#Override
protected void onCreate(Bundle savedInstanceState) {
super.onCreate(savedInstanceState);
progressBar = findViewById(R.id.progressBar);
ProgressRequestBody fileBody = new ProgressRequestBody(file, this);
MultipartBody.Part filePart = MultipartBody.Part.createFormData("image", file.getName(), fileBody);
Call<JsonObject> request = RetrofitClient.uploadImage(filepart);
request.enqueue(new Callback<JsonObject>{...});
}
#Override
public void onProgressUpdate(int percentage) {
// set current progress
progressBar.setProgress(percentage);
}
#Override
public void onError() {
// do something on error
}
#Override
public void onFinish() {
// do something on upload finished
// for example start next uploading at queue
progressBar.setProgress(100);
}
}
The problem I face is when I try to pass the "fileBody" object to
MultipartBody.Part filePart = MultipartBody.Part.createFormData("image",
file.getName(), fileBody);
I get an error as filebody cannot be passed in, it's a wrong argument.
I took reference from this post( https://stackoverflow.com/a/33384551 ) and coded it same but im getting erros.
You missing file paramater. Try it
File file = new File(your image path);
ProgressRequestBody fileBody = new ProgressRequestBody(file, this);
MultipartBody.Part filePart = MultipartBody.Part.createFormData("image", file.getName(), fileBody);
I'm using this method to download a file from Google Drive.
My code:
ByteArrayOutputStream byteArrayOutputStream = new ByteArrayOutputStream();
driveService.files().export(remoteFiles[0].getId(),"text/plain").executeMediaAndDownloadTo(byteArrayOutputStream);
FileOutputStream fileOutputStream= new FileOutputStream(new File(downloadsDirectory,remoteFiles[0].getName()));
byteArrayOutputStream.writeTo(fileOutputStream);
byteArrayOutputStream.flush();
byteArrayOutputStream.close();
fileOutputStream.close();
Is there a way to get the progress of the file download?
Answering my own question,found it on this page.
//custom listener for download progress
class DownloadProgressListener implements MediaHttpDownloaderProgressListener{
#Override
public void progressChanged(MediaHttpDownloader downloader) throws IOException {
switch (downloader.getDownloadState()){
//Called when file is still downloading
//ONLY CALLED AFTER A CHUNK HAS DOWNLOADED,SO SET APPROPRIATE CHUNK SIZE
case MEDIA_IN_PROGRESS:
//Add code for showing progress
break;
//Called after download is complete
case MEDIA_COMPLETE:
//Add code for download completion
break;
}
}
}
//create a Drive.Files.Get object,
//set a ProgressListener
//change chunksize(default chunksize seems absurdly high)
Drive.Files.Get request = driveService.files().get(remoteFiles[0].getId());
request.getMediaHttpDownloader().setProgressListener(new DownloadProgressListener()).setChunkSize(1000000);
request.executeMediaAndDownloadTo(outputStream);
The first thing that I can think of is wrapping the FileOutputStream in a custom Stream that overrides the write method to additionally notify a listener for the amount of bytes written. Like this:
public class ProgressOutputStream extends OutputStream {
IProgressListener _listener;
OutputStream _stream;
int _position;
public ProgressOutputStream(OutputStream stream, IProgressListener listener) {
_stream = stream;
_listener = listener;
}
#Override
public void write(byte[] b, int offset, int len) {
_stream.write(b, offset, len);
_position += len;
reportProgress();
}
private void reportProgress() {
_listener.onProgressUpdate(_position);
}
#Override
public void write(int b) {
_stream.write(b);
}
}
interface IProgressListener {
void onProgressUpdate(int progress);
}
The rest is to know the size of the file so that you can calculate the progress in percentages in your progress listener.
I want to write and read from file in the same time without errors.
For example, I will starting new Thread for writing to file from my running service.
In my activity i will starting new Thread for reading from the same file.
I wan't to do this synchronously. Some thing like this :
To wait execution of next thread until previous finished.
Next thread must not start until previous thread stops, irrespective of time consumption.
My code for read and write:
public static final String ROUTE_FILE_NAME = "route.txt";
public static void savePointToFile(Context context, String point) throws IOException {
FileOutputStream fOut = context.openFileOutput(ROUTE_FILE_NAME, Context.MODE_APPEND);
OutputStreamWriter osw = new OutputStreamWriter(fOut);
osw.write(point);
osw.flush();
osw.close();
}
public static String readRouteFromFile(Context context) {
StringBuffer fileContent = new StringBuffer(UIUtils.emptyString());
byte[] buffer = new byte[1024];
try {
FileInputStream fis = context.openFileInput(ROUTE_FILE_NAME);
int length;
while ((length = fis.read(buffer)) != -1) {
fileContent.append(new String(buffer));
}
} catch (FileNotFoundException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
}
return fileContent.toString();
}
Thanks in advance.
If you just want the read method called from a thread to wait for the write method called from another thread to be finished, and vice versa, just synchronize both methods on a common object:
private static final Object fileLock = new Object();
public static String readFile() {
synchronize(fileLock) {
[your current read code here]
}
}
public static void write(String data) {
synchronize(fileLock) {
[your current write code here]
}
}
You can look at a special thread pool executor service.
final ExecutorService threadpool = Executors.newSingleThreadExecutor();
Its fairly easy, just create runnables and put it in the threadpool. It contains a single thread so all your runnables are queued sequentially. Otherwise you could create a normal executorservice and set the threadpool to 1. Effectively its the same. Hope this helps
http://www.concretepage.com/java/newsinglethreadexecutor_java
So its like
WorkerThread.get(context).read()
WorkerThread.get(context).write()
You can even implement future calls instead of defining an explicit callback.
Just a general idea of how it can work. You need to save filepointers so you know where to pause and continue read/write. Other you will always start from the first data position in the file.
class WorkerThread {
interface Callback {
void onCompleteRead(String buffer, int pauseReadPointer);
void onCompleteWrite(int pauseWritePointer);
}
enum Action {
READ,
WRITE
}
private static WorkerThread singleton;
public static synchronized WorkerThread get(final Context context) {
if (instance == null) {
instance = new WorkerThread(context);
}
return instance;
}
private final Context context;
private final ExecutorService threadPool;
private WorkerThread(context) {
threadPool = Executors.newSingleThreadExecutor()
}
// PUBLIC READ CALL
public void read(int resumeReadPointer, Callback callback, "other params") {
queueJob(READ, null, resumeReadPointer, callback);
}
// PUBLIC WRITE CALL
public void write(String in, int resumeWritePointer, Callback callback, "other params") {
queueJob(WRITE, in, resumeWritePointer, callback);
}
private void queueJob(final Action action, String buffer, final int pointer, final Callback callback) {
/* Create handler in UI thread. */
final Handler handler = new Handler() {
#Override
public void handleMessage(Message msg) {
ResultPack pack = (ResultPack) msg.obj;
if (Action.READ == action) {
callback.onCompleteRead(pack.result, pack.pointer);
} else {
callback.onCompleteWrite(pack.pointer);
}
}
};
// single threadpool. everything is FIFO
threadPool.submit(new FileRunnable(action, buffer, handler, pointer));
}
private class ResultPack {
private final String result;
private final int pointer;
private ResultPack(String s, int p) {
this.result = s;
this.pointer = p;
}
}
private class FileRunnable implements Runnable {
private int pointer = 0;
private final Handler handler;
private final buffer = buffer;
FileRunnable(final Action action, String buffer, final Handler handler, final int pointer) {
this.pointer = pointer;
this.handler = handler;
this.buffer = buffer;
}
#Override
public void run() {
if (Action.READ == action) {
ResultPack pack = readRouteFromFile(..., pointer);
} else { // write
ResultPack pack = savePointToFile(..., buffer, pointer);
}
Message message = Message.obtain();
message.obj = pack;
handler.sendMessage(message);
}
}
}
I have two AsyncTasks, one is used to download xml file (DownloadTask), another is for parsing the file (ParseXMLTask).
There are two cases of using this tasks:
1) File doesn't exists > execute DownloadTask, onPostExecute > ParseXMLTask
2) File exists > execute only ParseXMLTask.
Everything is working, but the thing is, while performing the second case, there is a blocking the ui about 3 sec (black screen) that surely would make a user annoyed. This is absolutely confusing me, because the job in the second case seems to be easier.
So when I am testing my app, a situation is like that: I click on the button for the first time, file is being downloaded, saved on the sd card, parsed and finally opened. Then I go back and click on the button again. Now I see that lag while switching between activities.
Code:
Executing the tasks
private void downloadPack() {
if (packDownloaded) {
parseXML();
} else {
download = new DownloadFile(fileName, this, loadingBar);
download.execute(serverURL + fileName + ".xml");
}
}
private void parseXML() {
ParseXMLTask parseTask = new ParseXMLTask(this, this);
parseTask.execute(PATH + fileName + ".xml");
}
public void postDownload(File result) {
parseXML();
}
public void postParse() {
Intent packIntent = new Intent(this, PackActivity.class);
startActivity(packIntent);
}
ParseXMLTask.java
public class ParseXMLTask extends AsyncTask<String, Integer, Void> {
private Context context;
private XmlPullParser xpp;
private IPostParse iPostParse;
public ParseXMLTask(Context context, IPostParse iPostParse) {
this.context = context;
this.iPostParse = iPostParse;
}
#Override
protected Void doInBackground(String... params) {
File file = new File(params[0]);
/* doing the job */
}
#Override
protected void onPostExecute(Intent result) {
iPostParse.postParse(result);
}
}
DownloadFile.java
public class DownloadFile extends AsyncTask<String, Integer, File> {
private static final String PATH = Environment
.getExternalStorageDirectory().getPath() + "/.chgkgame/";;
private File dir;
private ProgressBar progressBar;
private String fileName;
private IPostDownload postDownload;
private boolean download;
public DownloadFile(String name, IPostDownload pDownload, ProgressBar pBar) {
progressBar = pBar;
fileName = name;
postDownload = pDownload;
}
#Override
protected File doInBackground(String... sUrl) {
URL url;
try {
url = new URL(sUrl[0]);
URLConnection urlConnection = url.openConnection();
urlConnection.connect();
int fileLength = urlConnection.getContentLength();
dir = new File(PATH + fileName + ".xml");
InputStream input = new BufferedInputStream(url.openStream());
OutputStream output = new FileOutputStream(dir);
byte[] data = new byte[1024];
long total = 0;
int count;
while ((count = input.read(data)) != -1) {
total += count;
publishProgress((int) (total * 100 / fileLength));
output.write(data, 0, count);
}
output.flush();
output.close();
input.close();
} catch (MalformedURLException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
}
return dir;
}
#Override
protected void onPostExecute(File result) {
if (postDownload != null) postDownload.postDownload(result);
}
#Override
protected void onProgressUpdate(Integer... values) {
super.onProgressUpdate(values);
if (progressBar != null) {
progressBar.setProgress(values[0]);
}
}
}
There is nothing wrong with the above.
The parsing is pretty fast that is why you only see the layout for a split second.
The black screen will be the PackActivity loading, check this activity for what is blocking the UI thread.
You could have also put LogCat messages in to show that the parsing has finished and onCreate of the next Activity is called.