My Android app (Java 8, min SDK 24, target SDK 27) downloads images from an FTP server, using Apache's FTPClient (version 3.6).
There are a couple of smaller .jpg images (around 12kb each) that always download completely but all of the bigger .png images (8+ mb each) show up black in my app. The files on the server look fine, so I downloaded the images from the emulator:
Windows' "Photo" app displays a black, horizontal bar at the bottom of all the affected images and they are all missing a couple of bytes (according to Windows' file explorer).
Reading the downloaded image with BitmapFactory.decodeFile(....) on a real device returns a null Bitmap, even though the file exists.
My code (runs on a background thread):
public void downloadFiles(String remoteFolder, String localFolder, ArrayList<String> filenames) {
//login here
ftpClient.setConnectTimeout(10000);
ftpClient.setDefaultTimeout(10000);
OutputStream out = null;
try {
ftpClient.enterLocalPassiveMode();
ftpClient.setFileType(FTP.BINARY_FILE_TYPE);
if(ftpClient.changeWorkingDirectory(remoteFolder)) {
for(String filename : filenames) {
FTPFile[] singleFile = ftpClient.listFiles(filename);
if(singleFile != null && singleFile.length > 0) { //check if file exists
String localPath = localFolder + File.separator + filename;
out = new FileOutputStream(localPath);
if(!ftpClient.retrieveFile(filename, out)) {
//Set error message here
out.close();
break;
}
out.close();
} else {
//Another error message here
break;
}
}
}
} catch (IOException e) {
//And another error message here
} finally {
try {
if(out!=null) { out.close(); }
} catch(IOException e) {
//Doesn't matter
}
}
//logout here
}
I tried out.flush() before closing the stream, a longer timeout (ftpClient.setDefaultTimeout(30000)) and even retrieveFileStream:
InputStream is = ftpClient.retrieveFileStream(filename);
BufferedInputStream bis = new BufferedInputStream(is);
byte[] buf = new byte[10000000];
int len;
while ((len = bis.read(buf)) > 0) {
out.write(buf, 0, len);
}
if(ftpClient.completePendingCommand()) {
Log.d(TAG,"Done"); //This is always printed, even though the files aren't complete
} else {
Log.d(TAG,"Not done");
}
out.close();
bis.close();
The problem persists. I added up len and and even ran bis.read a couple more times (if(fullLength<singleFile[0].getSize())) but no matter what, most of the time less than 10 bytes are missing in the downloaded file. Only rarely one of the images is actually downloaded completely but it seems to be pretty random.
How do I fix this, so it downloads all of the images completely?
Related
I am developing an app where time-cost of an algorithm matters a lot.
In the algorithm, I need to get path string to a file in assets folder. And I got answer from this question.
The file is a configuration file, which is ~400 bytes in size. The library I used requires path, but not some Java string.
My code is like:
public static File getCacheFile(String path, Context context) throws IOException {
File cacheFile = new File(context.getCacheDir(), path);
try {
InputStream inputStream = context.getAssets().open(path);
try {
FileOutputStream outputStream = new FileOutputStream(cacheFile);
try {
byte[] buf = new byte[1024];
int len;
while ((len = inputStream.read(buf)) > 0) {
outputStream.write(buf, 0, len);
}
} finally {
outputStream.close();
}
} finally {
inputStream.close();
}
} catch (IOException e) {
throw new IOException("Could not open file", e);
}
return cacheFile;
}
If I run the algorithm for the first time since I start my app, it will cost ~900ms.
If I run the algorithm again without restarting the app, it will cost ~400ms.
So I guess the time difference is that this function attempts to load the file into cache and read path from the cache? Maybe the file is already in the cache and that is why it is faster.
Is there any way to make it faster? E.g. preload this file to cache in onCreate(), maybe?
Edit: I tried to preload this file in onCreate() and it does not work.
#Override
protected void onCreate(Bundle savedInstanceState) {
super.onCreate(savedInstanceState);
setContentView(R.layout.activity_main);
try {
getCacheFile("a.properties", getApplicationContext());
} catch (IOException e) {
e.printStackTrace();
}
}
Edit2: Not sure whether it matters, but my algorithm is posted here.
My application allows users to select an image to upload. When users select an image from a picasa album my data intent comes back with dat=content://com.sec.android.gallery3d.provider/picasa/item/....
Apparently when selecting an image from a picasa folder, I must handle getting the image differently as noted in this answer.
But before I implement a fix, I want to be able to reproduce the crash so I can verify my fix actually works. So how can I get a Picasa folder on my new (marshmallow) Android test device since Picasa has been killed by Google?
The most guaranteed way of getting a file send inside an intent, is to open a stream to it and copy it over to a private folder on your app.
This way works for local file, content uri, picasa, all of it.
Something like that:
private File getSharedFile() {
Uri uri = intent.getExtras().getParcelable(Intent.EXTRA_STREAM);
// or using the new compat lib
Uri uri = ShareCompat.IntentReader(this).getStream();
InputStream is = null;
OutputStream os = null;
try {
File f = ... define here a temp file // maybe getCacheDir();
is = getContentResolver().openInputStream(uri);
os = new BufferedOutputStream(new FileOutputStream(f));
int read;
byte[] bytes = new byte[2048];
while ((read = is.read(bytes)) != -1) {
os.write(bytes, 0, read);
}
return f;
} catch (Exception e) {
... handle exceptions, buffer underflow, NPE, etc
} finally {
try { is.close(); } catch (Exception e) { /* u never know */ }
try {
os.flush();
os.close();
} catch (Exception e) { /* seriously can happen */ }
}
return null;
}
I'm trying to send PNG file from my android server to my python client.
The PNG image I try to send is a screenshot, around 4mb tops, usually under 2mb.
android code (sending):
File myFile = new File(imagePath);
FileInputStream fis = null;
try {
fis = new FileInputStream(myFile);
} catch (FileNotFoundException e) {
e.printStackTrace();
}
BufferedInputStream bis = new BufferedInputStream(fis);
Log.i("service", "sending file len");
try {
out.write("" +myFile.length());
out.flush();
} catch (Exception e) {
e.printStackTrace();
}
Log.i("service:", "waiteing for ok");
try {
msg = in.readLine();
} catch (Exception e) {
e.printStackTrace();
}
Log.i("service", "sending file");
byte[] outBuffer = new byte[(int) myFile.length()];
try {
bis.read(outBuffer, 0, outBuffer.length);
os = client.getOutputStream();
os.write(outBuffer, 0, outBuffer.length);
} catch (IOException e) {
e.printStackTrace();
}
python code (receiving):
print "waiting for responce's length"
MSGLEN = int(sock.recv(bufferLen))
print MSGLEN
sock.sendall("ok" +"\n")
chunks = []
bytes_recd = 0
while bytes_recd < MSGLEN:
chunk = sock.recv(min(MSGLEN - bytes_recd, bufferLen))
chunks.append(chunk)
bytes_recd = bytes_recd + len(chunk)
dataRecived = ''.join(chunks)
print 'data receieved'
print 'writing data to file'
fileout = open("D:\shots.png", 'w')
fileout.write(dataRecived)
fileout.close()
The file transfers from the android to my PC, but the file is corrupted.
When I compare it with the original image, almost everything is identical
except some empty lines here and there (not missing information, just empty line like someone added \n) and 1 or 2 big chunks of lines (15 lines or so) are missing.
Here you can see comparison between the tho files (left-original, right-file after sending).
I don't know why the file transfers corrupted, please help me.
Try coding it as Base64 and sending a simple string. Those missing lines are also part of image data - remember that those are binary.
Base64 or Bytestream is what u need
I am facing a problem downloading PNG images from my server to my Android app. The problem is specific to PNG images (JPG works fine), and the issue is that the downloaded files are corrupt images. I will explain in more details, below.
Scenario :
I need to download JPG and PNG images from my server, and display them to the user of the Android app.
Issue :
The JPG images get downloaded without an issue. But the downloaded PNG files are corrupt. I have double checked the source of the images at my server, and they are proper. Its only the downloaded PNG files, that are corrupt. So, the problem probably lies in the way I am downloading them in Android.
Code Sample :
URL imageURL;
File imageFile = null;
InputStream is = null;
FileOutputStream fos = null;
byte[] b = new byte[1024];
try {
// get the input stream and pass to file output stream
imageURL = new URL(image.getServerPath());
imageFile = new File(context.getExternalFilesDir(null), image.getLocalPath());
fos = new FileOutputStream(imageFile);
// get the input stream and pass to file output stream
is = imageURL.openConnection().getInputStream();
// also tried but gave same results :
// is = imageURL.openStream();
while(is.read(b) != -1)
fos.write(b);
} catch (FileNotFoundException e) {
} catch (MalformedURLException e) {
} catch (IOException e) {
} finally {
// close the streams
try {
if(fos != null)
fos.close();
if(is != null)
is.close();
} catch(IOException e){
}
}
Any pointers on how I can work on this, will be very appreciated.
Note :
Since this is happening in a service, there are no problems of doing this inside an AsyncTask.
The problem is here
while(is.read(b) != -1)
fos.write(b);
This is wrong, because in each iteration it will write the full buffer (1024 bytes) to the file. But the previous read could have read less bytes than than (almost surely on the last loop, unless the image lenght happens to be exactly a multiple of 1024). You should check how many bytes were read each time, and write that amount of bytes.
int bytesRead;
while( (bytesRead = is.read(b)) != -1)
fos.write(b,0,bytesRead );
Your error makes you write always files with sizes that are multiple of 1024 - which of course is not the case in general. Now, what happens when a image is saved with extra trailing bytes depends on the format and on the image reader. In some cases, it might work. Still, it's wrong.
BTW: never swallow exceptions - even if that's not the problem today, it might be tomorrow and you might spend hours finding the problem.
I am trying to copy a large pdf-file (3.7 mb) from my raw-folder to the external cache directory.
I a using the following piece of code:
int i = 0;
if (Environment.MEDIA_MOUNTED.equals(Environment.getExternalStorageState()))
{
InputStream input = getResources().openRawResource(pdfs[i]);
File file = new File(Environment.getExternalStorageDirectory(), "/Android/data/eu.app/cache/" + pdfNames[i]);
if(!file.exists())
{
try
{
new File(Environment.getExternalStorageDirectory().getAbsolutePath() + "/Android/data/eu.app/cache").mkdirs();
FileOutputStream fos = new FileOutputStream(file.toURI().getPath(), false);
OutputStream os = new BufferedOutputStream(fos);
byte[] buffer = new byte[1024];
int byteRead = 0;
while ((byteRead = input.read(buffer)) != -1) {
os.write(buffer, 0, byteRead);
}
fos.close();
}
catch(Exception ex)
{
Log.d("storage", ex.getMessage());
}
}
}
else
{
}
I don't get any errors, but the output-file is a few bytes smaller than the original and cannot be opened.
What do I need to do to fix this?
I think the main issue is that you close fos while you should close os. You also need to put the close operation in a finally block.
Update (now with a full keyboard ;)): You close the file stream (fos) before the buffered stream is flushed. What you should do is to close the buffered stream (os), and that will in turn flush its buffer and write those bytes that are missing, and then it will automatically close the underlying file stream. To fix it change fos.close() into os.close().
In addition, to make sure that you always close the stream you should place the close operation in a finally block. A typical pattern is the following:
BufferedInputStream in = null;
try {
in = new BufferedInputStream(anInputStream);
BufferedOutputStream out = null;
try {
out = new BufferedOutputStream(new FileOutputStream(aFile));
// Read and write what you should write
}
finally {
if (out != null) out.close();
}
} finally {
if (in != null) in.close();
}
You can easily add an input stream, but be careful to make sure all streams are closed. This can be handled by nesting finally blocks or nesting try-catch blocks inside the finally block.
Either you throw an IOException from this method and handle it outside (often preferred), or you wrap the above code in a new try-catch statement and handle it there. However, handling it within the method mixes UI with logic and the code is often clearer separating UI and logic.
A final note: 1024 is rather small. Play with different values. On the other hand the buffered stream will handle the buffering for you.
I've been using this function for reading from one stream to another for a few years and have never had any problems with the resulting file. Just open the source and target files as you are and pass their respective streams into this function:
public static void streamToStream(InputStream is, OutputStream os) {
int count = 0;
try {
while(count != -1) {
byte[] bytes = new byte[2048];
count = is.read(bytes);
if(count == -1) {
continue;
}
os.write(bytes, 0, count);
bytes = null;
}
} catch (Exception e) {
e.printStackTrace();
}
}