Develop Reference

File written by Android SAF has the tail of the original version

I made an app Tiddloid and it runs well on my phone (a Meizu with an os based on Android 8.1). It saves modified html files to the original file using the following code:
...
try (ByteArrayInputStream is = new ByteArrayInputStream(data.getBytes(StandardCharsets.UTF_8));
OutputStream os = getContentResolver().openOutputStream(uri)) {
if (os == null)
throw new FileNotFoundException(MainActivity.EXCEPTION_SAF_FILE_NOT_EXISTS);
int len = is.available();
int length, lengthTotal = 0;
byte[] b = new byte[MainActivity.BUF_SIZE];
while ((length = is.read(b)) != -1) {
os.write(b, 0, length);
lengthTotal += length;
}
os.flush();
if (lengthTotal != len)
throw new IOException(MainActivity.EXCEPTION_TRANSFER_CORRUPTED);
failed = false;
runOnUiThread(() -> {
if (tree != null && treeIndex != null) try {
syncTree(tree, id, treeIndex);
} catch (IOException e) {
e.printStackTrace();
}
getInfo(wv);
});
} catch (IOException e) {
e.printStackTrace();
Toast.makeText(TWEditorWV.this, R.string.failed, Toast.LENGTH_SHORT).show();
failed = true;
dumpOnFail(data.getBytes(StandardCharsets.UTF_8), uri);
}
...
Several weeks ago I'm testing it on an Android 11 AVD. I noticed that sometimes there are html codes at the end of the saved file, especially if the new file to be saved is smaller than original:
OOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOO <- this is the original
XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX <- this is the new file
XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXOOO <- the actual saved file
At that time I treated it as a bug of AVD, because this had never happened to other AVDs and my phone. But these days more and more people's posting issues on GitHub, complaining that the same thing's happening. Anyone have idea about this? I even made it throw exceptions if detected that written size not equal to the source bytearray, but it never throws, just making the file corrupted.

on android how to find the original file if got a hard link

in the app when receiving an intent which was created from other app and has a file path, it can access the file's content using the file path.
the question is if that path (call it as 'link-path') is a 'hard link' to the original file, is it possible to find the original file through this 'link-path'?
Searched and find some post like:
https://unix.stackexchange.com/questions/122333/how-to-tell-which-file-is-original-if-hard-link-is-created
they show some unix shell command. Not sure if there is some android file system support for this, anyone having suggestion?

You can use this code I made, based on this post. It will return the target path of any path. If path is not a symbolic link, it will return itself. If path doesn't exist it returns null.
public static String findLinkTarget(String path) {
try {
Process findTarget = Runtime.getRuntime().exec("readlink -f " + path);
BufferedReader br = new BufferedReader(new InputStreamReader(findTarget.getInputStream()));
return br.readLine();
} catch (IOException e) {
Log.w(TAG, "Couldn't find target file for link: " + path, e);
}
}
The code wasn't tested, but I tested the command on Termux and it worked.
EDIT: Try calling getCanonicalPath() on your file, I think it resolves the symlink.

find a way by comparing the inode, in api >21 android has Os to get it, otherwise using the command "ls -i" to get the inode. One issue though, tested on api<=18 the "ls -i" does not return any thing (tested on emulator), in that case maybe fallback to compare the file's size and timestamp.
static String getFileInode(File file) {
String inode = "-1";
if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.LOLLIPOP) {
StructStat st = null;
try {
ParcelFileDescriptor pfd = ParcelFileDescriptor.open(file,
ParcelFileDescriptor.parseMode("r"));
st = Os.fstat (pfd.getFileDescriptor());
if (st != null) {
inode = ""+st.st_ino;
}
} catch (Exception e) {
Log.e(TAG, "fstat() failed”+ e.getMessage());
}
} else {
BufferedReader reader = null;
try {
Process process = Runtime.getRuntime().exec(("ls -il " + path));
reader = new BufferedReader(
new InputStreamReader(process.getInputStream()));
int read;
char[] buffer = new char[4096];
StringBuffer output = new StringBuffer();
while ((read = reader.read(buffer)) > 0) {
output.append(buffer, 0, read);
}
process.waitFor();
String ret = output.toString();
if (!TextUtils.isEmpty(ret)) {
ret = ret.trim();
String[] splitArr = ret.split("\\s+");
if (splitArr.length>0) {
inode = splitArr[0];
}
}
} catch(Exception e) {
Log.e(TAG, "!!! Runtime.getRuntime().exec() exception, cmd:”+cmd);
} finally {
if (reader != null) {
try {
reader.close();
} catch (IOException e) {}
}
}
}
return inode;
}

0-byte files not detected when downloading files on Android

I have an app for Android which downloads hundreds of files from the Internet. Some files turn out to be 0-byte after download. The app attempts to detect such cases and delete such files after download but sometimes it fails. The problem is more frequent on Android 4.x devices.
Here is the method which does the downloading. I gets the number of actually read bytes from inputStream.read(buffer).
public class Utils
{
public static class DownloadFileData
{
int nTotalSize;
int nDownloadedSize;
}
public interface ProgressCallback
{
void onProgress(long nCurrent, long nMax);
}
public static boolean downloadFile(String sFileURL, File whereToSave, DownloadFileData fileData, ProgressCallback progressCallback)
{
InputStream inputStream = null;
FileOutputStream fileOutput = null;
try
{
URL url = new URL(sFileURL);
URLConnection connection = url.openConnection();
//set up some things on the connection
connection.setDoOutput(true);
connection.connect();
fileOutput = new FileOutputStream(whereToSave);
inputStream = connection.getInputStream();
fileData.nTotalSize = connection.getContentLength();
fileData.nDownloadedSize = 0;
byte[] buffer = new byte[1024];
int bufferLength = 0; //used to store a temporary size of the buffer
// now, read through the input buffer and write the contents to the file
while ((bufferLength = inputStream.read(buffer)) > 0)
{
// if interrupted, don't download the file further and return
// also restore the interrupted flag so that the caller stopped also
if (Thread.interrupted())
{
Thread.currentThread().interrupt();
return false;
}
// add the data in the buffer to the file in the file output stream
fileOutput.write(buffer, 0, bufferLength);
// add up the size so we know how much is downloaded
fileData.nDownloadedSize += bufferLength;
if (null != progressCallback && fileData.nTotalSize > 0)
{
progressCallback.onProgress(fileData.nDownloadedSize, fileData.nTotalSize);
}
}
return true;
}
catch (FileNotFoundException e)
{
return false; // swallow a 404
}
catch (IOException e)
{
return false; // swallow a 404
}
catch (Throwable e)
{
return false;
}
finally
{
// in any case close input and output streams
if (null != inputStream)
{
try
{
inputStream.close();
inputStream = null;
}
catch (Exception e)
{
}
}
if (null != fileOutput)
{
try
{
fileOutput.close();
fileOutput = null;
}
catch (Exception e)
{
}
}
}
}
Here is the piece of code which processes the downloads. Since sometimes the number of read bytes is incorrect (it is > 0 and the real file has the size 0 bytes) I check the size of the downloaded file with outputFile.length(). But this again gives a value > 0 even though the file is really 0 byte. I tried to also just create a new file and read its size with recheckSizeFile.length(). Still the size is determined as > 0 while it's really 0 byte.
Utils.DownloadFileData fileData = new Utils.DownloadFileData();
boolean bDownloadedSuccessully = Utils.downloadFile(app.sCurrenltyDownloadedFile, outputFile, fileData, new Utils.ProgressCallback()
{
... // progress bar is updated here
});
if (bDownloadedSuccessully)
{
boolean bIsGarbage = false;
File recheckSizeFile = new File(sFullPath);
long nDownloadedFileSize = Math.min(recheckSizeFile.length(), Math.min(outputFile.length(), fileData.nDownloadedSize));
// if the file is 0bytes, it's garbage
if (0 == nDownloadedFileSize)
{
bIsGarbage = true;
}
// if this is a video and if of suspiciously small size, it's
// garbage, too
else if (Utils.isStringEndingWith(app.sCurrenltyDownloadedFile, App.VIDEO_FILE_EXTENSIONS) && nDownloadedFileSize < Constants.MIN_NON_GARBAGE_VIDEO_FILE_SIZE)
{
bIsGarbage = true;
}
if (bIsGarbage)
{
++app.nFilesGarbage;
app.updateLastMessageInDownloadLog("File is fake, deleting: " + app.sCurrenltyDownloadedFile);
// delete the garbage file
if (null != outputFile)
{
if (!outputFile.delete())
{
Log.e("MyService", "Failed to delete garbage file " + app.sCurrenltyDownloadedFile);
}
}
}
else
{
... // process the normally downloaded file
}
I am not sure but I think there is a bug in Android with reading file size. Has anyone seen a similar problem? Or am I maybe doing something wrong here?
Thanks!
EDIT: how i determine that the files are 0-byte:
all the files which get downloaded go thru the described routines. When I then later view the download folder with a file browser (Ghost Commander), some of the files (like maybe 10%) are 0-byte. They can't be played by a video player (shown as "broken file" icon).

It looks to me like your problem is that you only check for "garbage" files if the Utils.downloadFile call returns true. If the download fails in the getInputStream call or the first read, you will have created a file with zero length which will never be deleted.

You should call flush() on your FileOutputStream to ensure that all data is written to the file. This should make your issue with 0-byte files occur less often.
To check for 0 byte files using File.length() should work properly. Can you open a shell (adb shell) on the device and run ls -l to see the byte count displayed by it is 0 (maybe your file manager has some weird issues). Also please debug (or put some log statements) that sFullPath contains the correct file paths. I can't see where sFullPath gets set in your code above and why you don't just use outputFile but recreate another File object.

"IOException: Try again" while using LocalServerSocket

Does anybody have more helpful information on the Exception "Try again"?
I'm sending Bitmaps between apps using LocalServerSocket and LocalSocket:
Output:
socket = new LocalSocket();
socket.connect(new LocalSocketAddress(SOCKET_NAME));
fos = new DataOutputStream(socket.getOutputStream());
...
public void onEvent() {
fos.writeInt(width);
fos.writeInt(height);
fos.writeInt(newBuffer.length);
fos.write(newBuffer);
}
Input:
server = new LocalServerSocket(SOCKET_NAME);
socket = server.accept();
socket.setSoTimeout(60);
while(true) {
int width = fis.readInt(); // IO Exception being thrown here
int height = fis.readInt();
int length = fis.readInt();
byte[] bytes = new byte[length];
fis.read(bytes);
}
[try/catch etc removed for clarity]
04-18 09:19:11.664: W/System.err(1268): java.io.IOException: Try again
04-18 09:19:11.664: W/System.err(1268): at android.net.LocalSocketImpl.readba_native(Native Method)
04-18 09:19:11.664: W/System.err(1268): at android.net.LocalSocketImpl.access$400(LocalSocketImpl.java:29)
04-18 09:19:11.664: W/System.err(1268): at android.net.LocalSocketImpl$SocketInputStream.read(LocalSocketImpl.java:92)
04-18 09:19:11.664: W/System.err(1268): at libcore.io.Streams.readFully(Streams.java:81)
04-18 09:19:11.664: W/System.err(1268): at java.io.DataInputStream.readInt(DataInputStream.java:124)
04-18 09:19:11.664: W/System.err(1268): at com.test.util.BitmapSendingUtils$BitmapReceiver$1.run(BitmapSendingUtils.java:105)

The exception you see is probably the java equivalent to the EAGAIN error. See for example this answer.
You should handle the exception and try the failed IO operation again.

I've since rengineered this as I couldn't find the solution. But while implementing it a different way I came across these errors in the original code:
byte[] bytes = new byte[length];
fis.read(bytes);
Should be:
byte[] content = new byte[length];
int read = is.read(content);
while(read < content.length) {
read += is.read(content, read, content.length - read);
}
as .read(byte[]) doesn't slurp the whole thing at once. I assumed that this continually slurped and blocked while it did so.
There is also this:
socket.setSoTimeout(60);
The arg is in millis rather than seconds so should be:
socket.setSoTimeout(60 * 1000);
I still don't know the cause of the above badly named exception though so hopefully someone will still answer this if they know!

try as flow, use mInputValid to control whether end the flow:
private int fill(byte[] buffer, int offset,int length) throws IOException {
int sum = 0, len;
while ((sum<length) && mInputValid) {
try{
len = is.read(buffer, offset + sum, length - sum);
if (len < 0) {
throw new IOException("End of stream");
} else{
sum += len;
Log.i(TAG, "is.read: " + len + " buffer:" + buffer[0]);
}
}
catch (IOException e){
e.printStackTrace();
Log.i(TAG, "read input fail, try again");
continue;
}
}
return sum;
}

I think the "Try again" IOException should actually be handled in the same way a SocketTimeoutException would be handled. This is a very poorly implemented API, but we are used to such crappy design on Android:
private int read(byte[] buffer) throws IOException {
while (true) {
try {
return fis.read(buffer);
} catch (SocketTimeoutException e) {
continue;
} catch (IOException e) {
String message = e.getMessage();
if (message != null && message.equals("Try again")) {
continue;
}
throw e;
}
}
}
private int readInt() throws IOException {
while (true) {
try {
return fis.readInt();
} catch (SocketTimeoutException e) {
continue;
} catch (IOException e) {
String message = e.getMessage();
if (message != null && message.equals("Try again")) {
continue;
}
throw e;
}
}
}

I know I am late to the party, but I just solved the same issue and there are a number of things that can cause this:
Not calling outputStream.flush() on sending. If you use writers, it's writer.flush(). This sends the last buffer. If you are not using buffers, it doesn't mean they aren't there. Stream send data in bytes, so unless you are sending a single byte, chances are there's some buffer down the line that doesn't get sent, and the receiver gets half an int.
If you are keeping the streams open, there is no way to detect the stream's end: so the Java will thrown an exception if you try to do things like reader.read(buffer), when size of the buffer exceeds the amount of data sent. In this case you must implement some kind of prototocle (header telling the length or some sort of end token, to know when to stop reading).
If you are closing streams after sending message and calling output.flush(), this exception can be caused by what you meantioned: socket timeouts, as well as disconnects.

Need help debugging my code

I need some input about my code.
Basically, I have a method to load music from Class A
public void onListItemClick(ListView parent, View v, int position, long id){
musicIndex = cursor.getColumnIndexOrThrow(MediaStore.Audio.Media.DATA);
cursor.moveToPosition(position);
filePath = cursor.getString(musicIndex);
fileName = new File(filePath).getName();
playMusic();//Play the selected music
}
public void playMusic(){
if(mPlayer.isPlaying()){
mPlayer.reset();
}
try{
mPlayer.setDataSource(filePath);
mPlayer.prepare();
mPlayer.start();
BeatDetection beatDetect = new BeatDetection();
beatDetect.init();
}catch (Exception e){
}
}
That method will call the init() method in Class B
public void init() throws Exception{
energy = 0;
variance = 0;
constant = 0;
isBeat = false;
sensitivity = 0;
dBuffer = new float[sampleRate / bufferSize];
eBuffer = new float[sampleRate / bufferSize];
timer = System.currentTimeMillis();
MusicLoad msc = new MusicLoad();
totalMs = 0;
seeking = true;
//msc.printText();
decode(msc.fileName, 25, 40);
}
In that method, it initializes everything and call the decode() method
public void decode(String path, int startMs, int maxMs)
throws IOException, javazoom.jl.decoder.DecoderException {
debug();
File in = new File(path);
InputStream inStream = new BufferedInputStream(new FileInputStream(in), 8 * 1024);
ByteArrayOutputStream outStream = new ByteArrayOutputStream(1024);
try {
Bitstream bitstream = new Bitstream(inStream);
Decoder decoder = new Decoder();
boolean done = false;
while (! done) {
Header frameHeader = bitstream.readFrame();
if (frameHeader == null) {
done = true;
} else {
totalMs += frameHeader.ms_per_frame();
if (totalMs >= startMs) {
seeking = false;
}
if (! seeking) {
SampleBuffer output = (SampleBuffer) decoder.decodeFrame(frameHeader, bitstream);
if (output.getSampleFrequency() != 44100 || output.getChannelCount() != 2) {
throw new javazoom.jl.decoder.DecoderException("mono or non-44100 MP3 not supported", null);
}
short[] pcm = output.getBuffer();
for (short s : pcm) {
outStream.write(s & 0xff);
outStream.write((s >> 8 ) & 0xff);
}
}
if (totalMs >= (startMs + maxMs)) {
done = true;
}
}
bitstream.closeFrame();
}
byte[] abAudioData = outStream.toByteArray();
calculation(abAudioData);
} catch (BitstreamException e) {
throw new IOException("Bitstream error: " + e);
} catch (DecoderException e) {
Log.w("Decoder error", e);
throw new javazoom.jl.decoder.DecoderException("Error",e);
} finally {
inStream.close();
}
}
Don't mind reading all the code lines. If you guys notice I put debug() in the beginning to see whether the method is called or not. At this point, the debug() is properly called. However, if I put the debug() after the line File in = new File(path);, the debug() will not be called anymore. It seems like the code is stop running at that point.
The ultimate result is, I can load and play the song without any problem. However, the decode() is not called and there is no error whatsoever. I'm stuck at pointing out the problem at this point. So if there's any input please help me.
EDIT: After I tried tracing the "path" variable, it returns NULL so the error is NullPointerException. Seems like the "fileName" variable from Class A is not passed to Class B. Any suggestion?

If you are using Eclipse with ADT then it's very easy to debug your Android apps, just add a breakpoint (probably in the new File(...) line) and see what happens.
My guess here is that File in = new File(path); probably is throwing a IOException in your decode method, that exception is bubbling first to init() and then to playMusic(), where it is caught by try catch block. Your catch is empty so you are not seeing anything. Try debugging as I said or add some logging info in the catch block.

This is just something to look at, but from the doc page
http://developer.android.com/reference/java/io/File.html#File%28java.lang.String%29
"The actual file referenced by a File may or may not exist. It may also, despite the name File, be a directory or other non-regular file."
If you had the path wrong, it may be trying to create the file and you may not have the correct permission to do so. Perhaps: WRITE_EXTERNAL_STORAGE.

I know this post is old, but I just wanted to show how to get the file path to read/write files for others that come across this post as I have:
String filePath = myContext.getFilesDir().getPath().toString() + "/sysout.log";
File file = new File(filePath);
These two lines will create (open if it exists, and overwrite) a file named "sysout.log" in the folder /data/data/com.app.name/files/; myContext is just the current context. Using this technique alleviates problems with defining your own path name. Hope this helps someone.