Develop Reference

Why is Android file creation suddenly failing when it worked on Android 11 previously?

In the last few days, my Android app is suddenly failing to download files from a web server to store in the app. This is the same for all users I have contacted. It was previously working in Android 11, so it's something that has only just changed. It's a (free) niche app for UK glider pilots to process NOTAMS, and has relatively large number of users who I don't want to let down.
The published app uses getExternalFilesDir(null) to return the directory in which to store the downloaded files, with android:requestLegacyExternalStorage set to "true" in the manifest.
I changed getExternalFilesDir(null) to getFilesDir() in Android Studio since that's what I understand should now be used for internal app data files. This returns /data/user/0/(my package name)/files. I'm running the Pixel 2 API 30 emulator for debugging, and the File Explorer shows that /data/data/(my package name)/files directory has been created. Everything I've read on here says that this is what is supposed to happen and it should all work. However no file was created when I attempted the download.
I changed android:requestLegacyExternalStorage to "false", and this time a file was created as expected. However it was empty and the download thread was giving an exception "unexpected end of stream on com.android.okhttp.Address#89599f3f".
This is the relevant code in my DownloadFile class which runs as a separate thread (comments removed for compactness):
public class DownloadFile implements Runnable
{
private String mUrlString;
private String mFileName;
private CountDownLatch mLatch;
public DownloadFile(String urlString, String fileName, CountDownLatch latch)
{
mUrlString = urlString;
mFileName = fileName;
mLatch = latch;
}
public void run()
{
HttpURLConnection urlConnection = null;
// Note for StackOverflow: following is a public static variable defined in the main activity
Spine.mDownloadStatus = false;
try
{
URL url = new URL(mUrlString);
urlConnection = (HttpURLConnection) url.openConnection();
urlConnection.setRequestMethod("GET");
urlConnection.setDoOutput(true);
urlConnection.setUseCaches(false);
urlConnection.connect();
File file = new File(Spine.dataDir, mFileName);
FileOutputStream fileOutput = new FileOutputStream(file);
InputStream inputStream = urlConnection.getInputStream();
byte[] buffer = new byte[1024];
int bufferLength = 0; // used to store a temporary size of the buffer
while ((bufferLength = inputStream.read(buffer)) > 0)
{
fileOutput.write(buffer, 0, bufferLength);
}
fileOutput.close();
Spine.mDownloadStatus = true;
}
// catch some possible errors...
catch (IOException e)
{
Spine.mErrorString = e.getMessage();
}
if (urlConnection != null)
urlConnection.disconnect();
// Signal completion
mLatch.countDown();
}
}
I now believe the problem lies with the URL connection, rather than the changes to local file storage access which is what I first thought. Incidentally, if I enter the full URL into my web browser the complete text file is displayed OK, so it's not a problem with the server.

The problem has been narrowed down to changes to the functionality of the website that hosts the data files to be downloaded. It's been made https secure and they are currently working on further changes.
I temporarily moved the hosting to my own website in Android Studio and everything worked so it's down to those website changes and nothing to do with my code (at least it may need changing later to support the upgrade to the main hosting site).
Thanks to all for responding.

complete event trigger before full download

In my app, I need to download few files form server. I used the following code:
function downloadFile(index:int):void
{
var urlLoader:URLLoader = new URLLoader();
urlLoader.dataFormat = URLLoaderDataFormat.BINARY;
urlLoader.addEventListener(Event.COMPLETE, onLoadFileComplete);
urlLoader.addEventListener(ProgressEvent.PROGRESS, onLoadFileProgress);
urlLoader.addEventListener(IOErrorEvent.IO_ERROR, onURLIOError);
urlLoader.load(new URLRequest("some url"));
fileNameToSave = "some name";
trace("file name:" + fileNameToSave);
downloading = true;
}
function onLoadFileProgress(e:ProgressEvent):void
{
var loadedPct:uint = Math.round(100 * (e.bytesLoaded / e.bytesTotal));
}
function onURLIOError(e:IOErrorEvent):void
{
trace("error msg");
e.target.removeEventListener(IOErrorEvent.IO_ERROR, onURLIOError);
}
function onLoadFileComplete(e:Event):void
{
trace("File downloaded");
var file:File;
file = File.documentsDirectory.resolvePath("somelocation/" + fileNameToSave);
if(file != null)
{
var fileStream:FileStream = new FileStream();
fileStream.openAsync(file, FileMode.WRITE);
fileStream.addEventListener(OutputProgressEvent.OUTPUT_PROGRESS, outputProgressHandler);
fileStream.addEventListener(Event.CLOSE, onSaveFile);
fileStream.writeBytes(e.target.data);
fileStream.close();
}
e.currentTarget.removeEventListener(Event.COMPLETE, onLoadFileComplete);
e.currentTarget.removeEventListener(ProgressEvent.PROGRESS, onLoadFileProgress);
e.currentTarget.removeEventListener(IOErrorEvent.IO_ERROR, onURLIOError);
}
function outputProgressHandler(e:OutputProgressEvent):void
{
if (e.bytesPending == 0)
{
trace("File is completely written");
}
}
function onSaveFile(e:Event):void
{
trace("Saved Complete");
loadfiles();
e.currentTarget.removeEventListener(Event.CLOSE, onSaveFile);
}
It works fine. But the problem I'm having is when the internet is slow, sometime it triggers complete event even if the file is not fully downloaded. Is there anyway to prevent this? Any help would be appreciated.
UPDATE: Is it a good practice to use progress event to check if the download is complete rather than using complete event. Like:
if(e.bytesLoaded >= e.bytesTotal)
{
//downloadComplete
}
else
{
//not complete
}

I can not comment because I have insufficient reputation.
I have previously had similar issue where filestream write reports as complete when in fact it has not. This occurs only on older devices and occurs randomly. My testing seemed to confirm that Flash reports file saved before the OS had actually completed. When a read then occurred it would report as file not found.
I solved it rather crudely using setTimeout and repeatedly checked if the data was complete.

In Unity, how can I populate this allRoundData array from a json file in a way that works on Android?

I am new to Unity and Android development, but please do not mark this as a duplicate - I've looked all over the stacks for an answer to this, but similar topics and questions haven't yielded any working solutions, or are wanting on details, or outdated, or seem not to fit my specific need.
Ok, I have a quiz app built from following part 1 & 2 of this tutorial. That link contains all the source files anyone might need for reference, and everything works fine on ios and in the unity editor, fyi.
The trouble is with the loadGameData function from the DataController script in Android. Again, everything works fine in iOS and the unity editor, but when I make an Android sdk, the quiz is blank and the console says the data couldn't be loaded.
Here is how the function is currently written (full file in tutorial link):
private void LoadGameData ()
{
string filePath = Path.Combine (Application.streamingAssetsPath, gameDataFileName);
if (File.Exists (result))
{
string dataAsJson = File.ReadAllText (result);
GameData loadedData = JsonUtility.FromJson<GameData> (dataAsJson);
allRoundData = loadedData.allRoundData;
} // #if(File.Exists...
else
{
Debug.LogError ("Cannot load game data!");
} // #else
} // #LoadGameData
If you check the same tutorial on youtube, you'll see lots of people have noted the same problem with the Android build and have been left unanswered. Same goes with unity forums - that's one reason why I don't think this question is a duplicate and could be helpful to others in a similar situation.
I've found that Android has always been sorta tricky with this and that there used to different ways of accessing a file based on platform, but these days "Application.streamingAssetsPath" should find the streaming assets directory on any platform, even Android.
What I've also learned is that in android, even if the path is correct, the file is compressed and will only return a url. So the url needs to be converted using unity's WWW class. I get that, but as of yet, I haven't been able to re-write my loadGameData function to work properly and load the allRoundData array.
Here's an example of some things I've tried:
IEnumerator androidData()
{
string filePath = Path.Combine (Application.streamingAssetsPath, gameDataFileName);
if (filePath.Contains("://"))
{
WWW www = new WWW(filePath);
yield return www;
result = www.text;
} // #if(filePath.Contains
} // #androidData
private void LoadGameData ()
{
androidData();
if (File.Exists (result))
{
string dataAsJson = File.ReadAllText (result);
GameData loadedData = JsonUtility.FromJson<GameData> (dataAsJson);
allRoundData = loadedData.allRoundData;
} // #if(File.Exists...
else
{
Debug.LogError ("Cannot load game data!");
} // #else
} // #LoadGameData
I know I'm close, and this is probably simple -- but I just can't seem to get to the finish line on this. Can some one help me figure out how to write the loadGameData function so it will load this allRoundData array on android?
An example code would be awesome and much appreciated, not just by me, but I'm sure many others would appreciate it also - Thank you!
UPDATE:
Based on the first answer, I've tested some code that works on the unity editor, but crashes in Android. In the Unity editor, I get the "file already exists" message. Here is the code I've tested:
Already had: private string gameDataFileName = "data.json";
I added the copyFile call above loadGameDate in Start() and wrote the copy file and loadGameData functions like so ..
int copyFileToPersistentDataPath(string gameDataFileName)
{
string persistentPath = Application.persistentDataPath + "/" + gameDataFileName;
try
{
//Copy only if gameDataFileName does not exist
if (!System.IO.File.Exists(persistentPath))
{
string path = System.IO.Path.Combine(Application.streamingAssetsPath, gameDataFileName);
WWW www = new WWW(path);
while (!www.isDone) { }
System.IO.File.WriteAllBytes(persistentPath, www.bytes);
Debug.Log(gameDataFileName + " Successfully Copied File to " + persistentPath);
return 1;
}
else
{
Debug.Log(gameDataFileName + " File already exist here. There is no need to copy it again");
return 0;
}
}
catch (Exception e)
{
Debug.Log(gameDataFileName + " Failed To Copy File. Reason: " + e.Message);
return -1;
}
}
private void LoadGameData ()
{
string tempPath = Path.Combine(Application.persistentDataPath, gameDataFileName);
string dataAsJson = File.ReadAllText(tempPath);
GameData loadedData = JsonUtility.FromJson<GameData>(dataAsJson);
allRoundData = loadedData.allRoundData;
} // #LoadGameData
This works with or without the call to copy the file in the editor, but crashes either way in Android.

I ended up putting the files in a Resources folder and going the resources.load a json file into a text asset and throw that into a string to parse route. I now have two different load functions, one that works in ios etc. and one that works in android. Here is the android function (the resourcesGameDataFile does not have the .json extension):
public void LoadDataForAndroid()
{
TextAsset dataFile = Resources.Load(resourcesGameDataFile) as TextAsset;
string dataAsJson = dataFile.ToString();
GameData loadedData = JsonUtility.FromJson<GameData>(dataAsJson);
allRoundData = loadedData.allRoundData;
Debug.Log ("Android data loaded with" + resourcesGameDataFile);
} // #LoadDataForAndroid
And this works in the unity editor and in Bluestacks (android simulator).

As for loading and saving games, this is a duplicate. I marked and remove this as a duplicate because the answer in the duplicated questions did not explain how to read from the StreamingAssets folder. It only talked about saving and loading data.
Make sure that Write Permission is set to External (SDCard).
The first thing to do is to copy the file from StreamingAssets to the persistentDataPath location.
I've found reading data from Application.streamingAssetsPath problematic but I use two methods to solve this.
int copyFileToPersistentDataPath(string fileNameWithExtensionName)
{
string persistentPath = Application.persistentDataPath + "/" + fileNameWithExtensionName;
try
{
//Copy only if fileNameWithExtensionName does not exist
if (!System.IO.File.Exists(persistentPath))
{
string path = System.IO.Path.Combine(Application.streamingAssetsPath, fileNameWithExtensionName);
WWW www = new WWW(path);
while (!www.isDone) { }
System.IO.File.WriteAllBytes(persistentPath, www.bytes);
Debug.Log(fileNameWithExtensionName + " Successfully Copied File to " + persistentPath);
return 1;
}
else
{
Debug.Log(fileNameWithExtensionName + " File already exist here. There is no need to copy it again");
return 0;
}
}
catch (Exception e)
{
Debug.Log(fileNameWithExtensionName + " Failed To Copy File. Reason: " + e.Message);
return -1;
}
}
If that does not work for you, use the method with WebClient below:
void copyFileToPersistentDataPath(string fileNameWithExtensionName)
{
string path = System.IO.Path.Combine(Application.streamingAssetsPath, fileNameWithExtensionName);
string persistentPath = Application.persistentDataPath + "/" + fileNameWithExtensionName;
Debug.Log("Dir: " + persistentPath);
WebClient webClient = new WebClient();
webClient.Proxy = null;
webClient.DownloadFileCompleted += new AsyncCompletedEventHandler(OnDownloadComplete);
webClient.DownloadProgressChanged += new DownloadProgressChangedEventHandler(OnUpdateDownloadProgress);
Uri uri = new Uri(path);
webClient.DownloadFileAsync(uri, persistentPath);
}
void OnDownloadComplete(object sender, AsyncCompletedEventArgs e)
{
Debug.Log("Finished Downloading: " + e.Error.Message);
}
void OnUpdateDownloadProgress(object sender, DownloadProgressChangedEventArgs e)
{
Debug.Log("Uploading Progreess: " + e.ProgressPercentage);
}
File Copy Usage:
copyFileToPersistentDataPath("yourFileName.txt");
After copying the file, you can then read and convert it to Json like this:
string fileNameWithExtensionName = "questionfile.txt";
string tempPath = Path.Combine(Application.persistentDataPath, fileNameWithExtensionName);
string dataAsJson = File.ReadAllText(fileNameWithExtensionName);
GameData loadedData = JsonUtility.FromJson<GameData>(dataAsJson);

Can't read lines from file by using StreamReader on Unity3d (Android)

I need to read a text stream by using StreamReader from file on android platform. File is about 100k lines, so even editor is getting stuck if i try to load it all to TextAsset or if i use WWW.
I simply need to read that file line by line without loading it all to a string. Then i'll do a tree generation from the lines that i got from the file. (But probably that part doesn't matter, i just need help on file reading part.)
I'm giving the code that i wrote down below. It works perfectly on editor, but fails on android.
I would be glad if anyone tell me, what am i missing.
(ps. english is not my native and this is my first question on the site. so sorry for the any mistakes that i may have done.)
private bool Load(string fileName)
{
try
{
string line;
string path = Application.streamingAssetsPath +"/";
StreamReader theReader = new StreamReader(path + fileName +".txt", Encoding.UTF8);
using (theReader)
{
{
line = theReader.ReadLine();
linesRead++;
if (line != null)
{
tree.AddWord(line);
}
}
while (line != null);
theReader.Close();
return true;
}
}
catch (IOException e)
{
Debug.Log("{0}\n" + e.Message);
exception = e.Message;
return false;
}
}

You can't use Application.streamingAssetsPath as a path on Android because streaming assets are stored within the JAR file with the application.
From http://docs.unity3d.com/Manual/StreamingAssets.html:
Note that on Android, the files are contained within a compressed .jar
file (which is essentially the same format as standard zip-compressed
files). This means that if you do not use Unity’s WWW class to
retrieve the file then you will need to use additional software to see
inside the .jar archive and obtain the file.
Use WWW like this in a coroutine:
WWW data = new WWW(Application.streamingAssetsPath + "/" + fileName);
yield return data;
if(string.IsNullOrEmpty(data.error))
{
content = data.text;
}
Or, if you really want to keep it simple (and your file is only a few 100k, stick it in a resource folder:
TextAsset txt = (TextAsset)Resources.Load(fileName, typeof(TextAsset));
string content = txt.text;

How to get the size of the data received via EXTRA_STREAM in an application handling the action send intent?

When another application is sending a file to my app, I get a Uri via the intent.getExtras().get(EXTRA_STREAM) property. I can then get the bytes of the file using an inputstream : new BufferedInputStream(activity.getContentResolver().openInputStream(uri));
Everything's OK and working so far. Now I'd like to show some kind of progress to my user, but I'm not sure of how to get the total number of bytes of the file without reading the stream completely beforehand (which would defeat the whole purpose of the progress bar) ...
I tried ParcelFileDescriptor fileDesc = activity.getContentResolver().openFileDescriptor(uri, "r"); but this only works with uris of type file://....
For example If I receive a file from Skydrive I get a content://.. Uri, as in : content://com.microsoft.skydrive.content.external/external_property/10C32CC94ECB90C4!155/Sunset.480p.mp4
On such Uri I get (unsurprisingly) a "FileNotFoundException : Not a whole file" exception.
Any sure fire way to get the total size of the stream of data I will get ?

Even though InputStream.available() is (almost) never a recommended way of getting file size, it might be a viable solution in your case.
The content is already available locally. A server is not involved. So, the following should return the exact file size:
try {
InputStream inputStream = getContentResolver().openInputStream(uri);
Log.i("TEST", "File Size: " + inputStream.available());
} catch (FileNotFoundException fnfe) {
fnfe.printStackTrace();
} catch (IOException ioe) {
ioe.printStackTrace();
}
I tested this with SkyDrive and Dropbox. The file sizes returned were correct.

There is no general solution for getting the size of a stream, other than reading the entire stream. This is easily proven: One could create a web server that, for some URL, generates a random stream of text that is terminated at a random time. (In fact, I'm sure such URLs exist, whether by design or not :-) In such a case, the size of the stream isn't known until the last byte has been generated, never mind received.
So, the stream size, if it is sent by the server at all, has to be sent in an application-specific manner.
I've never worked with SkyDrive, but a google search for its API turned up this link, which has the following example for Android Java apps:
public void readFile() {
String fileId = "file.a6b2a7e8f2515e5e.A6B2A7E8F2515E5E!141";
client.getAsync(fileId, new LiveOperationListener() {
public void onError(LiveOperationException exception, LiveOperation operation) {
resultTextView.setText("Error reading file: " + exception.getMessage());
}
public void onComplete(LiveOperation operation) {
JSONObject result = operation.getResult();
String text = "File info:" +
"\nID = " + result.optString("id") +
"\nName = " + result.optString("name");
resultTextView.setText(text);
}
});
}
Based on other examples on that page, I would guess that something like result.optString("size") (or maybe result.optInt("size") ?) would give you the size of the file.