My android program crashes on this line when the file size is very large. Is there any way I can prevent the program from crashing ?
byte[] myByteArray = new byte[(int)mFile.length()];
Additional details :-
I am trying to send a file to server.
error log-
E/dalvikvm-heap(29811): Out of memory on a 136309996-byte allocation.
You should use a stream when reading the file. Since you've mentioned sending to a server, you should stream that file to the server.
As others have mentioned, you should consider your data size (1GB seems excessive). I haven't tested this, but the basic approach in code would look something like:
// open a stream to the file
FileInputStream fileInputStream = new FileInputStream(filePath);
// open a stream to the server
HttpURLConnection connection = new URL(url).openConnection();
DataOutputStream outputStream = new DataOutputStream(connection.getOutputStream());
byte[] buffer = new byte[BUFFER_SIZE]; // pick some buffer size
int bytesRead = 0;
// continually read from the file into the buffer and immediately write that to output stream
while ((bytesRead = fileInputStream.read(buffer)) != -1) {
outputStream.write(buffer);
}
Hope that is clear enough for you to fit to your needs.
Yep. Don't try to read the whole file into memory at once...
If you really need the whole file in memory you might have more luck with allocating dynamic memory for each line and storing the lines in a list. (you might be able to get a bunch of smaller chunks of memory but not one big piece)
Without knowing the context we can't tell, but normally you would parse the file into data structs rather than just storing the whole file in memory.
In JDK 7 you can use Files.readAllBytes(Path).
Example:
import java.nio.file.Files;
import java.nio.file.Paths;
import java.nio.file.Path;
Path path = Paths.get("path/to/file");
byte[] myByteArray = Files.readAllBytes(path);
Don't try reading the complete file into memory. Instead open a stream and process the file line by line (is it's a text file) or in parts. How that has to be done depends on the problem you are trying to solve.
EDIT: You say you want to upload a file, so please check this question. You don't need to have the complete file in memory.
Related
I wrote an android application that part of it is to handle upload and download documents. Currently I am using the Microsoft Azure server to save the files on.
The way I am currently doing it is by turning the files to a string and saving it that way on the Azure server:
ByteArrayOutputStream baos = new ByteArrayOutputStream();
FileInputStream fis;
try {
fis = new FileInputStream(new File(Uridata.getPath()));
byte[] buf = new byte[1024];
int n;
while (-1 != (n = fis.read(buf)))
baos.write(buf, 0, n);
} catch (Exception e) {
e.printStackTrace();
}
byte[] bbytes = baos.toByteArray();
item.setStringFile(Base64.encodeToString(bbytes, Base64.URL_SAFE));
item.setName(Uridata.getLastPathSegment());
where item is my class that saves the string representation and the name of the file and is being loaded to the Azure, Uridata is an Uri instance of the file chosen.
I have one main problem with this solution and it is the limit on the file size.
I am searching for a good server to use instead of the Azure (maybe a RESET one) and if there is a better way to save files of all kinds (pdf, word...).
I will also want in the future to use the same data in a web interface
Does anybody have any suggestions on how to do it?
Thanks in advance!
To start, you don't have to transform the file into a string, you can just save it as a file. You have the possibility of losing data by continuing to do that. See: How do I save a stream to a file in C#?
If you're looking for another service to save files, then you should look into Azure Blob Storage. It will allow you to upload as much data as you want to a storage service for arbitrary files. See for example:
https://azure.microsoft.com/en-us/documentation/articles/storage-dotnet-how-to-use-blobs/
I am trying to create internal storage folder(s) and copy over my RAW files to those folder. I have following method
EDIT
private byte[] buffer = null;
private String DIR_NAME = "images/sample_images";
public void storeRAWFilesToInternalStorage()
{
buffer = new byte[3000000];
mFilesDir = tempContext.getFilesDir();
mImagesDir = new File(mFilesDir, DIR_NAME);
if (!mImagesDir.exists() && !mImagesDir.isDirectory()) dirCreated = mImagesDir.mkdirs();
InputStream fileStream = tempContext.getResources().openRawResource(R.raw.desert);
fileStream.read(buffer);
fileWithinMyDir = new File(mImagesDir, "my_sample_image.jpg");
FileOutputStream outputStream = new FileOutputStream(fileWithinMyDir);
outputStream.write(buffer);
outputStream.close();
}
it works fine, however I have following questions.
What is the buffer size should I assign since I dont know the bytes size of each file. If I assign more than required then I am wasting memory. If I am assigning less than required then image wont be saved properly.
Also, I see the image saved in my Internal Storage (using ES Fileexplorer to view files), but when I try to open it, it doesnt show up. Weird.
END EDIT
Also what if I have 100 RAW files which I would like to copy to my Internal Folder, say "images". I dont want to type following thousand of times.
InputStream fileStream = tempContext.getResources().openRawResource(R.raw.**asset_name**);
Is there a way to loop through all raw resources, read them and store them to Internal Storage folder? Can I see some code snippet?
I am reading a file with:
char [] buffer = new char[300];
FileInputStream istream = new FileInputStream(path);
InputStreamReader file = new InputStreamReader(istream);
size = file.read(buffer);
file.close();
After a few tries, it turns out that the file.read(buffer) reads exactly the number of chars allocated for buffer (in this case, 300, even that the file has much more characers in it).
Can I rely on read() always reading as much as it can, without generating any exception?
Or is this an undocumented feature?
The read method description says:
Reads characters from this reader and stores them in the character
array buf starting at offset 0. Returns the number of characters
actually read or -1 if the end of the reader has been reached.
There is no mention of the buffer allocation issue.
This is very important, and a good thing that it works this way, because it allows you to define the size of the buffer as you want/need and there is no need to guess, no need to code for exceptions. Actually, it is read(char[] buffer) but it works as read(char[] buffer, int size).
Yes you can rely on this call, unless an I/O error occurs, which is already mentionned in the api.
If you look at the code of read(char cbuf[]) you'll notice it calls the method public int read (char[] buffer, int offset, int length).
From Android source code:
public int read(char cbuf[]) throws IOException { read(cbuf, 0, cbuf.length);}
In your implementation, you need to continue reading the file with file.read(buffer) to obtain remaining bytes. The content of buffer needs to be appended to another buffer that will grow, depending on the size of the file you're reading.
You could also allocate that buffer with the size of the file with the method getTotalSpace()
I am finding that reading one line at a time from a text file on the SD card is rather slow. I imagine that it might be quicker if the file is in internal memory, so I want to copy files from the SD card to internal storage.
The file copy examples I can find on the web seem to involve copying one byte at a time from an InputStream to an OutputStream or from a FileReader to a FileWriter. Is this really the quickest and most efficient method?
If you are pulling the file in for use in your application what I suggest you do is read in the data then stuff the in memory data you have collected into some kind of reader (BufferedReader perhaps) so that you can then read the lines from there.
Here is an example of what I typically do:
// Assumption: I already have the file object I want to read
// Note: I'm not doing any error handling.
InputStream input = new FileInputStream(file);
ByteArrayOutputStream baos = new ByteArrayOutputStream();
byte[] buffer = new byte[1024];
int bytesRead = 0;
while( (bytesRead = input.read(buffer)) > 0){
baos.write(buffer, 0, bytesRead);
}
StringReader stringReader = new StringReader( new String(baos.toByteArray()) );
BufferedReader bufferedReader = new BufferedReader( stringReader );
for(String line : bufferedReader.readLine()){
// TODO: Handle each line appropriately or something
Log.d("Reading Data Example", line);
}
One of the truisims of CS that only becomes more true with time as CPUs get faster is: I/O is slow.
If you want speed, generally your best bet is to do as few I/O's as possible. Ideally, find out how big that file is, allocate that much memory, and then read the entire thing in one big I/O. Then you can just access the data from program memory. If you might not have enough RAM for every concievable file size then you might have to do a bit more work, but this is what you should strive for.
I'm writing a simple budget app for myself, and I'm having trouble figuring out how to write to internal storage. I don't seem to be writing to the file properly, and I can't find any more in depth examples than the Data Storage article on developer.android.com
Basically, I'm trying to write a test float to the MyBalance file, then read it into balance. In my actual code I use try/catch statements around the file in/out operations, but I skipped them to make the code more readable.
float test = 55;
float balance;
byte[] buffer = null;
FileOutputStream fos = openFileOutput( "MyBalance", Context.MODE_PRIVATE );
fos.write(Float.floatToRawIntBits(balance));
fis.read(buffer); //null pointer
ByteBuffer b = ByteBuffer.wrap(buffer);
balance=b.getFloat();
That's the gist of it, anyone see what I'm doing wrong?
Edit:
Thanks for the reply, I went ahead and converted to/from String like you suggested, but I still don't think the file is being created. I have an if statement that reads from it if it exists in onResume() and it isn't being run. Lemme post some of my code.
Here's how I'm writing the file, (setbal is an EditText and balanceview is a TextView):
balance = Float.valueOf(setbal.getText().toString());
balanceview.setText(setbal.getText());
balstring = String.valueOf(balance);
for (int i = 0; i < balstring.length(); ++i)
try {
fos.write((byte)balstring.charAt(i));
} catch (IOException e) {
e.printStackTrace();
}
I check if the file exists in onResume() like so:
File file = new File("data/data/com.v1nsai.mibudget/balance.txt");
Is that where an internal file for that context would be stored?
So this isn't exactly what you asked for, but this is how I have it working for Strings, and it may be helpful to you to see. (You could box the primatives and toString them of course if you wanted to use this code.)
Writing
FileOutputStream fos = context.openFileOutput("savedstate.txt", 0);
for (int i = 0; i < out.length(); ++i)
fos.write((byte)out.charAt(i));
Reading
StringBuilder inb = new StringBuilder();
FileInputStream fis = this.mContext.openFileInput("savedstate.txt");
int ch;
while((ch = fis.read()) != -1)
inb.append((char)ch);
Update
One thought that springs to mind is that you may not want to trust using a File object with a hand typed full path to the file. Instead, just use the FileInputStream with the context object and a relative path like in my code, then see if you get a String back of some length or something like that, or an exception that the file doesn't exist.
If you are really curious of where the file is created, or want to see it with your own eyes, I believe you can browse to it on your phone through the file manager in DDMS.
One last thing, I would suggest moving the try/catch block outside of your writing loop. Since it is an identical task being repeated, there is no need for the overhead of that approach, though it is typically good practice to minimize the size of your try/catch blocks.
Ok really one last thing, if you want to use the File object with the full path, you might want to have the path be the following:
File file = new File("/data/data/com.v1nsai.mibudget/balance.txt");
The beginning slash may make all the difference.