I am using this segment of copy a large file. Android crashes with "out of memory" at exactly 32 buffer loads. It is acting like dos.write is putting the data into a large buffer rather than spooling it out to the i/o device. No exception is thrown.
The bufferSize = 512*1024. bis is a BufferedInputStream. byteArray is a ByteArrayBuffer,
try {
FileOutputStream fos = new FileOutputStream(file);
dos = new DataOutputStream(fos);
int current = 0;
while((current = bis.read()) != -1){
byteArray.append((byte)current);
if (byteArray.isFull()){
byte[] b = byteArray.toByteArray();
dos.write(b, 0, bufferSize);
byteArray.clear();
}
}
int count = byteArray.length();
byte[] b = byteArray.toByteArray();
dos.write(b, 0, count);
dos.flush();
dos.close();
bis.close();
}
catch (Exception e) {
RunTimeError("Exception: " + e);
return false;
}
My guess is that byteArray.isFull() is always returning false for some reason. Then when you have loaded 16MB of data, you're out of memory. I wouldn't bother with a ByteArrayBuffer. (For that matter, 512KB is way too large a buffer for this kind of operation. You should try to match the file I/O block size. It probably varies by device, but 4K-8K is probably close.) You also don't need to wrap fos in a DataOutputStream; you're just writing bytes. A BufferedOutputStream, on the other hand, might be useful. And if bis is not buffered, wrapping it in a BufferedInputStream will also help.
I would rewrite your code like this:
BufferedOutputStream bos = null;
try {
bos = new BufferedOutputStream(new FileOutputStream(file), 8192);
byte[] buffer = new byte[1024];
int len = 0;
while((len = bis.read(buffer)) != -1) {
bos.write(buffer, 0, len);
}
} catch (Exception e) {
RunTimeError("Exception: " + e);
return false;
} finally {
try { bis.close(); } catch (Exception ignored) { }
try { bos.close(); } catch (Exception ignored) { }
}
Related
I am facing this issue in uploading the file to the google drive, i am uploading the recorded audio to the google drive at that time this exception is occurring
The code used for writing the content in the file
OutputStream outputStream = result.getDriveContents().getOutputStream();
FileInputStream fis;
try {
fis = new FileInputStream(file);
ByteArrayOutputStream baos = new ByteArrayOutputStream();
byte[] buf = new byte[1024];
int n;
while (-1 != (n = fis.read(buf)))
baos.write(buf, 0, n);
byte[] photoBytes = baos.toByteArray();
outputStream.write(photoBytes);
outputStream.close();
outputStream = null;
fis.close();
fis = null;
Log.e("File Size", "Size " + file.length());
} catch (FileNotFoundException e) {
Log.v("EXCEPTION", "FileNotFoundException: " + e.getMessage());
} catch (IOException e1) {
Log.v("EXCEPTION", "Unable to write file contents." + e1.getMessage());
}
The exception occurs in the line ` baos.write(buf, 0, n);
Please help me how to solve this error.`
Writing to a ByteArrayOutputStream first means that the complete file will end up in the JVM's heap. Depending on the file size and heap size this might not be possible, hence the exception. If you don't need the ByteArrayOutputStream for anything else just write directly to outputStream:
OutputStream outputStream = result.getDriveContents().getOutputStream();
FileInputStream fis;
try {
fis = new FileInputStream(file);
byte[] buf = new byte[1024];
int n;
while (-1 != (n = fis.read(buf)))
outputStream.write(buf, 0, n);
} catch (FileNotFoundException e) {
Log.v("EXCEPTION", "FileNotFoundException: " + e.getMessage());
} catch (IOException e1) {
Log.v("EXCEPTION", "Unable to write file contents." + e1.getMessage());
} finally {
outputStream.close();
fis.close();
Log.e("File Size", "Size " + file.length());
}
P.S.: nulling the references should not be necessary if they go out of scope soon...
You are getting OOM because you try to read full file into the memory before writing it to the google drive outputStream. The file may be too large to be stored in the memory. This way you need to write it part by part. It is easy to accomplish using this method:
private static final int BUFFER_SIZE = 1024;
public static long copy(InputStream from, OutputStream to)
throws IOException {
byte[] buffer = new byte[BUFFER_SIZE];
long total = 0;
while (true) {
int r = from.read(buffer);
if (r == -1) {
break;
}
to.write(buffer, 0, r);
total += r;
}
return total;
}
The method will return number of bytes copied.
I have tried to use code below in my application to encode a jpg file to Base64.
InputStream inputStream = null;
try {
inputStream = new FileInputStream(imagelocation);
} catch (FileNotFoundException e) {
e.printStackTrace();
}
byte[] bytes;
byte[] buffer = new byte[8192];
int bytesRead;
ByteArrayOutputStream output = new ByteArrayOutputStream();
try {
while ((bytesRead = inputStream.read(buffer)) != -1) {
output.write(buffer, 0, bytesRead);
}
} catch (IOException e) {
e.printStackTrace();
}
bytes = output.toByteArray();
String encodedString = Base64.encodeToString(bytes, Base64.DEFAULT);
Log.d(TAG, encodedString);
Output is following:
/9j/4Tj5RXhpZgAASUkqAAgAAAAPAA4BAgAgAAAAwgAAAA8BAgAgAAAA4gAAABABAgAgAAAAAgEA
ABIBAwABAAAAAQAAABoBBQABAAAAIgEAABsBBQABAAAAKgEAACgBAwABAAAAAgAAADEBAgAgAAAA
MgEAADIBAgAUAAAAUgEAABMCAwABAAAAAgAAACACAwABAAAAAAAAACECBAABAAAAAAAAACICAwAB
AAAAAAAAACMCAwABAAAAAAAAAGmHBAABAAAAZgEAAMQCAAAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgACAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAAICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgIABIAAAAAQAAAEgAAAABAAAATWVkaWFUZWsgQ2FtZXJhIEFwcGxpY2F0
aW9uAAAAAAAyMDE2OjAzOjMxIDIyOjMyOjEzABYAmoIFAAEAAAB0AgAAnYIFAAEAAAB8AgAAIogD
AAEAAAAAAAAAJ4gDAAEAAABQAQAAAJAHAAQAAAAwMjIwA5ACABQAAACEAgAABJACABQAAACYAgAA
AZEHAAQAAAABAgMABJIKAAEAAACsAgAAB5IDAAEAAAACAAAACJIDAAEAAAD/AAAACZIDAAEAAAAA
AAAACpIFAAEAAAC0AgAAAKAHAAQAAAAwMTAwAaADAAEAAAABAAAAAqAEAAEAAAAACgAAA6AEAAEA
AACABwAABaAEAAEAAAA6AwAAAqQDAAEAAAAAAAAAA6QDAAEAAAAAAAAABKQFAAEAAAC8AgAABqQD
AAEAAAAAAAAAAAAAAARxAgBAQg8AHAAAAAoAAAAyMDE2OjAzOjMxIDIyOjMyOjEzADIwMTY6MDM6
MzEgMjI6MzI6MTMAAAAAAAoAAABeAQAAZAAAAGQAAABkAAAACAADAQMAAQAAAAYAAAASAQMAAQAA
AAEAAAAaAQUAAQAAACoDAAAbAQUAAQAAADIDAAAoAQMAAQAAAAIAAAABAgQAAQAAAHEEAAACAgQA
AQAAAIA0AAATAgMAAQAAAAIAAAAAAAAASAAAAAEAAABIAAAAAQAAAAIAAQACAAQAAABSOTgAAgAH
AAQAAAAwMTAwAAAAAPP/8//0//T/8v/y/+z/7P/u/+7/9f/1/+r/6v/T/9P/wP/A/7z/vP/E/8T/
zf/N/8//z//Q/9D/1v/W/9r/2v/Y/9j/3P/c/9//3//R/9H/wv/C/7z/vP+5/7n/vv++/83/zf/R
/9H/zf/N/9T/1P/d/93/3v/e/9//3//g/+D/1f/V/7f/t/+c/5z/mv+a/57/nv+Y/5j/mv+a/6X/
pf+m/6b/qP+o/7z/vP/X/9f/5f/l/+3/7f/+//7/CAAIAP7//v/w//D/5P/k/8v/y/+4/7j/uP+4
/6z/rP+Y/5j/of+h/6z/rP+S/5L/gv+C/5n/mf+s/6z/uP+4/8H/wf+y/7L/of+h/6H/of+S/5L/
ev96/4P/g/+U/9j/4AAQSkZJRgABAQAAAQABAAD/2wBDAAIBAQEBAQIBAQECAgICAgQDAgICAgUE
BAMEBgUGBgYFBgYGBwkIBgcJBwYGCAsICQoKCgoKBggLDAsKDAkKCgr/2wBDAQICAgICAgUDAwUK
BwYHCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgr/wAAR
CACAAKADASEAAhEBAxEB/8QAHwAAAQUBAQEBAQEAAAAAAAAAAAECAwQFBgcICQoL/8QAtRAAAgED
AwIEAwUFBAQAAAF9AQIDAAQRBRIhMUEGE1FhByJxFDKBkaEII0KxwRVS0fAkM2JyggkKFhcYGRol
JicoKSo0NTY3ODk6Q0RFRkdISUpTVFVWV1hZWmNkZWZnaGlqc3R1dnd4eXqDhIWGh4iJipKTlJWW
l5iZmqKjpKWmp6ipqrKztLW2t7i5usLDxMXGx8jJytLT1NXW19jZ2uHi4+Tl5ufo6erx8vP09fb3
+Pn6/8QAHwEAAwEBAQEBAQEBAQAAAAAAAAECAwQFBgcICQoL/8QAtREAAgECBAQDBAcFBAQAAQJ3
AAECAxEEBSExBhJBUQdhcRMiMoEIFEKRobHBCSMzUvAVYnLRChYkNOEl8RcYGRomJygpKjU2Nzg5
OkNERUZHSElKU1RVVldYWVpjZGVmZ2hpanN0dXZ3eHl6goOEhYaHiImKkpOUlZaXmJmaoqOkpaan
qKmqsrO0tba3uLm6wsPExcbHyMnK0tPU1dbX2Nna4uPk5ebn6Onq8vP09fb3+Pn6/9oADAMBAAIR
AxEAPwD9iG+FPwttGZT8MpITk/NPrUqj7nmAs3lkACPdk8gOAg3EnGSunaNZXcsfhqJ7SxaUmKFn
3N9ST1J/+t2rwc0lCFFRirN/1+Z9Vw7RnUxMqstor8X/AMC/3iNCsjljMxxxuJFSR28QU4LH/gXU
183KKbPtuZ2sNEUDH5w2M4+93qSyNol7FuSQqJV3fN2yM1w1+xd5M/lP/b00zw78Jv22Pir8P7zR
vtU9p48vHitry4YR4MxOCMjapBHQE5z04r0P9n/9pjxf4p1O8j8QeHvAugxWUZDzeLPF2o28bRhS
zOTiXKpgZbHBIyO9fd4OhGeXUn/djt6I/PM2xVaGaVkn9qX4tv8AU4r9rj4n+HdY1K001tS+F/iK
2u8y/wDFFeKPt7BxjmUyWsbofQAkHmvC9U1bw1ctIX8CWjAy7twTIXsMHYP1r0oQly/5ni+0uzO1
jxcJVlOj6U9haxgZgtnUKme+GG5skE965rU9Zi1G48+c3ErhQo3bOQPXAHFXCFteoVKrnK40Xdla
3G+Oe6CtySFxnntirL6zZBsSz6gxHGWbkVTgpaiVapFWT0EGraWzFfOu8k5bzMHnviql3c6dLKpS
7kbPVp0H5fSrUbClVnLc2fAfwt8dfFXUrvTPhf4J1rxHeWNobu+t9D02S6kt4AyoZWVASqbmVc46
sB3rp7/9nn4k+GUWPxV8O/G2nXm91eC78E3ibSjEOQxTnaVIPuppuSj1Dmkb1lpFl4MjNhqfjddO
dXAVda8N31swYqHH316lWDY67WB6EGrK+KtPEjbfFejXyqn+tsXlDA98q4H45rlxGG+sL3v6/wCH
O6hmM6Nla5/X34/8Xw6fO/gzSbhZIrbZHdTRzOY32Z8uJQzH5Yw2Cc/M2Selc7ba0SgfyTnPOWFf
M4+t7XEO2y0R91kmBeHy+N95av57fhYn/taFh5bQsPckU19ZtowxWNm9MmvKnax7SoSbsiJ9eXaG
ETdepIql488Z6b8Mfhlr3xm8cpcaf4Z8M6TcanrWqywEJBbQoXdhnG44BwoOWJAHWvNnSq4ifLTT
b8grKlhoc9WSS8z+dD9tux+Ff/BRr9sfxj+1H4e+HmqeFNP1NbdzpsVws8srIChu7jahEbSHBZV+
VcgAkjNdFrn/AAQm8HeK/wBleP8AaL/Z/wDjV4q8VaxN4w0fQT4NPhgW+Wu8efJPcxySC2hRUlJn
kUR5KFmUPgfoeDjVwWDp0pa8qSZ+V5jiYY7HTqxVk3p+Rx5/4IcfHzxDr+veD9U+Et94CXQfC0vi
G51u81631S3uIEuba1W2SS3GyOV3uVZVdwzLG+AcHHh3xL/4Jy6/8O7hre4+I87gAhwbIk5HU8Ng
iun6171rXOJw0uea6z+zPNpn7xvHhlV32xO1kRuHGercdf5diDXOXfwa1K3n8pPEUTgtjd5DA/kT
/wDq71rGqpENMif4TanFIIR4kt1kORtaNwDj39c1Xb4cazGmY/EVs+cDAL8nuM47etUqibFuA+G3
iYweemr2zKTtOZHz3wOlRDwB4rdxHFdwMd2B++Pb04qudN2BnU/CrW/j18F/EMvi/wCFfjmLR9Qe
OOOW5hdHMsaTxzKrK8bKyebDExUgg7QGBBwfYLX/AIKE/wDBSuzht7KL9
Already tried to put this code into several decoders, but cannot convert this code back to a picture. What should be the problem?
I want to convert FileOutputStream to Byte array for passing binary data between two applications. please any one can help?
To convert a file to byte array, ByteArrayOutputStream class is used. This class implements an output stream in which the data is written into a byte array. The buffer automatically grows as data is written to it. The data can be retrieved using toByteArray() and toString().
To convert byte array back to the original file, FileOutputStream class is used. A file output stream is an output stream for writing data to a File or to a FileDescriptor.
The following code has been fully tested.
public static void main(String[] args) throws FileNotFoundException, IOException {
File file = new File("java.pdf");
FileInputStream fis = new FileInputStream(file);
//System.out.println(file.exists() + "!!");
//InputStream in = resource.openStream();
ByteArrayOutputStream bos = new ByteArrayOutputStream();
byte[] buf = new byte[1024];
try {
for (int readNum; (readNum = fis.read(buf)) != -1;) {
bos.write(buf, 0, readNum); //no doubt here is 0
//Writes len bytes from the specified byte array starting at offset off to this byte array output stream.
System.out.println("read " + readNum + " bytes,");
}
} catch (IOException ex) {
Logger.getLogger(genJpeg.class.getName()).log(Level.SEVERE, null, ex);
}
byte[] bytes = bos.toByteArray();
//below is the different part
File someFile = new File("java2.pdf");
FileOutputStream fos = new FileOutputStream(someFile);
fos.write(bytes);
fos.flush();
fos.close();
}
how to write a byte array to a file using a FileOutputStream. The FileOutputStream is an output stream for writing data to a File or to a FileDescriptor.
public static void main(String[] args) {
String s = "input text to be written in output stream";
File file = new File("outputfile.txt");
FileOutputStream fos = null;
try {
fos = new FileOutputStream(file);
// Writes bytes from the specified byte array to this file output stream
fos.write(s.getBytes());
}
catch (FileNotFoundException e) {
System.out.println("File not found" + e);
}
catch (IOException ioe) {
System.out.println("Exception while writing file " + ioe);
}
finally {
// close the streams using close method
try {
if (fos != null) {
fos.close();
}
}
catch (IOException ioe) {
System.out.println("Error while closing stream: " + ioe);
}
}
}
You may use ByteArrayOutputStream like that
private byte[] filetoByteArray(String path) {
byte[] data;
try {
InputStream input = new FileInputStream(path);
int byteReads;
ByteArrayOutputStream output = new ByteArrayOutputStream(1024);
while ((byteReads = input.read()) != -1) {
output.write(byteReads);
}
data = output.toByteArray();
output.close();
input.close();
return data;
} catch (Exception e) {
e.printStackTrace();
return null;
}
}
I am trying to convert a file from the sdcard to Base64 but it seems the file is too big and i get an OutOfMemoryError.
Here is my code :
InputStream inputStream = null;//You can get an inputStream using any IO API
inputStream = new FileInputStream(file.getAbsolutePath());
byte[] bytes;
byte[] buffer = new byte[8192];
int bytesRead;
ByteArrayOutputStream output = new ByteArrayOutputStream();
try {
while ((bytesRead = inputStream.read(buffer)) != -1) {
output.write(buffer, 0, bytesRead);
}
} catch (IOException e) {
e.printStackTrace();
}
bytes = output.toByteArray();
attachedFile = Base64.encodeToString(bytes, Base64.DEFAULT);
Is there a way to go around the OutOfMemoryError while filing the String attachedFile ?
Base64 encoding takes 3 input bytes and converts them to 4 bytes. So if you have 100 Mb file that will end up to be 133 Mb in Base64. When you convert it to Java string (UTF-16) it size will be doubled. Not to mention that during conversion process at some point you will hold multiple copies in memory. No matter how you turn this it is hardly going to work.
This is slightly more optimized code that uses Base64OutputStream and will need less memory than your code, but I would not hold my breath. My advice would be to improve that code further by skipping conversion to string, and using temporary file stream as output instead of ByteArrayOutputStream.
InputStream inputStream = null;//You can get an inputStream using any IO API
inputStream = new FileInputStream(file.getAbsolutePath());
byte[] buffer = new byte[8192];
int bytesRead;
ByteArrayOutputStream output = new ByteArrayOutputStream();
Base64OutputStream output64 = new Base64OutputStream(output, Base64.DEFAULT);
try {
while ((bytesRead = inputStream.read(buffer)) != -1) {
output64.write(buffer, 0, bytesRead);
}
} catch (IOException e) {
e.printStackTrace();
}
output64.close();
attachedFile = output.toString();
// Converting File to Base64.encode String type using Method
public String getStringFile(File f) {
InputStream inputStream = null;
String encodedFile = "", lastVal;
try {
inputStream = new FileInputStream(f.getAbsolutePath());
byte[] buffer = new byte[10240]; //specify the size to allow
int bytesRead;
ByteArrayOutputStream output = new ByteArrayOutputStream();
Base64OutputStream output64 = new Base64OutputStream(output, Base64.DEFAULT);
while ((bytesRead = inputStream.read(buffer)) != -1) {
output64.write(buffer, 0, bytesRead);
}
output64.close();
encodedFile = output.toString();
} catch (FileNotFoundException e1) {
e1.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
}
lastVal = encodedFile;
return lastVal;
}
This is my code for Reading .bin file. name:Testfile.bin location : Assets
In the byteRead(pathtobinfile) function I want to pass bin file path as a String.
how to get the bin file path. Any idea please!!!
public byte[] byteRead(String aInputFileName)
{
File file = new File(aInputFileName);
byte[] result = new byte[(int)file.length()];
try {
InputStream input = null;
try {
int totalBytesRead = 0;
input = new BufferedInputStream(new FileInputStream(file));
while(totalBytesRead < result.length){
int bytesRemaining = result.length - totalBytesRead;
//input.read() returns -1, 0, or more :
int bytesRead = input.read(result, totalBytesRead, bytesRemaining);
if (bytesRead > 0){
totalBytesRead = totalBytesRead + bytesRead;
}
}
}
finally {
//log("Closing input stream.");
input.close();
}
}
catch (FileNotFoundException ex) {
ex.printStackTrace();
}
catch (IOException ex) {
ex.printStackTrace();
}
Log.d("File Length", "Total No of bytes"+ result.length);
return result;
}
Any help?
Implement following code, which I modified as per your requirement. I have tested it and working very well.
public byte[] byteRead(String aInputFileName) {
ByteArrayOutputStream baos = new ByteArrayOutputStream();
try {
InputStream input = getResources().getAssets().open(aInputFileName);
try {
byte[] buffer = new byte[1024];
int read;
while ((read = input.read(buffer)) != -1) {
baos.write(buffer, 0, read);
}
} finally {
input.close();
}
} catch (FileNotFoundException ex) {
ex.printStackTrace();
} catch (IOException ex) {
ex.printStackTrace();
}
Log.d("Home", "Total No of bytes : " + baos.size());
return baos.toByteArray();
}
Input
You can use this function like this.
byte[] b = byteRead("myfile.txt");
String str = new String(b);
Log.d("Home", str);
Output
09-16 12:25:34.340: DEBUG/Home(4552): Total No of bytes : 10
09-16 12:25:34.340: DEBUG/Home(4552): hi Chintan
Its a very easy to read bin file from Asset folder.
Hope this will help someone.
InputStream input = context.getAssets().open("Testfile.bin");
// myData.txt can't be more than 2 gigs.
int size = input.available();
byte[] buffer = new byte[size];
input.read(buffer);
input.close();