I have an API call that returns a byte array. I currently stream the result into a byte array then make sure the checksums match and then write the ByteArrayOutputStream to File. The code is something like this and it works pretty well.
String path = "file.txt";
ByteArrayOutputStream byteBuffer = new ByteArrayOutputStream();
int bufferSize = 1024;
byte[] buffer = new byte[bufferSize];
int len = 0;
while ((len = inputStream.read(buffer)) != -1) {
byteBuffer.write(buffer, 0, len);
}
FileOutputStream stream = new FileOutputStream(path);
stream.write(byteBuffer.toByteArray());
My concern i that the result from inputstream could potentially be larger than the heap size in android and I could get OutOfMemory exceptions if the entire byte array is in memory. What is the most elegant way to write the inputStream to file in chunks, such that the byte array is never larger than the heap size?
Don't write to the ByteArrayOutputStream. Write directly to the FileOutputStream.
String path = "file.txt";
FileOutputStream output = new FileOutputStream(path);
int bufferSize = 1024;
byte[] buffer = new byte[bufferSize];
int len = 0;
while ((len = inputStream.read(buffer)) != -1) {
output.write(buffer, 0, len);
}
I went with the advice to skip the ByteArrayOutputStream and write to the FileOutputStream and this seems to address my concerns. With one quick adjustment, where the FileOutputStream is decorated by a BufferedOutputStream
String path = "file.txt";
OutputStream stream = new BufferedOutputStream(new FileOutputStream(path));
int bufferSize = 1024;
byte[] buffer = new byte[bufferSize];
int len = 0;
while ((len = is.read(buffer)) != -1) {
stream.write(buffer, 0, len);
}
if(stream!=null)
stream.close();
Related
I have tried for 200 MB of image file and converting it onto byte array but is crashed due to OOM, so how to read large file and converting into byte[]
Caused by: java.lang.OutOfMemoryError: Failed to allocate a 210288697 byte allocation with 4108138 free bytes and 186MB until OOM
byte[] fullyReadFileToBytes(File file) throws IOException {
int size = (int) file.length();
byte bytes[] = new byte[size];
byte tmpBuff[] = new byte[size];
FileInputStream fis = new FileInputStream(file);
try {
int read = fis.read(bytes, 0, size);
if (read < size) {
int remain = size - read;
while (remain > 0) {
read = fis.read(tmpBuff, 0, remain);
System.arraycopy(tmpBuff, 0, bytes, size - remain, read);
remain -= read;
}
}
} catch (IOException e) {
throw e;
} finally {
fis.close();
}
return bytes;
}
NOTE: I have tried for 100MB and it is working perfect but in case of size greater than 150MB it's creating crash.
Use Http post multipart transfer i.e.
ByteArrayInputStream fileInputStream = new ByteArrayInputStream(dataFile.getContent());
int bytesAvailable = fileInputStream.available();
int maxBufferSize = 1024 * 1024;//1 mb buffer - set size according to your need
int bufferSize = Math.min(bytesAvailable, maxBufferSize);
byte[] buffer = new byte[bufferSize];
int bytesRead = fileInputStream.read(buffer, 0, bufferSize);
while (bytesRead > 0) {
dataOutputStream.write(buffer, 0, bufferSize);
bytesAvailable = fileInputStream.available();
bufferSize = Math.min(bytesAvailable, maxBufferSize);
bytesRead = fileInputStream.read(buffer, 0, bufferSize);
}
dataOutputStream.writeBytes(lineEnd);
Use volley multi-transfer request. It will keep track of failures
https://gist.github.com/anggadarkprince/a7c536da091f4b26bb4abf2f92926594
The below code helps to compress the bitmap image and convert to string bytes.
public String BitMapToString(Bitmap bitmap) {
ByteArrayOutputStream baos = new ByteArrayOutputStream();
bitmap.compress(Bitmap.CompressFormat.JPEG, 50, baos);
byte[] b = baos.toByteArray();
String temp = null;
try {
System.gc();
temp = Base64.encodeToString(b, Base64.DEFAULT);
} catch (Exception e) {
e.printStackTrace();
} catch (OutOfMemoryError e) {
baos = new ByteArrayOutputStream();
bitmap.compress(Bitmap.CompressFormat.JPEG, 50, baos);
b = baos.toByteArray();
temp = Base64.encodeToString(b, Base64.DEFAULT);
}
return temp;
}
I have a method that takes an image's URL and attempts to save it in memory. I need to retrieve this saved file later, and want to do so by getting its file path. How do I get the file name from this save method?
InputStream input;
try {
URL url = new URL (strURL);
input = url.openStream();
byte[] buffer = new byte[1500];
OutputStream output = new FileOutputStream ("/sdcard/"+pos+".png");
try {
int bytesRead = 0;
while ((bytesRead = input.read(buffer, 0, buffer.length)) >= 0) {
output.write(buffer, 0, bytesRead);
}
}
finally {
output.close();
buffer = null;
}
}
I am using Android Studio.
i am writing the PDF's to sdcard and using the below code :
byte[] data = new byte[20000];
ByteArrayOutputStream buffer = new ByteArrayOutputStream();
int nRead;
InputStream fileInputStream = Thread.currentThread().getContextClassLoader().getResourceAsStream(fileName);
while ((nRead = fileInputStream.read(data, 0, data.length)) != -1) {
buffer.write(data, 0, nRead);
}
buffer.flush();
byte[] bytesToWrite = buffer.toByteArray();
fileInputStream.read(bytesToWrite);
fileInputStream.close();
FileOutputStream fileOutputStream = null;
String outputFileName = outputDirName + "/" + fileName;
fileOutputStream = new FileOutputStream(outputFileName);
BufferedOutputStream bos = new BufferedOutputStream(fileOutputStream);
bos.write(bytesToWrite);
bos.flush();
bos.close();
It works fine if i am trying to write 20 PDFs in one shot, but if its more than that it gives me OutOfMemory error.
what might be the issue ?
You are storing the entire file into RAM with your ByteArrayOutputStream, then copying it from RAM onto disk. This is likely what's causing your OutOfMemoryError.
It would be much more efficient to read a chunk into RAM, and then flush to disk immediately repeatedly. I've rewritten your code to do this.
byte[] data = new byte[20000];
FileOutputStream fileOutputStream = null;
String outputFileName = outputDirName + "/" + fileName;
fileOutputStream = new FileOutputStream(outputFileName);
int nRead;
InputStream fileInputStream = Thread.currentThread().getContextClassLoader().getResourceAsStream(fileName);
while ((nRead = fileInputStream.read(data, 0, data.length)) != -1) {
fileOutputStream.write(data,0,nRead);
}
fileInputStream.close();
fileOutputStream.flush();
fileOutputStream.close();
I have cheerapp.wav or cheerapp.mp3 or some other format.
InputStream in = context.getResources().openRawResource(R.raw.cheerapp);
BufferedInputStream bis = new BufferedInputStream(in, 8000);
// Create a DataInputStream to read the audio data from the saved file
DataInputStream dis = new DataInputStream(bis);
byte[] music = null;
music = new byte[??];
int i = 0; // Read the file into the "music" array
while (dis.available() > 0) {
// dis.read(music[i]); // This assignment does not reverse the order
music[i]=dis.readByte();
i++;
}
dis.close();
For the music byte array which takes the data from the DataInputStream. I don't know what the length of that to allocate.
This is raw file from resource not a file therefore I wouldn't know the size of that thing.
You do have byte array length as you can see:
InputStream inStream = context.getResources().openRawResource(R.raw.cheerapp);
byte[] music = new byte[inStream.available()];
And then you can read whole Stream into byte array easily.
Of course I would recommend that you do check when it comes to the size and use ByteArrayOutputStream with smaller byte[] buffer if needed:
public static byte[] convertStreamToByteArray(InputStream is) throws IOException {
ByteArrayOutputStream baos = new ByteArrayOutputStream();
byte[] buff = new byte[10240];
int i = Integer.MAX_VALUE;
while ((i = is.read(buff, 0, buff.length)) > 0) {
baos.write(buff, 0, i);
}
return baos.toByteArray(); // be sure to close InputStream in calling function
}
If you'll be doing lots of IO operations I recommend that you make use of org.apache.commons.io.IOUtils. That way you won't need to worry too much about quality of your IO implementation and once you import JAR into your project you would just do:
byte[] payload = IOUtils.toByteArray(context.getResources().openRawResource(R.raw.cheerapp));
Hope it will help.
Create an sdcard path:
String outputFile =
Environment.getExternalStorageDirectory().getAbsolutePath() + "/recording.3gp";
Convert as a file and have to call the byte array method:
byte[] soundBytes;
try {
InputStream inputStream =
getContentResolver().openInputStream(Uri.fromFile(new File(outputFile)));
soundBytes = new byte[inputStream.available()];
soundBytes = toByteArray(inputStream);
Toast.makeText(this, "Recordin Finished"+ " " + soundBytes, Toast.LENGTH_LONG).show();
} catch(Exception e) {
e.printStackTrace();
}
method:
public byte[] toByteArray(InputStream in) throws IOException {
ByteArrayOutputStream out = new ByteArrayOutputStream();
int read = 0;
byte[] buffer = new byte[1024];
while (read != -1) {
read = in.read(buffer);
if (read != -1)
out.write(buffer,0,read);
}
out.close();
return out.toByteArray();
}
In Kotlin use
InputStream.readBytes()
I am trying to share files between two Android phones using Socket programming. The problem is right now I have to hard code the file extension on the receiving end. Is there a way that I can automatically determine the extension of the file being received?
Here's my code.
Client Side
socket = new Socket(IP,4445);
File myFile = new File ("/mnt/sdcard/Pictures/A.jpg");
FileInputStream fis = null;
fis = new FileInputStream(myFile);
OutputStream os = null;
os = socket.getOutputStream();
int filesize = (int) myFile.length();
byte [] buffer = new byte [filesize];
int bytesRead =0;
while ((bytesRead = fis.read(buffer)) > 0) {
os.write(buffer, 0, bytesRead);
System.out.println("SO sendFile" + bytesRead);
}
os.flush();
os.close();
fis.close();
socket.close();
}
And the Server side
FileOutputStream fos = null;
File root = Environment.getExternalStorageDirectory();
fos = new FileOutputStream(new File(root,"B.jpg")); //Here I have to hardcode B.jpg with jpg extension.
BufferedOutputStream bos = new BufferedOutputStream(fos);
ServerS = new ServerSocket(4445);
clientSocket = ServerS.accept();
InputStream is = null;
is = clientSocket.getInputStream();
int bytesRead = 0;
int current = 0;
byte [] mybytearray = new byte [329];
do {
bos.write(mybytearray,0,bytesRead);
bytesRead = is.read(mybytearray, 0, mybytearray.length);
} while(bytesRead > -1);
bos.flush();
bos.close();
clientSocket.close();
}
You can find the file extension pretty easily by doing this:
String extension = filename.substring(filename.lastIndexOf('.'));