OutOfMemoryError Whilst Writing a String to a ByteArrayOutputStream - android

I have a method which I have an array of String objects which I need to write to a ByteArrayOutputStream which later on I will write into a ZipOutputStream.
Here is my code.
// SAVE THE STRING AS A BYTE ARRAY IN MEMORY INSTEAD OF CREATING AND SAVING TO A NEW FILE
ByteArrayOutputStream baos = new ByteArrayOutputStream();
DataOutputStream byteOut = new DataOutputStream(baos);
byte[] data;
if (i == 1) {
if (nodeStringArray != null) {
for (String s : nodeStringArray) {
byteOut.write(s.getBytes("UTF-8"));
}
}
} else {
data = stringArray.get(i).getBytes("UTF-8");
byteOut.write(data);
}
byteOut.close();
byte[] input = baos.toByteArray();
ZipEntry entry = new ZipEntry(fileName);
entry.setSize(input.length);
zos.putNextEntry(entry);
zos.write(input);
zos.closeEntry();
The issue is, and I have only received on crash report for it, I am getting an OutOfMemoryError on the line that reads byteOut.write(s.getBytes("UTF-8"));
Anyone know of a better way of doing this or can see if I am doing something wrong?
IDEA
Could I put a try catch statement around the problem and when it happens call byteOut.flush()?
Thanks in advance

You are making to many copies of the data, and this is causing you to run out of memory.
You should be directly writing into your ZipOutputStream in your first loop.
ZipEntry entry = new ZipEntry(fileName);
zos.putNextEntry(entry);
for (String s : nodeStringArray) {
zos.write(s.getBytes("UTF-8"));
}
zos.closeEntry();

Related

Android: Uploading image without losing Exif data

In our app users have been uploading millions of images for years using (roughly) this code:
BitmapFactory.Options bmOptions = new BitmapFactory.Options();
bmOptions.inJustDecodeBounds = true;
BitmapFactory.decodeFile(postFilePath, bmOptions);
Bitmap roughBitmap = BitmapFactory.decodeFile(postFilePath, bmOptions);
ByteArrayOutputStream stream = new ByteArrayOutputStream();
roughBitmap.compress(Bitmap.CompressFormat.JPEG, 70, stream);
InputStream fis = new ByteArrayInputStream(stream.toByteArray());
int fileSize = stream.toByteArray().length;
conn.setRequestProperty("Content-Length", Integer.toString(fileSize));
conn.setFixedLengthStreamingMode(fileSize);
...
if (fis != null) {
byte[] buf = new byte[10240];
int read;
while ((read = fis.read(buf)) > 0) {
os.write(buf, 0, read);
totalBytesRead += read;
if (uploadProgressListener != null) {
try {
uploadProgressListener.onBytesUploaded(read);
} catch (Exception e) {
Log.e(e);
}
}
}
fis.close();
}
Recently we saw the need to preserve the Exif data of uploaded images. The problem is that the image Exif data is lost when compressing the bitmap. I thought of using ExifInterface for extracting this data from the original file:
ExifInterface oldExif = new ExifInterface(postFilePath);
String value = oldExif.getAttribute(ExifInterface.TAG_DATETIME);
..and then adding it to the InputStream fis and then continue uploading the file. The problem is that ExifInterface cannot save Exif data to an InputStream.
How can Exif data be retained in the images when they'er uploaded to the server?
It's not a duplicate:
Just to clarify deeper, I've tried using the suggested duplicate question by using this method:
public static void copyExif(String originalPath, InputStream newStream) throws IOException {
String[] attributes = new String[]
{
ExifInterface.TAG_DATETIME,
ExifInterface.TAG_DATETIME_DIGITIZED,
ExifInterface.TAG_EXPOSURE_TIME,
ExifInterface.TAG_FLASH,
ExifInterface.TAG_FOCAL_LENGTH,
ExifInterface.TAG_GPS_ALTITUDE,
ExifInterface.TAG_GPS_ALTITUDE_REF,
ExifInterface.TAG_GPS_DATESTAMP,
ExifInterface.TAG_GPS_LATITUDE,
ExifInterface.TAG_GPS_LATITUDE_REF,
ExifInterface.TAG_GPS_LONGITUDE,
ExifInterface.TAG_GPS_LONGITUDE_REF,
ExifInterface.TAG_GPS_PROCESSING_METHOD,
ExifInterface.TAG_GPS_TIMESTAMP,
ExifInterface.TAG_MAKE,
ExifInterface.TAG_MODEL,
ExifInterface.TAG_ORIENTATION,
ExifInterface.TAG_SUBSEC_TIME,
ExifInterface.TAG_WHITE_BALANCE
};
ExifInterface oldExif = new ExifInterface(originalPath);
ExifInterface newExif = new ExifInterface(newStream);
if (attributes.length > 0) {
for (int i = 0; i < attributes.length; i++) {
String value = oldExif.getAttribute(attributes[i]);
if (value != null)
newExif.setAttribute(attributes[i], value);
}
newExif.saveAttributes();
}
}
.. but got the exception java.io.IOException: ExifInterface does not support saving attributes for the current input. after newExif.saveAttributes(); because I'm trying to save the attributes to an InputStream. How else can I do it?
My solution:
As #amuttsch and #CommonsWare suggested, I:
saved the scaled/compressed bitmap to a temp file
copied the exif from the original file to the temp file
converted the temp file to a byte array and sent it to upload
.. then I found out that the server strips the Exif again while generating image variants :-P but that's another story which server guys are now working to correct.
Main code:
...
// Copy original Exif to scaledBitmap
String tempFilePath = getTempFilePath(postFilePath);
try {
FileOutputStream out = new FileOutputStream(tempFilePath);
scaledBitmap.compress(Bitmap.CompressFormat.JPEG, 70, out);
copyExif(postFilePath, tempFilePath);
} catch (Exception e) {
e.printStackTrace();
}
// Get stream from temp (exif loaded) file
File tempFile = new File(tempFilePath);
byte[] byteFile = readFile(tempFile);
fis = new ByteArrayInputStream(byteFile);
// Remove the temp file
boolean deleted = tempFile.delete();
// Finalize
int fileSize = byteFile.length;
conn.setRequestProperty("Content-Length", Integer.toString(fileSize));
conn.setFixedLengthStreamingMode(fileSize);
...
getTempFilePath():
private String getTempFilePath(String filename) {
String temp = "_temp";
int dot = filename.lastIndexOf(".");
String ext = filename.substring(dot + 1);
if (dot == -1 || !ext.matches("\\w+")) {
filename += temp;
} else {
filename = filename.substring(0, dot) + temp + "." + ext;
}
return filename;
}
copyExif():
public static void copyExif(String originalPath, String newPath) throws IOException {
String[] attributes = new String[]
{
ExifInterface.TAG_DATETIME,
ExifInterface.TAG_DATETIME_DIGITIZED,
ExifInterface.TAG_EXPOSURE_TIME,
ExifInterface.TAG_FLASH,
ExifInterface.TAG_FOCAL_LENGTH,
ExifInterface.TAG_GPS_ALTITUDE,
ExifInterface.TAG_GPS_ALTITUDE_REF,
ExifInterface.TAG_GPS_DATESTAMP,
ExifInterface.TAG_GPS_LATITUDE,
ExifInterface.TAG_GPS_LATITUDE_REF,
ExifInterface.TAG_GPS_LONGITUDE,
ExifInterface.TAG_GPS_LONGITUDE_REF,
ExifInterface.TAG_GPS_PROCESSING_METHOD,
ExifInterface.TAG_GPS_TIMESTAMP,
ExifInterface.TAG_MAKE,
ExifInterface.TAG_MODEL,
ExifInterface.TAG_ORIENTATION,
ExifInterface.TAG_SUBSEC_TIME,
ExifInterface.TAG_WHITE_BALANCE
};
ExifInterface oldExif = new ExifInterface(originalPath);
ExifInterface newExif = new ExifInterface(newPath);
if (attributes.length > 0) {
for (int i = 0; i < attributes.length; i++) {
String value = oldExif.getAttribute(attributes[i]);
if (value != null)
newExif.setAttribute(attributes[i], value);
}
newExif.saveAttributes();
}
}
readFile():
public static byte[] readFile(File file) throws IOException {
// Open file
RandomAccessFile f = new RandomAccessFile(file, "r");
try {
// Get and check length
long longlength = f.length();
int length = (int) longlength;
if (length != longlength)
throw new IOException("File size >= 2 GB");
// Read file and return data
byte[] data = new byte[length];
f.readFully(data);
return data;
} finally {
f.close();
}
}
The problem is that the image Exif data is lost when compressing the bitmap
The EXIF data is lost when reading in the Bitmap. A Bitmap has no EXIF tags.
How can Exif data be retained in the images when they'er uploaded to the server?
Stop reading in the Bitmap. Just upload the contents of postFilePath as-is. It will contain whatever EXIF tags it contains.
My assumption is that you are reading in the Bitmap in the hope that saving it again in 70% JPEG quality will result in meaningful bandwidth savings. I suspect that you are not saving very much, and you may be increasing the bandwidth in some cases (e.g., postFilePath points to a PNG). Your costs are a chunk of CPU time, an increased risk of an OutOfMemoryError, and the loss of your EXIF tags.
If, instead, the convert-to-70%-JPEG is some sort of data normalization approach, do that work on the server, where you have more CPU power, more disk space, more RAM, and continuous power.
Source: https://stackoverflow.com/a/11572752/8252521
Answered by: https://stackoverflow.com/users/1592398/code-jaff
Convert the file to bitmap by
Bitmap bi = BitmapFactory.decode(filepath + "DSC00021.jpg");
You can specify options too, look at API documentation
Or if you want to exchange the meta data from one file to another,
sanselan will probably be the best choice. This would be much
helpful when you manipulating the image, for example re-size.
The sample code will guide you in a right direction.
You need to just create a new OutputStream to preserve the Exif Information. There is no need of creating a new File.

Sending video file in API

I need to make API in Json to send the video. I don't want to send the path of video. What is the best way to send the video in JSON which will be used by android and iPhone guys. If I use the base64 or byte[] then I am getting the memory exception error.
File file = new File("video.mp4");
FileInputStream fis = new FileInputStream(file);
ByteArrayOutputStream bos = new ByteArrayOutputStream();
byte[] buf = new byte[1024];
try {
for (int readNum; (readNum = fis.read(buf)) != -1;) {
bos.write(buf, 0, readNum); //no doubt here is 0
System.out.println("read " + readNum + " bytes,");
}
} catch (IOException ex) {
Logger.getLogger(genJpeg.class.getName()).log(Level.SEVERE, null, ex);
}
byte[] bytes = bos.toByteArray();
This is how you add a video byte by byte inside an byte array. You just then send the byte array as JSONOBject by following...
byte[] data; //array holding the video
String base64Encoded = DatatypeConverter.printBase64Binary(data); //You have encoded the array into String
now send that to server. (I am guessing you know how to)..
This is how you will decode your JSON to byteArray again.
byte[] base64Decoded = DatatypeConverter.parseBase64Binary(base64Encoded);
The typical way to send binary in json is to base64 encode it. Java provides different ways to Base64 encode and decode a byte[]. One of these is DatatypeConverter.
I hope it helps.
Cheers!
Edited:
You are getting OutOfMemoryException, because HeapMemory is 2Mb in size and your video is 2Mb, so when inserting into String, it's going out of memory. Even if you put it into an Object instances, you will EITHER have to re-initialize the heap or some way else. I will try to write an answer tomorrow. (Writing this half asleep, might be other way around_

Android compare/search multiple images

I got a list of images and an input image, I want to output true if the image exist in my list.
Images will consider to be equal if they got the same raw data.
Is there an easy way of doing this in Android?
I saw this method:
boolean Bitmap.sameAs(Bitmap)
But it require to test binary data against each of my images in the list and this is kind of expensive. Is there a better way doing this?
And what if I change the definition of equal images to "similar", in such way that if the images got over 90% visual similarity they will consider to be equal.
Here is the solution for the first part where equal images are those who got the same raw data.
You can hash each image and compare it by hash, this way you will test the image raw data only once.
String toSHA1(Bitmap bitmap){
ByteArrayOutputStream stream = new ByteArrayOutputStream();
bitmap.compress(Bitmap.CompressFormat.PNG, 100, stream);
byte[] byteArray = stream.toByteArray();
try {
String sha1 = toSHA1(byteArray);
stream.close();
return sha1;
} catch (Exception e) {
e.printStackTrace();
}
return null;
}
String toSHA1(byte[] data) throws NoSuchAlgorithmException {
MessageDigest md = MessageDigest.getInstance("SHA-1");
BigInteger bigInteger = new BigInteger(1, md.digest(data));
return bigInteger.toString(Character.MAX_RADIX);
}

How to copy large files in Android?

I am trying to copy a large pdf-file (3.7 mb) from my raw-folder to the external cache directory.
I a using the following piece of code:
int i = 0;
if (Environment.MEDIA_MOUNTED.equals(Environment.getExternalStorageState()))
{
InputStream input = getResources().openRawResource(pdfs[i]);
File file = new File(Environment.getExternalStorageDirectory(), "/Android/data/eu.app/cache/" + pdfNames[i]);
if(!file.exists())
{
try
{
new File(Environment.getExternalStorageDirectory().getAbsolutePath() + "/Android/data/eu.app/cache").mkdirs();
FileOutputStream fos = new FileOutputStream(file.toURI().getPath(), false);
OutputStream os = new BufferedOutputStream(fos);
byte[] buffer = new byte[1024];
int byteRead = 0;
while ((byteRead = input.read(buffer)) != -1) {
os.write(buffer, 0, byteRead);
}
fos.close();
}
catch(Exception ex)
{
Log.d("storage", ex.getMessage());
}
}
}
else
{
}
I don't get any errors, but the output-file is a few bytes smaller than the original and cannot be opened.
What do I need to do to fix this?
I think the main issue is that you close fos while you should close os. You also need to put the close operation in a finally block.
Update (now with a full keyboard ;)): You close the file stream (fos) before the buffered stream is flushed. What you should do is to close the buffered stream (os), and that will in turn flush its buffer and write those bytes that are missing, and then it will automatically close the underlying file stream. To fix it change fos.close() into os.close().
In addition, to make sure that you always close the stream you should place the close operation in a finally block. A typical pattern is the following:
BufferedInputStream in = null;
try {
in = new BufferedInputStream(anInputStream);
BufferedOutputStream out = null;
try {
out = new BufferedOutputStream(new FileOutputStream(aFile));
// Read and write what you should write
}
finally {
if (out != null) out.close();
}
} finally {
if (in != null) in.close();
}
You can easily add an input stream, but be careful to make sure all streams are closed. This can be handled by nesting finally blocks or nesting try-catch blocks inside the finally block.
Either you throw an IOException from this method and handle it outside (often preferred), or you wrap the above code in a new try-catch statement and handle it there. However, handling it within the method mixes UI with logic and the code is often clearer separating UI and logic.
A final note: 1024 is rather small. Play with different values. On the other hand the buffered stream will handle the buffering for you.
I've been using this function for reading from one stream to another for a few years and have never had any problems with the resulting file. Just open the source and target files as you are and pass their respective streams into this function:
public static void streamToStream(InputStream is, OutputStream os) {
int count = 0;
try {
while(count != -1) {
byte[] bytes = new byte[2048];
count = is.read(bytes);
if(count == -1) {
continue;
}
os.write(bytes, 0, count);
bytes = null;
}
} catch (Exception e) {
e.printStackTrace();
}
}

Android, Using FileInputStream for reading file from SDC, reporting null pointer error

I used FileInputStream class when I was trying to read something from my files reside in SDCard, the codes are as follows:
Code:
filepath = "/sdcard/myfile/testFile"
FileInputStream fileIn = null;
fileIn = new
FileInputStream(filepath); byte []
InBuf = new byte[1024];
fileIn.read(InBuf); Strubg
fileContent = new String(InBuf);
fileIn.close();
when I ran my program, android keep reporting null pointer error on my last line:"fileIn.close", I really didn't know the reason
Use it like this:
filePath=Environment.getExternalStorageDirectory().getAbsolutePath()+"/myfile/textfile.extenstion"
You must be missing the extension.
I worked till late last night, finally, I make it work, though I am not sure if it is exact reason. I simply tried the following methods, then no more null pointer report came,
try
{
InputStream fileIn = null;
filepath = "/sdcard/myfile/testFile";
fileIn = new BufferedInputStrream(new
FileInputStream(filepath));
byte [] InBuf = new byte[1024];
fileIn.read(InBuf);
...
...
...
} catch(Exception e) {
e.printStackTrace();
} finally {
if(fileIn != null)
{
try
{
fileIn.close();
} catch(IOException e) {
e.printStackTrace();
}
}
}
Any way, thank you so much for your help. ^^

Categories

Resources