I have an application in which I am using the code to decrypt the file which is already encrypted. The file location is "/mnt/sdcard/myfolder/test.mp4". The size of test.mp4 file is approx 20MB.
When I am using the following code to decrypt the encrypted files of small size, the files are successfully decrypted but when I am trying to decrypt the large video files, an exception of outOfMemoryException is occured.
Here is the code :
FileOutputStream fos = new FileOutputStream(outFilePath);
Cipher cipher = Cipher.getInstance("AES/CBC/PKCS5Padding");
byte[] keyBytes= new byte[16];
//byte[] b= key.getBytes(Charset.forName("UTF-8"));
byte[] b= key.getBytes("UTF-8");
Log.i("b",""+b);
int len= b.length;
Log.i("len",""+len);
if (len > keyBytes.length) len = keyBytes.length;
System.arraycopy(b, 0, keyBytes, 0, len);
SecretKeySpec keySpec = new SecretKeySpec(keyBytes, "AES");
IvParameterSpec ivSpec = new IvParameterSpec(keyBytes);
cipher.init(Cipher.DECRYPT_MODE,keySpec,ivSpec);
byte[] results = new byte[cipher.getOutputSize(abc.length)];
try
{
Log.i("output size:", ""+cipher.getOutputSize(abc.length));
***results = cipher.doFinal(abc);***
}
catch (Exception e) {
// TODO: handle exception
Log.e("EXCEPTION:", e.getMessage());
}
fos.write(results);
NOTE: byte[] abc = new byte[64]; contains the input byte array.
From your question, or at least from the code you posted, there is nothing that would couse OutOfMemoryException, especially since array abc is only 64 bytes long. But you said you get the exception when working with large files. So my inference,
Somewhere in your code (not in posted part), you are trying to read full file into any array, or trying to hold it in array. Android does impose a memory limit on application (16 MB for most devices), this limit includes the memory used for UI elements. So there is not much memory there for you to play with.
Now ideally, what you should do is to create a decrypt block, that works with streams. CipherInputStream does looks promising. And this stackoverflow thread, might be of interest if you are thinking of using CipherInputStream.
Related
I need to make API in Json to send the video. I don't want to send the path of video. What is the best way to send the video in JSON which will be used by android and iPhone guys. If I use the base64 or byte[] then I am getting the memory exception error.
File file = new File("video.mp4");
FileInputStream fis = new FileInputStream(file);
ByteArrayOutputStream bos = new ByteArrayOutputStream();
byte[] buf = new byte[1024];
try {
for (int readNum; (readNum = fis.read(buf)) != -1;) {
bos.write(buf, 0, readNum); //no doubt here is 0
System.out.println("read " + readNum + " bytes,");
}
} catch (IOException ex) {
Logger.getLogger(genJpeg.class.getName()).log(Level.SEVERE, null, ex);
}
byte[] bytes = bos.toByteArray();
This is how you add a video byte by byte inside an byte array. You just then send the byte array as JSONOBject by following...
byte[] data; //array holding the video
String base64Encoded = DatatypeConverter.printBase64Binary(data); //You have encoded the array into String
now send that to server. (I am guessing you know how to)..
This is how you will decode your JSON to byteArray again.
byte[] base64Decoded = DatatypeConverter.parseBase64Binary(base64Encoded);
The typical way to send binary in json is to base64 encode it. Java provides different ways to Base64 encode and decode a byte[]. One of these is DatatypeConverter.
I hope it helps.
Cheers!
Edited:
You are getting OutOfMemoryException, because HeapMemory is 2Mb in size and your video is 2Mb, so when inserting into String, it's going out of memory. Even if you put it into an Object instances, you will EITHER have to re-initialize the heap or some way else. I will try to write an answer tomorrow. (Writing this half asleep, might be other way around_
I tried to encrypt and save a file to the same location in the external storage with a different file name. But the way I have used is seems like wrong. Please help someone.
public static void encrypt(SecretKey secretKey, String filePath, IvParameterSpec iv){
try {
String file = "";
// Here you read the cleartext.
FileInputStream fis = new FileInputStream(filePath);
// This stream write the encrypted text. This stream will be wrapped by another stream.
//String filePath2 = filePath+"enc";
file = filePath.substring(0,filePath.length()-5)+"enc.jpeg";
FileOutputStream fos = new FileOutputStream(file);
Log.i(TAG, "Uri = "+file);
// Create cipher
Cipher cipher = Cipher.getInstance("AES/ECB/NoPadding");
cipher.init(Cipher.ENCRYPT_MODE, secretKey);
// Wrap the output stream
CipherOutputStream cos = new CipherOutputStream(fos, cipher);
// Write bytes
int b;
byte[] d = new byte[8];
while ((b = fis.read(d)) != -1) {
cos.write(d, 0, b);
}
// Flush and close streams.
cos.flush();
cos.close();
fis.close();
}catch(IOException e){
e.printStackTrace();
}catch (NoSuchAlgorithmException e){
e.printStackTrace();
}catch(NoSuchPaddingException e){
e.printStackTrace();
}catch(InvalidKeyException e){
e.printStackTrace();
}/*catch (InvalidAlgorithmParameterException e){
e.printStackTrace();
}*/
}
Manifest file containing both read and write permissions.
You specify "AES/ECB/NoPadding".
With ECB there is no iv and thus no need to supply one on calling the encrypt method. ECB mode is insecure, see ECB mode, scroll down to the Penguin.
AES is a block cipher and as such it encrypts block sized portions at a time thus the input needs to be a multiple of the block size. Padding accomplished this transparently but you have specified "NoPadding" so the input file size ,just be an exact multiple of the block size, for AES that is 16-bytes. Instead use PKCS#7 (some tined referred to as PKCS#5) padding.
The simplest solution is to use a library that puts all the elements of secure encryption together including password derivation, a random iv, padding, encryption authentication. Consider RNCryptor, it provides all of this plus versioning. See RNCryptor README and RNCryptor-Spec for more information.
I am developing an android app that secures images and videos like Vaulty and Keep safe. I am trying to use AES-128 encryption/decryption technique to store images and videos. I tried it by taking 3 sample images of size 5.13, 4.76 and 5.31 respectively. But the time it is consuming to encrypt is 25s, 22s, 27s respectively and time to decrypt is 31s, 30s, 34s respectively. I am testing it on HTC One X.
Such speed wont be feasible for my app as users will scroll and view images quickly without interruption. Can you please suggest me how can I improve the performance(speed) or should i switch to other algorithms? Can you please suggest me any other techniques through which i can encrypt/decrypt images and videos quickly without compromising security too much.
I tried Vaulty and Keep safe, and they are very quick. Vaulty is said to be using AES-256, but it is still very fast and responsive in terms of encrypting and viewing images. How is it possible that vaulty is that quick using AES-256?
Code I am using is:
static void encrypt(String filename) throws IOException, NoSuchAlgorithmException,
NoSuchPaddingException, InvalidKeyException {
// Here you read the cleartext.
File extStore = Environment.getExternalStorageDirectory();
startTime = System.currentTimeMillis();
Log.i("Encryption Started",extStore + "/5mbtest/"+filename);
FileInputStream fis = new FileInputStream(extStore + "/5mbtest/"+filename);
// This stream write the encrypted text. This stream will be wrapped by
// another stream.
FileOutputStream fos = new FileOutputStream(extStore + "/5mbtest/"+filename+".aes", false);
// Length is 16 byte
SecretKeySpec sks = new SecretKeySpec("MyDifficultPassw".getBytes(),
"AES");
// Create cipher
Cipher cipher = Cipher.getInstance("AES/CBC/PKCS7Padding");
cipher.init(Cipher.ENCRYPT_MODE, sks);
// Wrap the output stream
CipherOutputStream cos = new CipherOutputStream(fos, cipher);
// Write bytes
int b;
byte[] d = new byte[8];
while ((b = fis.read(d)) != -1) {
cos.write(d, 0, b);
}
// Flush and close streams.
cos.flush();
cos.close();
fis.close();
stopTime = System.currentTimeMillis();
Log.i("Encryption Ended",extStore + "/5mbtest/"+filename+".aes");
Log.i("Time Elapsed", ((stopTime - startTime)/1000.0)+"");
}
static void decrypt(String filename) throws IOException, NoSuchAlgorithmException,
NoSuchPaddingException, InvalidKeyException {
File extStore = Environment.getExternalStorageDirectory();
Log.i("Decryption Started",extStore + "/5mbtest/"+filename+".aes");
FileInputStream fis = new FileInputStream(extStore + "/5mbtest/"+filename+".aes");
FileOutputStream fos = new FileOutputStream(extStore + "/5mbtest/"+"decrypted"+filename,false);
SecretKeySpec sks = new SecretKeySpec("MyDifficultPassw".getBytes(),
"AES");
// Create cipher
Cipher cipher = Cipher.getInstance("AES/CBC/PKCS7Padding");
cipher.init(Cipher.DECRYPT_MODE, sks);
startTime = System.currentTimeMillis();
CipherInputStream cis = new CipherInputStream(fis, cipher);
int b;
byte[] d = new byte[8];
while ((b = cis.read(d)) != -1) {
fos.write(d, 0, b);
}
stopTime = System.currentTimeMillis();
Log.i("Decryption Ended",extStore + "/5mbtest/"+"decrypted"+filename);
Log.i("Time Elapsed", ((stopTime - startTime)/1000.0)+"");
fos.flush();
fos.close();
cis.close();
}
One thing that is making your code run slow is the size of your buffer:
byte[] d = new byte[8];
You should bump it up by a few orders of magnitude if you want it to run fast. Given the size of your files I would suggest using at least 1 MB but nowadays you can realistically set it to a few MBs, even on Android. Try changing it to:
byte[] d = new byte[1024 * 1024];
and let us know how much did that improve the speed by.
Use a larger buffer as suggested by #MikeLaren, and also wrap the FileOutputStream in a BufferedOutputStream. When decrypting, wrap the FileInputStream in a BufferedInputStream. Or do both in both cases: no harm done.
No need for heroic buffer sizes like a megabyte: 8k or 32k is sufficient.
Using Base64 from Apache commons
public byte[] encode(File file) throws FileNotFoundException, IOException {
byte[] encoded;
try (FileInputStream fin = new FileInputStream(file)) {
byte fileContent[] = new byte[(int) file.length()];
fin.read(fileContent);
encoded = Base64.encodeBase64(fileContent);
}
return encoded;
}
Exception in thread "AWT-EventQueue-0" java.lang.OutOfMemoryError: Java heap space
at org.apache.commons.codec.binary.BaseNCodec.encode(BaseNCodec.java:342)
at org.apache.commons.codec.binary.Base64.encodeBase64(Base64.java:657)
at org.apache.commons.codec.binary.Base64.encodeBase64(Base64.java:622)
at org.apache.commons.codec.binary.Base64.encodeBase64(Base64.java:604)
I'm making small app for mobile device.
You cannot just load the whole file into memory, like here:
byte fileContent[] = new byte[(int) file.length()];
fin.read(fileContent);
Instead load the file chunk by chunk and encode it in parts. Base64 is a simple encoding, it is enough to load 3 bytes and encode them at a time (this will produce 4 bytes after encoding). For performance reasons consider loading multiples of 3 bytes, e.g. 3000 bytes - should be just fine. Also consider buffering input file.
An example:
byte fileContent[] = new byte[3000];
try (FileInputStream fin = new FileInputStream(file)) {
while(fin.read(fileContent) >= 0) {
Base64.encodeBase64(fileContent);
}
}
Note that you cannot simply append results of Base64.encodeBase64() to encoded bbyte array. Actually, it is not loading the file but encoding it to Base64 causing the out-of-memory problem. This is understandable because Base64 version is bigger (and you already have a file occupying a lot of memory).
Consider changing your method to:
public void encode(File file, OutputStream base64OutputStream)
and sending Base64-encoded data directly to the base64OutputStream rather than returning it.
UPDATE: Thanks to #StephenC I developed much easier version:
public void encode(File file, OutputStream base64OutputStream) {
InputStream is = new FileInputStream(file);
OutputStream out = new Base64OutputStream(base64OutputStream)
IOUtils.copy(is, out);
is.close();
out.close();
}
It uses Base64OutputStream that translates input to Base64 on-the-fly and IOUtils class from Apache Commons IO.
Note: you must close the FileInputStream and Base64OutputStream explicitly to print = if required but buffering is handled by IOUtils.copy().
Either the file is too big, or your heap is too small, or you've got a memory leak.
If this only happens with really big files, put something into your code to check the file size and reject files that are unreasonably big.
If this happens with small files, increase your heap size by using the -Xmx command line option when you launch the JVM. (If this is in a web container or some other framework, check the documentation on how to do it.)
If the file recurs, especially with small files, the chances are that you've got a memory leak.
The other point that should be made is that your current approach entails holding two complete copies of the file in memory. You should be able to reduce the memory usage, though you'll typically need a stream-based Base64 encoder to do this. (It depends on which flavor of the base64 encoding you are using ...)
This page describes a stream-based Base64 encoder / decoder library, and includes lnks to some alternatives.
Well, do not do it for the whole file at once.
Base64 works on 3 bytes at a time, so you can read your file in batches of "multiple of 3" bytes, encode them and repeat until you finish the file:
// the base64 encoding - acceptable estimation of encoded size
StringBuilder sb = new StringBuilder(file.length() / 3 * 4);
FileInputStream fin = null;
try {
fin = new FileInputStream("some.file");
// Max size of buffer
int bSize = 3 * 512;
// Buffer
byte[] buf = new byte[bSize];
// Actual size of buffer
int len = 0;
while((len = fin.read(buf)) != -1) {
byte[] encoded = Base64.encodeBase64(buf);
// Although you might want to write the encoded bytes to another
// stream, otherwise you'll run into the same problem again.
sb.append(new String(buf, 0, len));
}
} catch(IOException e) {
if(null != fin) {
fin.close();
}
}
String base64EncodedFile = sb.toString();
You are not reading the whole file, just the first few kb. The read method returns how many bytes were actually read. You should call read in a loop until it returns -1 to be sure that you have read everything.
The file is too big for both it and its base64 encoding to fit in memory. Either
process the file in smaller pieces or
increase the memory available to the JVM with the -Xmx switch, e.g.
java -Xmx1024M YourProgram
This is best code to upload image of more size
bitmap=Bitmap.createScaledBitmap(bitmap, 100, 100, true);
ByteArrayOutputStream stream = new ByteArrayOutputStream();
bitmap.compress(Bitmap.CompressFormat.PNG, 100, stream); //compress to which format you want.
byte [] byte_arr = stream.toByteArray();
String image_str = Base64.encodeBytes(byte_arr);
Well, looks like your file is too large to keep the multiple copies necessary for an in-memory Base64 encoding in the available heap memory at the same time. Given that this is for a mobile device, it's probably not possible to increase the heap, so you have two options:
make the file smaller (much smaller)
Do it in a stram-based way so that you're reading from an InputStream one small part of the file at a time, encode it and write it to an OutputStream, without ever keeping the enitre file in memory.
In Manifest in applcation tag write following
android:largeHeap="true"
It worked for me
Java 8 added Base64 methods, so Apache Commons is no longer needed to encode large files.
public static void encodeFileToBase64(String inputFile, String outputFile) {
try (OutputStream out = Base64.getEncoder().wrap(new FileOutputStream(outputFile))) {
Files.copy(Paths.get(inputFile), out);
} catch (IOException e) {
throw new UncheckedIOException(e);
}
}
IN our application we are getting byte array from server if login gets success. We are converting those byte array into PDF format and storing those files into DB which using internal memory.If files are in KB , application works properly but of files size get increase in MB then application gives out of memory error.Please tell me how to handle this scenario?How to store files into SD card to maintain security also.It should not visible to outside user.
Please do help.
Thanks,
AA.
You should take a look at:
CipherInputStream and CipherOutputStream. They are used to encrypt and decrypt byte streams.
EDIT: So here you go!
I have a file named cleartext. The file contains:
Hi, I'm a clear text.
How are you?
That's awesome!
Now, you have an encrypt() function:
static void encrypt() throws IOException, NoSuchAlgorithmException, NoSuchPaddingException, InvalidKeyException {
// Here you read the cleartext.
FileInputStream fis = new FileInputStream("data/cleartext");
// This stream write the encrypted text. This stream will be wrapped by another stream.
FileOutputStream fos = new FileOutputStream("data/encrypted");
// Length is 16 byte
SecretKeySpec sks = new SecretKeySpec("MyDifficultPassw".getBytes(), "AES");
// Create cipher
Cipher cipher = Cipher.getInstance("AES");
cipher.init(Cipher.ENCRYPT_MODE, sks);
// Wrap the output stream
CipherOutputStream cos = new CipherOutputStream(fos, cipher);
// Write bytes
int b;
byte[] d = new byte[8];
while((b = fis.read(d)) != -1) {
cos.write(d, 0, b);
}
// Flush and close streams.
cos.flush();
cos.close();
fis.close();
}
After you execute this function, there should be a file names encrypted. The file contains the encrypted characters.
For decryption you have the decrypt function:
static void decrypt() throws IOException, NoSuchAlgorithmException, NoSuchPaddingException, InvalidKeyException {
FileInputStream fis = new FileInputStream("data/encrypted");
FileOutputStream fos = new FileOutputStream("data/decrypted");
SecretKeySpec sks = new SecretKeySpec("MyDifficultPassw".getBytes(), "AES");
Cipher cipher = Cipher.getInstance("AES");
cipher.init(Cipher.DECRYPT_MODE, sks);
CipherInputStream cis = new CipherInputStream(fis, cipher);
int b;
byte[] d = new byte[8];
while((b = cis.read(d)) != -1) {
fos.write(d, 0, b);
}
fos.flush();
fos.close();
cis.close();
}
After the execution of decrypt, there should be a file named decrypted. This file contains the free text.
Edit: You write you're a "noob" but depending on the use-case of encryption you could do a lot of harm if you're not doing it the right way. Know your tools!
Usage of CipherOutputStream Oracle documentation:
SecretKeySpec skeySpec = new SecretKeySpec(y.getBytes(), "AES");
FileInputStream fis;
FileOutputStream fos;
CipherOutputStream cos;
// File you are reading from
fis = new FileInputStream("/tmp/a.txt");
// File output
fos = new FileOutputStream("/tmp/b.txt");
// Here the file is encrypted. The cipher1 has to be created.
// Key Length should be 128, 192 or 256 bit => i.e. 16 byte
SecretKeySpec skeySpec = new SecretKeySpec("MyDifficultPassw".getBytes(), "AES");
Cipher cipher1 = Cipher.getInstance("AES");
cipher1.init(Cipher.ENCRYPT_MODE, skeySpec);
cos = new CipherOutputStream(fos, cipher1);
// Here you read from the file in fis and write to cos.
byte[] b = new byte[8];
int i = fis.read(b);
while (i != -1) {
cos.write(b, 0, i);
i = fis.read(b);
}
cos.flush();
Thus, the encryption should work. When you reverse the process, you should be able to read the decrypted bytes.
Storing on the SD card will make the file accessible to savvy users. Storing in the db will give you errors like you mentioned. Probably the best idea would be to go to internal storage. This isn't perfect (rooted users can browse to the files), but it's probably the best option.
http://developer.android.com/guide/topics/data/data-storage.html#filesInternal