Develop Reference

Incorrect duration and bitrate in ffmpeg-encoded audio

I am encoding raw data on Android using ffmpeg libraries. The native code reads the audio data from an external device and encodes it into AAC format in an mp4 container. I am finding that the audio data is successfully encoded (I can play it with Groove Music, my default Windows audio player). But the metadata, as reported by ffprobe, has an incorrect duration of 0.05 secs - it's actually several seconds long. Also the bitrate is reported wrongly as around 65kbps even though I specified 192kbps.
I've tried recordings of various durations but the result is always similar - the (very small) duration and bitrate. I've tried various other audio players such as Quicktime but they play only the first 0.05 secs or so of the audio.
I've removed error-checking from the following. The actual code checks every call and no problems are reported.
Initialisation:
void AudioWriter::initialise( const char *filePath )
{
AVCodecID avCodecID = AVCodecID::AV_CODEC_ID_AAC;
int bitRate = 192000;
char *containerFormat = "mp4";
int sampleRate = 48000;
int nChannels = 2;
mAvCodec = avcodec_find_encoder(avCodecID);
mAvCodecContext = avcodec_alloc_context3(mAvCodec);
mAvCodecContext->codec_id = avCodecID;
mAvCodecContext->codec_type = AVMEDIA_TYPE_AUDIO;
mAvCodecContext->sample_fmt = AV_SAMPLE_FMT_FLTP;
mAvCodecContext->bit_rate = bitRate;
mAvCodecContext->sample_rate = sampleRate;
mAvCodecContext->channels = nChannels;
mAvCodecContext->channel_layout = AV_CH_LAYOUT_STEREO;
avcodec_open2( mAvCodecContext, mAvCodec, nullptr );
mAvFormatContext = avformat_alloc_context();
avformat_alloc_output_context2(&mAvFormatContext, nullptr, containerFormat, nullptr);
mAvFormatContext->audio_codec = mAvCodec;
mAvFormatContext->audio_codec_id = avCodecID;
mAvOutputStream = avformat_new_stream(mAvFormatContext, mAvCodec);
avcodec_parameters_from_context(mAvOutputStream->codecpar, mAvCodecContext);
if (!(mAvFormatContext->oformat->flags & AVFMT_NOFILE))
{
avio_open(&mAvFormatContext->pb, filePath, AVIO_FLAG_WRITE);
}
if ( mAvFormatContext->oformat->flags & AVFMT_GLOBALHEADER )
{
mAvCodecContext->flags |= AV_CODEC_FLAG_GLOBAL_HEADER;
}
avformat_write_header(mAvFormatContext, NULL);
mAvAudioFrame = av_frame_alloc();
mAvAudioFrame->nb_samples = mAvCodecContext->frame_size;
mAvAudioFrame->format = mAvCodecContext->sample_fmt;
mAvAudioFrame->channel_layout = mAvCodecContext->channel_layout;
av_samples_get_buffer_size(NULL, mAvCodecContext->channels, mAvCodecContext->frame_size,
mAvCodecContext->sample_fmt, 0);
av_frame_get_buffer(mAvAudioFrame, 0);
av_frame_make_writable(mAvAudioFrame);
mAvPacket = av_packet_alloc();
}
Encoding:
// SoundRecording is a custom class with the raw samples to be encoded
bool AudioWriter::encodeToContainer( SoundRecording *soundRecording )
{
int ret;
int frameCount = mAvCodecContext->frame_size;
int nChannels = mAvCodecContext->channels;
float *buf = new float[frameCount*nChannels];
while ( soundRecording->hasReadableData() )
{
//Populate the frame
int samplesRead = soundRecording->read( buf, frameCount*nChannels );
// Planar data
int nFrames = samplesRead/nChannels;
for ( int i = 0; i < nFrames; ++i )
{
for (int c = 0; c < nChannels; ++c )
{
samples[c][i] = buf[nChannels*i +c];
}
}
// Fill a gap at the end with silence
if ( samplesRead < frameCount*nChannels )
{
for ( int i = samplesRead; i < frameCount*nChannels; ++i )
{
for (int c = 0; c < nChannels; ++c )
{
samples[c][i] = 0.0;
}
}
}
encodeFrame( mAvAudioFrame ) )
}
finish();
}
bool AudioWriter::encodeFrame( AVFrame *frame )
{
//send the frame for encoding
int ret;
if ( frame != nullptr )
{
frame->pts = mAudFrameCounter++;
}
avcodec_send_frame(mAvCodecContext, frame );
while (ret >= 0)
{
ret = avcodec_receive_packet(mAvCodecContext, mAvPacket);
if (ret == AVERROR(EAGAIN) || ret == AVERROR_EOF )
{
break;
}
else
if (ret < 0) {
return false;
}
av_packet_rescale_ts(mAvPacket, mAvCodecContext->time_base, mAvOutputStream->time_base);
mAvPacket->stream_index = mAvOutputStream->index;
av_interleaved_write_frame(mAvFormatContext, mAvPacket);
av_packet_unref(mAvPacket);
}
return true;
}
void AudioWriter::finish()
{
// Flush by sending a null frame
encodeFrame( nullptr );
av_write_trailer(mAvFormatContext);
}
Since the resultant file contains the recorded music, the code to manipulate the audio data seems to be correct (unless I am overwriting other memory somehow).
The inaccurate duration and bitrate suggest that information concerning time is not being properly managed. I set the pts of the frames using a simple increasing integer. I'm unclear what the code that sets the timestamp and stream index achieves - and whether it's even necessary: I copied it from supposedly working code but I've seen other code without it.
Can anyone see what I'm doing wrong?

The timestamp need to be correct. Set the time_base to 1/sample_rate and increment the timestamp by 1024 each frame. Note: 1024 is aac specific. If you change codecs, you need to change the frame size.

Getting SHOUTcast Meta Data Manually

I would like to scan for SHOUTcast meta data myself. I realise there cool classes such as IcyStreamMeta etc but I would like to know why I cannot see the data myself.
I am using this URL (have tried others too):
http://www.shoutcastunlimited.com:8512/
My understanding is that I should see meta data within the audio stream data - especially when a radio station changes the current tune.
What I've tried to do is output sequences of printable ASCII character to see if I can see keywords such as "StreamTitle" but all I can see if anything is "LAME".
My code below is less than ideal but is there are reason why I am not seeing "StreamTitle" or other meaningful words?
public void retreiveMetadata()
{
try
{
OkHttpClient client = new OkHttpClient();
Request request = new Request.Builder().url( mStreamUrl ).build();
Response response = client.newCall( request ).execute();
mStream = response.body().byteStream();
// This returns 200 as expected
ContextActivity.LogDebugf( "ICY RESPONSE: %d\n", response.code() );
if( abBuffer == null )
abBuffer = new byte[ nBufferSize ];
for( ;; )
{
// Read data INTO the buffer
int nRead = mStream.read( abBuffer, 0, nBufferSize );
//ContextActivity.LogDebugf( "ICY Data Read: %d\n", nRead );
int nPrintableStart = -1;
int nPrintableCount = 0;
for( int i = 0; i < nRead; i ++ )
{
// Look for printable chars only
if( ( abBuffer[ i ] >= ' ' ) && ( abBuffer[ i ] < '~' ) )
{
if( nPrintableStart < 0 )
{
nPrintableStart = i;
nPrintableCount = 0;
}
nPrintableCount ++;
}
else
{
// End of printable range
if( nPrintableCount >= 11 )
{
String sMeta = new String( abBuffer, nPrintableStart, nPrintableCount, "UTF-8" );
ContextActivity.LogDebugf( "ICY[%s]\n", sMeta );
}
nPrintableStart = -1;
nPrintableCount = 0;
}
}
}
}
catch( Exception e )
{
ContextActivity.LogDebugf( "ICY Exception[%s]\n", e.toString() );
}
}
Here are some "LAME" examples:
02-24 16:58:01.570: I/System.out(26965): ICY[LAME3.98.4UUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUU|]
02-24 16:58:01.580: I/System.out(26965): ICY[LAME3.98.4UUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUU|\]
02-24 16:58:01.590: I/System.out(26965): ICY[LAME3.98.4IUb\]

I don't see any place where you are actually requesting metadata. If you don't request it, the server won't send it. Add this header to your request:
Icy-MetaData: 1
In the response, you'll get a header back (assuming the server supports metadata) that says:
Icy-MetaInt: 8192
Whatever that number is (8192 in this case, which is a typical figure) is the number of bytes in between each metadata block.
The first byte in the metadata block indicates the size of the metadata block. If it's 0x00, then there is no metadata and it's back to audio data for the size of the interval. If it says 0x02 or some other non-zero value, multiply that by 16, and that's the number of bytes (NUL [0x00] padded) of text metadata, in a key="value" sort of format. StreamTitle is the only one that's all that meaningful these days. Some streams have been known to include other data, often for internal tracking.

Simple encoder Gstreamer on Android

I just try to make a simple demo with encoder in Gstreamer 1.0. However it does not work. Always return error when calling "push-buffer".
#define BUFFER_SIZE 1228800 //640x480x 4 byte/pixel
static void cb_need_video_data (GstElement *appsrc,guint unused_size, CustomData* user_data)
{
static gboolean white = FALSE;
static GstClockTime timestamp = 0;
GstBuffer *buffer;
guint size = BUFFER_SIZE;
GstFlowReturn ret;
user_data->num_frame++;
if (user_data->num_frame >= TOTAL_FRAMES) {
/* we are EOS, send end-of-stream and remove the source */
g_signal_emit_by_name (user_data->source, "end-of-stream", &ret);
g_print ("start TOTAL_FRAMES");
return ;
}
buffer = gst_buffer_new_allocate (NULL, size, NULL);
gst_buffer_memset (buffer, 0, white ? 0xff : 0x0, size);
white = !white;
GST_BUFFER_PTS (buffer) = timestamp;
GST_BUFFER_DURATION (buffer) = gst_util_uint64_scale_int (1, GST_SECOND, 2);
timestamp += GST_BUFFER_DURATION (buffer);
g_signal_emit_by_name (appsrc, "push-buffer", buffer, &ret);
if (ret != GST_FLOW_OK) {
/* something wrong, stop pushing */
GST_DEBUG("QUIT MAIN LOOP %d",ret);
g_main_loop_quit (user_data->main_loop);
}
}
void gst_set_pipeline(JNIEnv* env, jobject thiz,jstring uri){
CustomData *data = GET_CUSTOM_DATA (env, thiz, custom_data_field_id);
gchar *string = NULL;
string =
g_strdup_printf ("appsrc is-live=true name=testsource "
"queue ! videoconvert ! videoscale "
"add-borders=true ! videorate ! x264enc "
"! mpegtsmux "
"! filesink "
"location=/storage/emulated/0/test.avi sync=false append=true",
data->pipeline = gst_parse_launch (string, NULL);
g_free (string);
if (data->pipeline == NULL) {
g_print ("Bad sink\n");
return;
}
data->vsource = gst_bin_get_by_name (GST_BIN (data->pipeline), "testsource");
GstCaps *caps;
caps = gst_caps_new_simple ("video/x-raw",
"format", G_TYPE_STRING, "RGB",
"width", G_TYPE_INT, 640,
"height", G_TYPE_INT, 480,
"framerate", GST_TYPE_FRACTION, 25, 1,
NULL);
gst_app_src_set_caps(GST_APP_SRC(data->vsource), caps);
g_object_set (G_OBJECT(data->vsource),"stream-type", 0, "format", GST_FORMAT_TIME, NULL);
g_signal_connect (data->vsource, "need-data", G_CALLBACK (cb_need_video_data), data);
}
When I set state to GST_STATE_PLAYING, it go into cb_need_video_data once time and get ret != GST_FLOW_OK . I think the caps and input data don't match, but I don't know why.
I'm just newbie in Gstreamer. Please help me correct above code. Thanks all

Cannot extract file from ZIP archive created on Android (device/OS specific)

I am creating an archive on Android using the code like this:
OutputStream os = new FileOutputStream(zipFile);
ZipOutputStream zos = new ZipOutputStream(new BufferedOutputStream(os));
try
{
zos.setLevel(8);
byte[] buffer = new byte[32768];
for (VFile src : toPack)
{
ZipEntry entry = new ZipEntry(src.name);
zos.putNextEntry(entry);
src.pushToStream(zos, buffer);
src.close();
zos.closeEntry();
}
}
finally
{
zos.close();
}
I found that there's only one compression method available - DEFLATED (there's only STORED alternative available). This means that archive is always compressed with one method.
If I run this code on Android 2.3.4 - I can decompress files in Windows using 7Zip; if I run this on Android 3 (or Samsung Galaxy Tab; not sure who makes it wrong) - 7Zip shows archive list, but cannot decompress file saying Unsupported compression method. At the same time 7Zip shows Deflate as a compression method on file which means it can treat it properly.
Did anyone has this problem?
Thanks.
UDP: Found another topic with similar issue (may be not same though).

The #user1269737's answer is correct; almost. But it only works for a single-file archives.
Below is a code which parses the whole archive.
/**
* Replace wrong local file header byte
* http://sourceforge.net/tracker/?func=detail&aid=3477810&group_id=14481&atid=114481
* Applies to Android API 9-13
* #param zip file
* #throws IOException
*/
private static void fixInvalidZipFile(File zip) throws IOException
{
RandomAccessFile r = new RandomAccessFile(zip, "rw");
try
{
long eocd_offset = findEOCDRecord(r);
if (eocd_offset > 0)
{
r.seek(eocd_offset + 16); // offset of first CDE in EOCD
long cde_offset = readInt(r); // read offset of first Central Directory Entry
long lfh_offset = 0;
long fskip, dskip;
while (true)
{
r.seek(cde_offset);
if (readInt(r) != CDE_SIGNATURE) // got off sync!
return;
r.seek(cde_offset + 20); // compressed file size offset
fskip = readInt(r);
// fix the header
//
r.seek(lfh_offset + 7);
short localFlagsHi = r.readByte(); // hi-order byte of local header flags (general purpose)
r.seek(cde_offset + 9);
short realFlagsHi = r.readByte(); // hi-order byte of central directory flags (general purpose)
if (localFlagsHi != realFlagsHi)
{ // in latest versions this bug is fixed, so we're checking is bug exists.
r.seek(lfh_offset + 7);
r.write(realFlagsHi);
}
// calculate offset of next Central Directory Entry
//
r.seek(cde_offset + 28); // offset of variable CDE parts length in CDE
dskip = 46; // length of fixed CDE part
dskip += readShort(r); // file name
dskip += readShort(r); // extra field
dskip += readShort(r); // file comment
cde_offset += dskip;
if (cde_offset >= eocd_offset) // finished!
break;
// calculate offset of next Local File Header
//
r.seek(lfh_offset + 26); // offset of variable LFH parts length in LFH
fskip += readShort(r); // file name
fskip += readShort(r); // extra field
fskip += 30; // length of fixed LFH part
fskip += 16; // length of Data Descriptor (written after file data)
lfh_offset += fskip;
}
}
}
finally
{
r.close();
}
}
//http://www.pkware.com/documents/casestudies/APPNOTE.TXT
private static final int LFH_SIGNATURE = 0x04034b50;
private static final int DD_SIGNATURE = 0x08074b50;
private static final int CDE_SIGNATURE = 0x02014b50;
private static final int EOCD_SIGNATURE = 0x06054b50;
/** Find an offset of End Of Central Directory record in file */
private static long findEOCDRecord(RandomAccessFile f) throws IOException
{
long result = f.length() - 22; // 22 is minimal EOCD record length
while (result > 0)
{
f.seek(result);
if (readInt(f) == EOCD_SIGNATURE) return result;
result--;
}
return -1;
}
/** Read a 4-byte integer from file converting endianness. */
private static int readInt(RandomAccessFile f) throws IOException
{
int result = 0;
result |= f.read();
result |= (f.read() << 8);
result |= (f.read() << 16);
result |= (f.read() << 24);
return result;
}
/** Read a 2-byte integer from file converting endianness. */
private static short readShort(RandomAccessFile f) throws IOException
{
short result = 0;
result |= f.read();
result |= (f.read() << 8);
return result;
}

Need to be updated. It fixes single-file-zip. You have to look following sequence in zip file { 0, 0x08, 0x08, 0x08, 0 } and replace it to { 0, 0x08, 0x00, 0x08, 0 }
/**
* Replace wrong byte http://sourceforge.net/tracker/?func=detail&aid=3477810&group_id=14481&atid=114481
* #param zip file
* #throws IOException
*/
private static void replaceWrongZipByte(File zip) throws IOException {
RandomAccessFile r = new RandomAccessFile(zip, "rw");
int flag = Integer.parseInt("00001000", 2); //wrong byte
r.seek(7);
int realFlags = r.read();
if( (realFlags & flag) > 0) { // in latest versions this bug is fixed, so we're checking is bug exists.
r.seek(7);
flag = (~flag & 0xff);
// removing only wrong bit, other bits remains the same.
r.write(realFlags & flag);
}
r.close();
}
Update version :
Following code removes all wrong bytes in ZIP.
KMPMatch.java easy to find in google
public static void replaceWrongBytesInZip(File zip) throws IOException {
byte find[] = new byte[] { 0, 0x08, 0x08, 0x08, 0 };
int index;
while( (index = indexOfBytesInFile(zip,find)) != -1) {
replaceWrongZipByte(zip, index + 2);
}
}
private static int indexOfBytesInFile(File file,byte find[]) throws IOException {
byte fileContent[] = new byte[(int) file.length()];
FileInputStream fin = new FileInputStream(file);
fin.read(fileContent);
fin.close();
return KMPMatch.indexOf(fileContent, find);
}
/**
* Replace wrong byte http://sourceforge.net/tracker/?func=detail&aid=3477810&group_id=14481&atid=114481
* #param zip file
* #throws IOException
*/
private static void replaceWrongZipByte(File zip, int wrongByteIndex) throws IOException {
RandomAccessFile r = new RandomAccessFile(zip, "rw");
int flag = Integer.parseInt("00001000", 2);
r.seek(wrongByteIndex);
int realFlags = r.read();
if( (realFlags & flag) > 0) { // in latest versions this bug is fixed, so we're checking is bug exists.
r.seek(wrongByteIndex);
flag = (~flag & 0xff);
// removing only wrong bit, other bits remains the same.
r.write(realFlags & flag);
}
r.close();
}

Android append files to a zip file without having to re-write the entire zip file?

How can I append files to an existing zip file? I already have the code that can create a zip file and it works great except for one big problem. The way it works now, the user takes a bunch of pictures, and at the end, all the pictures get added to a zip file, which can take quite a while if you take enough pictures. :-( So I'm thinking, I have a very good and efficient solution. As the pictures are taken, I will simply add the each new picture to the zip file right after it's taken. Then when they're done taking pictures, finish up the zip file so it's usable and export it. :-)
The problem is, I can not get it to add files to an existing zip file. :-( Here's what I have so far. Also, please keep in mind, this is just a proof of concept, I do understand that re-initializing everything for every iteration of the for loop is very dumb. Each iteration of the loop is supposed to represent another file being added which will most likely be a long time later, maybe even an hour later, which is why I have everything resetting each iteration, because the app will be shut down between adding files. If I can get this working, then I will actually ditch the for loop and put this code into a function that gets called every time a picture gets taken. :-)
try {
for(int i=0; i < _files.size(); i++) {
//beginning of initial setup stuff
BufferedInputStream origin = null;
FileOutputStream dest = new FileOutputStream(_zipFile,false);
ZipOutputStream out = new ZipOutputStream(new BufferedOutputStream(dest));
byte data[] = new byte[BUFFER];
out.setLevel(0); //I added this because it makes it not compress the data
//at all and I hoped that it would allow the zip to be appended to
//end of initial setup stuff
//beginning of old for loop
Log.v("Compress", "Adding: " + _files[i]);
FileInputStream fi = new FileInputStream(_files[i]);
origin = new BufferedInputStream(fi, BUFFER);
ZipEntry entry = new ZipEntry(_files[i].substring(_files[i].lastIndexOf("/") + 1));
out.putNextEntry(entry);
int count;
while ((count = origin.read(data, 0, BUFFER)) != -1) {
out.write(data, 0, count);
}
origin.close();
//end of for old loop
//beginning of finishing stuff
out.close();
//end of finishing stuff
}
} catch(Exception e) {
Log.e("ZipCreation", "Error writing zip", e);
e.printStackTrace();
}
Also, I have experimented around with
FileOutputStream dest = new FileOutputStream(_zipFile,true);
If you notice, I set append to true, which will actually append the data to an existing file. And what's interesting is, it actually does append the data to the original file, however, after the file gets extracted on my computer, the last file written is all that gets extracted, which is bad. :-( So is there some way to start writing a zip file, and then later, add on to it, and finish up the zip file? I've even thought about possibly taking ZipOutputStream and modifying it to fit this model that I need. It should logically be possible somehow? :-)
Thanks in advance for the help! :-D
-Jared

Ok, thanks for all your suggestions, but I was able to get it working like I wanted.... it CAN be done, you CAN add files after closing the file, as long as you save your place!!! :-D
Here's how I was able to get it going working:
try {
for(int i=0; i < _files.size(); i++) {
//beginning of initial setup stuff
BufferedInputStream origin = null;
FileOutputStream dest = new FileOutputStream(_zipFile,true);
ZipOutputStreamNew out = new ZipOutputStreamNew(new BufferedOutputStream(dest));
byte data[] = new byte[BUFFER];
if (havePreviousData) {
out.setWritten(tempWritten);
out.setXentries(tempXentries);
}
//end of initial setup stuff
//beginning of for loop
Log.i("Compress", "Adding: " + _files.get(i));
FileInputStream fi = new FileInputStream(_files.get(i));
origin = new BufferedInputStream(fi, BUFFER);
TempString = _files.get(i).substring(_files.get(i).lastIndexOf("/") + 1);
ZipEntry entry = new ZipEntry(_paths.get(i) + TempString);
out.putNextEntry(entry);
int count;
while ((count = origin.read(data, 0, BUFFER)) != -1) {
out.write(data, 0, count);
}
origin.close();
out.closeEntry();
//end of for loop
//beginning of finishing stuff
if (i == (_files.size()-1)) {
//it's the last record so we should finish it off
out.closeAndFinish();
} else {
//close the file, but don't write the Central Directory
//first, back up where the zip file was...
tempWritten = out.getWritten();
tempXentries = out.getXentries();
havePreviousData = true;
//now close the file
out.close();
}
//end of finishing stuff
}
//zip succeeded
} catch(Exception e) {
Log.e("ZipCreation", "Error writing zip", e);
e.printStackTrace();
}
Also, keep in mind, this is not the only code I had to do. I also had to make my own copy of ZipOutputStream so that I could expose the following functions that I created within my ZipOutputStreamNew class....
getWritten()
getXentries()
as well as
setWritten(long mWritten)
setXentries(Vector<XEntry> mXEntries)
For the most part, all this does, is it starts writing like normal, then, instead of closing like normal, it backs up those two variables, and then for the next iteration, it restores just those variables.
Let me know if you have any questions about all this, but I knew it would work, all it has to do is save where it was. :-D
Thanks again for all the help everybody! :-)
At Raj's request, here is the source code for ZipOutputStreamNew:
/**
* This class implements an output stream filter for writing files in the
* ZIP file format. Includes support for both compressed and uncompressed
* entries.
*
* #author David Connelly
* #version %I%, %G%
*/
public class ZipOutputStreamNew extends DeflaterOutputStream implements ZipConstants {
public static class XEntry {
public final ZipEntry entry;
public final long offset;
public final int flag;
public XEntry(ZipEntry entry, long offset) {
this.entry = entry;
this.offset = offset;
this.flag = (entry.getMethod() == DEFLATED &&
(entry.getSize() == -1 ||
entry.getCompressedSize() == -1 ||
entry.getCrc() == -1))
// store size, compressed size, and crc-32 in data descriptor
// immediately following the compressed entry data
? 8
// store size, compressed size, and crc-32 in LOC header
: 0;
}
}
private XEntry current;
private Vector<XEntry> xentries = new Vector<XEntry>();
private HashSet<String> names = new HashSet<String>();
private CRC32 crc = new CRC32();
private long written = 0;
private long locoff = 0;
private String comment;
private int method = DEFLATED;
private boolean finished;
private boolean closed = false;
private boolean closeItPermanently = false;
private static int version(ZipEntry e) throws ZipException {
switch (e.getMethod()) {
case DEFLATED: return 20;
case STORED: return 10;
default: throw new ZipException("unsupported compression method");
}
}
/**
* Checks to make sure that this stream has not been closed.
*/
private void ensureOpen() throws IOException {
if (closed) {
throw new IOException("Stream closed");
}
}
/**
* Compression method for uncompressed (STORED) entries.
*/
public static final int STORED = ZipEntry.STORED;
/**
* Compression method for compressed (DEFLATED) entries.
*/
public static final int DEFLATED = ZipEntry.DEFLATED;
/**
* Creates a new ZIP output stream.
* #param out the actual output stream
*/
public ZipOutputStreamNew(OutputStream out) {
super(out, new Deflater(Deflater.DEFAULT_COMPRESSION, true));
usesDefaultDeflater = true;
}
/**
* Sets the ZIP file comment.
* #param comment the comment string
* #exception IllegalArgumentException if the length of the specified
* ZIP file comment is greater than 0xFFFF bytes
*/
public void setComment(String comment) {
if (comment != null && comment.length() > 0xffff/3
&& getUTF8Length(comment) > 0xffff) {
throw new IllegalArgumentException("ZIP file comment too long.");
}
this.comment = comment;
}
/**
* Sets the default compression method for subsequent entries. This
* default will be used whenever the compression method is not specified
* for an individual ZIP file entry, and is initially set to DEFLATED.
* #param method the default compression method
* #exception IllegalArgumentException if the specified compression method
* is invalid
*/
public void setMethod(int method) {
if (method != DEFLATED && method != STORED) {
throw new IllegalArgumentException("invalid compression method");
}
this.method = method;
}
/**
* Sets the compression level for subsequent entries which are DEFLATED.
* The default setting is DEFAULT_COMPRESSION.
* #param level the compression level (0-9)
* #exception IllegalArgumentException if the compression level is invalid
*/
public void setLevel(int level) {
def.setLevel(level);
}
/**
* Begins writing a new ZIP file entry and positions the stream to the
* start of the entry data. Closes the current entry if still active.
* The default compression method will be used if no compression method
* was specified for the entry, and the current time will be used if
* the entry has no set modification time.
* #param e the ZIP entry to be written
* #exception ZipException if a ZIP format error has occurred
* #exception IOException if an I/O error has occurred
*/
public void putNextEntry(ZipEntry e) throws IOException {
ensureOpen();
if (current != null) {
closeEntry(); // close previous entry
}
if (e.getTime() == -1) {
e.setTime(System.currentTimeMillis());
}
if (e.getMethod() == -1) {
e.setMethod(method); // use default method
}
switch (e.getMethod()) {
case DEFLATED:
break;
case STORED:
// compressed size, uncompressed size, and crc-32 must all be
// set for entries using STORED compression method
if (e.getSize() == -1) {
e.setSize(e.getCompressedSize());
} else if (e.getCompressedSize() == -1) {
e.setCompressedSize(e.getSize());
} else if (e.getSize() != e.getCompressedSize()) {
throw new ZipException(
"STORED entry where compressed != uncompressed size");
}
if (e.getSize() == -1 || e.getCrc() == -1) {
throw new ZipException(
"STORED entry missing size, compressed size, or crc-32");
}
break;
default:
throw new ZipException("unsupported compression method");
}
if (! names.add(e.getName())) {
throw new ZipException("duplicate entry: " + e.getName());
}
current = new XEntry(e, written);
xentries.add(current);
writeLOC(current);
}
/**
* Closes the current ZIP entry and positions the stream for writing
* the next entry.
* #exception ZipException if a ZIP format error has occurred
* #exception IOException if an I/O error has occurred
*/
public void closeEntry() throws IOException {
ensureOpen();
if (current != null) {
ZipEntry e = current.entry;
switch (e.getMethod()) {
case DEFLATED:
def.finish();
while (!def.finished()) {
deflate();
}
if ((current.flag & 8) == 0) {
// verify size, compressed size, and crc-32 settings
if (e.getSize() != def.getBytesRead()) {
throw new ZipException(
"invalid entry size (expected " + e.getSize() +
" but got " + def.getBytesRead() + " bytes)");
}
if (e.getCompressedSize() != def.getBytesWritten()) {
throw new ZipException(
"invalid entry compressed size (expected " +
e.getCompressedSize() + " but got " + def.getBytesWritten() + " bytes)");
}
if (e.getCrc() != crc.getValue()) {
throw new ZipException(
"invalid entry CRC-32 (expected 0x" +
Long.toHexString(e.getCrc()) + " but got 0x" +
Long.toHexString(crc.getValue()) + ")");
}
} else {
e.setSize(def.getBytesRead());
e.setCompressedSize(def.getBytesWritten());
e.setCrc(crc.getValue());
writeEXT(e);
}
def.reset();
written += e.getCompressedSize();
break;
case STORED:
// we already know that both e.size and e.csize are the same
if (e.getSize() != written - locoff) {
throw new ZipException(
"invalid entry size (expected " + e.getSize() +
" but got " + (written - locoff) + " bytes)");
}
if (e.getCrc() != crc.getValue()) {
throw new ZipException(
"invalid entry crc-32 (expected 0x" +
Long.toHexString(e.getCrc()) + " but got 0x" +
Long.toHexString(crc.getValue()) + ")");
}
break;
default:
throw new ZipException("invalid compression method");
}
crc.reset();
current = null;
}
}
/**
* Writes an array of bytes to the current ZIP entry data. This method
* will block until all the bytes are written.
* #param b the data to be written
* #param off the start offset in the data
* #param len the number of bytes that are written
* #exception ZipException if a ZIP file error has occurred
* #exception IOException if an I/O error has occurred
*/
public synchronized void write(byte[] b, int off, int len)
throws IOException
{
ensureOpen();
if (off < 0 || len < 0 || off > b.length - len) {
throw new IndexOutOfBoundsException();
} else if (len == 0) {
return;
}
if (current == null) {
throw new ZipException("no current ZIP entry");
}
ZipEntry entry = current.entry;
switch (entry.getMethod()) {
case DEFLATED:
super.write(b, off, len);
break;
case STORED:
written += len;
if (written - locoff > entry.getSize()) {
throw new ZipException(
"attempt to write past end of STORED entry");
}
out.write(b, off, len);
break;
default:
throw new ZipException("invalid compression method");
}
crc.update(b, off, len);
}
/**
* Finishes writing the contents of the ZIP output stream without closing
* the underlying stream. Use this method when applying multiple filters
* in succession to the same output stream.
* #exception ZipException if a ZIP file error has occurred
* #exception IOException if an I/O exception has occurred
*/
public void finish() throws IOException {
ensureOpen();
if (finished) {
return;
}
if (current != null) {
closeEntry();
}
if (xentries.size() < 1) {
throw new ZipException("ZIP file must have at least one entry");
}
if (closeItPermanently) {
// write central directory
long off = written;
for (XEntry xentry : xentries)
writeCEN(xentry);
writeEND(off, written - off);
finished = true;
//Log.e("ZipOutputStreamNew", "I just ran wrote the Central Directory Jared!");
}
//Log.e("ZipOutputStreamNew", "I just ran finish() Jared!");
}
/**
* Gets the value of the "xentries" variable (for later use)
* #return
*/
public Vector<XEntry> getXentries() {
return xentries;
//TODO convert this to primitive data types
}
/**
* Gets the value of the "written" variable (for later use)
* #return
*/
public long getWritten() {
return written;
}
/**
* Sets the value of the "xentries" variable (for later use)
* #return
*/
public void setXentries(Vector<XEntry> mXEntries) {
xentries = mXEntries;
//TODO convert this to primitive data types
}
/**
* Sets the value of the "written" variable (for later use)
* #return
*/
public void setWritten(long mWritten) {
written = mWritten;
}
/**
* Closes the ZIP output stream as well as the stream being filtered.
* #exception ZipException if a ZIP file error has occurred
* #exception IOException if an I/O error has occurred
*/
public void closeAndFinish() throws IOException {
if (!closed) {
closeItPermanently=true;
super.close();
closed = true;
}
}
/**
* Used to close the ZIP output stream as well as the stream being filtered.
* instead it does nothing :-P
* #exception ZipException if a ZIP file error has occurred
* #exception IOException if an I/O error has occurred
*/
public void close() throws IOException {
if (!closed) {
closeItPermanently=false;
super.close();
closed = true;
}
}
/*
* Writes local file (LOC) header for specified entry.
*/
private void writeLOC(XEntry xentry) throws IOException {
ZipEntry e = xentry.entry;
int flag = xentry.flag;
writeInt(LOCSIG); // LOC header signature
writeShort(version(e)); // version needed to extract
writeShort(flag); // general purpose bit flag
writeShort(e.getMethod()); // compression method
writeInt(e.getTime()); // last modification time
if ((flag & 8) == 8) {
// store size, uncompressed size, and crc-32 in data descriptor
// immediately following compressed entry data
writeInt(0);
writeInt(0);
writeInt(0);
} else {
writeInt(e.getCrc()); // crc-32
writeInt(e.getCompressedSize()); // compressed size
writeInt(e.getSize()); // uncompressed size
}
byte[] nameBytes = getUTF8Bytes(e.getName());
writeShort(nameBytes.length);
writeShort(e.getExtra() != null ? e.getExtra().length : 0);
writeBytes(nameBytes, 0, nameBytes.length);
if (e.getExtra() != null) {
writeBytes(e.getExtra(), 0, e.getExtra().length);
}
locoff = written;
}
/*
* Writes extra data descriptor (EXT) for specified entry.
*/
private void writeEXT(ZipEntry e) throws IOException {
writeInt(EXTSIG); // EXT header signature
writeInt(e.getCrc()); // crc-32
writeInt(e.getCompressedSize()); // compressed size
writeInt(e.getSize()); // uncompressed size
}
/*
* Write central directory (CEN) header for specified entry.
* REMIND: add support for file attributes
*/
private void writeCEN(XEntry xentry) throws IOException {
ZipEntry e = xentry.entry;
int flag = xentry.flag;
int version = version(e);
writeInt(CENSIG); // CEN header signature
writeShort(version); // version made by
writeShort(version); // version needed to extract
writeShort(flag); // general purpose bit flag
writeShort(e.getMethod()); // compression method
writeInt(e.getTime()); // last modification time
writeInt(e.getCrc()); // crc-32
writeInt(e.getCompressedSize()); // compressed size
writeInt(e.getSize()); // uncompressed size
byte[] nameBytes = getUTF8Bytes(e.getName());
writeShort(nameBytes.length);
writeShort(e.getExtra() != null ? e.getExtra().length : 0);
byte[] commentBytes;
if (e.getComment() != null) {
commentBytes = getUTF8Bytes(e.getComment());
writeShort(commentBytes.length);
} else {
commentBytes = null;
writeShort(0);
}
writeShort(0); // starting disk number
writeShort(0); // internal file attributes (unused)
writeInt(0); // external file attributes (unused)
writeInt(xentry.offset); // relative offset of local header
writeBytes(nameBytes, 0, nameBytes.length);
if (e.getExtra() != null) {
writeBytes(e.getExtra(), 0, e.getExtra().length);
}
if (commentBytes != null) {
writeBytes(commentBytes, 0, commentBytes.length);
}
}
/*
* Writes end of central directory (END) header.
*/
private void writeEND(long off, long len) throws IOException {
int count = xentries.size();
writeInt(ENDSIG); // END record signature
writeShort(0); // number of this disk
writeShort(0); // central directory start disk
writeShort(count); // number of directory entries on disk
writeShort(count); // total number of directory entries
writeInt(len); // length of central directory
writeInt(off); // offset of central directory
if (comment != null) { // zip file comment
byte[] b = getUTF8Bytes(comment);
writeShort(b.length);
writeBytes(b, 0, b.length);
} else {
writeShort(0);
}
}
/*
* Writes a 16-bit short to the output stream in little-endian byte order.
*/
private void writeShort(int v) throws IOException {
OutputStream out = this.out;
out.write((v >>> 0) & 0xff);
out.write((v >>> 8) & 0xff);
written += 2;
}
/*
* Writes a 32-bit int to the output stream in little-endian byte order.
*/
private void writeInt(long v) throws IOException {
OutputStream out = this.out;
out.write((int)((v >>> 0) & 0xff));
out.write((int)((v >>> 8) & 0xff));
out.write((int)((v >>> 16) & 0xff));
out.write((int)((v >>> 24) & 0xff));
written += 4;
}
/*
* Writes an array of bytes to the output stream.
*/
private void writeBytes(byte[] b, int off, int len) throws IOException {
super.out.write(b, off, len);
written += len;
}
/*
* Returns the length of String's UTF8 encoding.
*/
static int getUTF8Length(String s) {
int count = 0;
for (int i = 0; i < s.length(); i++) {
char ch = s.charAt(i);
if (ch <= 0x7f) {
count++;
} else if (ch <= 0x7ff) {
count += 2;
} else {
count += 3;
}
}
return count;
}
/*
* Returns an array of bytes representing the UTF8 encoding
* of the specified String.
*/
private static byte[] getUTF8Bytes(String s) {
char[] c = s.toCharArray();
int len = c.length;
// Count the number of encoded bytes...
int count = 0;
for (int i = 0; i < len; i++) {
int ch = c[i];
if (ch <= 0x7f) {
count++;
} else if (ch <= 0x7ff) {
count += 2;
} else {
count += 3;
}
}
// Now return the encoded bytes...
byte[] b = new byte[count];
int off = 0;
for (int i = 0; i < len; i++) {
int ch = c[i];
if (ch <= 0x7f) {
b[off++] = (byte)ch;
} else if (ch <= 0x7ff) {
b[off++] = (byte)((ch >> 6) | 0xc0);
b[off++] = (byte)((ch & 0x3f) | 0x80);
} else {
b[off++] = (byte)((ch >> 12) | 0xe0);
b[off++] = (byte)(((ch >> 6) & 0x3f) | 0x80);
b[off++] = (byte)((ch & 0x3f) | 0x80);
}
}
return b;
}
}

I believe it can't be done right now with the current API.
You can append data to any file, but that does not mean that you will end up with the right file format. A .zip file is not like a .tar file, and the compression requires imposes restrictions to the handling of the files (file positions, EOF, etc.). If you consider the structure of the file format (taken from wikipedia here) you will understand why just appending does not work.
There is a library called TrueZip that could work, although I do not know if it supports android. Take a look at this answer in another similar question:
Appending files to a zip file with Java .
Also, as a workaround, you could create individual .zip files and append them as a tarball (file format here). Compression might be slighty worst, but it would be much better in terms of time efficiency.
Update based on the comments (and possible solution)
You could separate the addition to each ZipEntry and leave the ZipOutputStream object open as long as you are still taking pictures. I can see risks with that approach, though, as any problem with the app while still taking pictures (a force close, run out of battery, etc) may render the whole file unusable. You will need to make sure to use the right try/catch/finally blocks to close the file and call closeZip() upon events such as onClose() and onDestroy(), but the idea would be the following:
import java.io.*;
import java.util.zip.*;
public class Zip {
static final int BUFFER = 2048;
ZipOutputStream out;
byte data[];
public Zip(String name) {
FileOutputStream dest = new FileOutputStream(name);
out = new ZipOutputStream(new BufferedOutputStream(dest));
data = new byte[BUFFER];
}
public void addFile (String name) {
FileInputStream fi = new FileInputStream(name);
BufferedInputStream origin = new BufferedInputStream(fi, BUFFER);
ZipEntry entry = new ZipEntry(name);
out.putNextEntry(entry);
int count;
while((count = origin.read(data, 0, BUFFER)) != -1) {
out.write(data, 0, count);
}
origin.close();
}
public void closeZip () {
out.close();
}
}