Related
I need to pass the FFMPEG 'raw' data back to my JAVA code in order to display it on the screen.
I have a native method that deals with FFMPEG and after that calls a method in java that takes Byte[] (so far) as an argument.
Byte Array that is passed is read by JAVA but when doing BitmapFactory.decodeByteArray(bitmap, 0, bitmap.length); it returns null. I have printed out the array and I get 200k of elements (which are expected), but cannot be decoded. So far what I'm doing is taking data from AvFrame->data casting it to unsigned char * and then casting that to jbyterArray. After all the casting, I pass the jbyteArray as argument to my JAVA method. Is there something I'm missing here? Why won't BitmapFactory decode the array into an image for displaying?
EDIT 1.0
Currently I am trying to obtain my image via
public void setImage(ByteBuffer bmp) {
bmp.rewind();
Bitmap bitmap = Bitmap.createBitmap(1920, 1080, Bitmap.Config.ARGB_8888);
bitmap.copyPixelsFromBuffer(bmp);
runOnUiThread(() -> {
ImageView imgViewer = findViewById(R.id.mSurfaceView);
imgViewer.setImageBitmap(bitmap);
});
}
But I keep getting an exception
JNI DETECTED ERROR IN APPLICATION: JNI NewDirectByteBuffer called with pending exception java.lang.RuntimeException: Buffer not large enough for pixels
at void android.graphics.Bitmap.copyPixelsFromBuffer(java.nio.Buffer) (Bitmap.java:657)
at void com.example.asmcpp.MainActivity.setSurfaceImage(java.nio.ByteBuffer)
Edit 1.1
So, here is the full code that is executing every time there is a frame incoming. Note that the ByteBuffer is created and passed from within this method
void VideoClientInterface::onEncodedFrame(video::encoded_frame_t &encodedFrame) {
AVFrame *filt_frame = av_frame_alloc();
auto frame = std::shared_ptr<video::encoded_frame_t>(new video::encoded_frame_t,
[](video::encoded_frame_t *p) { if (p) delete p; });
if (frame) {
frame->size = encodedFrame.size;
frame->ssrc = encodedFrame.ssrc;
frame->width = encodedFrame.width;
frame->height = encodedFrame.height;
frame->dataType = encodedFrame.dataType;
frame->timestamp = encodedFrame.timestamp;
frame->frameIndex = encodedFrame.frameIndex;
frame->isKeyFrame = encodedFrame.isKeyFrame;
frame->isDroppable = encodedFrame.isDroppable;
frame->data = new char[frame->size];
if (frame->data) {
memcpy(frame->data, encodedFrame.data, frame->size);
AVPacket packet;
av_init_packet(&packet);
packet.dts = AV_NOPTS_VALUE;
packet.pts = encodedFrame.timestamp;
packet.data = (uint8_t *) encodedFrame.data;
packet.size = encodedFrame.size;
int ret = avcodec_send_packet(m_avCodecContext, &packet);
if (ret == 0) {
ret = avcodec_receive_frame(m_avCodecContext, m_avFrame);
if (ret == 0) {
m_transform = sws_getCachedContext(
m_transform, // previous context ptr
m_avFrame->width, m_avFrame->height, AV_PIX_FMT_YUV420P, // src
m_avFrame->width, m_avFrame->height, AV_PIX_FMT_RGB24, // dst
SWS_BILINEAR, nullptr, nullptr, nullptr // options
);
auto decodedFrame = std::make_shared<video::decoded_frame_t>();
decodedFrame->width = m_avFrame->width;
decodedFrame->height = m_avFrame->height;
decodedFrame->size = m_avFrame->width * m_avFrame->height * 3;
decodedFrame->timeStamp = m_avFrame->pts;
decodedFrame->data = new unsigned char[decodedFrame->size];
if (decodedFrame->data) {
uint8_t *dstSlice[] = {decodedFrame->data,
0,
0};// outFrame.bits(), outFrame.bits(), outFrame.bits()
const int dstStride[] = {decodedFrame->width * 3, 0, 0};
sws_scale(m_transform, m_avFrame->data, m_avFrame->linesize,
0, m_avFrame->height, dstSlice, dstStride);
auto m_rawData = decodedFrame->data;
auto len = strlen(reinterpret_cast<char *>(m_rawData));
if (frameCounter == 10) {
jobject newArray = GetJniEnv()->NewDirectByteBuffer(m_rawData, len);
GetJniEnv()->CallVoidMethod(m_obj, setSurfaceImage, newArray);
frameCounter = 0;
}
frameCounter++;
}
} else {
av_packet_unref(&packet);
}
} else {
av_packet_unref(&packet);
}
}
}
}
I am not entirely sure I am even doing that part correctly. If you see any errors in this, feel free to point them out.
You cannot cast native byte arrays to jbyteArray and expect it to work. A byte[] is an actual object with length field, a reference count, and so on.
Use NewDirectByteBuffer instead to wrap your byte buffer into a Java ByteBuffer, from where you can grab the actual byte[] using .array().
Note that this JNI operation is relatively expensive, so if you expect to do this on a per-frame basis, you might want to pre-allocate some bytebuffers and tell FFmpeg to write directly into those buffers.
I'm trying to make app that transmits SuperpoweredAndroidAudioIO buffer from one android device to another. With the following code I've managed to send and receive short int buffer from the audio callback. However on the receiving side sound gets very distorted during playback.
Note: For brevity I have not included some code that doesen't seem to have to do with issue, including socket initialisation related functions. If needed I can add the code.
Sending side:
Mic.cpp
static bool SendBuffer(
int sd,
sockaddr_in address,
short int *buffer,
size_t bufferSize) {
// Send data buffer to the socket
ssize_t sentSize = sendto(sd,
buffer,
bufferSize,
0,
(struct sockaddr*)&address,
sizeof address);
// If send is failed
if (sentSize == -1){
__android_log_print(ANDROID_LOG_INFO, "Sent size ", "%i error %i",
sentSize , errno);
}
else if (sentSize > 0) {
__android_log_print(ANDROID_LOG_INFO, "DatagramSent : ", "%hi size: %hi",
buffer, sentSize);
}
return true;
}
// audio callback
static bool micProcessing(
void *clientdata,
short int *audioInputOutput,
int numberOfSamples,
int __unused samplerate) {
return SendBuffer(micSocket, dsocket, audioInputOutput, numberOfSamples);
}
// Constructor
Mic::Mic(JNIEnv *env,
unsigned int samplerate,
unsigned int buffersize,
unsigned int port){
micSocket = NewUdpSocket(env);
dsocket = initDestinationSocket(port); // where to send
__android_log_write(ANDROID_LOG_DEBUG, "Sockets", "initialised");
// init IO
microphone = new SuperpoweredAndroidAudioIO(samplerate,
buffersize,
true,
false,
micProcessing,
this,
-1,
SL_ANDROID_STREAM_MEDIA,
buffersize*2);
__android_log_write(ANDROID_LOG_DEBUG, "Mic", "initialised");
}
Receiving side consists of 2 parts: Mixer and Channel
Mixer.cpp
//audio callback
static bool mainprocess(
void *clientdata,
short int *audioInputOutput,
int numberOfSamples,
int __unused samplerate) {
return ((Mixer*)clientdata)->processMain(audioInputOutput, numberOfSamples);
}
// Setting up Mixer
Mixer::Mixer(JNIEnv *env,unsigned int samplerate, unsigned int buffersize) {
//Short int buffers for recieving
channel1 = new Channel(samplerate,buffersize);
//output buffer
outputFloat = ((float *)memalign(16, (buffersize + 16) * sizeof(float) * 2));
//volumes
outputLevel = 0.5f;
channel1level = 0.2f;
channel2level = 0.2f;
channel3level = 0.2f;
channel4level = 0.2f;
mainmixer = new SuperpoweredMonoMixer();
__android_log_print(ANDROID_LOG_INFO, "Mixer", " Created");
main = new SuperpoweredAndroidAudioIO(
samplerate,
buffersize,
false,
true,
mainprocess,
this,
-1,
SL_ANDROID_STREAM_MEDIA,
buffersize*2);
__android_log_write(ANDROID_LOG_INFO, "Main AudioIO created", " ");
main->stop();
SuperpoweredCPU::setSustainedPerformanceMode(true); // Prevents CPU drops
}
// processing.
bool Mixer::processMain(short int *outputbuffer, unsigned int numberOfSamples{
// a bit awkward
channel1->returnFloatBuffer();
inputs[0] = channel1->floatBuffer;
inputs[1] = NULL;
inputs[2] = NULL;
inputs[3] = NULL;
__android_log_print(ANDROID_LOG_INFO, "Channels are inside", " of mixer");
inputLevels[0] = channel1level;
inputLevels[1] = channel2level;
inputLevels[2] = channel3level;
inputLevels[3] = channel4level;
mainmixer->process(inputs,
outputFloat,
inputLevels,
outputLevel,
numberOfSamples);
SuperpoweredFloatToShortInt(outputFloat, outputbuffer, numberOfSamples);
return true;
}
Channel.cpp
//receiving buffer
static bool ReceiveDatagramFromSocket(int sd, short int *buffer, size_t bufferSize) {
ssize_t recvSize = recvfrom(sd, buffer, bufferSize, 0, NULL, NULL);
if (-1 == recvSize){ // If failed
__android_log_print(ANDROID_LOG_INFO, "receive failed", " %i ", errno);
}
else {
// If data is received
if (recvSize > 0) {
__android_log_print(ANDROID_LOG_INFO, "Received"," %i bytes: %hi", recvSize, buffer);
}
}
return true;
}
// init channel
Channel::Channel(unsigned int samplerate, unsigned int buffersize){
socketIn = NewUdpSocket();
BindSocketToPort(socketIn);
samplerRate = samplerate;
bufferSize = buffersize;
shortIntBuffer = (short int *)malloc((buffersize + 16) * sizeof(short int)*4);
floatBuffer = (float *)memalign(16, (buffersize + 16) * sizeof(float) * 2);
bandEQ = new Superpowered3BandEQ(samplerate);
bandEQ->enable(true);
__android_log_print(ANDROID_LOG_INFO, "Channel ", "created");
}
// this function is called from Mixer.cpp
void Channel::returnFloatBuffer(){
ReceiveDatagramFromSocket(socketIn, shortIntBuffer, bufferSize);
Converting the 16-bit integer samples to 32-bit floating point.
SuperpoweredShortIntToFloat(shortIntBuffer, floatBuffer, bufferSize, 1);
bandEQ->process(floatBuffer, floatBuffer, bufferSize );
__android_log_print(ANDROID_LOG_INFO, "EQ", " processing");
}
At first I thought that because AudioIO on both sided get initialised with different buffer sizes (different devices 240 and 512), but then I tried to hardcode 512 into both of them but it didn't help.
I also tried to increase buffer sizes in sendto() and recvfrom() up to 4096 and it made sound more recognizable but still too distorted.
I should also add that I'm a newbie in C++ and I stuck to 'naive' and 'whatever works' approaches which got me this far.
I want to understand whether I'm on the right track and what approach should I take in order to transmit audio without distortion.
There are two major problems with your approach:
Blocking functions, such as networking should be avoided from the audio processing callback. You need to perform networking (on both sides) from a different thread, and you need some buffering between the audio processing callback and the network thread to pass audio.
You need to "packetize" the transfers, you need to handle network packets on both sides. Network transfer is not fast nor reliable, so you need clever mechanisms to handle this.
In general, the implementation for such audio transfer is much, much more complex to your current code.
I am decoding a h264 video stream with the following code (original guide):
public void configure(Surface surface, int width, int height, ByteBuffer csd0) {
String VIDEO_FORMAT = "video/avc";
if (mConfigured) {
throw new IllegalStateException("Decoder is already configured");
}
MediaFormat format = MediaFormat.createVideoFormat(VIDEO_FORMAT, width, height);
// little tricky here, csd-0 is required in order to configure the codec properly
// it is basically the first sample from encoder with flag: BUFFER_FLAG_CODEC_CONFIG
format.setByteBuffer("csd-0", csd0);
try {
mCodec = MediaCodec.createDecoderByType(VIDEO_FORMAT);
} catch (IOException e) {
throw new RuntimeException("Failed to create codec", e);
}
mCodec.configure(format, surface, null, 0);
mCodec.start();
mConfigured = true;
}
#SuppressWarnings("deprecation")
public void decodeSample(byte[] data, int offset, int size, long presentationTimeUs, int flags) {
if (mConfigured && mRunning) {
int index = mCodec.dequeueInputBuffer(mTimeoutUs);
if (index >= 0) {
ByteBuffer buffer;
// since API 21 we have new API to use
if (Build.VERSION.SDK_INT < Build.VERSION_CODES.LOLLIPOP) {
buffer = mCodec.getInputBuffers()[index];
buffer.clear();
} else {
buffer = mCodec.getInputBuffer(index);
}
if (buffer != null) {
buffer.put(data, offset, size);
mCodec.queueInputBuffer(index, 0, size, presentationTimeUs, flags);
}
}
}
}
#Override
public void run() {
try {
MediaCodec.BufferInfo info = new MediaCodec.BufferInfo();
while (mRunning) {
if (mConfigured) {
int index = mCodec.dequeueOutputBuffer(info, mTimeoutUs);
if (index >= 0) {
// setting true is telling system to render frame onto Surface
mCodec.releaseOutputBuffer(index, true);
if ((info.flags & MediaCodec.BUFFER_FLAG_END_OF_STREAM) == MediaCodec.BUFFER_FLAG_END_OF_STREAM) {
break;
}
}
} else {
// just waiting to be configured, then decode and render
try {
Thread.sleep(10);
} catch (InterruptedException ignore) {
}
}
}
} finally {
if (mConfigured) {
mCodec.stop();
mCodec.release();
}
}
}
I can run this on both my Nexus 6 (api 22) and Samsung galaxy core (api 16) on low and medium quality. However when I switch to high quality (720p) it crashes on the Samsung after about 30 frames (but nothing is rendered to the screen).
E/ACodec﹕ [OMX.qcom.video.decoder.avc] ERROR(0x8000100a)
E/MediaCodec﹕ Codec reported an error. (omx error 0x8000100a, internalError -2147483648)
[...]
W/System.err﹕ java.lang.IllegalStateException
W/System.err﹕ at android.media.MediaCodec.dequeueInputBuffer(Native Method)
W/System.err﹕ at com.test.stream.VideoDecoder$Worker.decodeSample(VideoDecoder.java:95)
W/System.err﹕ at com.test.stream.VideoDecoder.decodeSample(VideoDecoder.java:24)
W/System.err﹕ at com.test.stream.VideoThread.run(VideoThread.java:160)
The error above is the first error that appears, the IllegalStateException is afterwards thrown on each frame.
My question is, is this a device specific problem (because of: older api/device, less powerful, etc.) or is something actually wrong?
and how should I deal with this?
For my Android h.264 decoder i do it slightly different to your setup. I think your using more modern api level. But for me it looks more like this:
public void startDecoder() {
// Initilize codec
mediaCodec = MediaCodec.createDecoderByType("video/avc");
mediaFormat = MediaFormat.createVideoFormat("video/avc", 0, 0);
bufferInfo = new MediaCodec.BufferInfo();
// STOPS unit-tests from crashing here from mocked out android
if (mediaCodec != null) {
mediaCodec.configure(mediaFormat, targetSurface, null, 0);
mediaCodec.start();
decoderThread = new Thread(this);
decoderThread.start();
}
}
// Decoder Thread refers to this class which does the decoder/render loop:
public void run() {
//mediaCodec input + output dequeue timeouts
long kInputBufferTimeoutMs = 50;
long kOutputBufferTimeoutMs = 50;
while (running && mediaCodec != null) {
synchronized (mediaCodec) {
// stop if not running.
if (!running || mediaCodec == null)
break;
// Only push in new data if there is data available in the queue
if (naluSegmentQueue.size() > 0) {
int inputBufferIndex = mediaCodec.dequeueInputBuffer(kInputBufferTimeoutMs);
if (inputBufferIndex >= 0) {
NaluSegment segment = naluSegmentQueue.poll();
codecInputBufferAvailable(segment, mediaCodec, inputBufferIndex);
}
}
// always check if output is available.
int outputBufferIndex = mediaCodec.dequeueOutputBuffer(bufferInfo, kOutputBufferTimeoutMs);
if (outputBufferIndex >= 0) {
// Try and render first
codecOuputBufferAvailable(mediaCodec, outputBufferIndex, bufferInfo);
} else if (outputBufferIndex == MediaCodec.INFO_OUTPUT_FORMAT_CHANGED) {
// Subsequent data will conform to new format.
// Can ignore if using getOutputFormat(outputBufferId)
mediaFormat = mediaCodec.getOutputFormat();
}
}
}
}
To put data into the decoder including the parameters. I don't bother with trying to use the csd-0/1 network streams can have changing format descriptions and its easier to just let it be picked up dynamically.
private void codecInputBufferAvailable(NaluSegment segment, MediaCodec codec, int index) {
int flags = (segment.getType() == NaluType.SPS
|| segment.getType() == NaluType.PPS
|| segment.getType() == NaluType.SUPP_ENHANCEMENT) ?
MediaCodec.BUFFER_FLAG_CODEC_CONFIG : MediaCodec.BUFFER_FLAG_SYNC_FRAME;
ByteBuffer[] buffers = codec.getInputBuffers();
ByteBuffer buffer = buffers[index];
// Can throw buffer overflow exception when buffer sizes are too small.
try {
buffer.put(segment.getBuffer());
codec.queueInputBuffer(index, 0, segment.getBufferSize(), 0, flags);
} catch(Exception e) {
Log.e(TAG, "Failed to push buffer to decoder");
}
}
IMPORTANT: buffer.put(segment.getBuffer());
getBuffer() here always returns a 4 byte annexb buffer. The android decoders do not understand 3 byte nal units. So if you have a 3 byte nal unit turn it into 4 bytes magic sequence with length + 1 and 0x00, 0x00, 0x00, 0x01 as the start magic sequence the rest of the buffer should be &buffer[headerLength].
Notice the try-catch here this doesn't give a compiler warning but it can throw a buffer overflow exception here if your have a very big payload and the byte-buffer is too small.
So long as your parse out your NAL units correctly this should work for you. But for my case i noticed that the NAL units can be 3 or 4 bytes for the magic header.
/**
* H264 is comprised of NALU segments.
*
* XXXX Y ZZZZZZZZ -> XXXX Y ZZZZZZZZ -> XXXX Y ZZZZZZZZ
*
* Each segment is comprised of:
*
* XXXX -> Magic byte header (0x00, 0x00, 0x00, 0x01) NOTE: this can be either 3 of 4 bytes
* Y -> The Nalu Type
* ZZZ... -> The Payload
*
* Notice there is no nalu length specified. To parse an nalu, you must
* read until the next magic-byte-sequence AKA the next segment to figure
* out the full nalu length
**/
public static List<NaluSegment> parseNaluSegments(byte[] buffer) throws NaluBufferException {
List<NaluSegment> segmentList = new ArrayList<>();
if (buffer.length < 6) {
return segmentList;
}
int lastStartingOffset = -1;
for (int i = 0; i < buffer.length - 10; ++i) {
**if (buffer[i] == 0x00 && buffer[i+1] == 0x00 && buffer[i+2] == 0x01)** {
int naluType = (buffer[i+3] & 0x1F);
NaluSegment segment = new NaluSegment(naluType, 3, i);
**if (i > 0 && buffer[i-1] == 0x00)** {
// This is actually a 4 byte segment
int currentSegmentOffset = segment.getOffset();
segment.setHeaderSize(4);
segment.setOffset(currentSegmentOffset - 1);
}
...
Create your own nalu-segment objects and don't forget the trailing NAL.
I hope this helps.
i want to send an audio stream from PC (C++ application, using FMOD-API to decode audio data and send via UDP Socket) to an android device. The communication already works and i can hear "sound" (100ms sound, followed by 900ms silence, alternating) on the android.
I don't know why the sound is stuttering - on the PC the same audio stream is played fine in nice quality. I think the problem is on the android..
Here is the code:
DatagramSocket sock = new DatagramSocket(12345);
byte []bSockBuffer = new byte[1024];
byte []bRecvBufTmp;
int iAudioBufSize, iCurAudioBufPos = 0;
sock.setReceiveBufferSize(bSockBuffer.length);
// Audio Stream initialisieren:
iAudioBufSize = AudioTrack.getMinBufferSize(44100, AudioFormat.CHANNEL_OUT_STEREO, AudioFormat.ENCODING_PCM_16BIT);
AudioTrack track = new AudioTrack(AudioManager.STREAM_MUSIC, 44100, AudioFormat.CHANNEL_OUT_STEREO,
AudioFormat.ENCODING_PCM_16BIT, iAudioBufSize, AudioTrack.MODE_STREAM);
track.play();
while (true)
{
DatagramPacket pack = new DatagramPacket(bSockBuffer, bSockBuffer.length);
// Paket empfangen:
sock.receive(pack);
track.write(pack.getData(), 0, pack.getLength());
}
I'm sure to set up 'AudioTrack' object correctly, settings compare to my settings in the c++ application.
An other step was pre-buffering the received socket-data in a temporary 'byte[]' variable and writing it to the AudioTrack-object when the size of the buffer 'iAudioBufSize' was reached.
This did not helped.
Any idears?
Thanks
[EDIT]
Code of C++ Application, used sample "manualdecode" of FMOD API examples:
FMOD_RESULT F_CALLBACK pcmreadcallback(FMOD_SOUND *sound, void *data, unsigned int datalen)
{
CCtrlSocket *cClientTmp = /* Obtaining target client sock here */;
FMOD_RESULT result;
unsigned int read, uSentTmp, uSizeTmp;
EnterCriticalSection(&decodecrit);
if (!decodesound)
return (FMOD_ERR_FILE_EOF);
result = decodesound->readData(data, datalen, &read);
if (result == FMOD_ERR_FILE_EOF)
{
// Handle looping:
decodesound->seekData(0);
datalen -= read;
result = decodesound->readData((char*) data + read, datalen, &read);
}
// Split package in multiple parts:
uSentTmp = 0;
do
{
uSizeTmp = (read - uSentTmp);
if (uSizeTmp > 1024)
uSizeTmp = 1024;
uSentTmp += cClientTmp->SendAudioData((char*) data + uSentTmp, uSizeTmp);
} while (uSentTmp < read);
LeaveCriticalSection(&decodecrit);
return (FMOD_OK);
}
I've done this problem.
The mess was an entry in a logfile that has cost lots of time creating a lag :(
Now i can hear the streamed music on my android client. But there are still some lags. I've experimented a LOT of values for socket and AudioTrack buffers.
I have compared the amount of sent and received bytes: In 20 secs sending 9170000 bytes of data results in receiving 8120000 bytes on android device. At first the stream is played fast for 3 secs (that means buffer's full?). After 30 secs the stream lags (which means buffer's empty?).
In general the music quality is very good, but there is a sizzling noise all the time (which indicates lost socket packages?).
My 'PlaybackStart()' function has changed - i'm not using a PCM read callback anymore:
FMOD_RESULT CAudioStream::PlaybackStart()
{
CCtrlSocket *cClientTmp;
unsigned int read, uSentTmp, uSizeTmp;
FMOD_RESULT result;
result = system->createStream("C:\\test.mp3", FMOD_OPENONLY | FMOD_ACCURATETIME, 0, &sound);
if(result != FMOD_OK)
return (result);
int iChannels, iBits;
FMOD_SOUND_FORMAT fFormat;
FMOD_SOUND_TYPE fType;
result = sound->getFormat(&fType, &fFormat, &iChannels, &iBits);
if(result != FMOD_OK)
return (result);
void *data;
unsigned int length = 0;
int iSampleSec = 1; // Playtime
int iSampleSize = (44100 * 2 * sizeof(signed short) * iSampleSec);
int iSleep = 6; // Sleep after sending a package
DWORD dSleepTotal;
result = sound->getLength(&length, FMOD_TIMEUNIT_PCMBYTES);
if(result != FMOD_OK)
return (result);
data = malloc(iSampleSize);
if (!data)
return (FMOD_RESULT_FORCEINT);
cClientTmp = (CCtrlSocket*) CCtrlSocket::cServerSock.GetClientSock(CCtrlSocket::cServerSock.GetClientSockCount() - 1);
do
{
result = sound->readData((char*) data, iSampleSize, &read);
if ((result != FMOD_OK) && (result != FMOD_ERR_FILE_EOF))
ASSERT(FALSE);
else if (read > 0)
{
dSleepTotal = 0;
for (int i = 0; i < read; i += NET_SVR_AUDIO_BUFFER)
{
// MIN_VAL_LIMITED ((MIN_VAL(VAL1, VAL2) <= LIMIT) ? LIMIT : MIN_VAL(VAL1, VAL2))
cClientTmp->SendAudioData((char*) data + i, MIN_VAL_LIMITED(NET_SVR_AUDIO_BUFFER, (read - i), 0));
// Sleep after sending every package:
Sleep(iSleep);
dSleepTotal += iSleep;
}
if (dSleepTotal < (iSampleSec * 1000))
{
dSleepTotal = (iSampleSec * 1000) - dSleepTotal;
// Sleep after sending every second playtime:
Sleep(dSleepTotal);
}
}
} while (read > 0);
result = sound->release();
if(result != FMOD_OK)
return (result);
result = system->close();
if(result != FMOD_OK)
return (result);
result = system->release();
if(result != FMOD_OK)
return (result);
return (result);
}
I have experimented with different sleep-timings, too.
How can I append files to an existing zip file? I already have the code that can create a zip file and it works great except for one big problem. The way it works now, the user takes a bunch of pictures, and at the end, all the pictures get added to a zip file, which can take quite a while if you take enough pictures. :-( So I'm thinking, I have a very good and efficient solution. As the pictures are taken, I will simply add the each new picture to the zip file right after it's taken. Then when they're done taking pictures, finish up the zip file so it's usable and export it. :-)
The problem is, I can not get it to add files to an existing zip file. :-( Here's what I have so far. Also, please keep in mind, this is just a proof of concept, I do understand that re-initializing everything for every iteration of the for loop is very dumb. Each iteration of the loop is supposed to represent another file being added which will most likely be a long time later, maybe even an hour later, which is why I have everything resetting each iteration, because the app will be shut down between adding files. If I can get this working, then I will actually ditch the for loop and put this code into a function that gets called every time a picture gets taken. :-)
try {
for(int i=0; i < _files.size(); i++) {
//beginning of initial setup stuff
BufferedInputStream origin = null;
FileOutputStream dest = new FileOutputStream(_zipFile,false);
ZipOutputStream out = new ZipOutputStream(new BufferedOutputStream(dest));
byte data[] = new byte[BUFFER];
out.setLevel(0); //I added this because it makes it not compress the data
//at all and I hoped that it would allow the zip to be appended to
//end of initial setup stuff
//beginning of old for loop
Log.v("Compress", "Adding: " + _files[i]);
FileInputStream fi = new FileInputStream(_files[i]);
origin = new BufferedInputStream(fi, BUFFER);
ZipEntry entry = new ZipEntry(_files[i].substring(_files[i].lastIndexOf("/") + 1));
out.putNextEntry(entry);
int count;
while ((count = origin.read(data, 0, BUFFER)) != -1) {
out.write(data, 0, count);
}
origin.close();
//end of for old loop
//beginning of finishing stuff
out.close();
//end of finishing stuff
}
} catch(Exception e) {
Log.e("ZipCreation", "Error writing zip", e);
e.printStackTrace();
}
Also, I have experimented around with
FileOutputStream dest = new FileOutputStream(_zipFile,true);
If you notice, I set append to true, which will actually append the data to an existing file. And what's interesting is, it actually does append the data to the original file, however, after the file gets extracted on my computer, the last file written is all that gets extracted, which is bad. :-( So is there some way to start writing a zip file, and then later, add on to it, and finish up the zip file? I've even thought about possibly taking ZipOutputStream and modifying it to fit this model that I need. It should logically be possible somehow? :-)
Thanks in advance for the help! :-D
-Jared
Ok, thanks for all your suggestions, but I was able to get it working like I wanted.... it CAN be done, you CAN add files after closing the file, as long as you save your place!!! :-D
Here's how I was able to get it going working:
try {
for(int i=0; i < _files.size(); i++) {
//beginning of initial setup stuff
BufferedInputStream origin = null;
FileOutputStream dest = new FileOutputStream(_zipFile,true);
ZipOutputStreamNew out = new ZipOutputStreamNew(new BufferedOutputStream(dest));
byte data[] = new byte[BUFFER];
if (havePreviousData) {
out.setWritten(tempWritten);
out.setXentries(tempXentries);
}
//end of initial setup stuff
//beginning of for loop
Log.i("Compress", "Adding: " + _files.get(i));
FileInputStream fi = new FileInputStream(_files.get(i));
origin = new BufferedInputStream(fi, BUFFER);
TempString = _files.get(i).substring(_files.get(i).lastIndexOf("/") + 1);
ZipEntry entry = new ZipEntry(_paths.get(i) + TempString);
out.putNextEntry(entry);
int count;
while ((count = origin.read(data, 0, BUFFER)) != -1) {
out.write(data, 0, count);
}
origin.close();
out.closeEntry();
//end of for loop
//beginning of finishing stuff
if (i == (_files.size()-1)) {
//it's the last record so we should finish it off
out.closeAndFinish();
} else {
//close the file, but don't write the Central Directory
//first, back up where the zip file was...
tempWritten = out.getWritten();
tempXentries = out.getXentries();
havePreviousData = true;
//now close the file
out.close();
}
//end of finishing stuff
}
//zip succeeded
} catch(Exception e) {
Log.e("ZipCreation", "Error writing zip", e);
e.printStackTrace();
}
Also, keep in mind, this is not the only code I had to do. I also had to make my own copy of ZipOutputStream so that I could expose the following functions that I created within my ZipOutputStreamNew class....
getWritten()
getXentries()
as well as
setWritten(long mWritten)
setXentries(Vector<XEntry> mXEntries)
For the most part, all this does, is it starts writing like normal, then, instead of closing like normal, it backs up those two variables, and then for the next iteration, it restores just those variables.
Let me know if you have any questions about all this, but I knew it would work, all it has to do is save where it was. :-D
Thanks again for all the help everybody! :-)
At Raj's request, here is the source code for ZipOutputStreamNew:
/**
* This class implements an output stream filter for writing files in the
* ZIP file format. Includes support for both compressed and uncompressed
* entries.
*
* #author David Connelly
* #version %I%, %G%
*/
public class ZipOutputStreamNew extends DeflaterOutputStream implements ZipConstants {
public static class XEntry {
public final ZipEntry entry;
public final long offset;
public final int flag;
public XEntry(ZipEntry entry, long offset) {
this.entry = entry;
this.offset = offset;
this.flag = (entry.getMethod() == DEFLATED &&
(entry.getSize() == -1 ||
entry.getCompressedSize() == -1 ||
entry.getCrc() == -1))
// store size, compressed size, and crc-32 in data descriptor
// immediately following the compressed entry data
? 8
// store size, compressed size, and crc-32 in LOC header
: 0;
}
}
private XEntry current;
private Vector<XEntry> xentries = new Vector<XEntry>();
private HashSet<String> names = new HashSet<String>();
private CRC32 crc = new CRC32();
private long written = 0;
private long locoff = 0;
private String comment;
private int method = DEFLATED;
private boolean finished;
private boolean closed = false;
private boolean closeItPermanently = false;
private static int version(ZipEntry e) throws ZipException {
switch (e.getMethod()) {
case DEFLATED: return 20;
case STORED: return 10;
default: throw new ZipException("unsupported compression method");
}
}
/**
* Checks to make sure that this stream has not been closed.
*/
private void ensureOpen() throws IOException {
if (closed) {
throw new IOException("Stream closed");
}
}
/**
* Compression method for uncompressed (STORED) entries.
*/
public static final int STORED = ZipEntry.STORED;
/**
* Compression method for compressed (DEFLATED) entries.
*/
public static final int DEFLATED = ZipEntry.DEFLATED;
/**
* Creates a new ZIP output stream.
* #param out the actual output stream
*/
public ZipOutputStreamNew(OutputStream out) {
super(out, new Deflater(Deflater.DEFAULT_COMPRESSION, true));
usesDefaultDeflater = true;
}
/**
* Sets the ZIP file comment.
* #param comment the comment string
* #exception IllegalArgumentException if the length of the specified
* ZIP file comment is greater than 0xFFFF bytes
*/
public void setComment(String comment) {
if (comment != null && comment.length() > 0xffff/3
&& getUTF8Length(comment) > 0xffff) {
throw new IllegalArgumentException("ZIP file comment too long.");
}
this.comment = comment;
}
/**
* Sets the default compression method for subsequent entries. This
* default will be used whenever the compression method is not specified
* for an individual ZIP file entry, and is initially set to DEFLATED.
* #param method the default compression method
* #exception IllegalArgumentException if the specified compression method
* is invalid
*/
public void setMethod(int method) {
if (method != DEFLATED && method != STORED) {
throw new IllegalArgumentException("invalid compression method");
}
this.method = method;
}
/**
* Sets the compression level for subsequent entries which are DEFLATED.
* The default setting is DEFAULT_COMPRESSION.
* #param level the compression level (0-9)
* #exception IllegalArgumentException if the compression level is invalid
*/
public void setLevel(int level) {
def.setLevel(level);
}
/**
* Begins writing a new ZIP file entry and positions the stream to the
* start of the entry data. Closes the current entry if still active.
* The default compression method will be used if no compression method
* was specified for the entry, and the current time will be used if
* the entry has no set modification time.
* #param e the ZIP entry to be written
* #exception ZipException if a ZIP format error has occurred
* #exception IOException if an I/O error has occurred
*/
public void putNextEntry(ZipEntry e) throws IOException {
ensureOpen();
if (current != null) {
closeEntry(); // close previous entry
}
if (e.getTime() == -1) {
e.setTime(System.currentTimeMillis());
}
if (e.getMethod() == -1) {
e.setMethod(method); // use default method
}
switch (e.getMethod()) {
case DEFLATED:
break;
case STORED:
// compressed size, uncompressed size, and crc-32 must all be
// set for entries using STORED compression method
if (e.getSize() == -1) {
e.setSize(e.getCompressedSize());
} else if (e.getCompressedSize() == -1) {
e.setCompressedSize(e.getSize());
} else if (e.getSize() != e.getCompressedSize()) {
throw new ZipException(
"STORED entry where compressed != uncompressed size");
}
if (e.getSize() == -1 || e.getCrc() == -1) {
throw new ZipException(
"STORED entry missing size, compressed size, or crc-32");
}
break;
default:
throw new ZipException("unsupported compression method");
}
if (! names.add(e.getName())) {
throw new ZipException("duplicate entry: " + e.getName());
}
current = new XEntry(e, written);
xentries.add(current);
writeLOC(current);
}
/**
* Closes the current ZIP entry and positions the stream for writing
* the next entry.
* #exception ZipException if a ZIP format error has occurred
* #exception IOException if an I/O error has occurred
*/
public void closeEntry() throws IOException {
ensureOpen();
if (current != null) {
ZipEntry e = current.entry;
switch (e.getMethod()) {
case DEFLATED:
def.finish();
while (!def.finished()) {
deflate();
}
if ((current.flag & 8) == 0) {
// verify size, compressed size, and crc-32 settings
if (e.getSize() != def.getBytesRead()) {
throw new ZipException(
"invalid entry size (expected " + e.getSize() +
" but got " + def.getBytesRead() + " bytes)");
}
if (e.getCompressedSize() != def.getBytesWritten()) {
throw new ZipException(
"invalid entry compressed size (expected " +
e.getCompressedSize() + " but got " + def.getBytesWritten() + " bytes)");
}
if (e.getCrc() != crc.getValue()) {
throw new ZipException(
"invalid entry CRC-32 (expected 0x" +
Long.toHexString(e.getCrc()) + " but got 0x" +
Long.toHexString(crc.getValue()) + ")");
}
} else {
e.setSize(def.getBytesRead());
e.setCompressedSize(def.getBytesWritten());
e.setCrc(crc.getValue());
writeEXT(e);
}
def.reset();
written += e.getCompressedSize();
break;
case STORED:
// we already know that both e.size and e.csize are the same
if (e.getSize() != written - locoff) {
throw new ZipException(
"invalid entry size (expected " + e.getSize() +
" but got " + (written - locoff) + " bytes)");
}
if (e.getCrc() != crc.getValue()) {
throw new ZipException(
"invalid entry crc-32 (expected 0x" +
Long.toHexString(e.getCrc()) + " but got 0x" +
Long.toHexString(crc.getValue()) + ")");
}
break;
default:
throw new ZipException("invalid compression method");
}
crc.reset();
current = null;
}
}
/**
* Writes an array of bytes to the current ZIP entry data. This method
* will block until all the bytes are written.
* #param b the data to be written
* #param off the start offset in the data
* #param len the number of bytes that are written
* #exception ZipException if a ZIP file error has occurred
* #exception IOException if an I/O error has occurred
*/
public synchronized void write(byte[] b, int off, int len)
throws IOException
{
ensureOpen();
if (off < 0 || len < 0 || off > b.length - len) {
throw new IndexOutOfBoundsException();
} else if (len == 0) {
return;
}
if (current == null) {
throw new ZipException("no current ZIP entry");
}
ZipEntry entry = current.entry;
switch (entry.getMethod()) {
case DEFLATED:
super.write(b, off, len);
break;
case STORED:
written += len;
if (written - locoff > entry.getSize()) {
throw new ZipException(
"attempt to write past end of STORED entry");
}
out.write(b, off, len);
break;
default:
throw new ZipException("invalid compression method");
}
crc.update(b, off, len);
}
/**
* Finishes writing the contents of the ZIP output stream without closing
* the underlying stream. Use this method when applying multiple filters
* in succession to the same output stream.
* #exception ZipException if a ZIP file error has occurred
* #exception IOException if an I/O exception has occurred
*/
public void finish() throws IOException {
ensureOpen();
if (finished) {
return;
}
if (current != null) {
closeEntry();
}
if (xentries.size() < 1) {
throw new ZipException("ZIP file must have at least one entry");
}
if (closeItPermanently) {
// write central directory
long off = written;
for (XEntry xentry : xentries)
writeCEN(xentry);
writeEND(off, written - off);
finished = true;
//Log.e("ZipOutputStreamNew", "I just ran wrote the Central Directory Jared!");
}
//Log.e("ZipOutputStreamNew", "I just ran finish() Jared!");
}
/**
* Gets the value of the "xentries" variable (for later use)
* #return
*/
public Vector<XEntry> getXentries() {
return xentries;
//TODO convert this to primitive data types
}
/**
* Gets the value of the "written" variable (for later use)
* #return
*/
public long getWritten() {
return written;
}
/**
* Sets the value of the "xentries" variable (for later use)
* #return
*/
public void setXentries(Vector<XEntry> mXEntries) {
xentries = mXEntries;
//TODO convert this to primitive data types
}
/**
* Sets the value of the "written" variable (for later use)
* #return
*/
public void setWritten(long mWritten) {
written = mWritten;
}
/**
* Closes the ZIP output stream as well as the stream being filtered.
* #exception ZipException if a ZIP file error has occurred
* #exception IOException if an I/O error has occurred
*/
public void closeAndFinish() throws IOException {
if (!closed) {
closeItPermanently=true;
super.close();
closed = true;
}
}
/**
* Used to close the ZIP output stream as well as the stream being filtered.
* instead it does nothing :-P
* #exception ZipException if a ZIP file error has occurred
* #exception IOException if an I/O error has occurred
*/
public void close() throws IOException {
if (!closed) {
closeItPermanently=false;
super.close();
closed = true;
}
}
/*
* Writes local file (LOC) header for specified entry.
*/
private void writeLOC(XEntry xentry) throws IOException {
ZipEntry e = xentry.entry;
int flag = xentry.flag;
writeInt(LOCSIG); // LOC header signature
writeShort(version(e)); // version needed to extract
writeShort(flag); // general purpose bit flag
writeShort(e.getMethod()); // compression method
writeInt(e.getTime()); // last modification time
if ((flag & 8) == 8) {
// store size, uncompressed size, and crc-32 in data descriptor
// immediately following compressed entry data
writeInt(0);
writeInt(0);
writeInt(0);
} else {
writeInt(e.getCrc()); // crc-32
writeInt(e.getCompressedSize()); // compressed size
writeInt(e.getSize()); // uncompressed size
}
byte[] nameBytes = getUTF8Bytes(e.getName());
writeShort(nameBytes.length);
writeShort(e.getExtra() != null ? e.getExtra().length : 0);
writeBytes(nameBytes, 0, nameBytes.length);
if (e.getExtra() != null) {
writeBytes(e.getExtra(), 0, e.getExtra().length);
}
locoff = written;
}
/*
* Writes extra data descriptor (EXT) for specified entry.
*/
private void writeEXT(ZipEntry e) throws IOException {
writeInt(EXTSIG); // EXT header signature
writeInt(e.getCrc()); // crc-32
writeInt(e.getCompressedSize()); // compressed size
writeInt(e.getSize()); // uncompressed size
}
/*
* Write central directory (CEN) header for specified entry.
* REMIND: add support for file attributes
*/
private void writeCEN(XEntry xentry) throws IOException {
ZipEntry e = xentry.entry;
int flag = xentry.flag;
int version = version(e);
writeInt(CENSIG); // CEN header signature
writeShort(version); // version made by
writeShort(version); // version needed to extract
writeShort(flag); // general purpose bit flag
writeShort(e.getMethod()); // compression method
writeInt(e.getTime()); // last modification time
writeInt(e.getCrc()); // crc-32
writeInt(e.getCompressedSize()); // compressed size
writeInt(e.getSize()); // uncompressed size
byte[] nameBytes = getUTF8Bytes(e.getName());
writeShort(nameBytes.length);
writeShort(e.getExtra() != null ? e.getExtra().length : 0);
byte[] commentBytes;
if (e.getComment() != null) {
commentBytes = getUTF8Bytes(e.getComment());
writeShort(commentBytes.length);
} else {
commentBytes = null;
writeShort(0);
}
writeShort(0); // starting disk number
writeShort(0); // internal file attributes (unused)
writeInt(0); // external file attributes (unused)
writeInt(xentry.offset); // relative offset of local header
writeBytes(nameBytes, 0, nameBytes.length);
if (e.getExtra() != null) {
writeBytes(e.getExtra(), 0, e.getExtra().length);
}
if (commentBytes != null) {
writeBytes(commentBytes, 0, commentBytes.length);
}
}
/*
* Writes end of central directory (END) header.
*/
private void writeEND(long off, long len) throws IOException {
int count = xentries.size();
writeInt(ENDSIG); // END record signature
writeShort(0); // number of this disk
writeShort(0); // central directory start disk
writeShort(count); // number of directory entries on disk
writeShort(count); // total number of directory entries
writeInt(len); // length of central directory
writeInt(off); // offset of central directory
if (comment != null) { // zip file comment
byte[] b = getUTF8Bytes(comment);
writeShort(b.length);
writeBytes(b, 0, b.length);
} else {
writeShort(0);
}
}
/*
* Writes a 16-bit short to the output stream in little-endian byte order.
*/
private void writeShort(int v) throws IOException {
OutputStream out = this.out;
out.write((v >>> 0) & 0xff);
out.write((v >>> 8) & 0xff);
written += 2;
}
/*
* Writes a 32-bit int to the output stream in little-endian byte order.
*/
private void writeInt(long v) throws IOException {
OutputStream out = this.out;
out.write((int)((v >>> 0) & 0xff));
out.write((int)((v >>> 8) & 0xff));
out.write((int)((v >>> 16) & 0xff));
out.write((int)((v >>> 24) & 0xff));
written += 4;
}
/*
* Writes an array of bytes to the output stream.
*/
private void writeBytes(byte[] b, int off, int len) throws IOException {
super.out.write(b, off, len);
written += len;
}
/*
* Returns the length of String's UTF8 encoding.
*/
static int getUTF8Length(String s) {
int count = 0;
for (int i = 0; i < s.length(); i++) {
char ch = s.charAt(i);
if (ch <= 0x7f) {
count++;
} else if (ch <= 0x7ff) {
count += 2;
} else {
count += 3;
}
}
return count;
}
/*
* Returns an array of bytes representing the UTF8 encoding
* of the specified String.
*/
private static byte[] getUTF8Bytes(String s) {
char[] c = s.toCharArray();
int len = c.length;
// Count the number of encoded bytes...
int count = 0;
for (int i = 0; i < len; i++) {
int ch = c[i];
if (ch <= 0x7f) {
count++;
} else if (ch <= 0x7ff) {
count += 2;
} else {
count += 3;
}
}
// Now return the encoded bytes...
byte[] b = new byte[count];
int off = 0;
for (int i = 0; i < len; i++) {
int ch = c[i];
if (ch <= 0x7f) {
b[off++] = (byte)ch;
} else if (ch <= 0x7ff) {
b[off++] = (byte)((ch >> 6) | 0xc0);
b[off++] = (byte)((ch & 0x3f) | 0x80);
} else {
b[off++] = (byte)((ch >> 12) | 0xe0);
b[off++] = (byte)(((ch >> 6) & 0x3f) | 0x80);
b[off++] = (byte)((ch & 0x3f) | 0x80);
}
}
return b;
}
}
I believe it can't be done right now with the current API.
You can append data to any file, but that does not mean that you will end up with the right file format. A .zip file is not like a .tar file, and the compression requires imposes restrictions to the handling of the files (file positions, EOF, etc.). If you consider the structure of the file format (taken from wikipedia here) you will understand why just appending does not work.
There is a library called TrueZip that could work, although I do not know if it supports android. Take a look at this answer in another similar question:
Appending files to a zip file with Java .
Also, as a workaround, you could create individual .zip files and append them as a tarball (file format here). Compression might be slighty worst, but it would be much better in terms of time efficiency.
Update based on the comments (and possible solution)
You could separate the addition to each ZipEntry and leave the ZipOutputStream object open as long as you are still taking pictures. I can see risks with that approach, though, as any problem with the app while still taking pictures (a force close, run out of battery, etc) may render the whole file unusable. You will need to make sure to use the right try/catch/finally blocks to close the file and call closeZip() upon events such as onClose() and onDestroy(), but the idea would be the following:
import java.io.*;
import java.util.zip.*;
public class Zip {
static final int BUFFER = 2048;
ZipOutputStream out;
byte data[];
public Zip(String name) {
FileOutputStream dest = new FileOutputStream(name);
out = new ZipOutputStream(new BufferedOutputStream(dest));
data = new byte[BUFFER];
}
public void addFile (String name) {
FileInputStream fi = new FileInputStream(name);
BufferedInputStream origin = new BufferedInputStream(fi, BUFFER);
ZipEntry entry = new ZipEntry(name);
out.putNextEntry(entry);
int count;
while((count = origin.read(data, 0, BUFFER)) != -1) {
out.write(data, 0, count);
}
origin.close();
}
public void closeZip () {
out.close();
}
}