Develop Reference

Android How to convert UDP recieving audio into wav file

I am trying to recieve an streaming audio from my app.
below is my code for recieving audio stream:
public class ClientListen implements Runnable {
private Context context;
public ClientListen(Context context) {
this.context = context;
}
#Override
public void run() {
boolean run = true;
try {
DatagramSocket udpSocket = new DatagramSocket(8765);
InetAddress serverAddr = null;
try {
serverAddr = InetAddress.getByName("127.0.0.1");
} catch (UnknownHostException e) {
e.printStackTrace();
}
while (run) {
try {
byte[] message = new byte[8000];
DatagramPacket packet = new DatagramPacket(message,message.length);
Log.i("UDP client: ", "about to wait to receive");
udpSocket.setSoTimeout(10000);
udpSocket.receive(packet);
String text = new String(packet.getData(), 0, packet.getLength());
Log.d("Received text", text);
} catch (IOException e) {
Log.e(" UDP clien", "error: ", e);
run = false;
udpSocket.close();
}
}
} catch (SocketException e) {
Log.e("Socket Open:", "Error:", e);
} catch (IOException e) {
e.printStackTrace();
}
}
}
In Received text logger i can see data as coming as
D/Received text: �������n�����������q�9�$�0�/�G�{�������s�����JiH&������d�����Z���������d�����E������C�+
��l��y�����������v���9����������u��f�j�������$�����K���������F��~R�2�����T��������������L�����!��G��8������s�;�"�,�R�����(��{�����*_��Z�������5������������\������x���j~������������/��=�����%�������
How can store this data into a wav file ?

What you see is the string representation of single udp packet after it was received and the received block has just being released.
It is a very small fraction of the sound you want to convert to wave.
Soon the while loop will continue and you will receive another packet and many more..
You need to collect all the packets in a buffer and then when you think it is ok - convert them to wave file.
Remember Wave is not just the sound bytes you get from udp but also 44 bytes of prefix you need to add to this file in order to be recognized by players.
Also if the udp is from another encoding format such as G711 - you must encode these bytes to PCM – if not you will hear heavy noise in the
Sound of the wave or the stream you play.
The buffer must be accurate. if it will be too big (many empty bytes in the end of the array) you will hear a sound of helicopter. if you know exactly what is the size of each packet then you can just write it to AudioTrack in order to play stream, or accumulate it and convert it to wave file when you will see fit. But If you are not sure about the size you can use this answer to get a buffer and then write the buffer to AudioTrack:
Android AudioRecord to Server over UDP Playback Issues.
they use Javax because it is very old answer but you just need to use AudioTrack instead in order to stream. It is not in this scope so I will just present the AudioTrack streaming replacements instead of Javax SourceDataLine:
final int SAMPLE_RATE = 8000; // Hertz
final int STREAM_TYPE = AudioManager.STREAM_NOTIFICATION;
int channelConfig = AudioFormat.CHANNEL_OUT_MONO;
int encodingFormat = AudioFormat.ENCODING_PCM_16BIT;
AudioTrack track = new AudioTrack(STREAM_TYPE, SAMPLE_RATE, channelConfig,
encodingFormat, BUF_SIZE, AudioTrack.MODE_STREAM);
track.play();
//.. then after receive UDP packets and the buffer is full:
if(track != null && packet != null){
track.write(audioStreamBuffer, 0, audioStreamBuffer.length);
}
You must not do this in the UI thread (I assume you know that).
In the code I will show you - I am getting udp of audio logs from PTT radio. It is encoded in G711 Ulaw . each packet is of 172 bytes exactly. First 12 bytes are for RTP and I need to offset (remove) them in order to eliminate small noises. rest 160 bytes are 20MS of sound.
I must decode the G711 Ulaw bytes to PCM shorts array. Then to take the short array and to make a wave file out of it. I am taking it after I see there was no packet receiving for more than one second (so I know the speech ended and the new block release is because of a new speech so I can take the old speech and make a wave file out of it). You can decide of a different buffer depends on what you are doing.
It works fine. After the decoding the sound of the wave is very good. If you have UDP with PCM so you don’t need to decode G711 - just skip this part.
Finally I want to mention I saw many old answers with code parts using javax.sound.sampled that seems great because it can convert easily an audio file or stream to wave format with AudioFileFormat
And also convert G711 to pcm with AudioFormat manipulations. But unfortunately it is not part of current java for android. We must count on android AudioTrack instead (and AudioRecord if we want to get the sound from the mic) but AudioTrack play only PCM and do not support G711 format – so when streaming G711 with AudioTrack the noise is terrible. We must decode it in our code before writing it to the track. Also we cannot convert to wave file using audioInputStream – I tried to do this easily with javax.sound.sampled jar file I added to my app but android keep giving me errors such as format not supported for wave, and mixer errors when try to stream – so I understood latest android cannot work with javax.sound.sampled and I went to look for law level decoding of G711 and law level creation of wave file out of the buffer of byte array received from the UDP packets .
A. in manifest add:
<uses-permission android:name="android.permission.WRITE_EXTERNAL_STORAGE"/>
<uses-permission android:name="android.permission.READ_EXTERNAL_STORAGE"/>
<uses-permission android:name="android.permission.INTERNET"/>
B. in the worker thread:
#Override
public void run(){
Log.i(TAG, "ClientListen thread started. Thread id: " + Thread.currentThread().getId());
try{
udpSocket = new DatagramSocket(port);
}catch(SocketException e){
e.printStackTrace();
}
byte[] messageBuf = new byte[BUF_SIZE];
Log.i(TAG, "waiting to receive packet in port: " + port);
if(udpSocket != null){
// here you can create new AudioTrack and play.track
byte pttSession[] = null;
while (running){
packet = new DatagramPacket(messageBuf, 0, messageBuf.length);
Log.d(TAG, "inside while running loop");
try{
Log.d(TAG, "receive block: waiting for user to press on
speaker(listening now inside udpSocket for DatagramPacket..)");
//get inside receive block until packet will arrive through this socket
long timeBeforeBlock = System.currentTimeMillis();
udpSocket.receive(packet);
Log.d(TAG, "client received a packet, receive block stopped)");
//this is for sending msg handler to the UI tread (you may skip this)
sendState("getting UDP packets...");
/* if previous block release happened more than one second ago - so this
packet release is for a new speech. so let’s copy the previous speech
to a wave file and empty the speech */
if(System.currentTimeMillis() - timeBeforeBlock > 1000 && pttSession != null){
convertBytesToFile(pttSession);
pttSession = null;
}
/* let’s take the packet that was released and start new speech or add it to the ongoing speech. */
byte[] slice = Arrays.copyOfRange(packet.getData(), 12, packet.getLength());
if(null == pttSession){
pttSession = slice;
}else{
pttSession = concat(pttSession, slice);
Log.d(TAG, "pttSession:" + Arrays.toString(pttSession));
}
}catch(IOException e){
Log.e(TAG, "UDP client IOException - error: ", e);
running = false;
}
}
// let’s take the latest speech and make a last wave file out of it.
if(pttSession != null){
convertBytesToFile(pttSession);
pttSession = null;
}
// if running == false then stop listen.
udpSocket.close();
handler.sendEmptyMessage(MainActivity.UdpClientHandler.UPDATE_END);
}else{
sendState("cannot bind datagram socket to the specified port:" + port);
}
}
private void convertBytesToFile(byte[] byteArray){
//decode the bytes from G711U to PCM (outcome is a short array)
G711UCodec decoder = new G711UCodec();
int size = byteArray.length;
short[] shortArray = new short[size];
decoder.decode(shortArray, byteArray, size, 0);
String newFileName = "speech_" + System.currentTimeMillis() + ".wav";
//convert short array to wav (add 44 prefix shorts) and save it as a .wav file
Wave wave = new Wave(SAMPLE_RATE, (short) 1, shortArray, 0, shortArray.length - 1);
if(wave.writeToFile(Environment.getExternalStoragePublicDirectory
(Environment.DIRECTORY_DOWNLOADS),newFileName)){
Log.d(TAG, "wave.writeToFile successful!");
sendState("create file: "+ newFileName);
}else{
Log.w(TAG, "wave.writeToFile failed");
}
}
C. encoding/decoding G711 U-Law class:
taken from: https://github.com/thinktube-kobe/airtube/blob/master/JavaLibrary/src/com/thinktube/audio/G711UCodec.java
/**
* G.711 codec. This class provides u-law conversion.
*/
public class G711UCodec {
// s00000001wxyz...s000wxyz
// s0000001wxyza...s001wxyz
// s000001wxyzab...s010wxyz
// s00001wxyzabc...s011wxyz
// s0001wxyzabcd...s100wxyz
// s001wxyzabcde...s101wxyz
// s01wxyzabcdef...s110wxyz
// s1wxyzabcdefg...s111wxyz
private static byte[] table13to8 = new byte[8192];
private static short[] table8to16 = new short[256];
static {
// b13 --> b8
for (int p = 1, q = 0; p <= 0x80; p <<= 1, q += 0x10) {
for (int i = 0, j = (p << 4) - 0x10; i < 16; i++, j += p) {
int v = (i + q) ^ 0x7F;
byte value1 = (byte) v;
byte value2 = (byte) (v + 128);
for (int m = j, e = j + p; m < e; m++) {
table13to8[m] = value1;
table13to8[8191 - m] = value2;
}
}
}
// b8 --> b16
for (int q = 0; q <= 7; q++) {
for (int i = 0, m = (q << 4); i < 16; i++, m++) {
int v = (((i + 0x10) << q) - 0x10) << 3;
table8to16[m ^ 0x7F] = (short) v;
table8to16[(m ^ 0x7F) + 128] = (short) (65536 - v);
}
}
}
public int decode(short[] b16, byte[] b8, int count, int offset) {
for (int i = 0, j = offset; i < count; i++, j++) {
b16[i] = table8to16[b8[j] & 0xFF];
}
return count;
}
public int encode(short[] b16, int count, byte[] b8, int offset) {
for (int i = 0, j = offset; i < count; i++, j++) {
b8[j] = table13to8[(b16[i] >> 4) & 0x1FFF];
}
return count;
}
public int getSampleCount(int frameSize) {
return frameSize;
}
}
D. Converting to wave file:
Taken from here:
https://github.com/google/oboe/issues/320
import java.io.File;
import java.io.FileNotFoundException;
import java.io.FileOutputStream;
import java.io.IOException;
public class Wave
{
private final int LONGINT = 4;
private final int SMALLINT = 2;
private final int INTEGER = 4;
private final int ID_STRING_SIZE = 4;
private final int WAV_RIFF_SIZE = LONGINT+ID_STRING_SIZE;
private final int WAV_FMT_SIZE = (4*SMALLINT)+(INTEGER*2)+LONGINT+ID_STRING_SIZE;
private final int WAV_DATA_SIZE = ID_STRING_SIZE+LONGINT;
private final int WAV_HDR_SIZE = WAV_RIFF_SIZE+ID_STRING_SIZE+WAV_FMT_SIZE+WAV_DATA_SIZE;
private final short PCM = 1;
private final int SAMPLE_SIZE = 2;
int cursor, nSamples;
byte[] output;
public Wave(int sampleRate, short nChannels, short[] data, int start, int end)
{
nSamples=end-start+1;
cursor=0;
output=new byte[nSamples*SMALLINT+WAV_HDR_SIZE];
buildHeader(sampleRate,nChannels);
writeData(data,start,end);
}
/*
by Udi for using byteArray directly
*/
public Wave(int sampleRate, short nChannels, byte[] data, int start, int end)
{
int size = data.length;
short[] shortArray = new short[size];
for (int index = 0; index < size; index++){
shortArray[index] = (short) data[index];
}
nSamples=end-start+1;
cursor=0;
output=new byte[nSamples*SMALLINT+WAV_HDR_SIZE];
buildHeader(sampleRate,nChannels);
writeData(shortArray,start,end);
}
// ------------------------------------------------------------
private void buildHeader(int sampleRate, short nChannels)
{
write("RIFF");
write(output.length);
write("WAVE");
writeFormat(sampleRate, nChannels);
}
// ------------------------------------------------------------
public void writeFormat(int sampleRate, short nChannels)
{
write("fmt ");
write(WAV_FMT_SIZE-WAV_DATA_SIZE);
write(PCM);
write(nChannels);
write(sampleRate);
write(nChannels * sampleRate * SAMPLE_SIZE);
write((short)(nChannels * SAMPLE_SIZE));
write((short)16);
}
// ------------------------------------------------------------
public void writeData(short[] data, int start, int end)
{
write("data");
write(nSamples*SMALLINT);
for(int i=start; i<=end; write(data[i++]));
}
// ------------------------------------------------------------
private void write(byte b)
{
output[cursor++]=b;
}
// ------------------------------------------------------------
private void write(String id)
{
if(id.length()!=ID_STRING_SIZE){
}
else {
for(int i=0; i<ID_STRING_SIZE; ++i) write((byte)id.charAt(i));
}
}
// ------------------------------------------------------------
private void write(int i)
{
write((byte) (i&0xFF)); i>>=8;
write((byte) (i&0xFF)); i>>=8;
write((byte) (i&0xFF)); i>>=8;
write((byte) (i&0xFF));
}
// ------------------------------------------------------------
private void write(short i)
{
write((byte) (i&0xFF)); i>>=8;
write((byte) (i&0xFF));
}
// ------------------------------------------------------------
public boolean writeToFile(File fileParent , String filename)
{
boolean ok=false;
try {
File path=new File(fileParent, filename);
FileOutputStream outFile = new FileOutputStream(path);
outFile.write(output);
outFile.close();
ok=true;
} catch (FileNotFoundException e) {
e.printStackTrace();
ok=false;
} catch (IOException e) {
ok=false;
e.printStackTrace();
}
return ok;
}
/**
* by Udi for test: write file with temp name so if you write many packets each packet will be written to a new file instead of deleting
* the previous file. (this is mainly for debug)
* #param fileParent
* #param filename
* #return
*/
public boolean writeToTmpFile(File fileParent , String filename)
{
boolean ok=false;
try {
File outputFile = File.createTempFile(filename, ".wav",fileParent);
FileOutputStream fileoutputstream = new FileOutputStream(outputFile);
fileoutputstream.write(output);
fileoutputstream.close();
ok=true;
} catch (FileNotFoundException e) {
e.printStackTrace();
ok=false;
} catch (IOException e) {
ok=false;
e.printStackTrace();
}
return ok;
}
}

Pause and Seek in android stagefright Media Player

I have RK3288 Hardware Board and I am trying to use software decoder on it..
I have already integrated my software decoder with the android stage-fright player..
It is perfectly integrated and video is also running completely..
But now the problem is when I pause or seek video ,pop up window says
"Can't Play this video"
When I click on pause button it again calls awesomeplayer's constructor(I checked this out with the printf).
So please Can anybody point where i am missing in my code? or is there any other changes required?
My Onqueuefilled Function(which is similar to SoftAVC.cpp)
void IntdecHEVC::onQueueFilled(OMX_U32 portIndex) {
UNUSED(portIndex);
if (checkp) {
return;
}
IAccessUnit pic_val;
DisplayPic Display_val;
int32_t got_frame;
int32_t timeDelay, timeTaken;
unsigned int p;
if (mSignalledError) {
return;
}
if (mOutputPortSettingsChange != NONE) {
return;
}
List<BufferInfo *> &inQueue = getPortQueue(kInputPortIndex);
List<BufferInfo *> &outQueue = getPortQueue(kOutputPortIndex);
while (!outQueue.empty()) {
BufferInfo *inInfo;
OMX_BUFFERHEADERTYPE *inHeader = NULL;
BufferInfo *outInfo;
OMX_BUFFERHEADERTYPE *outHeader = NULL;
uint8_t *pBuf;
inInfo = NULL;
inHeader = NULL;
Display_val.eosFlag = false;
if (!mIsInFlush) {
if (!inQueue.empty()) {
inInfo = *inQueue.begin();
inHeader = inInfo->mHeader;
} else {
break;
}
}
outInfo = *outQueue.begin();
outHeader = outInfo->mHeader;
outHeader->nFlags = 0;
outHeader->nTimeStamp = 0;
outHeader->nOffset = 0;
if (inHeader != NULL && (inHeader->nFlags & OMX_BUFFERFLAG_EOS)) {
mReceivedEOS = true;
Display_val.eosFlag = true;
if (inHeader->nFilledLen == 0) {
inQueue.erase(inQueue.begin());
inInfo->mOwnedByUs = false;
notifyEmptyBufferDone(inHeader);
inHeader = NULL;
setFlushMode();
}
}
if (inHeader != NULL) {
pic_val.bufferData = inHeader->pBuffer + inHeader->nOffset;
pic_val.bufferLength = inHeader->nFilledLen;
pic_val.pts = inHeader->nTimeStamp;
}
if (outHeader)
pBuf = outHeader->pBuffer;
Display_val.pobLumaSl = pBuf;
Display_val.pobCromaCb = pBuf + (mWidth * mHeight);
Display_val.pobCromaCr = //pBuf + (mWidth*mHeight) + (m_lOriginalWidthUV*m_lHeightUV);
Display_val.pobCromaCb +(m_lOriginalWidthUV*m_lHeightUV);
GETTIME(&mTimeStart, NULL);
/* Compute time elapsed between end of previous decode()
* to start of current decode() */
TIME_DIFF(mTimeEnd, mTimeStart, timeDelay);
// set the height and width of the picture */
pic_val.width = mWidth;
pic_val.height = mHeight;
mInterface.api_inte265Decode(&pic_val); //decoding data
handlePortSettingChangeEvent(&pic_val);
GETTIME(&mTimeEnd, NULL);
/* Compute time taken for decode() */
TIME_DIFF(mTimeStart, mTimeEnd, timeTaken);
got_frame = mInterface.api_inte265GetPic(&Display_val);
if (got_frame) {
FormatIYUV(&Display_val); //copy data to outputbuffer 1-4
/* added to check decoded data by pooja*/
if (1) {
unsigned int i;
unsigned int MAX_SIZE = Display_val.width * Display_val.height;
for (i = 0; i < MAX_SIZE; i++) {
fwrite(&Display_val.pobLumaSl[i], sizeof(uint8_t), 1, fb);
}
MAX_SIZE = (Display_val.width >> 1) * (Display_val.height >> 1);
for (i = 0; i < MAX_SIZE; i++) {
fwrite(&Display_val.pobCromaCb[i], sizeof(uint8_t), 1, fb);
}
for (i = 0; i < MAX_SIZE; i++) {
fwrite(&Display_val.pobCromaCr[i], sizeof(uint8_t), 1, fb);
}
}
if (outHeader != NULL) {
outHeader->nFilledLen = (Display_val.width * Display_val.height * 3) / 2;
outHeader->nTimeStamp = Display_val.pts;
outInfo->mOwnedByUs = false;
outQueue.erase(outQueue.begin());
outInfo = NULL;
notifyFillBufferDone(outHeader);
outHeader = NULL;
}
} else if (mIsInFlush) {
/* If in flush mode and no output is returned by the codec,
* then come out of flush mode */
mIsInFlush = false;
/* If EOS was recieved on input port and there is no output
* from the codec, then signal EOS on output port */
if (mReceivedEOS) {
if (outHeader != NULL) {
outHeader->nFilledLen = 0;
outHeader->nFlags |= OMX_BUFFERFLAG_EOS; // 12-4-2016
outInfo->mOwnedByUs = false;
outQueue.erase(outQueue.begin());
outInfo = NULL;
notifyFillBufferDone(outHeader);
outHeader = NULL;
resetPlugin();
resetDecoder(); // added
checkp = true; //15-4-2016
// break; //13-4-2016
return; //14-4-2016
}
}
}
/* If input EOS is seen and decoder is not in flush mode,
* set the decoder in flush mode.
* There can be a case where EOS is sent along with last picture data
* In that case, only after decoding that input data, decoder has to be
* put in flush. This case is handled here */
if (mReceivedEOS && !mIsInFlush) {
setFlushMode();
}
/* Notify to client that empty buffer is done - pooja*/
if (inHeader != NULL) {
//inHeader->nFilledLen = 0;
inInfo->mOwnedByUs = false;
inQueue.erase(inQueue.begin());
inInfo = NULL;
notifyEmptyBufferDone(inHeader);
inHeader = NULL;
}
} // end !outQueue.empty()) loop
}

Write to file in Android but failing to get an OuputStream

I need to write to a file which is in the assets folder. I must use
context.getResources().getAssets()
but when I use the open(path_to_my_txt) I get an InputStream, and I have found nothing nice to convert it to an OutputStream
How can I write to this file using the getResources method ?

this below class is taken from Unviersal imageloader libraray
import java.io.Closeable;
import java.io.IOException;
import java.io.InputStream;
import java.io.OutputStream;
/**
* Provides I/O operations
*
* #author Sergey Tarasevich (nostra13[at]gmail[dot]com)
* #since 1.0.0
*/
public final class IoUtils {
/** {#value} */
public static final int DEFAULT_BUFFER_SIZE = 32 * 1024; // 32 KB
/** {#value} */
public static final int DEFAULT_IMAGE_TOTAL_SIZE = 500 * 1024; // 500 Kb
/** {#value} */
public static final int CONTINUE_LOADING_PERCENTAGE = 75;
private IoUtils() {
}
/**
* Copies stream, fires progress events by listener, can be interrupted by
* listener. Uses buffer size = {#value #DEFAULT_BUFFER_SIZE} bytes.
*
* #param is
* Input stream
* #param os
* Output stream
* #param listener
* null-ok; Listener of copying progress and controller of
* copying interrupting
* #return <b>true</b> - if stream copied successfully; <b>false</b> - if
* copying was interrupted by listener
* #throws IOException
*/
public static boolean copyStream(InputStream is, OutputStream os,
CopyListener listener) throws IOException {
return copyStream(is, os, listener, DEFAULT_BUFFER_SIZE);
}
/**
* Copies stream, fires progress events by listener, can be interrupted by
* listener.
*
* #param is
* Input stream
* #param os
* Output stream
* #param listener
* null-ok; Listener of copying progress and controller of
* copying interrupting
* #param bufferSize
* Buffer size for copying, also represents a step for firing
* progress listener callback, i.e. progress event will be fired
* after every copied <b>bufferSize</b> bytes
* #return <b>true</b> - if stream copied successfully; <b>false</b> - if
* copying was interrupted by listener
* #throws IOException
*/
public static boolean copyStream(InputStream is, OutputStream os,
CopyListener listener, int bufferSize) throws IOException {
int current = 0;
int total = is.available();
if (total <= 0) {
total = DEFAULT_IMAGE_TOTAL_SIZE;
}
final byte[] bytes = new byte[bufferSize];
int count;
if (shouldStopLoading(listener, current, total))
return false;
while ((count = is.read(bytes, 0, bufferSize)) != -1) {
os.write(bytes, 0, count);
current += count;
if (shouldStopLoading(listener, current, total))
return false;
}
os.flush();
return true;
}
private static boolean shouldStopLoading(CopyListener listener,
int current, int total) {
if (listener != null) {
boolean shouldContinue = listener.onBytesCopied(current, total);
if (!shouldContinue) {
if (100 * current / total < CONTINUE_LOADING_PERCENTAGE) {
return true; // if loaded more than 75% then continue
// loading anyway
}
}
}
return false;
}
/**
* Reads all data from stream and close it silently
*
* #param is
* Input stream
*/
public static void readAndCloseStream(InputStream is) {
final byte[] bytes = new byte[DEFAULT_BUFFER_SIZE];
try {
while (is.read(bytes, 0, DEFAULT_BUFFER_SIZE) != -1)
;
} catch (IOException e) {
// Do nothing
} finally {
closeSilently(is);
}
}
public static void closeSilently(Closeable closeable) {
try {
closeable.close();
} catch (Exception e) {
// Do nothing
}
}
/** Listener and controller for copy process */
public static interface CopyListener {
/**
* #param current
* Loaded bytes
* #param total
* Total bytes for loading
* #return <b>true</b> - if copying should be continued; <b>false</b> -
* if copying should be interrupted
*/
boolean onBytesCopied(int current, int total);
}
}
AssetManager assetManager = getAssets();
InputStream is = null;
try {
is = assetManager.open("filename");
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
try {
OutputStream os = null;
IoUtils.copyStream(is, os);
} catch (IOException e1) {
// TODO Auto-generated catch block
e1.printStackTrace();
}

you can't write to a file in assets fold.But you can write it to file system.

Performance issue with Volley's DiskBasedCache

In my Photo Collage app for Android I'm using Volley for loading images.
I'm using the DiskBasedCache (included with volley) with 50 mb storage to prevent re-downloading the same images multiple times.
Last time I checked the DiskBasedCache contained about 1000 cache entries.
When my app starts Volley calls mCache.initialize() and it will spend about 10 seconds (!) on my Galaxy S4 to do the following:
List all files in cache folder
Open each and every file and read the header section.
I find that reading 1000+ files at startup is not a very efficient way to load the cache index! :-)
From volley/toolbox/DiskBasedCache.java:
#Override
public synchronized void initialize() {
if (!mRootDirectory.exists()) {
if (!mRootDirectory.mkdirs()) {
VolleyLog.e("Unable to create cache dir %s", mRootDirectory.getAbsolutePath());
}
return;
}
File[] files = mRootDirectory.listFiles();
if (files == null) {
return;
}
for (File file : files) {
FileInputStream fis = null;
try {
fis = new FileInputStream(file);
CacheHeader entry = CacheHeader.readHeader(fis);
entry.size = file.length();
putEntry(entry.key, entry);
} catch (IOException e) {
if (file != null) {
file.delete();
}
} finally {
try {
if (fis != null) {
fis.close();
}
} catch (IOException ignored) { }
}
}
}
I'm looking for a fast and scalable solution. Perhaps an alternative DiskBasedCache implementation or suggestions on how to improve the volley library.
Update: (2014-01-06)
Noticing that the Volley cache used a lot of small (1 byte) IO read/writes. I cloned DiskBasedCache.java and encapsulating all FileInputStreams and FileOutputStreams with BufferedInputStream and BufferedOutputStreams. I found that that this optimization gave me a 3-10 times speed up.
This modification has a low risks of bugs compared to writing a new disk cache with a central index file.
Update: (2014-01-10)
Here is new class BufferedDiskBasedCache.java that I'm using now.
package no.ludde.android.ds.android.volley;
/*
* Copyright (C) 2011 The Android Open Source Project
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
import android.os.SystemClock;
import com.android.volley.Cache;
import com.android.volley.VolleyLog;
import java.io.BufferedInputStream;
import java.io.BufferedOutputStream;
import java.io.EOFException;
import java.io.File;
import java.io.FileInputStream;
import java.io.FileOutputStream;
import java.io.FilterInputStream;
import java.io.IOException;
import java.io.InputStream;
import java.io.OutputStream;
import java.util.Collections;
import java.util.HashMap;
import java.util.Iterator;
import java.util.LinkedHashMap;
import java.util.Map;
/**
* Cache implementation that caches files directly onto the hard disk in the specified
* directory. The default disk usage size is 5MB, but is configurable.
*/
public class BufferedDiskBasedCache implements Cache {
/** Map of the Key, CacheHeader pairs */
private final Map<String, CacheHeader> mEntries =
new LinkedHashMap<String, CacheHeader>(16, .75f, true);
/** Total amount of space currently used by the cache in bytes. */
private long mTotalSize = 0;
/** The root directory to use for the cache. */
private final File mRootDirectory;
/** The maximum size of the cache in bytes. */
private final int mMaxCacheSizeInBytes;
/** Default maximum disk usage in bytes. */
private static final int DEFAULT_DISK_USAGE_BYTES = 5 * 1024 * 1024;
/** High water mark percentage for the cache */
private static final float HYSTERESIS_FACTOR = 0.9f;
/** Magic number for current version of cache file format. */
private static final int CACHE_MAGIC = 0x20120504;
/**
* Constructs an instance of the DiskBasedCache at the specified directory.
* #param rootDirectory The root directory of the cache.
* #param maxCacheSizeInBytes The maximum size of the cache in bytes.
*/
public BufferedDiskBasedCache(File rootDirectory, int maxCacheSizeInBytes) {
mRootDirectory = rootDirectory;
mMaxCacheSizeInBytes = maxCacheSizeInBytes;
}
/**
* Constructs an instance of the DiskBasedCache at the specified directory using
* the default maximum cache size of 5MB.
* #param rootDirectory The root directory of the cache.
*/
public BufferedDiskBasedCache(File rootDirectory) {
this(rootDirectory, DEFAULT_DISK_USAGE_BYTES);
}
/**
* Clears the cache. Deletes all cached files from disk.
*/
#Override
public synchronized void clear() {
File[] files = mRootDirectory.listFiles();
if (files != null) {
for (File file : files) {
file.delete();
}
}
mEntries.clear();
mTotalSize = 0;
VolleyLog.d("Cache cleared.");
}
/**
* Returns the cache entry with the specified key if it exists, null otherwise.
*/
#Override
public synchronized Entry get(String key) {
CacheHeader entry = mEntries.get(key);
// if the entry does not exist, return.
if (entry == null) {
return null;
}
File file = getFileForKey(key);
CountingInputStream cis = null;
try {
cis = new CountingInputStream(new BufferedInputStream(new FileInputStream(file)));
CacheHeader.readHeader(cis); // eat header
byte[] data = streamToBytes(cis, (int) (file.length() - cis.bytesRead));
return entry.toCacheEntry(data);
} catch (IOException e) {
VolleyLog.d("%s: %s", file.getAbsolutePath(), e.toString());
remove(key);
return null;
} finally {
if (cis != null) {
try {
cis.close();
} catch (IOException ioe) {
return null;
}
}
}
}
/**
* Initializes the DiskBasedCache by scanning for all files currently in the
* specified root directory. Creates the root directory if necessary.
*/
#Override
public synchronized void initialize() {
if (!mRootDirectory.exists()) {
if (!mRootDirectory.mkdirs()) {
VolleyLog.e("Unable to create cache dir %s", mRootDirectory.getAbsolutePath());
}
return;
}
File[] files = mRootDirectory.listFiles();
if (files == null) {
return;
}
for (File file : files) {
BufferedInputStream fis = null;
try {
fis = new BufferedInputStream(new FileInputStream(file));
CacheHeader entry = CacheHeader.readHeader(fis);
entry.size = file.length();
putEntry(entry.key, entry);
} catch (IOException e) {
if (file != null) {
file.delete();
}
} finally {
try {
if (fis != null) {
fis.close();
}
} catch (IOException ignored) { }
}
}
}
/**
* Invalidates an entry in the cache.
* #param key Cache key
* #param fullExpire True to fully expire the entry, false to soft expire
*/
#Override
public synchronized void invalidate(String key, boolean fullExpire) {
Entry entry = get(key);
if (entry != null) {
entry.softTtl = 0;
if (fullExpire) {
entry.ttl = 0;
}
put(key, entry);
}
}
/**
* Puts the entry with the specified key into the cache.
*/
#Override
public synchronized void put(String key, Entry entry) {
pruneIfNeeded(entry.data.length);
File file = getFileForKey(key);
try {
BufferedOutputStream fos = new BufferedOutputStream(new FileOutputStream(file));
CacheHeader e = new CacheHeader(key, entry);
e.writeHeader(fos);
fos.write(entry.data);
fos.close();
putEntry(key, e);
return;
} catch (IOException e) {
}
boolean deleted = file.delete();
if (!deleted) {
VolleyLog.d("Could not clean up file %s", file.getAbsolutePath());
}
}
/**
* Removes the specified key from the cache if it exists.
*/
#Override
public synchronized void remove(String key) {
boolean deleted = getFileForKey(key).delete();
removeEntry(key);
if (!deleted) {
VolleyLog.d("Could not delete cache entry for key=%s, filename=%s",
key, getFilenameForKey(key));
}
}
/**
* Creates a pseudo-unique filename for the specified cache key.
* #param key The key to generate a file name for.
* #return A pseudo-unique filename.
*/
private String getFilenameForKey(String key) {
int firstHalfLength = key.length() / 2;
String localFilename = String.valueOf(key.substring(0, firstHalfLength).hashCode());
localFilename += String.valueOf(key.substring(firstHalfLength).hashCode());
return localFilename;
}
/**
* Returns a file object for the given cache key.
*/
public File getFileForKey(String key) {
return new File(mRootDirectory, getFilenameForKey(key));
}
/**
* Prunes the cache to fit the amount of bytes specified.
* #param neededSpace The amount of bytes we are trying to fit into the cache.
*/
private void pruneIfNeeded(int neededSpace) {
if ((mTotalSize + neededSpace) < mMaxCacheSizeInBytes) {
return;
}
if (VolleyLog.DEBUG) {
VolleyLog.v("Pruning old cache entries.");
}
long before = mTotalSize;
int prunedFiles = 0;
long startTime = SystemClock.elapsedRealtime();
Iterator<Map.Entry<String, CacheHeader>> iterator = mEntries.entrySet().iterator();
while (iterator.hasNext()) {
Map.Entry<String, CacheHeader> entry = iterator.next();
CacheHeader e = entry.getValue();
boolean deleted = getFileForKey(e.key).delete();
if (deleted) {
mTotalSize -= e.size;
} else {
VolleyLog.d("Could not delete cache entry for key=%s, filename=%s",
e.key, getFilenameForKey(e.key));
}
iterator.remove();
prunedFiles++;
if ((mTotalSize + neededSpace) < mMaxCacheSizeInBytes * HYSTERESIS_FACTOR) {
break;
}
}
if (VolleyLog.DEBUG) {
VolleyLog.v("pruned %d files, %d bytes, %d ms",
prunedFiles, (mTotalSize - before), SystemClock.elapsedRealtime() - startTime);
}
}
/**
* Puts the entry with the specified key into the cache.
* #param key The key to identify the entry by.
* #param entry The entry to cache.
*/
private void putEntry(String key, CacheHeader entry) {
if (!mEntries.containsKey(key)) {
mTotalSize += entry.size;
} else {
CacheHeader oldEntry = mEntries.get(key);
mTotalSize += (entry.size - oldEntry.size);
}
mEntries.put(key, entry);
}
/**
* Removes the entry identified by 'key' from the cache.
*/
private void removeEntry(String key) {
CacheHeader entry = mEntries.get(key);
if (entry != null) {
mTotalSize -= entry.size;
mEntries.remove(key);
}
}
/**
* Reads the contents of an InputStream into a byte[].
* */
private static byte[] streamToBytes(InputStream in, int length) throws IOException {
byte[] bytes = new byte[length];
int count;
int pos = 0;
while (pos < length && ((count = in.read(bytes, pos, length - pos)) != -1)) {
pos += count;
}
if (pos != length) {
throw new IOException("Expected " + length + " bytes, read " + pos + " bytes");
}
return bytes;
}
/**
* Handles holding onto the cache headers for an entry.
*/
// Visible for testing.
static class CacheHeader {
/** The size of the data identified by this CacheHeader. (This is not
* serialized to disk. */
public long size;
/** The key that identifies the cache entry. */
public String key;
/** ETag for cache coherence. */
public String etag;
/** Date of this response as reported by the server. */
public long serverDate;
/** TTL for this record. */
public long ttl;
/** Soft TTL for this record. */
public long softTtl;
/** Headers from the response resulting in this cache entry. */
public Map<String, String> responseHeaders;
private CacheHeader() { }
/**
* Instantiates a new CacheHeader object
* #param key The key that identifies the cache entry
* #param entry The cache entry.
*/
public CacheHeader(String key, Entry entry) {
this.key = key;
this.size = entry.data.length;
this.etag = entry.etag;
this.serverDate = entry.serverDate;
this.ttl = entry.ttl;
this.softTtl = entry.softTtl;
this.responseHeaders = entry.responseHeaders;
}
/**
* Reads the header off of an InputStream and returns a CacheHeader object.
* #param is The InputStream to read from.
* #throws IOException
*/
public static CacheHeader readHeader(InputStream is) throws IOException {
CacheHeader entry = new CacheHeader();
int magic = readInt(is);
if (magic != CACHE_MAGIC) {
// don't bother deleting, it'll get pruned eventually
throw new IOException();
}
entry.key = readString(is);
entry.etag = readString(is);
if (entry.etag.equals("")) {
entry.etag = null;
}
entry.serverDate = readLong(is);
entry.ttl = readLong(is);
entry.softTtl = readLong(is);
entry.responseHeaders = readStringStringMap(is);
return entry;
}
/**
* Creates a cache entry for the specified data.
*/
public Entry toCacheEntry(byte[] data) {
Entry e = new Entry();
e.data = data;
e.etag = etag;
e.serverDate = serverDate;
e.ttl = ttl;
e.softTtl = softTtl;
e.responseHeaders = responseHeaders;
return e;
}
/**
* Writes the contents of this CacheHeader to the specified OutputStream.
*/
public boolean writeHeader(OutputStream os) {
try {
writeInt(os, CACHE_MAGIC);
writeString(os, key);
writeString(os, etag == null ? "" : etag);
writeLong(os, serverDate);
writeLong(os, ttl);
writeLong(os, softTtl);
writeStringStringMap(responseHeaders, os);
os.flush();
return true;
} catch (IOException e) {
VolleyLog.d("%s", e.toString());
return false;
}
}
}
private static class CountingInputStream extends FilterInputStream {
private int bytesRead = 0;
private CountingInputStream(InputStream in) {
super(in);
}
#Override
public int read() throws IOException {
int result = super.read();
if (result != -1) {
bytesRead++;
}
return result;
}
#Override
public int read(byte[] buffer, int offset, int count) throws IOException {
int result = super.read(buffer, offset, count);
if (result != -1) {
bytesRead += result;
}
return result;
}
}
/*
* Homebrewed simple serialization system used for reading and writing cache
* headers on disk. Once upon a time, this used the standard Java
* Object{Input,Output}Stream, but the default implementation relies heavily
* on reflection (even for standard types) and generates a ton of garbage.
*/
/**
* Simple wrapper around {#link InputStream#read()} that throws EOFException
* instead of returning -1.
*/
private static int read(InputStream is) throws IOException {
int b = is.read();
if (b == -1) {
throw new EOFException();
}
return b;
}
static void writeInt(OutputStream os, int n) throws IOException {
os.write((n >> 0) & 0xff);
os.write((n >> 8) & 0xff);
os.write((n >> 16) & 0xff);
os.write((n >> 24) & 0xff);
}
static int readInt(InputStream is) throws IOException {
int n = 0;
n |= (read(is) << 0);
n |= (read(is) << 8);
n |= (read(is) << 16);
n |= (read(is) << 24);
return n;
}
static void writeLong(OutputStream os, long n) throws IOException {
os.write((byte)(n >>> 0));
os.write((byte)(n >>> 8));
os.write((byte)(n >>> 16));
os.write((byte)(n >>> 24));
os.write((byte)(n >>> 32));
os.write((byte)(n >>> 40));
os.write((byte)(n >>> 48));
os.write((byte)(n >>> 56));
}
static long readLong(InputStream is) throws IOException {
long n = 0;
n |= ((read(is) & 0xFFL) << 0);
n |= ((read(is) & 0xFFL) << 8);
n |= ((read(is) & 0xFFL) << 16);
n |= ((read(is) & 0xFFL) << 24);
n |= ((read(is) & 0xFFL) << 32);
n |= ((read(is) & 0xFFL) << 40);
n |= ((read(is) & 0xFFL) << 48);
n |= ((read(is) & 0xFFL) << 56);
return n;
}
static void writeString(OutputStream os, String s) throws IOException {
byte[] b = s.getBytes("UTF-8");
writeLong(os, b.length);
os.write(b, 0, b.length);
}
static String readString(InputStream is) throws IOException {
int n = (int) readLong(is);
byte[] b = streamToBytes(is, n);
return new String(b, "UTF-8");
}
static void writeStringStringMap(Map<String, String> map, OutputStream os) throws IOException {
if (map != null) {
writeInt(os, map.size());
for (Map.Entry<String, String> entry : map.entrySet()) {
writeString(os, entry.getKey());
writeString(os, entry.getValue());
}
} else {
writeInt(os, 0);
}
}
static Map<String, String> readStringStringMap(InputStream is) throws IOException {
int size = readInt(is);
Map<String, String> result = (size == 0)
? Collections.<String, String>emptyMap()
: new HashMap<String, String>(size);
for (int i = 0; i < size; i++) {
String key = readString(is).intern();
String value = readString(is).intern();
result.put(key, value);
}
return result;
}
}

Yes, the way DiskBasedCache works it needs to open all the files in initialize(). Which is simply.... not a good idea :-(
You need to make a different implementation that doesent open all the files at startup.
Take a copy of DiskBasedCache and change initialize() to
#Override
public synchronized void initialize() {
if (!mRootDirectory.exists()) {
if (!mRootDirectory.mkdirs()) {
VolleyLog.e("Unable to create cache dir %s", mRootDirectory.getAbsolutePath());
}
}
}
And change get() so it makes an additional check for if the file exists on the file system, like
#Override
public synchronized Entry get(String key) {
CacheHeader entry = mEntries.get(key);
File file = getFileForKey(key);
if (entry == null && !file.exists()) { // EXTRA CHECK
// if the entry does not exist, return.
VolleyLog.d("DrVolleyDiskBasedCache miss for " + key);
return null;
}
...
I use this approach in https://play.google.com/store/apps/details?id=dk.dr.radio and it works fine - its robustness have been tested by ~300000 users :-)
You can download a full version of the file from https://code.google.com/p/dr-radio-android/source/browse/trunk/DRRadiov3/src/dk/dr/radio/net/volley/DrDiskBasedCache.java (you'll have to delete some DR Radio specific stuff)

In the streamToBytes(), first it will new bytes by the cache file length, does your cache file was too large than application maximum heap size ?
private static byte[] streamToBytes(InputStream in, int length) throws IOException {
byte[] bytes = new byte[length];
...
}
public synchronized Entry get(String key) {
CacheHeader entry = mEntries.get(key);
File file = getFileForKey(key);
byte[] data = streamToBytes(..., file.length());
}
If you want to clear the cache, you could keep the DiskBasedCache reference, after clear time's came, use ClearCacheRequest and pass that cache instance in :
File cacheDir = new File(context.getCacheDir(), DEFAULT_CACHE_DIR);
DiskBasedCache cache = new DiskBasedCache(cacheDir);
RequestQueue queue = new RequestQueue(cache, network);
queue.start();
// clear all volley caches.
queue.add(new ClearCacheRequest(cache, null));
this way will clear all caches, so I suggest you use it carefully. of course, you can doing conditional check, just iterating the cacheDir files, estimate which was too large then remove it.
for (File cacheFile : cacheDir.listFiles()) {
if (cacheFile.isFile() && cacheFile.length() > 10000000) cacheFile.delete();
}
Volley wasn't design as a big data cache solution, it's common request cache, don't storing large data anytime.
------------- Update at 2014-07-17 -------------
In fact, clear all caches is final way, also isn't wise way, we should suppressing these large request use cache when we sure it would be, and if not sure? we still can determine the response data size whether large or not, then call setShouldCache(false) to disable it.
public class TheRequest extends Request {
#Override
protected Response<String> parseNetworkResponse(NetworkResponse response) {
// if response data was too large, disable caching is still time.
if (response.data.length > 10000) setShouldCache(false);
...
}
}

My initial thought was to use the DiskLruCache written by Jake Wharton by writing a com.android.volley.Cache wrapper over it.
But I finally implemented a singleton pattern for the Volley, combined with the cache creation in an AsyncTask called from the Application context
public static synchronized VolleyClient getInstance(Context context)
{
if (mInstance == null)
{
mInstance = new VolleyClient(context);
}
return mInstance;
}
private VolleyClient(Context context)
{
this.context = context;
VolleyCacheInitializer volleyCacheInitializer = new VolleyCacheInitializer();
volleyCacheInitializer.execute();
}
private class VolleyCacheInitializer extends AsyncTask<Void, Void, Boolean>
{
#Override
protected Boolean doInBackground(Void... params)
{
// Instantiate the cache with 50MB Cache Size
Cache diskBasedCache = new DiskBasedCache(context.getCacheDir(), 50 * 1024 * 1024);
// Instantiate the RequestQueue with the cache and network.
mRequestQueue = new RequestQueue(diskBasedCache, network);
// Start the queue which calls the DiskBasedCache.initialize()
mRequestQueue.start();
return true;
}
#Override
protected void onPostExecute(Boolean aBoolean)
{
super.onPostExecute(aBoolean);
if(aBoolean)
Log.d(TAG, "Volley request queue initialized");
else
Log.d(TAG, "Volley request queue initialization failed");
}
}
Inside MyApplication class
#Override
public void onCreate()
{
super.onCreate();
// Initialize an application level volley request queue
VolleyClient volleyHttpClient = VolleyClient.getInstance(this);
}

VM aborting Getting error while running in emulator

I am using phonegap with socket programming.
When I check on emulator I am getting VM aborting error.
I am just calling my server and checking it to my emulator.
Here is my code.
package com.example.test;
import org.apache.cordova.DroidGap;
import android.os.Bundle;
import android.app.Activity;
import android.view.Menu;
public class MainActivity extends DroidGap {
#Override
public void onCreate(Bundle savedInstanceState) {
super.onCreate(savedInstanceState);
//setContentView(R.layout.activity_main);
super.loadUrl("file:///android_asset/www/index.html");
appView.addJavascriptInterface(new WebSocketFactory(appView), "WebSocketFactory");
}
}
---------------------------------------------------------------------
package com.example.test;
import java.io.IOException;
import java.io.UnsupportedEncodingException;
import java.net.InetSocketAddress;
import java.net.URI;
import java.nio.ByteBuffer;
import java.nio.channels.NotYetConnectedException;
import java.nio.channels.SelectionKey;
import java.nio.channels.Selector;
import java.nio.channels.SocketChannel;
import java.security.MessageDigest;
import java.security.NoSuchAlgorithmException;
import java.util.Iterator;
import java.util.Random;
import java.util.Set;
import java.util.concurrent.BlockingQueue;
import java.util.concurrent.LinkedBlockingQueue;
import android.util.Log;
import android.webkit.WebView;
public class WebSocket implements Runnable {
/**
* Enum for WebSocket Draft
*/
public enum Draft {
DRAFT75, DRAFT76
}
// //////////////// CONSTANT
/**
* The connection has not yet been established.
*/
public final static int WEBSOCKET_STATE_CONNECTING = 0;
/**
* The WebSocket connection is established and communication is possible.
*/
public final static int WEBSOCKET_STATE_OPEN = 1;
/**
* The connection is going through the closing handshake.
*/
public final static int WEBSOCKET_STATE_CLOSING = 2;
/**
* The connection has been closed or could not be opened.
*/
public final static int WEBSOCKET_STATE_CLOSED = 3;
/**
* An empty string
*/
private static String BLANK_MESSAGE = "";
/**
* The javascript method name for onOpen event.
*/
private static String EVENT_ON_OPEN = "onopen";
/**
* The javascript method name for onMessage event.
*/
private static String EVENT_ON_MESSAGE = "onmessage";
/**
* The javascript method name for onClose event.
*/
private static String EVENT_ON_CLOSE = "onclose";
/**
* The javascript method name for onError event.
*/
private static String EVENT_ON_ERROR = "onerror";
/**
* The default port of WebSockets, as defined in the spec.
*/
public static final int DEFAULT_PORT = 80;
/**
* The WebSocket protocol expects UTF-8 encoded bytes.
*/
public static final String UTF8_CHARSET = "UTF-8";
/**
* The byte representing Carriage Return, or \r
*/
public static final byte DATA_CR = (byte) 0x0D;
/**
* The byte representing Line Feed, or \n
*/
public static final byte DATA_LF = (byte) 0x0A;
/**
* The byte representing the beginning of a WebSocket text frame.
*/
public static final byte DATA_START_OF_FRAME = (byte) 0x00;
/**
* The byte representing the end of a WebSocket text frame.
*/
public static final byte DATA_END_OF_FRAME = (byte) 0xFF;
// //////////////// INSTANCE Variables
/**
* The WebView instance from Phonegap DroidGap
*/
private final WebView appView;
/**
* The unique id for this instance (helps to bind this to javascript events)
*/
private String id;
/**
* The URI this client is supposed to connect to.
*/
private URI uri;
/**
* The port of the websocket server
*/
private int port;
/**
* The Draft of the WebSocket protocol the Client is adhering to.
*/
private Draft draft;
/**
* The <tt>SocketChannel</tt> instance to use for this server connection.
* This is used to read and write data to.
*/
private SocketChannel socketChannel;
/**
* The 'Selector' used to get event keys from the underlying socket.
*/
private Selector selector;
/**
* Keeps track of whether or not the client thread should continue running.
*/
private boolean running;
/**
* Internally used to determine whether to recieve data as part of the
* remote handshake, or as part of a text frame.
*/
private boolean handshakeComplete;
/**
* The 1-byte buffer reused throughout the WebSocket connection to read
* data.
*/
private ByteBuffer buffer;
/**
* The bytes that make up the remote handshake.
*/
private ByteBuffer remoteHandshake;
/**
* The bytes that make up the current text frame being read.
*/
private ByteBuffer currentFrame;
/**
* Queue of buffers that need to be sent to the client.
*/
private BlockingQueue<ByteBuffer> bufferQueue;
/**
* Lock object to ensure that data is sent from the bufferQueue in the
* proper order
*/
private Object bufferQueueMutex = new Object();
/**
* Number 1 used in handshake
*/
private int number1 = 0;
/**
* Number 2 used in handshake
*/
private int number2 = 0;
/**
* Key3 used in handshake
*/
private byte[] key3 = null;
/**
* The readyState attribute represents the state of the connection.
*/
private int readyState = WEBSOCKET_STATE_CONNECTING;
private final WebSocket instance;
/**
* Constructor.
*
* Note: this is protected because it's supposed to be instantiated from {#link WebSocketFactory} only.
*
* #param appView
* {#link android.webkit.WebView}
* #param uri
* websocket server {#link URI}
* #param draft
* websocket server {#link Draft} implementation (75/76)
* #param id
* unique id for this instance
*/
protected WebSocket(WebView appView, URI uri, Draft draft, String id) {
this.appView = appView;
this.uri = uri;
this.draft = draft;
// port
port = uri.getPort();
if (port == -1) {
port = DEFAULT_PORT;
}
// Id
this.id = id;
this.bufferQueue = new LinkedBlockingQueue<ByteBuffer>();
this.handshakeComplete = false;
this.remoteHandshake = this.currentFrame = null;
this.buffer = ByteBuffer.allocate(1);
this.instance = this;
}
// //////////////////////////////////////////////////////////////////////////////////////
// /////////////////////////// WEB SOCKET API Methods
// ///////////////////////////////////
// //////////////////////////////////////////////////////////////////////////////////////
/**
* Starts a new Thread and connects to server
*
* #throws IOException
*/
public Thread connect() throws IOException {
this.running = true;
this.readyState = WEBSOCKET_STATE_CONNECTING;
// open socket
socketChannel = SocketChannel.open();
socketChannel.configureBlocking(false);
// set address
socketChannel.connect(new InetSocketAddress(uri.getHost(), port));
// start a thread to make connection
// More info:
// http://groups.google.com/group/android-developers/browse_thread/thread/45a8b53e9bf60d82
// http://stackoverflow.com/questions/2879455/android-2-2-and-bad-address-family-on-socket-connect
System.setProperty("java.net.preferIPv4Stack", "true");
System.setProperty("java.net.preferIPv6Addresses", "false");
selector = Selector.open();
socketChannel.register(selector, SelectionKey.OP_CONNECT);
Log.v("websocket", "Starting a new thread to manage data reading/writing");
Thread th = new Thread(this);
th.start();
// return thread object for explicit closing, if needed
return th;
}
public void run() {
while (this.running) {
try {
_connect();
} catch (IOException e) {
this.onError(e);
}
}
}
/**
* Closes connection with server
*/
public void close() {
this.readyState = WebSocket.WEBSOCKET_STATE_CLOSING;
// close socket channel
try {
this.socketChannel.close();
} catch (IOException e) {
this.onError(e);
}
this.running = false;
selector.wakeup();
// fire onClose method
this.onClose();
this.readyState = WebSocket.WEBSOCKET_STATE_CLOSED;
}
/**
* Sends <var>text</var> to server
*
* #param text
* String to send to server
*/
public void send(final String text) {
new Thread(new Runnable() {
#Override
public void run() {
if (instance.readyState == WEBSOCKET_STATE_OPEN) {
try {
instance._send(text);
} catch (IOException e) {
instance.onError(e);
}
} else {
instance.onError(new NotYetConnectedException());
}
}
}).start();
}
/**
* Called when an entire text frame has been received.
*
* #param msg
* Message from websocket server
*/
public void onMessage(final String msg) {
appView.post(new Runnable() {
#Override
public void run() {
appView.loadUrl(buildJavaScriptData(EVENT_ON_MESSAGE, msg));
}
});
}
public void onOpen() {
appView.post(new Runnable() {
#Override
public void run() {
appView.loadUrl(buildJavaScriptData(EVENT_ON_OPEN, BLANK_MESSAGE));
}
});
}
public void onClose() {
appView.post(new Runnable() {
#Override
public void run() {
appView.loadUrl(buildJavaScriptData(EVENT_ON_CLOSE, BLANK_MESSAGE));
}
});
}
public void onError(final Throwable t) {
appView.post(new Runnable() {
#Override
public void run() {
appView.loadUrl(buildJavaScriptData(EVENT_ON_ERROR, t.getMessage()));
}
});
}
public String getId() {
return id;
}
/**
* #return the readyState
*/
public int getReadyState() {
return readyState;
}
/**
* Builds text for javascript engine to invoke proper event method with
* proper data.
*
* #param event
* websocket event (onOpen, onMessage etc.)
* #param msg
* Text message received from websocket server
* #return
*/
private String buildJavaScriptData(String event, String msg) {
String _d = "javascript:WebSocket." + event + "(" + "{" + "\"_target\":\"" + id + "\"," + "\"data\":'" + msg.replaceAll("'", "\\\\'")
+ "'" + "}" + ")";
return _d;
}
// //////////////////////////////////////////////////////////////////////////////////////
// /////////////////////////// WEB SOCKET Internal Methods
// //////////////////////////////
// //////////////////////////////////////////////////////////////////////////////////////
private boolean _send(String text) throws IOException {
if (!this.handshakeComplete) {
throw new NotYetConnectedException();
}
if (text == null) {
throw new NullPointerException("Cannot send 'null' data to a WebSocket.");
}
// Get 'text' into a WebSocket "frame" of bytes
byte[] textBytes = text.getBytes(UTF8_CHARSET.toString());
ByteBuffer b = ByteBuffer.allocate(textBytes.length + 2);
b.put(DATA_START_OF_FRAME);
b.put(textBytes);
b.put(DATA_END_OF_FRAME);
b.rewind();
// See if we have any backlog that needs to be sent first
if (_write()) {
// Write the ByteBuffer to the socket
this.socketChannel.write(b);
}
// If we didn't get it all sent, add it to the buffer of buffers
if (b.remaining() > 0) {
if (!this.bufferQueue.offer(b)) {
throw new IOException("Buffers are full, message could not be sent to"
+ this.socketChannel.socket().getRemoteSocketAddress());
}
return false;
}
return true;
}
// actual connection logic
private void _connect() throws IOException {
// Continuous loop that is only supposed to end when "close" is called.
selector.select();
Set<SelectionKey> keys = selector.selectedKeys();
Iterator<SelectionKey> i = keys.iterator();
while (i.hasNext()) {
SelectionKey key = i.next();
i.remove();
if (key.isConnectable()) {
if (socketChannel.isConnectionPending()) {
socketChannel.finishConnect();
}
socketChannel.register(selector, SelectionKey.OP_READ);
_writeHandshake();
}
if (key.isReadable()) {
try {
_read();
} catch (NoSuchAlgorithmException nsa) {
this.onError(nsa);
}
}
}
}
private void _writeHandshake() throws IOException {
String path = this.uri.getPath();
if (path.indexOf("/") != 0) {
path = "/" + path;
}
String host = uri.getHost() + (port != DEFAULT_PORT ? ":" + port : "");
String origin = "*"; // TODO: Make 'origin' configurable
String request = "GET " + path + " HTTP/1.1\r\n" + "Upgrade: WebSocket\r\n" + "Connection: Upgrade\r\n"
+ "Host: " + host + "\r\n" + "Origin: " + origin + "\r\n";
// Add random keys for Draft76
if (this.draft == Draft.DRAFT76) {
request += "Sec-WebSocket-Key1: " + this._randomKey() + "\r\n";
request += "Sec-WebSocket-Key2: " + this._randomKey() + "\r\n";
request += "\r\n";
this.key3 = new byte[8];
(new Random()).nextBytes(this.key3);
// Convert to bytes early so last eight bytes don't get jacked
byte[] bRequest = request.getBytes(UTF8_CHARSET);
byte[] bToSend = new byte[bRequest.length + 8];
// Copy in the Request bytes
System.arraycopy(bRequest, 0, bToSend, 0, bRequest.length);
// Now tack on key3 bytes
System.arraycopy(this.key3, 0, bToSend, bRequest.length, this.key3.length);
// Now we can send all keys as a single frame
_write(bToSend);
return;
}
request += "\r\n";
_write(request.getBytes(UTF8_CHARSET));
}
private boolean _write() throws IOException {
synchronized (this.bufferQueueMutex) {
ByteBuffer buffer = this.bufferQueue.peek();
while (buffer != null) {
this.socketChannel.write(buffer);
if (buffer.remaining() > 0) {
return false; // Didn't finish this buffer. There's more to
// send.
} else {
this.bufferQueue.poll(); // Buffer finished. Remove it.
buffer = this.bufferQueue.peek();
}
}
return true;
}
}
private void _write(byte[] bytes) throws IOException {
this.socketChannel.write(ByteBuffer.wrap(bytes));
}
private void _read() throws IOException, NoSuchAlgorithmException {
this.buffer.rewind();
int bytesRead = -1;
try {
bytesRead = this.socketChannel.read(this.buffer);
} catch (Exception ex) {
}
if (bytesRead == -1) {
close();
} else if (bytesRead > 0) {
this.buffer.rewind();
if (!this.handshakeComplete) {
_readHandshake();
} else {
_readFrame();
}
}
}
private void _readFrame() throws UnsupportedEncodingException {
byte newestByte = this.buffer.get();
if (newestByte == DATA_START_OF_FRAME) { // Beginning of Frame
this.currentFrame = null;
} else if (newestByte == DATA_END_OF_FRAME) { // End of Frame
String textFrame = null;
// currentFrame will be null if END_OF_FRAME was send directly after
// START_OF_FRAME, thus we will send 'null' as the sent message.
if (this.currentFrame != null) {
textFrame = new String(this.currentFrame.array(), "ISO-8859-1");
}
// fire onMessage method
this.onMessage(textFrame);
} else { // Regular frame data, add to current frame buffer
ByteBuffer frame = ByteBuffer.allocate((this.currentFrame != null ? this.currentFrame.capacity() : 0)
+ this.buffer.capacity());
if (this.currentFrame != null) {
this.currentFrame.rewind();
frame.put(this.currentFrame);
}
frame.put(newestByte);
this.currentFrame = frame;
String textFrame = new String(this.currentFrame.array(),"ISO-8859-1");
this.onMessage(textFrame);
}
}
private void _readHandshake() throws IOException, NoSuchAlgorithmException {
ByteBuffer ch = ByteBuffer.allocate((this.remoteHandshake != null ? this.remoteHandshake.capacity() : 0)
+ this.buffer.capacity());
if (this.remoteHandshake != null) {
this.remoteHandshake.rewind();
ch.put(this.remoteHandshake);
}
ch.put(this.buffer);
this.remoteHandshake = ch;
byte[] h = this.remoteHandshake.array();
// If the ByteBuffer contains 16 random bytes, and ends with
// 0x0D 0x0A 0x0D 0x0A (or two CRLFs), then the client
// handshake is complete for Draft 76 Client.
if ((h.length >= 20 && h[h.length - 20] == DATA_CR && h[h.length - 19] == DATA_LF
&& h[h.length - 18] == DATA_CR && h[h.length - 17] == DATA_LF)) {
_readHandshake(new byte[] { h[h.length - 16], h[h.length - 15], h[h.length - 14], h[h.length - 13],
h[h.length - 12], h[h.length - 11], h[h.length - 10], h[h.length - 9], h[h.length - 8],
h[h.length - 7], h[h.length - 6], h[h.length - 5], h[h.length - 4], h[h.length - 3],
h[h.length - 2], h[h.length - 1] });
// If the ByteBuffer contains 8 random bytes,ends with
// 0x0D 0x0A 0x0D 0x0A (or two CRLFs), and the response
// contains Sec-WebSocket-Key1 then the client
// handshake is complete for Draft 76 Server.
} else if ((h.length >= 12 && h[h.length - 12] == DATA_CR && h[h.length - 11] == DATA_LF
&& h[h.length - 10] == DATA_CR && h[h.length - 9] == DATA_LF)
&& new String(this.remoteHandshake.array(), UTF8_CHARSET).contains("Sec-WebSocket-Key1")) {// ************************
_readHandshake(new byte[] { h[h.length - 8], h[h.length - 7], h[h.length - 6], h[h.length - 5],
h[h.length - 4], h[h.length - 3], h[h.length - 2], h[h.length - 1] });
// Consider Draft 75, and the Flash Security Policy
// Request edge-case.
} else if ((h.length >= 4 && h[h.length - 4] == DATA_CR && h[h.length - 3] == DATA_LF
&& h[h.length - 2] == DATA_CR && h[h.length - 1] == DATA_LF)
&& !(new String(this.remoteHandshake.array(), UTF8_CHARSET).contains("Sec"))
|| (h.length == 23 && h[h.length - 1] == 0)) {
_readHandshake(null);
}else{
_readHandshake(null);
}
}
private void _readHandshake(byte[] handShakeBody) throws IOException, NoSuchAlgorithmException {
// byte[] handshakeBytes = this.remoteHandshake.array();
// String handshake = new String(handshakeBytes, UTF8_CHARSET);
// TODO: Do some parsing of the returned handshake, and close connection
// in received anything unexpected!
this.handshakeComplete = true;
boolean isConnectionReady = true;
if (this.draft == WebSocket.Draft.DRAFT76) {
if (handShakeBody == null) {
isConnectionReady = true;
} else{
byte[] challenge = new byte[] { (byte) (this.number1 >> 24), (byte) ((this.number1 << 8) >> 24),
(byte) ((this.number1 << 16) >> 24), (byte) ((this.number1 << 24) >> 24),
(byte) (this.number2 >> 24), (byte) ((this.number2 << 8) >> 24),
(byte) ((this.number2 << 16) >> 24), (byte) ((this.number2 << 24) >> 24), this.key3[0],
this.key3[1], this.key3[2], this.key3[3], this.key3[4], this.key3[5], this.key3[6], this.key3[7] };
MessageDigest md5 = MessageDigest.getInstance("MD5");
byte[] expected = md5.digest(challenge);
for (int i = 0; i < handShakeBody.length; i++) {
if (expected[i] != handShakeBody[i]) {
isConnectionReady = true;
}
}
}
}
if (isConnectionReady) {
this.readyState = WEBSOCKET_STATE_OPEN;
// fire onOpen method
this.onOpen();
} else {
close();
}
}
private String _randomKey() {
Random r = new Random();
long maxNumber = 4294967295L;
long spaces = r.nextInt(12) + 1;
int max = new Long(maxNumber / spaces).intValue();
max = Math.abs(max);
int number = r.nextInt(max) + 1;
if (this.number1 == 0) {
this.number1 = number;
} else {
this.number2 = number;
}
long product = number * spaces;
String key = Long.toString(product);
int numChars = r.nextInt(12);
for (int i = 0; i < numChars; i++) {
int position = r.nextInt(key.length());
position = Math.abs(position);
char randChar = (char) (r.nextInt(95) + 33);
// exclude numbers here
if (randChar >= 48 && randChar <= 57) {
randChar -= 15;
}
key = new StringBuilder(key).insert(position, randChar).toString();
}
for (int i = 0; i < spaces; i++) {
int position = r.nextInt(key.length() - 1) + 1;
position = Math.abs(position);
key = new StringBuilder(key).insert(position, "\u0020").toString();
}
return key;
}
}
----------------------------------------------------------------------------
package com.example.test;
import java.net.URI;
import java.util.Random;
import android.webkit.WebView;
/**
* The <tt>WebSocketFactory</tt> is like a helper class to instantiate new
* WebSocket instaces especially from Javascript side. It expects a valid
* "ws://" URI.
*
* #author Animesh Kumar
*/
public class WebSocketFactory {
/** The app view. */
WebView appView;
/**
* Instantiates a new web socket factory.
*
* #param appView
* the app view
*/
public WebSocketFactory(WebView appView) {
this.appView = appView;
}
public WebSocket getInstance(String url) {
// use Draft75 by default
return getInstance(url, WebSocket.Draft.DRAFT76);
}
public WebSocket getInstance(String url, WebSocket.Draft draft) {
WebSocket socket = null;
Thread th = null;
try {
socket = new WebSocket(appView, new URI(url), draft, getRandonUniqueId());
th = socket.connect();
return socket;
} catch (Exception e) {
//Log.v("websocket", e.toString());
if(th != null) {
th.interrupt();
}
}
return null;
}
/**
* Generates random unique ids for WebSocket instances
*
* #return String
*/
private String getRandonUniqueId() {
return "WEBSOCKET." + new Random().nextInt(100);
}
}
-------------------------------------------------------------
<!DOCTYPE html>
<html>
<head>
<meta charset="utf-8" />
<meta name="format-detection" content="telephone=no" />
<meta name="viewport" content="user-scalable=no, initial-scale=1, maximum-scale=1, minimum-scale=1, width=device-width, height=device-height, target-densitydpi=device-dpi" />
<link rel="stylesheet" type="text/css" href="css/index.css" />
<title>Hello World</title>
</head>
<body>
<script type="text/javascript" src="cordova-2.7.0.js"></script>
<script type="text/javascript" src="js/index.js"></script>
<script type="text/javascript" src="js/websocket.js"></script>
<script type="text/javascript">
var socket = new WebSocket('ws://192.168.1.3:8101/');
// push a message after the connection is established.
socket.onopen = function() {
socket.send('Hello World')
};
// alerts message pushed from server
socket.onmessage = function(msg) {
console.log("TTTTT"+JSON.stringify(msg));
};
// alert close event
socket.onclose = function() {
alert('closed');
};
</script>
</body>
</html>
From the script I am calling my functions.
But when I check it on emulator it is showing that error.
Please help me removing this error.
Thanks.