Android parsing ints efficiency - android

I am trying to parse some unsigned integers from a file stored on the memory card.
Since java does not provide unsigned types I use longs to store the obtained values.
The performance seems to be influenced by the Debugger:
Without the debugger connected I can parse 20000 values in ~820ms.
With the debugger ~5800ms (on HTC Desire).
Can you reproduce this on your devices? Any clue why the debugger has such an overhead?
Here is the code:
public class IntParsingTest extends Activity {
public static final String TEST_FILE = "yourfile";
public static final int LOOPS = 20000;
private Handler handler = new Handler() {
#Override
public void handleMessage(Message msg) {
super.handleMessage(msg);
Toast.makeText(IntParsingTest.this, (CharSequence) msg.obj, Toast.LENGTH_LONG).show();
}
};
#Override
public void onCreate(Bundle savedInstanceState) {
super.onCreate(savedInstanceState);
setContentView(R.layout.main);
try {
Runnable r = new Runnable() {
#Override
public void run() {
Message m = new Message();
File path = Environment.getExternalStorageDirectory();
File f = new File(path, TEST_FILE);
long startTime = SystemClock.currentThreadTimeMillis();
try {
InputStream is = new FileInputStream(f);
for (int i = 0; i < LOOPS; i++) {
readUnsignedInt(is);
}
long total = SystemClock.currentThreadTimeMillis() - startTime;
m.obj = "Time to parse\r\n"
+ LOOPS
+ " unsigned integers:\r\n"
+ total + " ms";
handler.sendMessage(m);
} catch (Exception e) {
m.obj = e.getMessage();
handler.sendMessage(m);
}
}
};
new Thread(r).start();
} catch (Exception e) {
}
}
public static long readUnsignedInt(InputStream input) throws IOException {
long b1 = input.read();
long b2 = input.read();
long b3 = input.read();
long b4 = input.read();
return (b1 << 24) + (b2 << 16) + (b3 << 8) + b4;
}
}
Edit: It is also interesting that only the shift opperations are slow. If I return one of the bytes or a sum of them it works much faster. It may be the way shift is implemented !?

I'm guessing here - with debugger connected JVM cannot do just-in-time compilation; to be able to detect breakpoints it has to do bytecode interpretation all the time. Debugger also connects to device log stream, which may cause some overhead.

Related

Android How to convert UDP recieving audio into wav file

I am trying to recieve an streaming audio from my app.
below is my code for recieving audio stream:
public class ClientListen implements Runnable {
private Context context;
public ClientListen(Context context) {
this.context = context;
}
#Override
public void run() {
boolean run = true;
try {
DatagramSocket udpSocket = new DatagramSocket(8765);
InetAddress serverAddr = null;
try {
serverAddr = InetAddress.getByName("127.0.0.1");
} catch (UnknownHostException e) {
e.printStackTrace();
}
while (run) {
try {
byte[] message = new byte[8000];
DatagramPacket packet = new DatagramPacket(message,message.length);
Log.i("UDP client: ", "about to wait to receive");
udpSocket.setSoTimeout(10000);
udpSocket.receive(packet);
String text = new String(packet.getData(), 0, packet.getLength());
Log.d("Received text", text);
} catch (IOException e) {
Log.e(" UDP clien", "error: ", e);
run = false;
udpSocket.close();
}
}
} catch (SocketException e) {
Log.e("Socket Open:", "Error:", e);
} catch (IOException e) {
e.printStackTrace();
}
}
}
In Received text logger i can see data as coming as
D/Received text: �������n�����������q�9�$�0�/�G�{�������s�����JiH&������d�����Z���������d�����E������C�+
��l��y�����������v���9����������u��f�j�������$�����K���������F��~R�2�����T��������������L�����!��G��8������s�;�"�,�R�����(��{�����*_��Z�������5������������\������x���j~������������/��=�����%�������
How can store this data into a wav file ?
What you see is the string representation of single udp packet after it was received and the received block has just being released.
It is a very small fraction of the sound you want to convert to wave.
Soon the while loop will continue and you will receive another packet and many more..
You need to collect all the packets in a buffer and then when you think it is ok - convert them to wave file.
Remember Wave is not just the sound bytes you get from udp but also 44 bytes of prefix you need to add to this file in order to be recognized by players.
Also if the udp is from another encoding format such as G711 - you must encode these bytes to PCM – if not you will hear heavy noise in the
Sound of the wave or the stream you play.
The buffer must be accurate. if it will be too big (many empty bytes in the end of the array) you will hear a sound of helicopter. if you know exactly what is the size of each packet then you can just write it to AudioTrack in order to play stream, or accumulate it and convert it to wave file when you will see fit. But If you are not sure about the size you can use this answer to get a buffer and then write the buffer to AudioTrack:
Android AudioRecord to Server over UDP Playback Issues.
they use Javax because it is very old answer but you just need to use AudioTrack instead in order to stream. It is not in this scope so I will just present the AudioTrack streaming replacements instead of Javax SourceDataLine:
final int SAMPLE_RATE = 8000; // Hertz
final int STREAM_TYPE = AudioManager.STREAM_NOTIFICATION;
int channelConfig = AudioFormat.CHANNEL_OUT_MONO;
int encodingFormat = AudioFormat.ENCODING_PCM_16BIT;
AudioTrack track = new AudioTrack(STREAM_TYPE, SAMPLE_RATE, channelConfig,
encodingFormat, BUF_SIZE, AudioTrack.MODE_STREAM);
track.play();
//.. then after receive UDP packets and the buffer is full:
if(track != null && packet != null){
track.write(audioStreamBuffer, 0, audioStreamBuffer.length);
}
You must not do this in the UI thread (I assume you know that).
In the code I will show you - I am getting udp of audio logs from PTT radio. It is encoded in G711 Ulaw . each packet is of 172 bytes exactly. First 12 bytes are for RTP and I need to offset (remove) them in order to eliminate small noises. rest 160 bytes are 20MS of sound.
I must decode the G711 Ulaw bytes to PCM shorts array. Then to take the short array and to make a wave file out of it. I am taking it after I see there was no packet receiving for more than one second (so I know the speech ended and the new block release is because of a new speech so I can take the old speech and make a wave file out of it). You can decide of a different buffer depends on what you are doing.
It works fine. After the decoding the sound of the wave is very good. If you have UDP with PCM so you don’t need to decode G711 - just skip this part.
Finally I want to mention I saw many old answers with code parts using javax.sound.sampled that seems great because it can convert easily an audio file or stream to wave format with AudioFileFormat
And also convert G711 to pcm with AudioFormat manipulations. But unfortunately it is not part of current java for android. We must count on android AudioTrack instead (and AudioRecord if we want to get the sound from the mic) but AudioTrack play only PCM and do not support G711 format – so when streaming G711 with AudioTrack the noise is terrible. We must decode it in our code before writing it to the track. Also we cannot convert to wave file using audioInputStream – I tried to do this easily with javax.sound.sampled jar file I added to my app but android keep giving me errors such as format not supported for wave, and mixer errors when try to stream – so I understood latest android cannot work with javax.sound.sampled and I went to look for law level decoding of G711 and law level creation of wave file out of the buffer of byte array received from the UDP packets .
A. in manifest add:
<uses-permission android:name="android.permission.WRITE_EXTERNAL_STORAGE"/>
<uses-permission android:name="android.permission.READ_EXTERNAL_STORAGE"/>
<uses-permission android:name="android.permission.INTERNET"/>
B. in the worker thread:
#Override
public void run(){
Log.i(TAG, "ClientListen thread started. Thread id: " + Thread.currentThread().getId());
try{
udpSocket = new DatagramSocket(port);
}catch(SocketException e){
e.printStackTrace();
}
byte[] messageBuf = new byte[BUF_SIZE];
Log.i(TAG, "waiting to receive packet in port: " + port);
if(udpSocket != null){
// here you can create new AudioTrack and play.track
byte pttSession[] = null;
while (running){
packet = new DatagramPacket(messageBuf, 0, messageBuf.length);
Log.d(TAG, "inside while running loop");
try{
Log.d(TAG, "receive block: waiting for user to press on
speaker(listening now inside udpSocket for DatagramPacket..)");
//get inside receive block until packet will arrive through this socket
long timeBeforeBlock = System.currentTimeMillis();
udpSocket.receive(packet);
Log.d(TAG, "client received a packet, receive block stopped)");
//this is for sending msg handler to the UI tread (you may skip this)
sendState("getting UDP packets...");
/* if previous block release happened more than one second ago - so this
packet release is for a new speech. so let’s copy the previous speech
to a wave file and empty the speech */
if(System.currentTimeMillis() - timeBeforeBlock > 1000 && pttSession != null){
convertBytesToFile(pttSession);
pttSession = null;
}
/* let’s take the packet that was released and start new speech or add it to the ongoing speech. */
byte[] slice = Arrays.copyOfRange(packet.getData(), 12, packet.getLength());
if(null == pttSession){
pttSession = slice;
}else{
pttSession = concat(pttSession, slice);
Log.d(TAG, "pttSession:" + Arrays.toString(pttSession));
}
}catch(IOException e){
Log.e(TAG, "UDP client IOException - error: ", e);
running = false;
}
}
// let’s take the latest speech and make a last wave file out of it.
if(pttSession != null){
convertBytesToFile(pttSession);
pttSession = null;
}
// if running == false then stop listen.
udpSocket.close();
handler.sendEmptyMessage(MainActivity.UdpClientHandler.UPDATE_END);
}else{
sendState("cannot bind datagram socket to the specified port:" + port);
}
}
private void convertBytesToFile(byte[] byteArray){
//decode the bytes from G711U to PCM (outcome is a short array)
G711UCodec decoder = new G711UCodec();
int size = byteArray.length;
short[] shortArray = new short[size];
decoder.decode(shortArray, byteArray, size, 0);
String newFileName = "speech_" + System.currentTimeMillis() + ".wav";
//convert short array to wav (add 44 prefix shorts) and save it as a .wav file
Wave wave = new Wave(SAMPLE_RATE, (short) 1, shortArray, 0, shortArray.length - 1);
if(wave.writeToFile(Environment.getExternalStoragePublicDirectory
(Environment.DIRECTORY_DOWNLOADS),newFileName)){
Log.d(TAG, "wave.writeToFile successful!");
sendState("create file: "+ newFileName);
}else{
Log.w(TAG, "wave.writeToFile failed");
}
}
C. encoding/decoding G711 U-Law class:
taken from: https://github.com/thinktube-kobe/airtube/blob/master/JavaLibrary/src/com/thinktube/audio/G711UCodec.java
/**
* G.711 codec. This class provides u-law conversion.
*/
public class G711UCodec {
// s00000001wxyz...s000wxyz
// s0000001wxyza...s001wxyz
// s000001wxyzab...s010wxyz
// s00001wxyzabc...s011wxyz
// s0001wxyzabcd...s100wxyz
// s001wxyzabcde...s101wxyz
// s01wxyzabcdef...s110wxyz
// s1wxyzabcdefg...s111wxyz
private static byte[] table13to8 = new byte[8192];
private static short[] table8to16 = new short[256];
static {
// b13 --> b8
for (int p = 1, q = 0; p <= 0x80; p <<= 1, q += 0x10) {
for (int i = 0, j = (p << 4) - 0x10; i < 16; i++, j += p) {
int v = (i + q) ^ 0x7F;
byte value1 = (byte) v;
byte value2 = (byte) (v + 128);
for (int m = j, e = j + p; m < e; m++) {
table13to8[m] = value1;
table13to8[8191 - m] = value2;
}
}
}
// b8 --> b16
for (int q = 0; q <= 7; q++) {
for (int i = 0, m = (q << 4); i < 16; i++, m++) {
int v = (((i + 0x10) << q) - 0x10) << 3;
table8to16[m ^ 0x7F] = (short) v;
table8to16[(m ^ 0x7F) + 128] = (short) (65536 - v);
}
}
}
public int decode(short[] b16, byte[] b8, int count, int offset) {
for (int i = 0, j = offset; i < count; i++, j++) {
b16[i] = table8to16[b8[j] & 0xFF];
}
return count;
}
public int encode(short[] b16, int count, byte[] b8, int offset) {
for (int i = 0, j = offset; i < count; i++, j++) {
b8[j] = table13to8[(b16[i] >> 4) & 0x1FFF];
}
return count;
}
public int getSampleCount(int frameSize) {
return frameSize;
}
}
D. Converting to wave file:
Taken from here:
https://github.com/google/oboe/issues/320
import java.io.File;
import java.io.FileNotFoundException;
import java.io.FileOutputStream;
import java.io.IOException;
public class Wave
{
private final int LONGINT = 4;
private final int SMALLINT = 2;
private final int INTEGER = 4;
private final int ID_STRING_SIZE = 4;
private final int WAV_RIFF_SIZE = LONGINT+ID_STRING_SIZE;
private final int WAV_FMT_SIZE = (4*SMALLINT)+(INTEGER*2)+LONGINT+ID_STRING_SIZE;
private final int WAV_DATA_SIZE = ID_STRING_SIZE+LONGINT;
private final int WAV_HDR_SIZE = WAV_RIFF_SIZE+ID_STRING_SIZE+WAV_FMT_SIZE+WAV_DATA_SIZE;
private final short PCM = 1;
private final int SAMPLE_SIZE = 2;
int cursor, nSamples;
byte[] output;
public Wave(int sampleRate, short nChannels, short[] data, int start, int end)
{
nSamples=end-start+1;
cursor=0;
output=new byte[nSamples*SMALLINT+WAV_HDR_SIZE];
buildHeader(sampleRate,nChannels);
writeData(data,start,end);
}
/*
by Udi for using byteArray directly
*/
public Wave(int sampleRate, short nChannels, byte[] data, int start, int end)
{
int size = data.length;
short[] shortArray = new short[size];
for (int index = 0; index < size; index++){
shortArray[index] = (short) data[index];
}
nSamples=end-start+1;
cursor=0;
output=new byte[nSamples*SMALLINT+WAV_HDR_SIZE];
buildHeader(sampleRate,nChannels);
writeData(shortArray,start,end);
}
// ------------------------------------------------------------
private void buildHeader(int sampleRate, short nChannels)
{
write("RIFF");
write(output.length);
write("WAVE");
writeFormat(sampleRate, nChannels);
}
// ------------------------------------------------------------
public void writeFormat(int sampleRate, short nChannels)
{
write("fmt ");
write(WAV_FMT_SIZE-WAV_DATA_SIZE);
write(PCM);
write(nChannels);
write(sampleRate);
write(nChannels * sampleRate * SAMPLE_SIZE);
write((short)(nChannels * SAMPLE_SIZE));
write((short)16);
}
// ------------------------------------------------------------
public void writeData(short[] data, int start, int end)
{
write("data");
write(nSamples*SMALLINT);
for(int i=start; i<=end; write(data[i++]));
}
// ------------------------------------------------------------
private void write(byte b)
{
output[cursor++]=b;
}
// ------------------------------------------------------------
private void write(String id)
{
if(id.length()!=ID_STRING_SIZE){
}
else {
for(int i=0; i<ID_STRING_SIZE; ++i) write((byte)id.charAt(i));
}
}
// ------------------------------------------------------------
private void write(int i)
{
write((byte) (i&0xFF)); i>>=8;
write((byte) (i&0xFF)); i>>=8;
write((byte) (i&0xFF)); i>>=8;
write((byte) (i&0xFF));
}
// ------------------------------------------------------------
private void write(short i)
{
write((byte) (i&0xFF)); i>>=8;
write((byte) (i&0xFF));
}
// ------------------------------------------------------------
public boolean writeToFile(File fileParent , String filename)
{
boolean ok=false;
try {
File path=new File(fileParent, filename);
FileOutputStream outFile = new FileOutputStream(path);
outFile.write(output);
outFile.close();
ok=true;
} catch (FileNotFoundException e) {
e.printStackTrace();
ok=false;
} catch (IOException e) {
ok=false;
e.printStackTrace();
}
return ok;
}
/**
* by Udi for test: write file with temp name so if you write many packets each packet will be written to a new file instead of deleting
* the previous file. (this is mainly for debug)
* #param fileParent
* #param filename
* #return
*/
public boolean writeToTmpFile(File fileParent , String filename)
{
boolean ok=false;
try {
File outputFile = File.createTempFile(filename, ".wav",fileParent);
FileOutputStream fileoutputstream = new FileOutputStream(outputFile);
fileoutputstream.write(output);
fileoutputstream.close();
ok=true;
} catch (FileNotFoundException e) {
e.printStackTrace();
ok=false;
} catch (IOException e) {
ok=false;
e.printStackTrace();
}
return ok;
}
}

ThreadpoolExecutor data getting mixed up

I am using android's thread pool executor framework (initialized as below).
BlockingQueue<Runnable> taskQueue = new LinkedBlockingQueue<>();
ExecutorService executorService = new ThreadPoolExecutor(totalCores, totalCores * 3, 10, TimeUnit.SECONDS, taskQueue);
Now, consider the following function onFrameProcessed -
public void onFrameProcessed(RenderedImage renderedImage) {
String timeNow = new SimpleDateFormat("d-M-Y_HH_mm_ss_SSS").format(new Date()).toString();
CustomRunnable3 customRunnable3 = new CustomRunnable3(renderedImage, timeNow);
executorService.execute(customRunnable3);
}
Definition of CustomRunnable3 is as follows:
class CustomRunnable3 implements Runnable {
RenderedImage renderedImageLocal;
String basePath, timeNowCopy;
int hashCode;
CustomRunnable3(RenderedImage renderedImage, String timeNow) {
renderedImageLocal = renderedImage;
this.basePath = Environment.getExternalStoragePublicDirectory(Environment.DIRECTORY_PICTURES).toString();
this.timeNowCopy = timeNow;
hashCode = renderedImageLocal.hashCode();
}
#Override
public void run() {
if (renderedImageLocal.imageType() == RenderedImage.ImageType.ThermalRadiometricKelvinImage) {
int[] thermalData = renderedImageLocal.thermalPixelValues();
String dataPath = basePath + "/" + this.timeNowCopy + ".csv";
try {
PrintWriter printWriter = new PrintWriter(dataPath);
int dataLen = thermalData.length;
for (int i = 0; i < dataLen; i++) {
printWriter.println(thermalData[i]);
}
printWriter.close();
} catch (IOException e) {
e.printStackTrace();
}
String imgPath = basePath + "/" + this.timeNowCopy + ".jpg";
try {
if (hashCode != renderedImageLocal.hashCode()) {
Log.e("Checking", "Hash code changed..");
}
renderedImageLocal.getFrame().save(new File(imgPath), frameProcessor);
if (hashCode != renderedImageLocal.hashCode()) {
Log.e("Checking", "Hash code changed after writing..");
}
} catch (IOException e) {
e.printStackTrace();
}
}
}
}
Usage Scenario : onFrameReceived is being called multiple times per second(like 4-5 times). In each call to onFrameReceived, I am saving two files from renderedImage object (1 csv file, 1 jpg file). Both of these files must be related to each other because both are created from one parent and have same name(except the extension).
Problem : But that is not happening and somehow I am ending up with jpg file content from 1 renderedImage and csv content from another renderedImage object.
What are the possible reasons for this problem, please share your opinion.

Android Bluetooth input stream not reading full array

I'm creating an app to read string values over Bluetooth serial port. My data receiving but in two parts. If I send $F00,A,B,0,M# via bluetooth it only reads $ in first part and F00,A,B,0,M# in next part. I provided my code here. Please do correct me if I'm wrong.
InputStream inputStream=null;
int avilableBytes=0;
public ConnectedThread(BluetoothSocket socket){
InputStream temp=null;
try{
temp=socket.getInputStream();
}catch (IOException e){
e.printStackTrace();
}
inputStream=temp;
}
public void run() {
try{
int bytes;
while (true){
try{
avilableBytes=inputStream.available();
if (avilableBytes>0){
byte[] buffer=new byte[avilableBytes];
bytes=inputStream.read(buffer);
final String readMessage=new String(buffer,0,bytes);
bt_handler.obtainMessage(handlerState,bytes,-1,readMessage).sendToTarget();
Log.d("PRAVEEN",readMessage);
}
}catch (IOException e){
e.printStackTrace();
}
}
}catch (Exception e){
e.printStackTrace();
}
}
Data are like stream bytes and can not be processed immediately when it comes with a few bytes. Data will not come all at once as a single packet. You have to use the other byte[] buffer (MainBuffer) in which you will gradually save incoming byte and move the index in that buffer. Then, from time to time (e.g. in the timer once per second) take data from the main buffer and processed it. By default you must implement some data frame with a separator (eg. Data * data * data * - Many ways to do it good or bad). I dealt with this in .net via Xamarin, but just as an example it may be helpfull :
update example, format
In ConnectedThread :
public override void Run()
{
while (true)
{
try
{
int readBytes = 0;
lock (InternaldataReadLock)
{
readBytes = clientSocketInStream.Read(InternaldataRead, 0, InternaldataRead.Length);
Array.Copy(InternaldataRead, TempdataRead, readBytes);
}
if (readBytes > 0)
{
lock (dataReadLock)
{
dataRead = new byte[readBytes];
for (int i = 0; i < readBytes; i++)
{
dataRead[i] = TempdataRead[i];
}
}
Bundle dataBundle = new Bundle();
dataBundle.PutByteArray("Data", dataRead);
Message message = btlManager.sourceHandler.ObtainMessage();
message.What = 1;
message.Data = dataBundle;
btlManager.sourceHandler.SendMessage(message);
}
}
catch (System.Exception e)
{
btlManager.btlState = BTLService.BTLState.Nothing;
}
}
}
In BTLHandler :
public override void HandleMessage(Message msg)
{
switch (msg.What)
{
case 1:
{
byte[] data = msg.Data != null ? msg.Data.GetByteArray("Data") : new byte[0];
btlService.BTLReceiveData(data);
}
break;
}
}
public void BTLReceiveData(byte[] data)
{
lock (dataReadLock)
{
for (int i = 0; i < data.Length; i++)
{
dataRead[dataReadWriteCursor] = data[i];
dataReadWriteCursor++;
}
}
}
In Timer :
int tmpWriteCursor = dataReadWriteCursor;
int tmpReadCursor = dataReadReadCursor;
lock (dataReadLock)
{
int newBytes = dataReadWriteCursor - dataReadReadCursor;
for (int i = 0; i < newBytes; i++)
{
dataReadMain[dataReadReadCursor] = dataRead[dataReadReadCursor++];
}
}
bool odradkovani = false;
string tmpRadek = "";
int lastLineIndex = 0;
List<string> list = new List<string>();
for (int i = LastWriteLineIndex; i < tmpWriteCursor; i++)
{
if (dataReadMain[i] >= 32 && dataReadMain[i] <= 255)
{
tmpRadek += (char)dataReadMain[i];
}
else if (dataReadMain[i] == 13) odradkovani = true;
else if (dataReadMain[i] == 10)
{
if (odradkovani)
{
odradkovani = false;
list.Add(Utils.GetFormatedDateTime(DateTime.Now) + " " + tmpRadek);
tmpRadek = "";
lastLineIndex = i + 1;
}
}
else
{
tmpRadek += "?" + dataReadMain[i].ToString() + "?";
}
}
WriteDataToLog(list);
LastWriteLineIndex = lastLineIndex;

Android-Buffering values to minimize the number of disk writes?

In my Android application I am reading brain data values from an EEG headset. These values are then being written to a Text File.
The problem with this is that the values are being produced 500 times a seconds. As a result they are also being written to the text file 500 times a second, which I don't want.
I only want the values to be displayed in the text file once a second. I was reading about buffering to do so.
How could I use buffering in my case to solve the problem?
Below is my current android code, and also rough pseudo code for what i'm trying to achieve.
Current Android code:
Method used to save the data to file:
public void writeToFileRawData(String data) {
// creating the file where the contents will be written to
File file = new File(dir, fileNameRaw + ".txt");
FileOutputStream os;
try {
boolean append = true;
os = new FileOutputStream(file, append);
String writeMe = data + "\n";
os.write(writeMe.getBytes());
os.close();
} catch (FileNotFoundException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
}
}
Calling the above method within the handler for the Headset Values:
Note: in code below "order" is irrelevant, it is just a method used for formatting the txt.
final Handler handler = new Handler() {
#Override
public void handleMessage(Message msg) {
// msg.what determines the type of each message
switch (msg.what) {
case TGDevice.MSG_RAW_DATA:
// creating the string to be written to file
String line2 = order(seconds2 + "") + order("" + msg.arg1)
+ "\n";
// write the string to file
writeToFileRawData(line2);
break;
Rough Pseudo code for what I am trying to achieve:
brainWaveRaw
time voltage
xxx yyyy
xxx yyyy
[and there should be 500 of these per second]
(buffer these to minimize number of disk writes)
// initialize
timeOfLastRawWrite = timeNow
rawWriteStringBuffer = ''
rawEvent(raw (list of last one or few raw samples))
eventTime = timeNow
for every entry r in raw
rawWriteStringBuffer.append(printf("%d %d\n", eventTime, r))
if timeNow-timeOfLastRawWrite > one second
write rawWriteStringBuffer to file
rawWriteStringBuffer = ''
timeOfLastRawWrite = timeNow
// e.g. if last set of raw values was [123, 456, 678], arrived at time
9876
9876 123
9876 456
9876 678
Don't open the file every time.
Use BufferedOutputStream.
You might want to flush the stream every several times.
Handler mHandler; // member of your Activity class
#Override
public void onCreate(Bundle savedInstanceState) {
super.onCreate(savedInstanceState);
final String dir = ...;
final String fileNameRaw = ...;
mHandler = new Handler() {
#Override
public void handleMessage(Message msg) {
// msg.what determines the type of each message
switch (msg.what) {
case TGDevice.MSG_RAW_DATA:
// creating the string to be written to file
String line2 = order(seconds2 + "") + order("" + msg.arg1)
+ "\n";
// write the string to file
writeToFileRawData(line2);
break;
}
}
// members of your custom Handler class
private File mFile = new File(dir, fileNameRaw + ".txt");
private BufferedOutputStream mOs = new BufferedOutputStream(new FileOutputStream(mFile, true));
private int mWriteCnt = 0;
// moved this function from Activity to your custom Handler class as well
private void writeToFileRawData(String data) {
try {
mOs.write(data.getBytes());
mOs.write("\n".getBytes());
if (++mWriteCnt == 500) {
mOs.flush();
mWriteCnt = 0;
}
} catch (IOException e) {
e.printStackTrace();
}
}
}
}

Compare contents of two files line by line

public class MainActivity extends Activity {
FileOutputStream fos;
FileInputStream fOne, fTwo;
ArrayList<String> arr1 = new ArrayList<String>();
ArrayList<String> arr2 = new ArrayList<String>();
ArrayList<String> words = new ArrayList<String>();
ArrayList<String> wordsTwo = new ArrayList<String>();
int count = 0;
int countTwo = 0;
int countThree = 0;
#Override
protected void onCreate(Bundle savedInstanceState) {
super.onCreate(savedInstanceState);
setContentView(R.layout.activity_main);
Button fileOne = (Button)findViewById(R.id.file1);
Button fileTwo = (Button)findViewById(R.id.file2);
Button compare = (Button)findViewById(R.id.compare);
arr1.add("1");
arr1.add("2");
arr1.add("3");
arr1.add("4");
//arr1.add("3");
arr2.add("1");
arr2.add("2");
fileOne.setOnClickListener(new OnClickListener() {
#Override
public void onClick(View v) {
// TODO Auto-generated method stub
try
{
fos = openFileOutput("File1", Context.MODE_PRIVATE);
for(int temp = 0; temp< arr1.size(); temp++)
{
fos.write((arr1.get(temp).getBytes()) );
fos.write(System.getProperty("line.separator").getBytes());
}
fos.close();
fos.flush();
}
catch(Exception e)
{
}
}
});
fileTwo.setOnClickListener(new OnClickListener() {
#Override
public void onClick(View v) {
// TODO Auto-generated method stub
try
{
fos = openFileOutput("File2", Context.MODE_PRIVATE);
for(int temp = 0; temp< arr2.size(); temp++)
{
fos.write((arr2.get(temp).getBytes()) );
fos.write(System.getProperty("line.separator").getBytes());
}
fos.close();
fos.flush();
}
catch(Exception e)
{
}
}
});
compare.setOnClickListener(new OnClickListener() {
#Override
public void onClick(View v) {
// TODO Auto-generated method stub
try
{
fOne = openFileInput("File1");
fTwo = openFileInput("File2");
} catch (FileNotFoundException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
Scanner scanFile = new Scanner(new DataInputStream(fOne));
Scanner scanFileT = new Scanner(new DataInputStream(fTwo));
words = new ArrayList<String>();
wordsTwo = new ArrayList<String>();
while (scanFile.hasNextLine())
{
if(scanFile.nextLine()!=null)
{
count++;
}
while(scanFileT.hasNextLine())
{
if(scanFileT.nextLine()!=null)
{
countTwo++;
}
}
}
try
{
fOne.close();
fTwo.close();
scanFile.close();
scanFileT.close();
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
Toast.makeText(getBaseContext(), "One : " + count, 1000).show();
Toast.makeText(getBaseContext(), "Two : " + countTwo, 1000).show();
Toast.makeText(getBaseContext(), "Three : " + countThree, 1000).show();
count = 0 ;
countTwo = 0;
countThree = 0;
}
});
}
}
Above is the code to write and read the file. What I did here, write two files and read the contents..Now I have to compare contents of files line by line. What needs to be done?
Try following code. This will give you desired output. I took files from asset directory. So you need to replace that line of code if you are taking files from other directory.
private void compareFiles() throws Exception {
String s1 = "";
String s2 = "", s3 = "", s4 = "";
String y = "", z = "";
// Reading the contents of the files
BufferedReader br = new BufferedReader(new InputStreamReader(
getAssets().open("first.txt")));
BufferedReader br1 = new BufferedReader(new InputStreamReader(
getAssets().open("second.txt")));
while ((z = br1.readLine()) != null) {
s3 += z;
s3 += System.getProperty("line.separator");
}
while ((y = br.readLine()) != null) {
s1 += y;
s1 += System.getProperty("line.separator");
}
// String tokenizing
StringTokenizer st = new StringTokenizer(s1);
String[] a = new String[10000];
for (int l = 0; l < 10000; l++) {
a[l] = "";
}
int i = 0;
while (st.hasMoreTokens()) {
s2 = st.nextToken();
a[i] = s2;
i++;
}
StringTokenizer st1 = new StringTokenizer(s3);
String[] b = new String[10000];
for (int k = 0; k < 10000; k++) {
b[k] = "";
}
int j = 0;
while (st1.hasMoreTokens()) {
s4 = st1.nextToken();
b[j] = s4;
j++;
}
// comparing the contents of the files and printing the differences, if
// any.
int x = 0;
for (int m = 0; m < a.length; m++) {
if (a[m].equals(b[m])) {
} else {
x++;
Log.d("Home", a[m] + " -- " + b[m]);
}
}
Log.d("Home", "No. of differences : " + x);
if (x > 0) {
Log.d("Home", "Files are not equal");
} else {
Log.d("Home", "Files are equal. No difference found");
}
}
Input File 1
Hi
Hello
Chintan
Rathod
Input File 2
Hi
HellO
Chintan
RathoD
Output
08-26 12:07:58.219: DEBUG/Home(2350): Hello3. -- HellO3.
08-26 12:07:58.219: DEBUG/Home(2350): Rathod -- RathoD
08-26 12:07:58.229: DEBUG/Home(2350): No. of differences : 2
08-26 12:07:58.229: DEBUG/Home(2350): Files are not equal
Edit
To get Difference between two files
Use StringUtils library which is provide by Apache and check this Documentation for more about that library.
And modify following lines of code.
int x = 0;
for (int m = 0; m < a.length; m++) {
if (a[m].equals(b[m])) {
} else {
x++;
Log.d("Home", a[m] + " -- " + b[m]);
//to print difference
if (a[m].length() < b[m].length())
Log.d("Home", "" + StringUtils.difference(a[m], b[m]));
else
Log.d("Home", "" + StringUtils.difference(b[m], a[m]));
}
}
Output
08-26 17:51:26.949: DEBUG/Home(17900): 12 -- 123
08-26 17:51:26.949: DEBUG/Home(17900): Difference String : 3
08-26 17:51:26.949: DEBUG/Home(17900): No. of differences : 1
08-26 17:51:26.949: DEBUG/Home(17900): Files are not equal
Try using java.util.Scanner
while (sc1.hasNext() && sc2.hasNext()) {
String str1 = sc1.next();
String str2 = sc2.next();
if (!str1.equals(str2))
System.out.println(str1 + " != " + str2);
}
Change your while loop to the following:
while (scanFile.hasNextLine() && scanFileT.hasNextLine())
{
if(scanFileT.nextLine().equals(scanFile.nextLine()))
{
// The lines are equal.
} else {
// The lines are not equal.
}
}
if(scanFile.hasNextLine() || scanFileT.hasNextLine())
{
// If more lines remain in one of the files, they are not equal.
} else {
// If no content remains in both files, they are equal.
}
Depending on the size of your file, I would recommend some optimisation like checking the file sizes before you go through them line by line.
The overall logic reads as follows; if both have another line, compare it to see if it is equal. If they don't have another line, check if one of them has lines remaining, if so, they are not equal.
Update
After clarifying the objective of the comparison in chat, see the comments to this question, I have come to the conclusion that another comparison would be more effective and, as a matter of fact, correct. The comparison algorithm above works great if comparing the structure of text but not if comparing a data vector which may or may not be sorted. After some discussion, we came to the conclusion that data needs to be sorted or the comparison will blow the complexity to at least O(n^2)which could be done in O(2n) if the data is sorted. Here the algorithm's skeleton:
if(! scanGroupFriends.hasNextLine())
{
//simple sanity check to see if we need to compare at all. In this case, add all friends.
} else {
String nextFriend = scanGroupFriends.nextLine();
while(scanAllFriends.hasNextLine())
{
if(scanAllFriends.nextLine().equals(nextFriend))
{
// Friend already figures, do not add him and advance the list of group friends.
if(scanGroupFriends.hasNextLine())
{
nextFriend = scanGroupFriends.nextLine();
} else {
// There are no more friends in the group, add all remaining friends to list to show.
break; // Terminate the `while` loop.
}
}
}
}
However, I personally think it is bad to make to many assumptions. What I would suggest is that the friends be saved in a Set, a TreeSet for example. Then, serialize the object rather than manually writing it to file. Sets are neat because they hold several interesting objects. For example, you could easily use the following code to remove all friends in a group from the set of all friends:
allFriends.removeAll(groupFriends);
However, be aware that this removes it from the set completely so you should make a copy beforehand.

Categories

Resources