Develop Reference

Android SQLite Insert Performance

I might just be hitting native performance issues with SQLite, but there seems to be significant overhead associated with inserting in SQLite even using Transaction processing and pre-compiled statements. Or perhaps I am using them incorrectly.
I have a requirement to read a file line by line (via a URL) and create rows. The mechanism is generic, so any table can be created (within reason) and any number of rows added. I am seeing pretty decent performance for larger inserts, but with smaller ones it seems I have a minimum of +400ms. Since I may have over 100 of these to load, that small overhead is resulting in large load times.
For example some timings:
34 records Test #1 ----> 490 ms
36238 records Test #2 ----> 3021 ms
4 records Test #3 ----> 520 ms
Here is the code (I cut out all the try/catches and other code to boil it down to the actual insert code):
InputStream input = new BufferedInputStream(url.openStream());
// Create the file reader
BufferedReader br = new BufferedReader(new InputStreamReader(input, "UTF-8"));
StringBuffer insertSql = null;
// Build the SQL to bind
insertSql = new StringBuffer("INSERT INTO " + fileName + " (");
String sep = "";
insertSql.append("[" + getDbColumnNames().replaceAll(" ", "").replaceAll(",", "],[") + "]");
String[] columns = getDbColumnNames().split(",");
insertSql.append(") VALUES (");
for (#SuppressWarnings("unused") String col : columns) {
insertSql.append(sep.trim() + "?");
sep = ",";
}
insertSql.append(");");
this.open();
sqlDB.beginTransaction();
SQLiteStatement stmt = = sqlDB.compileStatement(insertSql.toString());
String line = "";
// Read the file line by line
while ((line = br.readLine()) != null) {
String[] tokens = line.split(",");
// Build the bindings and insert the data
int bindcnt = 1;
for (String token : tokens) {
stmt.bindString(bindcnt++, token.trim());
}
long entryID = stmt.executeInsert();
if (entryID < 0) {
success = false;
}
stmt.clearBindings();
}
sqlDB.setTransactionSuccessful();
sqlDB.endTransaction();
I have verified that the read buffer overhead is not significant for most of the reads.

fetching individual running process Start-Time

I am using below code to get all currently running process's on an Android device.
// Get running processes
ActivityManager manager = (ActivityManager) getSystemService(ACTIVITY_SERVICE);
List<ActivityManager.RunningAppProcessInfo> runningProcesses = manager.getRunningAppProcesses();
And I am also trying to use the below code to get all individual process start time in the Android Device from each individual pid- "process id" in the following file directory:"/proc/[PID]/stat" obtained from linux:
public static long getStartTime(final int pid) throws IOException {
final String path = ("/proc/" + pid + "/stat");
final String stat;
final String field2End = ") ";
final String fieldSep = " ";
final BufferedReader reader = new BufferedReader(new FileReader(path));
try {
stat = reader.readLine();
System.out.println("******Stat******"+ stat);
} finally {
reader.close();
}
try {
final String[] fields = stat.substring(stat.lastIndexOf(field2End)).split(fieldSep);
final long processstartTime = .....;
...(change processstartTime from clock tick to seconds & return processstartTime value)...
}
}
I do need to get the process StartTime from the following Linux directory path:"/proc/pid/stat" for each individual running processes in an Android Device. At this point, when I debug the statement for the following Linux directory path:"/proc/[PID]/stat", in the following code line: System.out.println("******Stat******"+ stat);, I am getting the output debug as :
******Stat******642 (flipboard.app) S 2848 2848 0 0 -1 4194624 126020 0 1019 0 2441 632 0 0 20 0 101 0 7040346 1079652352 7233 4294967295 1 1 0 0 0 0 4612 0 38120 4294967295 0 0 17 1 0 0 0 0 0 0 0 0
Furthermore, I do know that the start_time of the process is measure in clock ticks, hence to convert it to seconds, I will need to cal the following as "start_time/hertz".
Now the issue is, How Do I get the Running Process Start Time in "/proc/[PID]/stat"? Can anyone please help? Thanks.

loggin of individual process time can be done by this way:
public static long getStartTime(final int pid) throws
IOException {
final long SYSTEM_CLK_TCK= 100; //needed as value in /proc/[PID]/stat file driectory is in clock ticks,100 is used to convert clock ticks to secs
final int fieldStartTime = 20; //column 20 of the /proc/[PID]/stat file driectory
try {
System.out.println("******String path******"+ path);
stat = reader.readLine();
System.out.println("******String stat******"+ stat);
} finally {
reader.close();
}
try {
final String[] fields = stat.substring(stat.lastIndexOf(field2End)).split(fieldSep);
final long startTime = Long.parseLong(fields[fieldStartTime]);
System.out.println("******fieldstarttime based on clock ticks******"+ startTime);
return startTime * msInSec / SYSTEM_CLK_TCK;
}

broadcast to have application launch count

How can I get the launch count of all applications? I have the complete list of installed apps, and I have a broadcast when an application is installed, but I need the launch count of any apps.
I see this app with this. You have the cpu time, the foreground time, and the launch count... how do they do it??

Finally i do that! i create a AlarmManager that every minute check the running applications, if an application in running (background or active) i check the last time that i saw it. if this time is greater than one minute i increase the count.
Now i'm trying to have how many data the application sent to an external server, i have this data, but do you know if this data is from i have installed my application or from when i boot my smartphone?
Long txByte = TrafficStats.getUidTxBytes(listApp.getAppsRunning().get(i).getPid());
this code is for get the count time
for(int i=0; i< listApp.getAppsRunning().size(); i++)
{
String pName = listApp.getAppsRunning().get(i).getPackageName();
String Ldate = "0";
int Nrun = 0;
Long Ntime = null, Ndata = null ;
Cursor c=db.fetchInstalled(pName);
if(c.moveToFirst())
{
Nrun = c.getInt(2);
Ldate = c.getString(3);
Ntime = c.getLong(4);
Ndata = c.getLong(5);
Log.d("db", "last time: " + Nrun+ " time: " + Ldate);
}
if(Ldate.equalsIgnoreCase("0"))
{
Nrun++;
db.updateLaunchAndTime(Nrun, lastUpdated, pName, Ntime, Ndata);
}
else
{
SimpleDateFormat dateFormat = new SimpleDateFormat("MMMM dd, yyyy h:mmaa");
Date lastDate = null;
Date currentDate = null;
try {
lastDate = dateFormat.parse(Ldate);
currentDate = dateFormat.parse(lastUpdated);
} catch (ParseException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
//int pid = android.os.Process.getUidForName(listApp.getAppsRunning().get(i).getPid());
Long txByte = TrafficStats.getUidTxBytes(listApp.getAppsRunning().get(i).getPid());
Log.d("pid process", "pid: " + listApp.getAppsRunning().get(i).getPid());
Ndata = txByte;
Log.d("data send", "send: " + Ndata);
long diff = currentDate.getTime() - lastDate.getTime();
if(diff > 100* 1000)
{
Log.d("db", "difference plus 1 min app: " + pName);
Nrun++;
}
Ntime = Ntime+diff;
db.updateLaunchAndTime(Nrun, lastUpdated, pName, Ntime, Ndata);
}
//db.insertRunningP(pName , lastUpdated);
}
db.close()
I checked the power consume of this code and is less than 3% of total battery, so for now this is the best solution that i have found

I've never done it before, but I'm pretty sure http://developer.android.com/reference/android/app/ActivityManager.html provides the information you need.

if you had rooted your device, you also can read the usage stats files in /data/system/usagestats/usage-* for detail infomation.

Capturing part of string from a txt file?

I have a text file like this, separated by ";"
1022-3, 1603-4, 2012-5, 2489-6;
Gotta catch the first part before the "-" and pass to variable, and compare with milliseconds, if is equal the number, capture the number after of "-".
And do so with the next number after the semicolon, and so front.
public static long MilliSeconds() {
// get Calendar instance
Calendar now = Calendar.getInstance();
return now.getTimeInMillis();
}
And the beginning of the code to do what I need this here
private void LerArquivo() {
String lstrNomeArq;
File arq;
String lstrlinha;
long tempoInicio = 0;
long tempoDecorrido = 0;
try {
tempoDecorrido = (RecordSound.MilliSeconds() - tempoInicio);
lstrNomeArq = "/Android/data/br.com.couldsys.drumspro/cache/GravaSound.TXT";
String conteudotexto = "";
arq = new File(Environment.getExternalStorageDirectory(),
lstrNomeArq);
BufferedReader br = new BufferedReader(new FileReader(arq));
// pega o conteudo do arquivo texto
conteudotexto = br.readLine();
String capturaIndex = ("Conteudo do texto: "
+ conteudotexto.substring(
conteudotexto.indexOf("-") + 1,
conteudotexto.indexOf(";",
conteudotexto.lastIndexOf("-"))));
if (tempoDecorrido == capturatempo) {
DrumsProActivity.vsm.playSound(capturaindex);
// ler a nova linha
// se chegar ao final do string então para o while
if (conteudotexto.length() > 0) {
executar = false;
}
}
} catch (Exception e) {
trace("Erro : " + e.getMessage());
}
}

use this simpler code : create an array of substrings each contain a string formated ####-#
string[] MyStr = conteudotexto.split(',');
string sss= MyStr[0];
string sss2= MyStr[1];
....
now sss is 1022-3
sss2 is 1603-4 and so on ....
then reuse split function:
string[] MyStr2 = sss.split('-');
now we have :
MyStr2[0] = 1022
MyStr2[1] = 3

Maybe not particular elegant but just pragmatic for me - use the String split method. First split with ","
String[] parts = conteudotexto.split(",");
and then with each of the parts (here for the first)
String[] subParts = parts[0].split("-");
Just gives you everything in the pieces you need to look at and no danger get mixed up with positions etc.

What I need now is know how to catch part of text.
arq = new File(Environment.getExternalStorageDirectory(),
lstrNomeArq);
BufferedReader br = new BufferedReader(new FileReader(arq));
// pega o conteudo do arquivo texto
conteudotexto = br.readLine();
The text may vary more separation has two, the first number is separated from the second number by a "-" (dash) which is then separated by "," (comma)
549-8,1019-9,1404-3,1764-3,2208-10,2593-5,2938-9,3264-6,3700-0,4174-7,4585-8,4840-2,5192-9,5540-10,5932-0,
As has been shown
String[] parte1 = conteudotexto.split(",");
e em seguida
String[] parte2 = parte1[0].split("-");
The rule I'm trying to do is: Turn within a millisecond while the method and compare with the first part of the text
type
**If valor_milissegundos first part is equal to the number of text then
---> enters and runs the function playSound (the second number of the text);
------> goes to the next number in the text loop, capturing the second number of the text and compares it to the millisecond, if equal enters the IF block and catch the second number after the dash, and so on, until you reach the end of the text.**
Method return milliseconds calculates milliseconds
public static long MilliSeconds() {
// get Calendar instance
Calendar now = Calendar.getInstance();
return now.getTimeInMillis();
}
I hope understand why I used the google translator
thank you

too much time in reading sms from database and storing it in to text file

I'm reading sms record from database and write them into text file. But it takes too much time 3 to 4 mins to read n write 3500 records. If records are much more than that it takes plenty of time which is not appreciable. My code is:
final Cursor cur1 = c.getContentResolver().query(Uri.parse("content://sms/"), null, null, null, "date ASC");
final int size = cur1.getCount();
final int sleeptimer = size;
final SMS [] sms = new SMS[size];
final String baseDir = Environment.getExternalStorageDirectory().getAbsolutePath()+ File.separator + "account.txt";
FileWriter fw = null;
try {
fw = new FileWriter(baseDir);
} catch (IOException e1) {
// TODO Auto-generated catch block
e1.printStackTrace();
}
final BufferedWriter writer = new BufferedWriter(fw);
Thread myThread = new Thread()
{
public void run()
{
try
{
int currentwait = 0;
int j=0;
while(currentwait < sleeptimer)
{
sleep(200);
currentwait+=200;
for(int i = 0 ; i < 200 ; i++)
{
if(!cur1.moveToNext())
{
break;
}
ContactInfo p = new ContactInfo();
String content = cur1.getString(cur1.getColumnIndex("body"));
String number = cur1.getString(cur1.getColumnIndex("address"));
long date = cur1.getLong(cur1.getColumnIndex("date"));
String protocol = cur1.getString(cur1.getColumnIndex("protocol"));
String name = p.getName(number, c);
String type = null;
Calendar cal=Calendar.getInstance();
cal.clear();
cal.setTimeInMillis(date);
String date_time=String.format("%1$te %1$tB %1$tY,%1$tI:%1$tM:%1$tS %1$Tp",cal);
if( protocol == null )
{
type = "Outbox";
}
else
type = "Inbox";
try{
writer.write("Type: " + type);
writer.newLine();
writer.write("Name: " + number+"<"+name+">");
writer.newLine();
writer.write("Date: " + date_time);
writer.newLine();
writer.write("Content: " + content);
writer.newLine();
writer.newLine();
} catch (IOException e) {
Log.i("INFO", e.getMessage().toString());
}
//Log.i("INFO", content+" "+j);
sms[j] = new SMS(type , name , number , date_time , content );
j++;
}
}
}
catch(Exception e)
{
}
finally{
try{
writer.flush();
writer.close();
}
catch (Exception e) {
}
uploadtoserver(baseDir);
}
}
};
myThread.start();
any idea to improve it ???thanks :)))

Just remove this line:
sleep(200);

Call setMaxSqlCacheSize to increase the cache size. Default is 10. Try setting 20 first and see whether the time reduces to half..
Or/And
You can execute this sqlite query before performing any operation on db and see if that improves the speed. Changing the temp store to memory should improve the read write speed..
PRAGMA temp_store = 2; /* 0 | DEFAULT | 1 | FILE | 2 | MEMORY; */
When temp_store is MEMORY (2) temporary tables and indices are kept in as if they were pure in-memory databases memory
Also
PRAGMA page_size = bytes;
Query or set the page size of the database. The page size must be a power of two between 512 and 65536 inclusive.
Probably you can pass these statements either to execSQL(String sql) or to query(). Try and let me know how it works.
Check out other PRAGMAs that Sqlite supports: http://www.sqlite.org/
Update:
From documentation of query API:
For best performance, the caller should follow these guidelines:
Provide an explicit projection, to prevent reading data from storage that aren't going to be used.
Use question mark parameter markers such as 'phone=?' instead of explicit values in the selection parameter, so that queries that differ only by those values will be recognized as the same for caching purposes.