Exporting SQLite data to CSV - android

My task is to export data to CSV file. I am trying to generate a query before exporting data to a CSV File. Each student has a unique ID. As for now, I'm able to export the data in different tables separately and its working well.
Here is my current schema :
1) Student Table
studID | name | teacher | activity
SQ202 Mich Lisa Hearing Test
FG91A Lim Hanson Behavioural Test
2) Session Table (Each student can have more than one session - not fixed)
studID | startTime | endTime | sessionNo | status
SQ202 10:00 10:40 1 Completed
SQ202 13:00 13:50 2 Completed
FG91A 14:20 15:00 1 Completed
3) Interval Table (Each session can have more than four intervals - not fixed)
studID |sessionNum |intervalNo | score
SQ202 1 1 10/10
SQ202 1 2 7/10
.
.
.
This is how I exported data from a specific table :
File exportDir = new File(Environment.getExternalStorageDirectory(), "");
if (!exportDir.exists()) {
exportDir.mkdirs();
}
file = new File(exportDir, “ChildObservation" + " (D)" + date + " (T)" + time +".csv");
try {
file.createNewFile();
CSVWriter csvWrite = new CSVWriter(new FileWriter(file));
// sqlite core query
SQLiteDatabase db = myNewGradingDB.getReadableDatabase();
Cursor curChild = db.rawQuery("SELECT * FROM Student ORDER BY name", null);
csvWrite.writeNext(“Student ID”, “Name”, “Teacher”, “Activity");
while (curChild.moveToNext()) {
String arrStr[] = {
curChild.getString(0), curChild.getString(1),
curChild.getString(2), curChild.getString(3),
};
csvWrite.writeNext(arrStr);
}
csvWrite.writeNext();
csvWrite.close();
return true;
}
catch (IOException e) {
Log.e(“Child", e.getMessage(), e);
return false;
}
However, my goal is to retrieve and display all the data tied to the student in a single row. Each row in the exported Excel sheet will contain all data from the Student table & Session table as well as from the respective interval number and scores from the Interval table. I have tried using inner joins and different kind of queries but there are still some duplicated values in those rows. Each cell must only contain one value, so in this case I won't be able to concatenate the columns in my query.
Here is what I want to achieve :
studID | name | teacher | activity | startTime | endTime | sessionNo | status | intervalNo | score | intervalNo | score ... (repeat intervalNo and score)
SQ202 Mich Lisa Hearing Test 10:00 10:40 1 Completed 1 10/10 2 7/10 ...
FG91A Lim Hanson Behavioural Test 12:30 1:40 2 Completed 1 8/10 2 6/10 ...
At this stage, its impossible for me to alter the database design. Perhaps there are other ways to resolve this, whether by SQL or another. Any help or suggestions will be greatly appreciated. Thank you.

if you want a sql only solution then
the trick to solve you problem is to join the interval table several times using an alias
select
st.studID, st.name,
se.startTime,
i1.intervalNo as intervalNo1, i1.score as score1,
i2.intervalNo as intervalNo2, i2.score as score2,
i3.intervalNo as intervalNo3, i3.score as score3,
i4.intervalNo as intervalNo4, i4.score as score4
from Student as st
left join Session as se on st.studID = se.studID
left join Interval as i1 on st.studID = i1.studID and i1.sessionNo=1
left join Interval as i2 on st.studID = i2.studID and i2.sessionNo=2
left join Interval as i3 on st.studID = i3.studID and i3.sessionNo=3
left join Interval as i4 on st.studID = i4.studID and i4.sessionNo=4
but this only works if you know in advance how many interval-s there are

Related

Sqlite Trigger to Update table 1 when certain rows of table 2 is updated

I have two tables in sqlite in android:
Table1: Outlets
id pid OutletName Status
1 1 Tesco 1
2 1 Coors 1
3 2 Coors 1
Table2: projectparams
ppid projectname outletname param projStatus
1 RA_Pepsi Tesco shelfstrip 2
2 RA_Pepsi Tesco shelfleft 2
3 RA_Cocola Coors shelfstrip 1
Table1: For every PID (ProjectID) there are multiple Outlets stored in OutletName.
Table2: For each outlet there are multiple params stored in project params.
Whenever the user completes a task the param in Table2 is updated to 2. In the above example, when the user completes two tasks in Tesco, the status would be updated to 2 upon completion.
I am trying to set the outlets status to 2 based on completion of tasks for every outlet. In this example, when the status is updated to 2 in projectparam table, I want the Status in Outlets to be updated to 2 for the outlet Tesco.
I am using the following codes:
triggered when the task gets completed.
private void updateTaskStatus() {
SQLiteDatabase db = sqLiteHelper.getWritableDatabase();
String spinTextRecovered = spnTxt.getString("spinText", null);
String updateTable = "update projectparams set projStatus=2 where param='"+spinTextRecovered+"' and projectName='"+projectName+"' and OutletName='"+outletName+"'";
db.execSQL(updateTable);
takePicButton.setEnabled( false );
}
Triggered every time when the task gets completed and updates the outlet status when all the tasks for an outlet is completed.
private void updateOutletStatus() {
String query = "select OutletName from projectparams where projectName= '"+projectName+"' and OutletName= '"+outletName+"' group by projStatus";
if (sqLiteHelper.getData(query).getCount()==1) {
Toast.makeText( getApplicationContext(), "No more tasks in this outlet!", Toast.LENGTH_SHORT ).show();
SQLiteDatabase db = sqLiteHelper.getWritableDatabase();
String outletUpdate = "update outlets set Status=2 where OutletName='"+outletName+"' and pid = (select id from projects where ProjectName = '"+projectName+"' ) ";
db.execSQL(outletUpdate);
}
}
The above code works...however since I am using an intent to display the projects, outlets and task parameters, many times i find the updation of outlet not happening.
Can i write this better? Can someone guide me to use triggers?
I believe your core issue is that the query
String query = "select OutletName from projectparams where projectName= '"+projectName+"' and OutletName= '"+outletName+"' group by projStatus";
Will return 1 not only if all the projects have completed but also if none have been completed (i.e. all the projStatus values are 1).
Additionally, as it stands if very much appears that the Outlets table is superfluous. That is all columns (bar the id, which serves no purpose) are duplicated in the projectparams table.
You can ascertain is all the tasks for a project by multiplying the number of params by 2 (the status completion value) against the sum of all of the projstatus values. If they match then all params have been set to 2.
Consider the following which is your data (but status 1 for RA_Perpsi/Tesco rows) plus some additional data where the only fully completed is for the Completed Example project (highlighted) as per :-
The using (to process all results)
-- WHERE clause removed group by projectname added (to show all results)
SELECT OutletName FROM projectparams GROUP BY projectname,projStatus;
The result is :-
That is Ra_Pepsi/Tesco has no params completed (nothing done) but there is only one row so the project is detected as being completed. Likewise for RA_Cocola and for Some other outlet.
Completed Example produces 1 row so is complete as it should be. Incomplete example (where it is part way through) is the only one that comes up as incomplete.
Using the above data consider the following (noting that no reference is needed to the Outlets table AND that the WHERE clause has been omitted and the GROUP BY according to projectname to show all projects) :-
SELECT
OutletName,
CASE
WHEN (count() * 2 - sum(projStatus)) = 0 THEN 2 ELSE 1
END AS Status,
(count() * 2 - sum(projStatus)) ||' of '||count() AS outstanding
FROM projectparams
GROUP BY projectname
This results in :-
As such there is no need for the Outlet Table the projectparams table, at least according to your example, is sufficient and as such there is no need for a trigger. Whenever an update is made you simply refresh the display using the one table and a query such as the last example.
To demonstrate your scenario step by step consider the following code (based upon the data above i.e. the RA_Pepsi has been added but nothing done) :-
-- First Query
SELECT
OutletName,
CASE
WHEN (count() * 2 - sum(projStatus)) = 0 THEN 2 ELSE 1
END AS Status,
(count() * 2 - sum(projStatus)) ||' of '||count() AS outstanding
FROM projectparams
WHERE projectname = 'RA_Pepsi'AND outletName = 'Tesco'
GROUP BY projectname
;
-- First Update
UPDATE projectparams SET projStatus = 2 WHERE param = 'shelfstrip' AND projectname = 'RA_Pepsi' AND outletName = 'Tesco';
-- 2nd Query
SELECT
OutletName,
CASE
WHEN (count() * 2 - sum(projStatus)) = 0 THEN 2 ELSE 1
END AS Status,
(count() * 2 - sum(projStatus)) ||' of '||count() AS outstanding
FROM projectparams
WHERE projectname = 'RA_Pepsi'AND outletName = 'Tesco'
GROUP BY projectname
;
-- 2nd Update (all completed)
UPDATE projectparams SET projStatus = 2 WHERE param = 'shelfleft' AND projectname = 'RA_Pepsi' AND outletName = 'Tesco';
-- 3rd Query
SELECT
OutletName,
CASE
WHEN (count() * 2 - sum(projStatus)) = 0 THEN 2 ELSE 1
END AS Status,
(count() * 2 - sum(projStatus)) ||' of '||count() AS outstanding
FROM projectparams
WHERE projectname = 'RA_Pepsi'AND outletName = 'Tesco'
GROUP BY projectname
;
The first query shows the project with nothing done as per :-
The 2nd query (after changing the status of the shelfstrip to 2) shows :-
The 3rd query (after changing the status of the shelfleft to 2) shows :-
For Android
The foloowing code demonstrates applying the above on Android and additional uses the recommended convenience methods (they build much of the SQL, off protection against SQL injection and add additional functionality e.g. update returns the number of rows updated) :-
Two methods (the first using the query above, the second updating as per yours but using the update convenience method :-
public String getOutletStatus(String projectname, String outletname) {
String rv = "Ooops nothing found!"; // default if nothing found
String[] columns = new String[]{"outletname",
"CASE " +
"WHEN (count() * 2 - sum(projStatus)) = 0 THEN 2 ELSE 1 " +
"END AS Status", //<<<<<<<<< the column name will be Status
"(count() * 2 - sum(projStatus)) ||' of '||count() AS outstanding" // The column name will be outstanding
};
String whereclause = "projectname=? AND outletName=?";
String[] whereargs = new String[]{projectname,outletname};
String groupby = "projectname"; // not needed
Cursor csr = sqliteHelper.getWritableDatabase().query("projectparams",columns,whereclause,whereargs,null,null,null);
if (csr.moveToFirst()) {
int statuscode = csr.getInt(csr.getColumnIndex("Status"));
String outstanding = csr.getString(csr.getColumnIndex("outstanding"));
String outlet = csr.getColumnName(csr.getColumnIndex("outletname"));
String statusdescription = "incomplete";
if (statuscode == 2) {
statusdescription = "complete";
}
rv = "The status of project " + projectname + " for outlet " + outlet + " is " + statusdescription + ". With " + outstanding + ".";
}
csr.close();
return rv;
}
public void updateStatus(String projectname, String outletname, String param) {
String whereclause = "projectname=? AND outletname=? AND param=?";
String[] whereargs = new String[]{projectname,outletname,param};
ContentValues cv = new ContentValues();
cv.put("projStatus",2);
int number_of_rows_updated = sqliteHelper.getWritableDatabase().update("projectparams",cv,whereclause,whereargs);
}
They have been tested (using the base data show above) and reflect the final 3 queries with updates between them :-
String status = getOutletStatus("RA_Pepsi","Tesco"); // Before anything
Log.d("RESULT",status);
updateStatus("RA_Pepsi","Tesco","shelfstrip"); //shelfstrip completed
status = getOutletStatus("RA_Pepsi","Tesco");
Log.d("RESULT",status);
updateStatus("RA_Pepsi","Tesco","shelfleft"); //shelfleft completed
status = getOutletStatus("RA_Pepsi","Tesco");
Log.d("RESULT",status);
Result in the Log :-
04-29 12:46:09.615 20471-20471/? D/RESULT: The status of project RA_Pepsi for outlet outletname is incomplete. With 2 of 2.
04-29 12:46:09.621 20471-20471/? D/RESULT: The status of project RA_Pepsi for outlet outletname is incomplete. With 1 of 2.
04-29 12:46:09.625 20471-20471/? D/RESULT: The status of project RA_Pepsi for outlet outletname is complete. With 0 of 2.

Android sqlite db query can't get correct data

I have a question.
Have a table like this:
(table name: times)
+-------+-----------+---------+
| block | startTime | endTime |
+-------+-----------+---------+
| 1 | 08:00 | 10:00 |
+-------+-----------+---------+
when I use rawquery like this:
String ctime = "'09:45'";
cursor = sdb.rawQuery(String.format("select block from times where startTime < time(%s) and endTime> time(%s)", ctime,ctime), null);
I Can get the correct data "1".
But when I use query like this:
cursor = sdb.query("times", new String[]{"block"}, "startTime <= time(?) and EndTime >= time(?)", new String[]{ctime,ctime}, null, null, null);
I can't get the correct data.
The Cursor count is 0.
Why??
A valid time string must not contain quotes.
The quotes in your ctime variable are correct when you are constructing the SQL statement by hand, but they are used to delimit the SQL string and are not part of the value of the SQL string.
When you are using parameters, you do not need to put quotes around strings (unless you actually want them to be part of the string's value).

How to insert a table in to group in Corona SDK (.Lua)?

I get error message when i try to insert a table into a group
My table code is containing images
Here is the code i am using for the table
local myJoints = {}
for i = 1,5 do
local link = {}
for j = 1,17 do
link[j] = display.newImage( "link.png" )
link[j].x = 121 + (i*34)
link[j].y = 55 + (j*17)
physics.addBody( link[j], { density=2.0, friction=0, bounce=0 } )
-- Create joints between links
if (j > 1) then
prevLink = link[j-1] -- each link is joined with the one above it
else
prevLink = wall -- top link is joined to overhanging beam
end
myJoints[#myJoints + 1] = physics.newJoint( "pivot", prevLink, link[j], 121 + (i*34), 46 + (j*17) )
end
end
and here is the code for group
GUI:insert(myJoints);
i have my background image in the GUI group and it is covering the table.
I don't know if it is actually possible to insert table into a group
Any help please
Thanks in Advance!
You can't insert a table into a group using the "insert" method because that method is looking for a display object. Try calling GUI.myJoints = myJoints. Also keep in mind that your table just references your display objects, which is different from having them in a group.

How to iterate through a joined table with multiple results?

I have 3 tables, USER, ENTRY (for entered products, not necessary to create a PRODUCT table), and USER_COLLECTION, which is a table inbetween USER and ENTRY, because an entry can have multiple users.
Basically:
User = USERID | USER_NAME
Entry = ENTRYID | ENTRY_NAME | ENTRYPRICE | ENTRY_DATE
Collection = COLLECTIONID | ENTRYID | USERID
I have a table with users that persist throughout the project. They can create entries (which is usually some kind of product with a price) and they can link multiple users to a certain entry (which can be selected from a list, hence the users persist throughout the project).
So for instance, my tables look like this:
User
--------------------------
user_id | user_name
--------------------------
1 | 'FOO'
2 | 'BAR'
3 | 'FOOBAR'
ENTRY
-----------------------------------------------------------------------
entryid | entry_name | entry_price | entry_date
-----------------------------------------------------------------------
0 | 'Banana' | 2.50 | 12/12/2012
COLLECTION
---------------------------------------
collectionid | entryid | userid
----------------------------------------
0 | 1 | 1
1 | 1 | 2
2 | 1 | 3
I have a Banana, with a price of 2.50 and 3 users linked to it, Foo, Bar and Foobar.
Now, I want to use this in my app and get the data; except I don't know where to start. I tried selecting the entry data, using that id to loop through the collection data, but that would mean I have two cursors open and it wouldn't work. Tried creating a join but I couldn't really make a good one, mainly because:
JOIN
---------------------------------------
collectionid | entryname | username
----------------------------------------
0 | Banana | FOO
1 | Banana | BAR
2 | Banana | FOOBAR
I can't iterate through this, because I would create multiple of the same entry objects in my Android code...
Hope I'm being clear on this.
if (cursor2.moveToFirst()) {
do {
Item i = new Item(<GET STUFF FROM CURSOR>);
i.addUser(new Person(<GET STUFF FROM CURSOR>)));
Log.d("TAG", i.getUsersPaying().size() + "");
} while (cursor2.moveToNext());
}
If I use this, I create mulitple instances of Item i. They'll all be Banana, whilst I should only have 1 item Banana, with multiple users added to it.
First, you might want to consider returning the IDs from your tables in your join query. Things would be a little easier if you returned the entryid column.
Just make a Map<Integer, Item> to store items that you have seen already in your loop. As you examine each cursor, check the map to see if you already have an instance. If you don't, just make a new one and insert it.
Let's assume your query results are:
JOIN
----------------------------------------------------
collectionid | entryname | entryname | username
----------------------------------------------------
0 | 1 | Banana | FOO
1 | 1 | Banana | BAR
2 | 1 | Banana | FOOBAR
2 | 2 | Apple | FOOBAR
You can modify your code as follows:
Map<Integer, Item> items = new HashMap<Integer, Item>();
if (cursor2.moveToFirst()) {
do {
int itemId = cursor2.getInt(1);
Item i;
if (items.containsKey(itemId))
i = items.get(itemId);
else
{
i = new Item(<GET STUFF FROM CURSOR>);
items.put(itemId, i);
}
i.addUser(new Person(<GET STUFF FROM CURSOR>)));
Log.d("TAG", i.getUsersPaying().size() + "");
} while (cursor2.moveToNext());
}
You need to maintain a dictionnary of your entities which are already loaded in memory. For instance in a background fragment which would be retained.
Basically you would do:
Item i = cacheFragment.createOrGetEntry( cursor.getLong( ENTRY_ID_COLUMN_INDEX ) );
Person p = cacheFragment.createOrGetPerson( cursor.getLong( PERSON_ID_COLUMN_INDEX ) );
Of course, your query must also return the IDs of all the rows you need (entryId and personId). But a join query is the way to do it efficiently, so keep what you did about that and just add the two missing ID columns.
a createOrGetPerson method would look like:
public Person createOrGetPerson(long id) {
Entry<Long, Person> p = personDictionnary.get( id ); // can be a HashMap or even better, a SparseArray
if (p==null) {
p = new Person(id);
personDictionnary.put(p); // Remember it for next time
}
return p;
}
You should also have a look at data persistence frameworks or ORM frameworks which are made to deal with this kind of problem (e.g. Hibernate, even though I don't know if that is working with Android).

Bad SQLite performance on external storage in Android

I'm using the external storage for storing events in a database while they are waiting to be sent to the server.
I'm seeing really bad performance when inserting records.
I know the external memory can be slow but I wanted to see some number so I wrote a small app which tests it.
Here is the code:
public static final int INSERTS = 100;
File dbFile = new File(Environment.getExternalStorageDirectory(), "test.sqlite3");
// File dbFile = new File(getFilesDir(), "test.sqlite3");
dbFile.delete();
SQLiteDatabase db = SQLiteDatabase.openOrCreateDatabase(dbFile, null);
db.execSQL("CREATE TABLE events (_id integer primary key autoincrement, event_type TEXT NOT NULL, timestamp BIGINT, data TEXT);");
db.execSQL("CREATE INDEX mainIndex ON events (event_type, timestamp ASC);");
InsertHelper helper = new InsertHelper(db, "events");
final int eventTypeCol = helper.getColumnIndex("event_type");
final int timestampCol = helper.getColumnIndex("timestamp");
final int dataCol = helper.getColumnIndex("data");
long start = System.currentTimeMillis();
String eventType = "foo", data = "bar";
long timestamp = 4711;
for(int i = 0; i < INSERTS; ++i) {
helper.prepareForInsert();
helper.bind(eventTypeCol, eventType);
helper.bind(timestampCol, timestamp);
helper.bind(dataCol, data);
helper.execute();
}
long end = System.currentTimeMillis();
Log.i("Test", String.format("InsertHelper, Speed: %d ms, Records per second: %.2f", (int)(end-start), 1000*(double)INSERTS/(double)(end-start)));
db.close();
dbFile.delete();
db = SQLiteDatabase.openOrCreateDatabase(dbFile, null);
db.execSQL("CREATE TABLE events (_id integer primary key autoincrement, event_type TEXT NOT NULL, timestamp BIGINT, data TEXT);");
db.execSQL("CREATE INDEX mainIndex ON events (event_type, timestamp ASC);");
start = System.currentTimeMillis();
ContentValues cv = new ContentValues();
for(int i = 0; i < INSERTS; ++i) {
cv.put("event_type", eventType);
cv.put("timestamp", timestamp);
cv.put("data", data);
db.insert("events", null, cv);
}
end = System.currentTimeMillis();
Log.i("Test", String.format("Normal, Speed: %d ms, Records per second: %.2f", end-start, 1000*(double)INSERTS/(double)(end-start)));
db.close();
dbFile.delete();
The database is exactly as the one my real app is using, I tried removing the index but it made no difference.
Here are the results:
Nexus One, Internal memory
Method | Records | Time (ms) | Records per second
-------------+---------+-----------+--------------------
Normal | 100 | 2072 | 48.26
InsertHelper | 100 | 1662 | 60.17
Nexus One, External memory:
Method | Records | Time (ms) | Records per second
-------------+---------+-----------+--------------------
Normal | 100 | 7390 | 13.53
InsertHelper | 100 | 7152 | 13.98
Emulator, Internal memory:
Method | Records | Time (ms) | Records per second
-------------+---------+-----------+--------------------
Normal | 100 | 1803 | 55.46
InsertHelper | 100 | 3075 | 32.52
Emulator, External memory:
Method | Records | Time (ms) | Records per second
-------------+---------+-----------+--------------------
Normal | 100 | 5742 | 17.42
InsertHelper | 100 | 7164 | 13.96
As you can see the emulator cannot be trusted, InsertHelper should be faster if anything.
This is, of course, to be expected, the test was mostly done out of curiosity.
What have me concerned however is the bad performance on my phone when using external memory, have I missed some crucial aspect of SQLiteDatabase or is it simply so that the SD card will be slow?
I can add that in my real app I've disabled locking and it makes little difference.
CommonsWare is correct in his comment. Something that makes a big difference for db performance is using transactions. Wrap your insert loop in a transaction. I'm not 100% sure if it would work with the InsertHelper but you can try replacing your for loop with this:
db.beginTransaction();
try {
for(int i = 0; i < INSERTS; ++i) {
helper.prepareForInsert();
helper.bind(eventTypeCol, eventType);
helper.bind(timestampCol, timestamp);
helper.bind(dataCol, data);
helper.execute();
}
db.setTransactionSuccessful();
} finally {
db.endTransaction();
}
I have some db performance issues so I used your code to measure the inserts per second on my system. But I also added wrapping in {begin,end}Transaction().
In the emulator. I got:
InsertHelper-Internal-Trans, Speed: 67 ms, Records per second: 1492.54
InsertHelper-External-Trans, Speed: 70 ms, Records per second: 1428.57
Normal-Internal-Trans, Speed: 148 ms, Records per second: 675.68
Normal-External-Trans, Speed: 152 ms, Records per second: 657.89
InsertHelper-Internal-NoTrans, Speed: 514 ms, Records per second: 194.55
Normal-Internal-NoTrans, Speed: 519 ms, Records per second: 192.68
InsertHelper-External-NoTrans, Speed: 590 ms, Records per second: 169.49
Normal-External-NoTrans, Speed: 618 ms, Records per second: 161.81
And on a Samsung Galaxy Note:
InsertHelper-External-Trans, Speed: 52 ms, Records per second: 1923.08
InsertHelper-Internal-Trans, Speed: 52 ms, Records per second: 1923.08
Normal-External-Trans, Speed: 77 ms, Records per second: 1298.70
Normal-Internal-Trans, Speed: 121 ms, Records per second: 826.45
Normal-External-NoTrans, Speed: 4562 ms, Records per second: 21.92
Normal-Internal-NoTrans, Speed: 4855 ms, Records per second: 20.60
InsertHelper-External-NoTrans, Speed: 5997 ms, Records per second: 16.68
InsertHelper-Internal-NoTrans, Speed: 8361 ms, Records per second: 11.96

Categories

Resources