Develop Reference

How to insert an accent into a sqlite table in android?

I am creating a database in Android, all was going nice, but when I was testing the queries retrieving the correct data I've got an error.
E/AndroidRuntime(14126): java.lang.IndexOutOfBoundsException: Invalid index 0, size is 0
I know that means that there is no data matching the query, but the fact is that I inserted the data by query and it actually has a match. And the same query works with all the codes that doesn't have accents.
These are the queries for inserting the rows.
INSERT INTO "codigo" VALUES('A','PEATÓN');
INSERT INTO "codigo" VALUES('B','PEATÓN');
INSERT INTO "codigo" VALUES('C','PEATÓN');
So I did a query that gets the values of the field that I was replacing, like this:
String selectCode = "select distinct c.tipo from codigo c";
Cursor cursor = database.rawQuery(selectCodigo, new String[] {});
cursor.moveToFirst();
while (!cursor.isAfterLast()) {
String codigo= cursor.getString(0);
codigos.add(codigo);
System.out.println(codigo);
cursor.moveToNext();
}
cursor.close();
return codigos;
And the result was this:
10-14 16:40:32.140: I/System.out(13716): PEAT�N
I have the text file in the /raw folder and I edited from the Eclipse so I make sure it wasn't the table export I did, but I have the same results.
This is the code that reads the file:
public int insertFromFile(SQLiteDatabase db,Context context, int resourceId) throws IOException {
// Reseting Counter
int result = 0;
// Open the resource
InputStream insertsStream = context.getResources().openRawResource(resourceId);
BufferedReader insertReader = new BufferedReader(new InputStreamReader(insertsStream));
// Iterate through lines (assuming each insert has its own line and theres no other stuff)
while (insertReader.ready()) {
String insertStmt = insertReader.readLine();
db.execSQL(insertStmt);
result++;
}
insertReader.close();
// returning number of inserted rows
return result;
}
How could I get that accent working?
I am using a lot of them, so, replacing the word is not a way out.
Please help, is the only thing I need to finish this project.
UPDATE:
Had the same problems two more times later. The first I fixed it by working with a .sql file coded in cp1525 and opened it in Eclipse with the default editor and find/replaced the wrong characters.
The last time I've got this error I made it the right way, and found that if you are working with SQLiteManager it imports files encoded in UTF-8 but it export files in ANSI, so I opened the file with Notepad++, change the enconding of the .sql file from ANSI to UTF-8 and it works fine, all the characters were shown fine.

The InputStreamReader constructor documentation says:
Constructs a new InputStreamReader on the InputStream in. This constructor sets the character converter to the encoding specified in the "file.encoding" property and falls back to ISO 8859_1 (ISO-Latin-1) if the property doesn't exist.
If the file is encoded in UTF-8, you have to tell the reader this.

in your SQLQueryString just add before
PRAGMA encoding = "UTF-8";
example
"PRAGMA encoding = \"UTF-8\"; INSERT INTO MYTABLE (mytablefield) VALUES ('value'); "

Lots of inserts into SQLite database from CSV file

When the user downloads my app for the first time, the app downloads a CSV file which contains about 100,000 rows of data.
Then, I would like to populate my SQLite database with it.
Here is my code:
InputStream inputStream = activity.openFileInput(file);
InputStreamReader inputStreamReader = new InputStreamReader(inputStream);
BufferedReader bufferedReader = new BufferedReader(inputStreamReader);
String line = null;
dbHelper.beginTransaction();
while((line = bufferedReader.readLine()) != null) {
String[] values = line.replaceAll("'", "''").split(",");
dbHelper.insert(values);
}
dbHelper.setTransactionSuccessful();
dbHelper.endTransaction();
dbHelper.close();
In my DBHelper class, here is the method insert:
public void insert(String[] data) {
ContentValues values = new ContentValues();
values.put(DBHelper.SHAPE_ID, data[0]);
values.put(DBHelper.SHAPE_PT_LAT, data[1]);
values.put(DBHelper.SHAPE_PT_LON, data[2]);
values.put(DBHelper.SHAPE_PT_SEQUENCE, data[3]);
myDB.insert(DBHelper.TABLE_SHAPES, null, values);
}
I tried this code, it worked BUT it took 7 minutes to do it...
Am I doing something wrong ? Is there a better way (read faster) to populate a SQLite database?

You should download the SQLite database file itself.
If there is other data in the database that you want to keep, you could either
copy that other data into the downloaded database (since it is not as much data, this should be fast); or
keep the downloaded database and the other database separated (and ATTACH one to the other if you need to do joins between them).

Use a prepared statement, that should give a performance boost.
Something like:
SQLiteStatement statement = db.compileStatement("INSERT INTO " + TABLE_SHAPES + " VALUES(?, ?));
and change your insert() method to do something like:
statement.bindString(1, data[0]);
statement.bindString(2, data[1]);
statement.executeInsert();

According to this, SQLite3 has facilities to import a CSV file built-in:
.separator ','
.import '/path/to/csv/data' [table]
I'd imagine this can be done on Android using the DatabaseUtils.InsertHelper:
DatabaseUtils.InsertHelper helper = new DatabaseUtils.InsertHelper(dbFilePath, dbTablename);
helper.prepareForInsert();
// use the bind* methods to bind your columns of CSV data to the helper in a loop
helper.execute();
I also see the the class is now marked as deprecated (API level 17) in lieu of SQLiteStatement, which may or may not be relevant to your application. If you have further questions, please leave a comment.

Displaying a unicode character from SQLite database

I have a listView which is populated by data from an SQLite database. One of my fields is a marker with a unicode character of either a full circle: \u25cf or open circle: \u25cb. Both characters display properly when I use a hardcoded string in a text field. However, in my listView I am seeing the text encoding instead of the character.
Does anyone know why this is... and how I get the unicode characters to display?
Thanks.
Update:
The insertion code is
private void loadRecords() throws IOException {
final Resources resources = mContext.getResources();
InputStream inputStream = resources.openRawResource(R.raw.records);
BufferedReader reader = new BufferedReader(new InputStreamReader(inputStream));
try {
String line;
while ((line = reader.readLine()) != null) {
String[] strings = TextUtils.split(line, ",");
if (strings.length <2) continue;
long id = addRecord(strings[0].trim(),strings[1].trim(),strings[2].trim(),strings[3].trim(),
strings[4].trim(),strings[5].trim(),strings[6].trim());
}
} finally {
reader.close();
}
}
with the resource being a csv file with one line being e.g.
primaryKey,name,surname,address,phone,email,\u25CB

In your text file, \u25CB are six characters.
To correctly use Unicode characters, put them directly into the file:
primaryKey,name,surname,address,phone,email,●
However, you must ensure that the .csv file has the same encoding as that used by your InputStreamReader (set the second parameter of its constructor).

How to execute the query in a plain text file in Android?

I can read a text file which is in the assets folder. But now I have a DB creation query in the text file; how can I execute that query in Android? Please don't ask why I want to keep queries in a plain text file.

You should use a SqliteOpenHelper and override the onCreate method.
Inside that method you should read the file from resources and execute it.
See this for how to use an SqliteOpenHelper
Then for reading file from resources just do this (from here):
#Override
public void onCreate(SQLiteDatabase database) {
Resources res = getResources();
InputStream in_s = res.openRawResource(R.raw.sql);
byte[] b = new byte[in_s.available()];
in_s.read(b);
String sql = new String(b);
database.execSQL(sql);
}
Then you also need to implement onUpgrade (since it is marked as abstract) but you don't need to do anything in that method until you decide to change the structure of your database.

Reading a textfile database and displaying the results

I have a database thats in the form of a text file, my job is to parse the txt file and display the data in a listview. I have no idea where to start.
Heres an example entry.
"|9251115|,|0|,|DETAILS|,||,||,|Heading Price Text Text |,||,||
Where each || represents a field. There are also html tags between heading price and the text (p,b)
My first idea would be to parse it similarly to an xml document, i.e have it create a new line where it starts with a "|", fill it with everything in between and end the line when it reaches the next "|". But I still have no concrete idea on how to do this.
EDIT:
Taking it one step at a time for now. Using stringtokenizer to read it line by line and remove "," for a start. Ran into a problem, the textview to display the results is displaying false for some reason instead of the scanned text. here's my code if anyone needs a good headscratcher.
Context myContext;
#Override
public void onCreate(Bundle savedInstanceState) {
super.onCreate(savedInstanceState);
TextView t = (TextView) findViewById(R.id.text);
st = new ArrayList<property>();
try
{
InputStream is;
is = myContext.getAssets().open("rooms.txt");
BufferedReader br = new BufferedReader(new InputStreamReader(is,"UTF-8"));
String read = br.readLine();
while( read != null)
{
StringTokenizer st = new StringTokenizer(read,",");
{
while(st.hasMoreTokens())
{
String a = st.nextToken();
String b = st.nextToken();
String c = st.nextToken();
String d = st.nextToken();
String e = st.nextToken();
String f = st.nextToken();
String g = st.nextToken();
String h = st.nextToken();
t.setText(a+" "+b+" "+c+" "+d+" "+e+" "+f+" "+g+" "+h);
}
}
}
}
catch(Exception e)
{
e.printStackTrace();
}

While you can definitely implement your own parser using pure Java code, you should consider using a SQLite database (1) to store your data, which will allow you to insert, delete, modify and query much more easily.
If you database comes in that format from an external source, I'd write a one-time parser that parses the data and inserts it into the SQLite database for future use.
Remember that the CPU on Android devices is slower than your average PC CPU, so if you are parsing large amounts of data in this format all the time, your app might become very slow. Hence my suggestion of converting it to a database.
Another option you have in this case is using XML like you said, because there are ready-to-use parsers out there. But the advice about performance remains: you should really avoid reparsing the data all the time and, instead, store it in a ready-to-use format.
(1): http://developer.android.com/reference/android/database/sqlite/package-summary.html

Here is how I would do,
Have an object with getter/setter
Have a list intialized
1) You need to use StreamReaders/Bufferedreader to read the file
2) If each is not empty
2a) Use StringTokenizer to parse the string with "," as delimiter
2b) Set tokenized values to object
2c) Add object to list
3) return the list created in above step.
Note: If large data you need to be careful while reading entire file, you may get OutofMemoryError.

Bruno Oliveira gave very good advice.
You can parse your file by reading it line by line and then use string.split method, as result you will have all your data in an array where you can easily read and put into a list view or move it to a sqlite database.