Android Importing Raw JSON into Database without a lot of coding - android

I have a sql lite database in my android project that gets fed its data from a web service giving JSON. The JSON is straight forward, no tags. It simply has the rows of table data in the same field order as the table in the database.
JSON
[["123","ny","45"],
["456","nj","76"],
["778","ca","33"]]
SQL Lite Customer Table CustomerID,State,NumofItems
How can I import this into the database without the overhead of creating classes for each table etc.. some tables have over 50 fields.
I am using Xamarin if it somehow makes things easier

It depends upon what you mean by without a lot of coding and how many rows you are populating ;-)
Using dynamics and Json.Net you can use DeserializeObject to obtain an array of dynamics and loop through those array elements and construct a SQL Insert statement bypassing column names and class/model definitions of each table... Using dynamics is slower but unless you are populating a hundred thousand rows every import/app launch/.... this works fine...
This is how I am populating a bunch of tables in a case very much like yours to import numeric sequences for drawing SparkLines within RecyclerView/CardViews, I removed the error checking to simplify the answer:
Batteries_V2.Init();
var sqlConn = new SQLiteConnection("foobar.db");
sqlConn.CreateCommand("create table Customer (CustomerID INT PRIMARY KEY NOT NULL, State TEXT NOT NULL, NumofItems INT NOT NULL)").ExecuteNonQuery();
var jsonString = #"
[[""123"",""ny"",""45""],
[""456"",""nj"",""76""],
[""778"",""ca"",""33""]]"
;
dynamic jsonResponse = JsonConvert.DeserializeObject(jsonString);
sqlConn.BeginTransaction();
foreach (var item in jsonResponse)
{
string aBunchOfValues = "";
var noOfColumns = item.Count;
for (int i = 0; i < noOfColumns; i++)
{
var isString = !((string)item[i]).TryParse(out float n);
aBunchOfValues += $"{(isString ? '"' : ' ')}{item[i]}{(isString ? '"' : ' ')} {(i < noOfColumns - 1 ? ',' : ' ')}";
}
var sqlInsertString = $"INSERT INTO Customer VALUES ({aBunchOfValues});";
sqlConn.CreateCommand(sqlInsertString).ExecuteNonQuery();
}
sqlConn.Commit();
Note: This is using FKrueger's sqlite-net to provide the SQLiteConnection, CreateCommand and the transaction, this is on top of ESink's SQLitePCL.raw, adjust accordingly...

Related

Fetch a single column from Realm Database (Android)

I'm a beginner in Realm.
I have a table with 3 columns which named Id, Name, Email,Address.
To get the data of Name column, we use a query like 'SELECT Name from table_name' for SQLite.
If we using Realm in Android, then which method do we have to use for fetching the data of only one column?
I searched alot on Google & documentation but to no avail.
Could anyone help me?
Update:
What I am tried:
RealmResults<User> results = query.findAll();
ArrayList<String> name = new Arraylist();
for(i=0; i<results.size; i++){
name.add(result.get(i).getName();
}
My problem:
results.size() > 10k. So I want to avoid 10k iteration
for(i=0; i<results.size; i++){
}
Look at queries section at the documentation:
All fetches (including queries) are lazy in Realm, and the data is never copied.
This mean, that data of particular column (property) will be fetched when you call getMyProperty() method. Not after call of finadAll() method of RealmQuery object
If we using Realm in Android, then which method do we have to use for fetching the data of only one column?
You can't, because Realm is an object store, it doesn't have concept of "columns".
My problem:
results.size() > 10k. So I want to avoid 10k iteration
for(i = 0; i < results.size(); i++){
}
Solution: don't iterate?
RealmResults<User> results = query.findAll();
//List<String> name = new ArrayList<>();
//for(i = 0; i < results.size(); i++){
// name.add(result.get(i).getName();
//}
return results;
// ...
String name = results.get(position).getName();

Android Dictionary app

I am going to create an android dictionary application which goal is to translate my mother language (Zazaki) to Turkish.
I have 2 documents which have a format like below:
Zazaki to Turkish word docs
ban: ev ====> home
Turkish to Zazaki
ev: ban, çe ===> home
In my mind, first taking word before colon as key, after colon as value.
Then putting it into a database.
My question is:
"Do I have to do this operation for every time?
How can I make this database available for every download by not importing the word list documents into my database"?
First, if you have excel, it's quite easy to make a word document into a database , by splitting by ' ' and deleting the resulting columns you don't need. You can then export it, e.g. into a .csv file.
For your request, I would use the following pseudo-code as a guide for my code:
int currIndex = 0;
string key, value;
while(!EndOfFile){
currIndex = find('=');
key = getWordBeforeCurrentIndex();
currIndex = find('>');
value = getWordAfterCurrentIndex();
myDictionary.Add(key, value);
}
Make that into real code and voila - you got yourself an app parsing a document into a dictionary.
If you know that your document has an exact format like:
ev ====> home
word2 ===> translation2
word3 ===> translation3
You can of course leverage this and go
Dictionary myDict = new HashTable<string, string>();
InputStream wordDictionary = getResources().openRawResource(R.raw.wordDictionary);
DataInputStream myDIS = new DataInputStream(wordDictionary);
ArrayList<string> lines = new ArrayList<Lines>();
string currLine;
//Read document into arraylist:
while((myLine=myDIS.readline())!=null) list.add(myLine);
//Take each line, add left word as key, add right word as value:
foreach(line : lines){
myDict.add(line.substring(0, line.indexOf(" ")), line.substring(line.indexOf(" ", line.indexOf(" ")+1));
}

Why am I unable to use obtain any results by using SQLite's 'LIKE' clause in an AIR Desktop App?

I am trying to build a simple Library Management System for a class project. I am using SQLite to do that.
The app looks like this:
At first I select and load everything from the table, which works fine as seen in the above image:
Here is the create table query:
"CREATE TABLE IF NOT EXISTS books (id INTEGER PRIMARY KEY AUTOINCREMENT, title TEXT, isbn TEXT, category TEXT, authors TEXT, publisher TEXT, edition INTEGER, copies INTEGER, pdf TEXT, cover TEXT)";
Here is the query that selects the whole table:
"SELECT id, title, isbn, authors, publisher, edition, copies, pdf, cover FROM books ORDER BY id";
To implement the search functionality, I tried using the 'LIKE' clause, like this:
"SELECT id, title, isbn, category, authors, publisher, edition, copies, pdf, cover FROM books WHERE title LIKE %'"+userInput+"'% ORDER BY id";
When I search with the above query, I get this error:
TypeError: Error #1009: Cannot access a property or method of a null object reference.
at (address-removed)/BooksContainer.as:124]
And here is line 124:
var numResults:uint = result.data.length;
I tried to change the code but after working with for about a week, I still get the same error.
Yesterday, however I tried to approach it in a different as I was really frustrated and I was sure this approach would work. What I did is I selected the entire table like this:
public function searchBooks(userInput:String)
{
var stat:SQLStatement = new SQLStatement();
stat.sqlConnection = _connection;
stat.text = "SELECT id, title, isbn, category, authors, publisher, edition, copies, pdf, cover FROM books ORDER BY id";
//stat.parameters['#search'] = "%" + userInput + "%";
stat.execute();
trace(stat.text);
searchedString = userInput;
stat.addEventListener(SQLEvent.RESULT, createSearchedContent, false, 0, true);
}
private function createSearchedContent(e:SQLEvent):void
{
var result:SQLResult = e.target.getResult();
_allBooks = result;
trace(result.data);
var numResults:uint = result.data.length;
var currentY:Number = 0;
var xIterator:Number = 0;
for (var i:int = 0; i < numResults; i++)
{
var row:Object = result.data[i];
var bookTitle:String = String(result.data[i].title);
if (bookTitle.toLowerCase().indexOf(searchedString.toLowerCase()) > -1)
{
var book:Book = new Book(row.title, row.isbn, row.category, row.authors, row.publisher, row.edition, row.copies, row.pdf, row.cover);
book.x = book.width * xIterator + 10;
book.y = currentY;
addChild(book);
xIterator++;
if (xIterator == 7)
{
xIterator = 0;
currentY += book.height + 10;
}
}
}
}
Surprisingly enough, I got the same error. I didn't get it at all. Then I put this in the above loop:
var a:String = _main.search_txt.text.toLowerCase();
var b:String = String(row.title.toLowerCase());
trace(b.indexOf(a), a, b);
What I found out is that , in the trace I always get -1 for the searched term.
Here is the output window:
-1 data dark matter and the dinosaures
-1 data a history of religious ideas
-1 data steve jobs
-1 data digital logic design
-1 data thomas calculus
-1 data Data structures and algorithms in java
-1 data data structures in c++
-1 data data structures and algorithms in java
-1 data data structures and algorithms in java
So can any one help me out here?? I've only got a week to finish this project. Thanks in advance.
It looks that percent operators position issue.
Collect query
where something like '%foo%'
Your query
where something like %'foo'%
Put the percent operators inside single quotations.
"SELECT id, title, isbn, category, authors, publisher, edition, copies, pdf, cover FROM books WHERE title LIKE '%"+userInput+"%' ORDER BY id";
You also able to use like clause with parameters.
ActionScript and SQLite parameters on Select using Like

ORMLite joins queries and Order by

I'm tring to make join in two tables and get all columns in both, I did this:
QueryBuilder<A, Integer> aQb = aDao.queryBuilder();
QueryBuilder<B, Integer> bQb = bDao.queryBuilder();
aQb.join(bQb).prepare();
This equates to:
SELECT 'A'.* FROM A INNER JOIN B WHERE A.id = B.id;
But I want:
SELECT * FROM A INNER JOIN B WHERE A.id = B.id;
Other problem is when taking order by a field of B, like:
aQb.orderBy(B.COLUMN, true);
I get an error saying "no table column B".
When you are using the QueryBuilder, it is expecting to return B objects. They cannot contain all of the fields from A in B. It will not flesh out foreign sub-fields if that is what you mean. That feature has not crossed the lite barrier for ORMLite.
Ordering on join-table is also not supported. You can certainly add the bQb.orderBy(B.COLUMN, true) but I don't think that will do what you want.
You can certainly use raw-queries for this although it is not optimal.
Actually, I managed to do it without writing my whole query as raw query. This way, I didn't need to replace my query builder codes (which is pretty complicated). To achieve that, I followed the following steps:
(Assuming I have two tables, my_table and my_join_table and their daos, I want to order my query on my_table by the column order_column_1 of the my_join_table)
1- Joined two query builders & used QueryBuilder.selectRaw(String... columns) method to include the original table's + the columns I want to use in foreign sort. Example:
QueryBuilder<MyJoinTable, MyJoinPK> myJoinQueryBuilder = myJoinDao.queryBuilder();
QueryBuilder<MyTable, MyPK> myQueryBuilder = myDao.queryBuilder().join(myJoinQueryBuilder).selectRaw("`my_table`.*", "`my_join_table`.`order_column` as `order_column_1`");
2- Included my order by clauses like this:
myQueryBuilder.orderByRaw("`order_column_1` ASC");
3- After setting all the select columns & order by clauses, it's time to prepare the statement:
String statement = myQueryBuilder.prepare().getStatement();
4- Get the table info from the dao:
TableInfo tableInfo = ((BaseDaoImpl) myDao).getTableInfo();
5- Created my custom column-to-object mapper which just ignores the unknown column names. We avoid the mapping error of our custon columns (order_column_1 in this case) by doing this. Example:
RawRowMapper<MyTable> mapper = new UnknownColumnIgnoringGenericRowMapper<>(tableInfo);
6- Query the table for the results:
GenericRawResults<MyTable> results = activityDao.queryRaw(statement, mapper);
7- Finally, convert the generic raw results to list:
List<MyTable> myObjects = new ArrayList<>();
for (MyTable myObject : results) {
myObjects.add(myObject);
}
Here's the custom row mapper I created by modifying (just swallowed the exception) com.j256.ormlite.stmt.RawRowMapperImpl to avoid the unknown column mapping errors. You can copy&paste this into your project:
import com.j256.ormlite.dao.RawRowMapper;
import com.j256.ormlite.field.FieldType;
import com.j256.ormlite.table.TableInfo;
import java.sql.SQLException;
public class UnknownColumnIgnoringGenericRowMapper<T, ID> implements RawRowMapper<T> {
private final TableInfo<T, ID> tableInfo;
public UnknownColumnIgnoringGenericRowMapper(TableInfo<T, ID> tableInfo) {
this.tableInfo = tableInfo;
}
public T mapRow(String[] columnNames, String[] resultColumns) throws SQLException {
// create our object
T rowObj = tableInfo.createObject();
for (int i = 0; i < columnNames.length; i++) {
// sanity check, prolly will never happen but let's be careful out there
if (i >= resultColumns.length) {
continue;
}
try {
// run through and convert each field
FieldType fieldType = tableInfo.getFieldTypeByColumnName(columnNames[i]);
Object fieldObj = fieldType.convertStringToJavaField(resultColumns[i], i);
// assign it to the row object
fieldType.assignField(rowObj, fieldObj, false, null);
} catch (IllegalArgumentException e) {
// log this or do whatever you want
}
}
return rowObj;
}
}
It's pretty hacky & seems like overkill for this operation but I definitely needed it and this method worked well.

OrmLite Select within select using SelectArg to escape characters

I'm using the QueryBuilder to construct the inner SQL that later is used in a raw SQL to avoid escaping invalid characters manually.
SelectArg friendsIN = new SelectArg(friendsUsernames);
QueryBuilder<MyObject, Integer> qb = myObjectDao.queryBuilder();
qb.selectRaw("username", "MAX(time) AS latestTime").groupBy("username").where()
.in("username", friendsIN);
String innerSelect = pq.getStatement();
friendsUsernames is defined as ArrayList<String>.
Then I use the innerSelect to build the outer select:
String select = "SELECT w.id FROM (" + innerSelect +") AS x INNER JOIN myObject AS w on w.username = x.username AND w.time = x.latestTime";
GenericRawResults<String[]> results = myObjectDao.queryRaw(select);
But, as expected, the innerString has '?' and when I call queryRaw on myObjectDao I don't get any result. I tried to give friendsUsername as an array to queryRaw:
GenericRawResults<String[]> results =
myObjectrDao.queryRaw(select,
friendsUsernames.toArray(new String[friendsUsernames.size()]));
But I get the following error:
android.database.sqlite.SQLiteBindOrColumnIndexOutOfRangeException:
bind or column index out of range: handle 0x17a22e8
Any suggestions on how to accomplish this kind of queries with OrmLite?
Yeah that's not going to work. There is only one ? in your query and yet you are trying to pass in an array of user-names. There must be a 1-to-1 correspondence between the number of ? SQL arguments and the number of arguments passed to the queryRaw(...) method exactly.
If the friendsUsernames is a fixed size then you should be able to do something like the following which will generate SQL something like "in (?, ?, ?, ?)":
List<SelectArg> friendsInList = new ArrayList<SelectArg>();
for (int i = 0; i < NUM_FRIENDS; i++) {
// it doesn't matter what the value is since you just want the ?
fieldsInList.add(new SelectArg());
}
...in("name", friendsInList);
However if the list of names is dynamic then you are going to have to do this on the fly since, again, the number of ? must match the number of arguments passed to the queryRaw(...) method exactly.

Categories

Resources