I am just porting some JDBC code (intended for use with HSQLDB) to Android’s own SQLite implementation. I have a snippet where I delete records based on a particular field matching one of the values in a String[] array in my Java code.
Here is the JDBC code for the DELETE statement:
String[] ids = getIdsSomehow();
PreparedStatement stmtD = db.prepareStatement("delete from message where id in (unnest(?))");
Array delIdArray = stmtD.getConnection().createArrayOf("VARCHAR", ids);
stmtD.setArray(1, delIdArray);
stmtD.execute();
stmtD.close();
Another snippet does a SELECT instead of DELETE and has the values in a List<String> instead of an array.
How would I accomplish this with the methods offered by SQLiteDatabase, preferably in a way that does not open up any SQL injection vulnerabilities?
The main “ingredients” to make this work with HSQLDB over JDBC, namely the unnest() function of the DBMS, and the ability to pass array values to SQL statements, are not available with the android.sqlite stack, making a workaround necessary:
String[] ids = getIdsSomehow();
String whereClause = "id in (" + TextUtils.join(",", Collections.nCopies(ids.size(), "?")) + ")";
db.delete("message", whereClause, ids);
This builds a where clause à la id in (?,?,?,?) with the correct number of question marks.
For the selection, use rawQuery() with a SELECT statement built in the same manner. The List<String> can be converted to an array like this:
ids.toArray(new String[]{})
Input came from this answer and its comments.
Related
Conclusion: Android's database APIs work but the documentation is horribly incomplete.
I have recently run into a brain wrecking situation due to the flexibility Sqlite provides by not forcing you to specify the data type when creating the table. The problem was my mindset that assumed that every data type would be a general character sequence if not specified and therefore the way to talk to database is through java.lang.String.
But you can't blame me either when you see methods like the below:
int delete (String table,
String whereClause,
String[] whereArgs)
in the SqlDatabase class from Android docs.
I have a table consisting of Phone No(that I stored as java.lang.String) and Timestamp as a long field. When I tried deleting a record using this method, it just never got deleted despite countless debugging.
I checked everything and query was alright and table is existent and all the checklist until by chance, I discovered that removing the '' around the timestamp while querying in a raw manner instead of using the above method yields a successful deletion, something like this:
DELETE FROM messages_records_table WHERE messageTimestamp = 1508494606000;
instead of the following:
DELETE FROM messages_records_table WHERE messageTimestamp = '1508494606000';
or,
DELETE FROM messages_records_table WHERE messageTimestamp = "1508494606000";
Phone No isn't a problem; it's the timestamp that was creating the problem in INSERTION/DELETION
So, I tried running a raw deletion query with quotes removed(that are required with a string/varchar type) and it yielded successful deletion. I used the following method for this:
db.execSQL(String sql, Object[] whereArgs)
The key thing to notice here is that Object[] is different from String[] when compared to delete(). I passed a Long to Object to make it work but passing a Long.toString() in delete() seems to be useless.
So my question is, Is my analysis correct and delete() API is basically useless or have I missed some bigger picture..after all, it's provided by Android team carefully?
SQLite supports multiple data types; and while column types are not strictly enforced, values might be automatically converted in some cases (this is called affinity).
When your values are stored as numbers, you should access them as numbers, not as strings.
The Android database API does not allow you to use parameter types other than strings in most functions. This is a horrible design bug.
To search for a number, either use execSQL(), which allows you to use number parameters, or convert the string value back into a number:
db.delete(..., "timestamp = CAST(? AS NUMBER)",
new String[]{ String.valueOf(ts) });
The problem was my mindset that assumed that every data type would be
a general character sequence if not specified and therefore the way to
talk to database is through java.lang.String.
I think that's the real issue.
If you specify no type e.g.
CREATE TABLE mytable (col1,col2,col3)
Then according to Determination of Column Affinity(3.1) rule 3:-
3) If the declared type for a column contains the string "BLOB" or if no
type is specified then the column has affinity BLOB.
And then according to Section 3
A column with affinity BLOB does not prefer one storage class over
another and no attempt is made to coerce data from one storage class
into another.
I've personally never had an issue with delete. However I do have a tendency to always delete according to rowid.
Here's a working example usage that shows that delete isn't useless and is deleting according to a long. However the columns are all of type INTEGER :-
int pudeletes;
int sldeletes;
int rdeletes;
int pdeletes;
if(doesProductExist(productid)) {
// if not in a transaction then begin a transaction
if(!intransaction) {
db.beginTransaction();
}
String whereargs[] = { Long.toString(productid)};
// Delete ProductUsage rows that use this product
pudeletes = db.delete(
DBProductusageTableConstants.PRODUCTUSAGE_TABLE,
DBProductusageTableConstants.PRODUCTUSAGE_PRODUCTREF_COL +
" = ?",
whereargs
);
// Delete ShopList rows that use this product
sldeletes = db.delete(
DBShopListTableConstants.SHOPLIST_TABLE,
DBShopListTableConstants.SHOPLIST_PRODUCTREF_COL +
" = ?",
whereargs
);
// Delete Rules rows that use this product
rdeletes = db.delete(
DBRulesTableConstants.RULES_TABLE,
DBRulesTableConstants.RULES_PRODUCTREF_COL +
" = ?",
whereargs
);
// Delete the Product
pdeletes = db.delete(
DBProductsTableConstants.PRODUCTS_TABLE,
DBProductsTableConstants.PRODUCTS_ID_COL +
" = ?",
whereargs
);
// if originally not in a transaction then as one was started
// complete and end the transaction
if(!intransaction) {
db.setTransactionSuccessful();
db.endTransaction();
}
}
Question: Is it possible to use a variable as your table name without having to use string constructors to do so?
Info:
I'm working on a project right now that catalogs data from a star simulation of mine. To do so I'm loading all the data into a sqlite database. It's working pretty well, but I've decided to add a lot more flexibility, efficiency, and usability to my db. I plan on later adding planetoids to the simulation, and wanted to have a table for each star. This way I wouldn't have to query a table of 20m some planetoids for the 1-4k in each solar system.
I've been told using string constructors is bad because it leaves me vulnerable to a SQL injection attack. While that isn't a big deal here as I'm the only person with access to these dbs, I would like to follow best practices. And also this way if I do a project with a similar situation where it is open to the public, I know what to do.
Currently I'm doing this:
cursor.execute("CREATE TABLE StarFrame"+self.name+" (etc etc)")
This works, but I would like to do something more like:
cursor.execute("CREATE TABLE StarFrame(?) (etc etc)",self.name)
though I understand that this would probably be impossible. though I would settle for something like
cursor.execute("CREATE TABLE (?) (etc etc)",self.name)
If this is not at all possible, I'll accept that answer, but if anyone knows a way to do this, do tell. :)
I'm coding in python.
Unfortunately, tables can't be the target of parameter substitution (I didn't find any definitive source, but I have seen it on a few web forums).
If you are worried about injection (you probably should be), you can write a function that cleans the string before passing it. Since you are looking for just a table name, you should be safe just accepting alphanumerics, stripping out all punctuation, such as )(][;, and whitespace. Basically, just keep A-Z a-z 0-9.
def scrub(table_name):
return ''.join( chr for chr in table_name if chr.isalnum() )
scrub('); drop tables --') # returns 'droptables'
For people searching for a way to make the table as a variable, I got this from another reply to same question here:
It said the following and it works. It's all quoted from mhawke:
You can't use parameter substitution for the table name. You need to add the table name to the query string yourself. Something like this:
query = 'SELECT * FROM {}'.format(table)
c.execute(query)
One thing to be mindful of is the source of the value for the table name. If that comes from an untrusted source, e.g. a user, then you need to validate the table name to avoid potential SQL injection attacks. One way might be to construct a parameterised query that looks up the table name from the DB catalogue:
import sqlite3
def exists_table(db, name):
query = "SELECT 1 FROM sqlite_master WHERE type='table' and name = ?"
return db.execute(query, (name,)).fetchone() is not None
I wouldn't separate the data into more than one table. If you create an index on the star column, you won't have any problem efficiently accessing the data.
Try with string formatting:
sql_cmd = '''CREATE TABLE {}(id, column1, column2, column2)'''.format(
'table_name')
db.execute(sql_cmd)
Replace 'table_name' with your desire.
To avoid hard-coding table names, I've used:
table = "sometable"
c = conn.cursor()
c.execute('''CREATE TABLE IF NOT EXISTS {} (
importantdate DATE,
somename VARCHAR,
)'''.format(table))
c.execute('''INSERT INTO {} VALUES (?, ?)'''.format(table),
(datetime.strftime(datetime.today(), "%Y-%m-%d"),
myname))
As has been said in the other answers, "tables can't be the target of parameter substitution" but if you find yourself in a bind where you have no option, here is a method of testing if the table name supplied is valid.
Note: I have made the table name a real pig in an attempt to cover all of the bases.
import sys
import sqlite3
def delim(s):
delims="\"'`"
use_delim = []
for d in delims:
if d not in s:
use_delim.append(d)
return use_delim
db_name = "some.db"
db = sqlite3.connect(db_name)
mycursor = db.cursor()
table = 'so""m ][ `etable'
delimiters = delim(table)
if len(delimiters) < 1:
print "The name of the database will not allow this!"
sys.exit()
use_delimiter = delimiters[0]
print "Using delimiter ", use_delimiter
mycursor.execute('SELECT name FROM sqlite_master where (name = ?)', [table])
row = mycursor.fetchall()
valid_table = False
if row:
print (table,"table name verified")
valid_table = True
else:
print (table,"Table name not in database", db_name)
if valid_table:
try:
mycursor.execute('insert into ' +use_delimiter+ table +use_delimiter+ ' (my_data,my_column_name) values (?,?) ',(1,"Name"));
db.commit()
except Exception as e:
print "Error:", str(e)
try:
mycursor.execute('UPDATE ' +use_delimiter+ table +use_delimiter+ ' set my_column_name = ? where my_data = ?', ["ReNamed",1])
db.commit()
except Exception as e:
print "Error:", str(e)
db.close()
you can use something like this
conn = sqlite3.connect()
createTable = '''CREATE TABLE %s (# );''' %dateNow)
conn.execute(createTable)
basically, if we want to separate the data into several tables according to the date right now, for example, you want to monitor a system based on the date.
createTable = '''CREATE TABLE %s (# );''' %dateNow) means that you create a table with variable dateNow which according to your coding language, you can define dateNow as a variable to retrieve the current date from your coding language.
You can save your query in a .sql or txt file and use the open().replace() method to use variables in any part of your query. Long time reader but first time poster so I apologize if anything is off here.
```SQL in yoursql.sql```
Sel *
From yourdbschema.tablenm
```SQL to run```
tablenm = 'yourtablename'
cur = connect.cursor()
query = cur.execute(open(file = yoursql.sql).read().replace('tablenm',tablenm))
You can pass a string as the SQL command:
import sqlite3
conn = sqlite3.connect('db.db')
c = conn.cursor()
tablename, field_data = 'some_table','some_data'
query = 'SELECT * FROM '+tablename+' WHERE column1=\"'+field_data+"\""
c.execute(query)
I've had a look around and found a few similar cases but none where there is a need for specificity in the entries to sum. So here I am.
I have a method filterPayments that returns all entries in my PayTable based on a specific GroupID and is then displayed in my GridView. From there I want to sum the values of 2 of the 5 columns in PayTable, specifically my Interest and Due columns. I'm not sure how to do this in a query, let alone do it only for specific columns.
Question
How do I add the values of all entries in a specific column.
Is it possible to do this in a SQLite query? If so how do I use the returned value of filterPayments and perform the summation only on specific columns? If it isn't then how can I do this?
Below are my code snippets.
filterPayments
Cursor filterPayments(String Payment) {
SQLiteDatabase db = this.getWritableDatabase();
String[] columns = new String[]{"_id", colGroupID, colPayBal, colInterest, colDue, colDateDue, colPaid};
Cursor c = db.query(viewPmnts, columns, colGroupID + "=?", new String[]{Payment}, null, null, null);
return c;
}
GridView
public void Paygrid() {
dbHelper = new DatabaseHelper(this);
String Payment = String.valueOf(txt.getText());
Cursor a = dbHelper.filterPayments(Payment);
startManagingCursor(a);
String[] from = new String[]{DatabaseHelper.colPayBal, DatabaseHelper.colInterest, DatabaseHelper.colDue, DatabaseHelper.colDateDue, DatabaseHelper.colPaid};
int[] to = new int[]{R.id.Amount, R.id.Interest, R.id.Due, R.id.DateDue, R.id.Paid};
SimpleCursorAdapter saa = new SimpleCursorAdapter(this, R.layout.paygrid, a, from, to);
intgrid.setAdapter(saa);
}
I suggest pulling all the data from column and then sum them in Java or android. That would be the simplest way.
There are no core sqlite functions that does it.
https://www.sqlite.org/lang_corefunc.html
You can however create custom sqlite functions. Look below.
How to create custom functions in SQLite
I hope I get your question right. But if you have two columns with the column names interest and due you can get the sum of both columns with the SQL query
SELECT interest + due FROM PayTable;
This also applies for multiplication (and its inverse counterparts). Unfortunately it gets more tricky for non-integer exponentiation (like square root). As far as I know, you need the already mentioned own SQLite function. If you are lucky you can load a module wrapping the math.h from the C standard lib (search for extension-functions.c)
For other ways of summing in tables look at this question for PostgreSQL (It's the same for SQLite)
I have a sqlite database where all rows have a UUID as the primary key, the db column is defined as a BLOB.
The keys are inserted as byte[] instead of Strings to avoid wasting storage and index spaces. I can insert, update and delete rows using SQLiteDatabase.compileStatement and using bindBlob on the SQLiteStatement but I can't find anyway to bind a blob to the parameters of a SELECT query.
Both SQLiteDatabase.query and .rawQuery expects my WHERE arguments to be Strings which will never match my byte array blobs. I can find my row if I construct my WHERE manually using a BLOB literal like this:
final Cursor query = db.query(getTableName(),
getColumns(),
"id = X'" + bytesToHex(getByteArrayFromUUID(id)) + "'" ,
null,
null,
null,
null);
But then I am vulnerable to SQL injections...
In every other language I have used SQLite in this is not a problem, is there really no way to get a standard prepared SELECT statement with android SQLite?
Most of the APIs in Android sqlite expect the parameter to be strings.
You can use compileStatement() to prepare your query and then use bindBlob() to bind a blob argument to it. Getting useful results out from the query is not easy though, SQLiteStatement has methods for only a few 1x1 result sets.
On the other hand, using a blob as a key doesn't seem like a good idea.
I am trying to learn the SQL Database stuff for SQLite using the android. I have seen a couple examples of the Queries....
I have a two part question about sqlite queries in android.
Part 1
Say I want to delete something. and I use the following Query.
db.delete(MY_DB_TABLE, "CustomerName = ?", new String[] { customerName });
what would happen if the Customer name had a bad character in it.
For example. If I use the following Query
db.execSQL("delete from " + MY_DB_TABLE +
" where customername = '" + customerName + "';");
and say for this example the name of my customer was "Arby's".
That query would blow up because the ' is a special character and the query would not be formatted correctly.
Part 2
does this format allow me to specify as many paramaters as I want.
Example:
db.delete(MYTABLE, "val1 = ? and val2 != ?", new String[] { "test", "test2" } );
Please refer to my post here:
Storing Lists to A Database, and Retrieving Them All Together : Android
and short answer to your question, yes.
Each '?' means that an argument will be expected, so for each '?' you WILL have an exact number of arguments to pass in unless you want an exception :) !