Question: Is it possible to use a variable as your table name without having to use string constructors to do so?
Info:
I'm working on a project right now that catalogs data from a star simulation of mine. To do so I'm loading all the data into a sqlite database. It's working pretty well, but I've decided to add a lot more flexibility, efficiency, and usability to my db. I plan on later adding planetoids to the simulation, and wanted to have a table for each star. This way I wouldn't have to query a table of 20m some planetoids for the 1-4k in each solar system.
I've been told using string constructors is bad because it leaves me vulnerable to a SQL injection attack. While that isn't a big deal here as I'm the only person with access to these dbs, I would like to follow best practices. And also this way if I do a project with a similar situation where it is open to the public, I know what to do.
Currently I'm doing this:
cursor.execute("CREATE TABLE StarFrame"+self.name+" (etc etc)")
This works, but I would like to do something more like:
cursor.execute("CREATE TABLE StarFrame(?) (etc etc)",self.name)
though I understand that this would probably be impossible. though I would settle for something like
cursor.execute("CREATE TABLE (?) (etc etc)",self.name)
If this is not at all possible, I'll accept that answer, but if anyone knows a way to do this, do tell. :)
I'm coding in python.
Unfortunately, tables can't be the target of parameter substitution (I didn't find any definitive source, but I have seen it on a few web forums).
If you are worried about injection (you probably should be), you can write a function that cleans the string before passing it. Since you are looking for just a table name, you should be safe just accepting alphanumerics, stripping out all punctuation, such as )(][;, and whitespace. Basically, just keep A-Z a-z 0-9.
def scrub(table_name):
return ''.join( chr for chr in table_name if chr.isalnum() )
scrub('); drop tables --') # returns 'droptables'
For people searching for a way to make the table as a variable, I got this from another reply to same question here:
It said the following and it works. It's all quoted from mhawke:
You can't use parameter substitution for the table name. You need to add the table name to the query string yourself. Something like this:
query = 'SELECT * FROM {}'.format(table)
c.execute(query)
One thing to be mindful of is the source of the value for the table name. If that comes from an untrusted source, e.g. a user, then you need to validate the table name to avoid potential SQL injection attacks. One way might be to construct a parameterised query that looks up the table name from the DB catalogue:
import sqlite3
def exists_table(db, name):
query = "SELECT 1 FROM sqlite_master WHERE type='table' and name = ?"
return db.execute(query, (name,)).fetchone() is not None
I wouldn't separate the data into more than one table. If you create an index on the star column, you won't have any problem efficiently accessing the data.
Try with string formatting:
sql_cmd = '''CREATE TABLE {}(id, column1, column2, column2)'''.format(
'table_name')
db.execute(sql_cmd)
Replace 'table_name' with your desire.
To avoid hard-coding table names, I've used:
table = "sometable"
c = conn.cursor()
c.execute('''CREATE TABLE IF NOT EXISTS {} (
importantdate DATE,
somename VARCHAR,
)'''.format(table))
c.execute('''INSERT INTO {} VALUES (?, ?)'''.format(table),
(datetime.strftime(datetime.today(), "%Y-%m-%d"),
myname))
As has been said in the other answers, "tables can't be the target of parameter substitution" but if you find yourself in a bind where you have no option, here is a method of testing if the table name supplied is valid.
Note: I have made the table name a real pig in an attempt to cover all of the bases.
import sys
import sqlite3
def delim(s):
delims="\"'`"
use_delim = []
for d in delims:
if d not in s:
use_delim.append(d)
return use_delim
db_name = "some.db"
db = sqlite3.connect(db_name)
mycursor = db.cursor()
table = 'so""m ][ `etable'
delimiters = delim(table)
if len(delimiters) < 1:
print "The name of the database will not allow this!"
sys.exit()
use_delimiter = delimiters[0]
print "Using delimiter ", use_delimiter
mycursor.execute('SELECT name FROM sqlite_master where (name = ?)', [table])
row = mycursor.fetchall()
valid_table = False
if row:
print (table,"table name verified")
valid_table = True
else:
print (table,"Table name not in database", db_name)
if valid_table:
try:
mycursor.execute('insert into ' +use_delimiter+ table +use_delimiter+ ' (my_data,my_column_name) values (?,?) ',(1,"Name"));
db.commit()
except Exception as e:
print "Error:", str(e)
try:
mycursor.execute('UPDATE ' +use_delimiter+ table +use_delimiter+ ' set my_column_name = ? where my_data = ?', ["ReNamed",1])
db.commit()
except Exception as e:
print "Error:", str(e)
db.close()
you can use something like this
conn = sqlite3.connect()
createTable = '''CREATE TABLE %s (# );''' %dateNow)
conn.execute(createTable)
basically, if we want to separate the data into several tables according to the date right now, for example, you want to monitor a system based on the date.
createTable = '''CREATE TABLE %s (# );''' %dateNow) means that you create a table with variable dateNow which according to your coding language, you can define dateNow as a variable to retrieve the current date from your coding language.
You can save your query in a .sql or txt file and use the open().replace() method to use variables in any part of your query. Long time reader but first time poster so I apologize if anything is off here.
```SQL in yoursql.sql```
Sel *
From yourdbschema.tablenm
```SQL to run```
tablenm = 'yourtablename'
cur = connect.cursor()
query = cur.execute(open(file = yoursql.sql).read().replace('tablenm',tablenm))
You can pass a string as the SQL command:
import sqlite3
conn = sqlite3.connect('db.db')
c = conn.cursor()
tablename, field_data = 'some_table','some_data'
query = 'SELECT * FROM '+tablename+' WHERE column1=\"'+field_data+"\""
c.execute(query)
Related
I am fetching a string from Database using the column id. When I enter a query in SQLite DB Browser it returns what is need but the same query returns nothing when coded through Java.
My Data Base contains a table named drugs which has 3 columns i.e. drug_id, drug_name and drug_overview. Using drug_id i am fetching drug_overview. I have tried the query in db browser which returns me the correct string from drug_overview but the same query returns nothing when coded through java.
SQLite DB Browser query:
SELECT * FROM drugs Where drug_id = 50;
JAVA CODE:
String query105 = "SELECT * FROM drugs Where drug_id = " + drug_id;
Log.e("TESTDB1","Drugs table query: " + query105);
Cursor c105 = db.rawQuery(query105,null);
if (c105 != null){
while (c105.moveToNext()){
String overview = c105.getString(c105.getColumnIndexOrThrow("drug_overview"));
Log.e("TESTDB1","Overview: " + overview);
}
c105.close();
}
Expected result is Overview: Acyclovir is an antiviral drug. It slows the growth and spread of the herpes virus in the body. It will not cure herpes, but it can lessen the symptoms of the infection.Acyclovir is used to treat infections caused by herpes viruses, such as genital herpes, cold sores, shingles, and chicken pox, as well as varicella (chickenpox), and cytomegalovirus.Acyclovir may also be used for purposes not listed in this medication guide.
But the actual result is Overview:
empty
. When i change the id in my query it gives the correct result from a different drug.
I am afraid that your problem may be with the actual data itself as Mike said in comment, I think your database in the files is old and you haven't copied the latest to folder. Try to re-install and delete old database
Your query returns 0 or 1 lines so I think you should use c105.moveToFirst() instead of c105.moveToNext(). moveToNext is supposed to be used for a list, not for a single entry. Do something like:
if (c105.moveToFirst()){
String overview = c105.getString(c105.getColumnIndex("drug_overview"));
// do something with the result
}
c105.close();
It continues to occur error... what is the problem?
every variables are String.
String insertSQL = "INSERT INTO " + DBHelper.getTableName() + " VALUES (\''"+entry.getKey()+"\'', \''"+images+"\'')";
Error message
INSERT INTO LABELING RESULT VALUES (''Sky'',
''["/storage/emulated/0/DCIM/CandyCam/IMG_20171009_164101723.jpg","/storage/emulated/0/DCIM/Pictail/IMG_20180305_000218777.jpg","/storage/emulated/0/DCIM/Pictail/IMG_20180401_235850170.jpg","/storage/emulated/0/DCIM/Pictail/IMG_20180518_194252232.jpg"]''))
My table has three column : ID(Integer), LABEL(TEXT), IMAGES(TEXT)
Issue 1 you have specified a table name with a space i.e. LABELING RESULT, if this is the table name then you would need to enclose it e.g. [LABELING RESULT].
Issue 2 you have double single quotes when you should only have 1.
Issue 3 you have an extra closing parenthesis.
I believe the following is what you want :-
INSERT INTO [LABELING RESULT] VALUES ('Sky', '["/storage/emulated/0/DCIM/CandyCam/IMG_20171009_164101723.jpg","/storage/emulated/0/DCIM/Pictail/IMG_20180305_000218777.jpg","/storage/emulated/0/DCIM/Pictail/IMG_20180401_235850170.jpg","/storage/emulated/0/DCIM/Pictail/IMG_20180518_194252232.jpg"]'))
Which would (not checked though) be :-
String insertSQL = "INSERT INTO [" + DBHelper.getTableName() + "] VALUES ('"+entry.getKey()+"', '"+images+"')";
That assumes that Sky goes into the LABEL column and the 2 comma separated image paths go into the IMAGES column.
Additional re enclosing (the [ ] ):-
If you want to use a keyword as a name, you need to quote it. There
are four ways of quoting keywords in SQLite:
'keyword' A keyword in single quotes is a string literal.
"keyword" A keyword in double-quotes is an identifier.
[keyword] A keyword enclosed in square brackets is an identifier. This is not
standard SQL. This quoting mechanism is used by MS Access and SQL
Server and is included in SQLite for compatibility.
keyword A keyword enclosed in grave accents (ASCII code 96) is an identifier.
This is not standard SQL. This quoting mechanism is used by MySQL and
is included in SQLite for compatibility.
SQL As Understood By SQLite - SQLite Keywords
Conclusion: Android's database APIs work but the documentation is horribly incomplete.
I have recently run into a brain wrecking situation due to the flexibility Sqlite provides by not forcing you to specify the data type when creating the table. The problem was my mindset that assumed that every data type would be a general character sequence if not specified and therefore the way to talk to database is through java.lang.String.
But you can't blame me either when you see methods like the below:
int delete (String table,
String whereClause,
String[] whereArgs)
in the SqlDatabase class from Android docs.
I have a table consisting of Phone No(that I stored as java.lang.String) and Timestamp as a long field. When I tried deleting a record using this method, it just never got deleted despite countless debugging.
I checked everything and query was alright and table is existent and all the checklist until by chance, I discovered that removing the '' around the timestamp while querying in a raw manner instead of using the above method yields a successful deletion, something like this:
DELETE FROM messages_records_table WHERE messageTimestamp = 1508494606000;
instead of the following:
DELETE FROM messages_records_table WHERE messageTimestamp = '1508494606000';
or,
DELETE FROM messages_records_table WHERE messageTimestamp = "1508494606000";
Phone No isn't a problem; it's the timestamp that was creating the problem in INSERTION/DELETION
So, I tried running a raw deletion query with quotes removed(that are required with a string/varchar type) and it yielded successful deletion. I used the following method for this:
db.execSQL(String sql, Object[] whereArgs)
The key thing to notice here is that Object[] is different from String[] when compared to delete(). I passed a Long to Object to make it work but passing a Long.toString() in delete() seems to be useless.
So my question is, Is my analysis correct and delete() API is basically useless or have I missed some bigger picture..after all, it's provided by Android team carefully?
SQLite supports multiple data types; and while column types are not strictly enforced, values might be automatically converted in some cases (this is called affinity).
When your values are stored as numbers, you should access them as numbers, not as strings.
The Android database API does not allow you to use parameter types other than strings in most functions. This is a horrible design bug.
To search for a number, either use execSQL(), which allows you to use number parameters, or convert the string value back into a number:
db.delete(..., "timestamp = CAST(? AS NUMBER)",
new String[]{ String.valueOf(ts) });
The problem was my mindset that assumed that every data type would be
a general character sequence if not specified and therefore the way to
talk to database is through java.lang.String.
I think that's the real issue.
If you specify no type e.g.
CREATE TABLE mytable (col1,col2,col3)
Then according to Determination of Column Affinity(3.1) rule 3:-
3) If the declared type for a column contains the string "BLOB" or if no
type is specified then the column has affinity BLOB.
And then according to Section 3
A column with affinity BLOB does not prefer one storage class over
another and no attempt is made to coerce data from one storage class
into another.
I've personally never had an issue with delete. However I do have a tendency to always delete according to rowid.
Here's a working example usage that shows that delete isn't useless and is deleting according to a long. However the columns are all of type INTEGER :-
int pudeletes;
int sldeletes;
int rdeletes;
int pdeletes;
if(doesProductExist(productid)) {
// if not in a transaction then begin a transaction
if(!intransaction) {
db.beginTransaction();
}
String whereargs[] = { Long.toString(productid)};
// Delete ProductUsage rows that use this product
pudeletes = db.delete(
DBProductusageTableConstants.PRODUCTUSAGE_TABLE,
DBProductusageTableConstants.PRODUCTUSAGE_PRODUCTREF_COL +
" = ?",
whereargs
);
// Delete ShopList rows that use this product
sldeletes = db.delete(
DBShopListTableConstants.SHOPLIST_TABLE,
DBShopListTableConstants.SHOPLIST_PRODUCTREF_COL +
" = ?",
whereargs
);
// Delete Rules rows that use this product
rdeletes = db.delete(
DBRulesTableConstants.RULES_TABLE,
DBRulesTableConstants.RULES_PRODUCTREF_COL +
" = ?",
whereargs
);
// Delete the Product
pdeletes = db.delete(
DBProductsTableConstants.PRODUCTS_TABLE,
DBProductsTableConstants.PRODUCTS_ID_COL +
" = ?",
whereargs
);
// if originally not in a transaction then as one was started
// complete and end the transaction
if(!intransaction) {
db.setTransactionSuccessful();
db.endTransaction();
}
}
Here is the table where i want to insert the value:
" create table if not exists "+CipherCongfigTable +
" ( DATABSE_NAME **TEXT** PRIMARY KEY NOT NULL,DATABSE_KEY **TEXT** NOT NULL);";
when i want to insert
String **configDBPassword**= "**x\'2DD29CA89\'**"
through statement
"insert into "+CipherCongfigTable+ " values("+DataBaseName+","+**configDBPassword**+")"
I am getting exception:
unrecognized token: "\":
I need the password in the same format i.e. having escape charecter. Is there any way to do it????
Thanks
Don't manually build INSERT (or any other if you can avoid it) queries on Android (or any other database wrapper as long as there is a predefined API to get what you want). It opens up your application to quoting problems like the one from your question and --at the worst-- to SQL injection attacks from outside of your application.
For example, setting configDBPassword = "\"; DROP TABLE <tablename>; --" I could possibly wreak havoc on your database as long as configDBPassword can be entered by the user.
Also, SQLite uses double quotes ("), backticks (`, borrowed from MySQL), or square brackets ([], borrowed from MS SQL) to quote identifiers (e.g. column or table names with spaces in them), string literals are canonically quoted with single (') quotes. SQLite is a quite liberal in allowing to mix both quoting types, but it is significantly more readable to use the proper quoting style whereever appropriate. From the documentation:
Programmers are cautioned not to use the two exceptions described in the previous bullets. We emphasize that they exist only so that old and ill-formed SQL statements will run correctly. Future versions of SQLite might raise errors instead of accepting the malformed statements covered by the exceptions above.
As a matter of fact, you should avoid doing the quoting by yourself whenever possible. For inserting values, please instead use SQLiteDatabase.insert() which is the proper way of inserting values into an SQLiteDatabase on Android. It also does proper quoting of the arguments, too:
db.beginTransaction();
try {
final ContentValues values = new ContentValues();
values.put("DATABSE_NAME", DataBaseName);
values.put("DATABSE_KEY", configDBPassword);
db.insert(CipherCongfigTable, null, values);
db.setTransactionSuccessful();
} finally {
db.endTransaction();
}
Always properly quote your SQL parameters.
Try this .
"INSERT INTO "+CipherCongfigTable+ " values('DataBaseName','configDBPassword')”
You are not adding Single quotes(') in your insert query
Edit :
If you need to insert string with Single qotes means use like this before insert.
configDBPassword = configDBPassword.replace ("'", "''");
Also change your **
" create table if not exists "+CipherCongfigTable +
" ( DATABSE_NAME TEXT PRIMARY KEY NOT NULL,DATABSE_KEY TEXT NOT NULL);";
Well I'm trying to select all entities where entitie_id = *some id*
I do this with this rule:
List<NodeSugar> nodesugars = NodeSugar.find(NodeSugar.class, "entitie_id = ? ", String.valueOf(nodeid));
I know I should get some results back and I get an error.
this error:
E/SQLiteLog﹕ (1) no such column: entitie_id
I know the column exists because I can set it in a different part of code.
Where am I going wrong?
You should be able to query with "entitieid".
Sugar ORM does some conversion when it creates the columns in the tables. Here is a basic example:
String firstName; //creates column first_name
int entitie_id; //creates column entitieid
The documentation only specifies the conversion from camel case to underscore separation. I had to figure the other one out.
You can use NamingHelper.toSQLNameDefault() to wrap column names when building queries. Try:
List<NodeSugar> nodesugars = NodeSugar.find(NodeSugar.class, NamingHelper.toSQLNameDefault("entitie_id") + " = ? ", String.valueOf(nodeid));
NamingHelper is used in the Sugar ORM library to create tables and column names:
https://github.com/satyan/sugar/blob/master/library/src/main/java/com/orm/helper/NamingHelper.java
SugarORM currently uses toSQLName() method of the StringUtil class to generate names for columns
For example in your case:
List<NodeSugar> nodesugars = NodeSugar.find(NodeSugar.class, StringUtil.toSQLName("entitie_id")+" = ? ", String.valueOf(nodeid));
The other and neat way to do it would be to use the Select class:
List<NodeSugar> nodesugars = Select.from(NodeSugar.class).where(Condition.prop(StringUtil.toSQLName("entitie_id")).eq(nodeid)).list();
https://satyan.github.io/sugar/query.html