I have a database full of values and one of the columns in an origin (where the particular item comes from). I am trying to populate a spinner with all the origins listed in the database, but obviously I only want them repeated once. I am trying to use the sql statement:
String sql = "SELECT _id, origin FROM items GROUP BY origins";
But I am also using a custom SimpleCursorAdapter to populate the spinner and because of that I also need to query the _id column. When I add the _id column to the query it produces a query with all the repeated origins because the id makes it to where no row is a duplicate. What do I need to do to pass both columns, but remove the duplicates? Being able to organize them alphabetically would also be great!
You get a view with duplicate origin values, since every origin value can "belong" to several ids.
I don't think that the following is a "clean" solution but it gives you no duplicate values along with the highest id corresponding to each origin value:
Select origin, max(_id) from items group by origin
You might also consider normalizing your database. Rather than repeating the origins multiple times in the table, you could create a separate origins reference table. Then, in your original table, you could reference the origin by it's primary key. For example:
origins
***********************
id name
-----------------------
1 origin_1
2 origin_2
3 origin_3
***********************
detail_data
**************************
id origin_id data
1 1 ...
2 1 ...
3 3 ...
4 3 ...
**************************
Then if you only wanted origins for which data existed, you could do something like:
SELECT DISTINCT o.*
FROM origins AS o
JOIN detail_data AS dd ON (dd.origin_id = o.id)
ORDER BY o.name ASC;
I generally prefer a normalized database because normalization can offer advantages such as a reduced footprint size, cleaner data, and a more flexible structure. I will admit, however, that normalization seems less beneficial with Android compared to traditional database systems (i.e., desktops, servers). And normalization has disadvantages, too, such as more complex queries (as we can see above). For maximizing flexibility, maintainability and robustness for the long-term, though, normalization may be the better approach.
To add to Preli's answer, if you want them in alphabetical order, just change it to this:
SELECT origin, MAX(_id) FROM items GROUP BY origin ORDER BY origin
Related
I have an SQLite DB where I perform a query like
Select * from table where col_name NOT IN ('val1','val2')
Basically I'm getting a huge list of values from server and I need to select the ones which is not present in the list given.
Currently its working fine, No issues. But the number of values from server becomes huge as the server DB is getting updated frequently.
So, I may get thousands of String values which I need to pass to the NOT IN
My question is, Will it cause any perfomance issue in the future? Does the NOT IN parameter have any size restriction? (like max 10000 values you can check)?
Will it cause any crash at some point?
This is an official reference about various limitation in sqlite. I think the Maximum Length Of An SQL Statement may related to your case. Default value is 1000000, and it is adjustable.
Except this I don't think any limitation existed for numbers of parameter of NOT IN clause.
With more than a few values to test for, you're better off putting them in a table that has an index on the column holding them. Then things like
SELECT *
FROM table
WHERE col_name NOT IN (SELECT value_col FROM value_table);
or
SELECT *
FROM table AS t
WHERE NOT EXISTS (SELECT 1 FROM value_table WHERE value_col = t.col_name);
will be reasonably efficient no matter how many records are in value_table because that index will be used to find entries.
Plus, of course, it makes it a lot easier to re-use prepared statements because you don't have to create a new one and re-bind every value (You are using prepared statements with placeholders for these values, right, and not trying to put their contents inline into a string?) every time you add a value to the ones you need to check. You just insert it into value_table instead.
Yes, there is a limit of 999 arguments as reported in the official documentation: https://www.sqlite.org/limits.html#max_variable_number
There are a lot of questions about splitting a BigQuery, MySQL column, but I can't find one that fits my situation.
I am processing a large dataset (3rd party) that includes a freeform location field to normalize it for my Android app. When I run a select I'd like to split the column data by commas, take only the last segment and trim it of whitespace.
So far I've come up with the following by Googling documentation:
SELECT RTRIM(LOWER(SPLIT(location, ',')[OFFSET(-1)])) FROM `users` WHERE location <> ''
But the -1 trick to split at last element does not work (with either offset or ordinal). I can't use ARRAY_LENGTH with the same array inline and I'm not exactly sure how to structure a nested query and know the last column index of the row.
I might be approaching this from the wrong angle, I work with Android and NoSQL now so I haven't used MySQL in a long time
How do I structure this query correctly?
I'd like to split the column data by commas, take only the last segment ...
You can use below approach (BigQuery Standard SQL)
SELECT ARRAY_REVERSE(SPLIT(location))[SAFE_OFFSET(0)]
Below is an example illustrating it:
#standardSQL
WITH `project.dataset.table` AS (
SELECT '1,2,3,4,5' location UNION ALL
SELECT '6,7,8'
)
SELECT location, ARRAY_REVERSE(SPLIT(location))[SAFE_OFFSET(0)] last_segment
FROM `project.dataset.table`
with result
Row location last_segment
1 1,2,3,4,5 5
2 6,7,8 8
For trimming - you can use LTRIM(RTRIM()) - like in
SELECT LTRIM(RTRIM(ARRAY_REVERSE(SPLIT(location))[SAFE_OFFSET(0)]))
To get the last part of the split string, I use the len(string) - len(replace(string,delimeter,'')) trick to count the number of delimiters:
split(<string>,'-')[OFFSET(length(<string>)-length(replace(<string>,'-',''))]
For my Android app, I want to save data using sqlite with this format:
name, date, attr1, attr2, attr3,...
These are the requirements:
each date can only contain each name once
there can be a variable number of attributes(numbers) for each name
each specific name has the same number of attributes
The app will be used to track events throughout the day. Events can have zero or more numeric properties.
The questions are: is sqlite the best way to store things here? If so how do I design my database? What other ways are there to store this kind of data?
is sqlite the best way to store things here?
This will depend on a number of other factors, such as how the data will be queried and used, the volume of transactions, data growth and retention, etc. From what you've described, though, SQLite is a great option, offering functionality out-of-the-box that supports some of your requirements directly, and is commonly used in such cases.
If you don't have much experience with relational databases, implementing this functionality may seem difficult at first, but like learning a new language or framework, it will get easier with time.
If so how do I design my database?
Let's step through each of your enumerated requirements...
each date can only contain each name once
SQLite supports the UNIQUE constraint. For example, if your columns were named name and date, you could add the following to your CREATE TABLE statement:
UNIQUE(name, date)
(A more complete CREATE TABLE statement is in the next example below, and it includes this constraint.)
This constraint prevents the insertion of rows with name/date pairs that already exist. Using android.database.sqlite.SQLiteDatabase, if you attempt to insert a row into the table with a duplicate name/date pair, a SQLiteConstraintException will be thrown at runtime. You will need to handle this exception in your java code.
there can be a variable number of attributes(numbers) for each name
This is a textbook case for normalizing the database, putting your data into multiple tables. For example:
CREATE TABLE names (
name_id INTEGER PRIMARY KEY AUTOINCREMENT NOT NULL,
name TEXT NOT NULL,
date DATETIME,
UNIQUE(name, date));
CREATE TABLE attrs (
name_id INTEGER NOT NULL,
attr_value INTEGER NOT NULL,
FOREIGN KEY(attr_value) REFERENCES names(name_id));
Your queries that retrieve attribute data would then JOIN the two tables. Since you indicated that "Events can have zero or more numeric properties", a LEFT OUTER JOIN might be most appropriate, as it will return names and dates even if there are no attributes.
Here's an example query, by name:
SELECT n.name, n.date, a.attr_value
FROM names AS n
LEFT OUTER JOIN attrs AS a
ON n.name_id = a.name_id
WHERE n.name = 'SMITH'
ORDER BY n.name, n.date, a.attr_value;
This query would return results like the following:
name date attr_value
--------------- ---------- ------------
SMITH 2015-02-13 1027
SMITH 2015-02-13 4426
SMITH 2015-02-13 8390
SMITH 2015-02-20 4426
SMITH 2015-02-20 8152
SMITH 2015-02-20 9328
You can then iterate through and process these results in java. If your results include multiple names and/or dates, then in your loop you should keep track of the last used name and date. If the name/date in the current record is the same, the attribute belongs to the current one. If the name/date is different, then this is a new one.
Note that this approach to your database design is flexible, allowing you to query on the attributes, for instance, to see what name/date pairs are associated.
Also note that there is a FOREIGN KEY constraint on the attrs table, meaning that if you attempt to insert a record into that table with a name_id that does not exist in the names table, a SQLiteConstraintException will be thrown at runtime. You will need to handle this exception in your java code.
each specific name has the same number of attributes
You will need to accommodate this requirement in your java code, probably doing some checks in the database prior to performing an INSERT.
What other ways are there to store this kind of data?
Flat files, JSON, XML, third-party data stores (with their own libraries), to name a few.
I'm not sure but I think the best way to achieve your requirement is to use sqlite and to solve your problem you can have 3 columns only. One for the name and one for the date and the other contains a JSON array that represents the rest of the attributes.
I have in my db 2 table with a many to many relationship.
TAB_ARTICLES: {_ID, TITLE, BODY, DATE}
TAB_TAG: {_ID, NAME, COLOR, DATE}
TAB_ART_TAG: {_ID, ARTICLE_ID, TAG_ID}
I need to populate a ListView, one row for article and in every row I need to have a TextView for every label linked to that article. Like the following image
I think 2 solutions.
a. I use a CursorAdapter with a cursor made only on TAB_ARTICLE and than in every row I do a query to join the other 2 tables looking for all tags related at this article. This solution require a lot of db accesses.
b. I realize a temporary table
TABLE_TEMP: {ARTICLE_TITLE, ARTICLE_BODY, ARTICLE_DATE, TAG1_NAME, TAG1_COLOR, TAG2_NAME, TAG2_COLOR, ...}
and I use a query on this table as cursor for custom adapter. This solution use more space and have a limitation on possible displayed tags due to table columns.
Are there other ways?
Well, actually, it's a multicriterion thing: time, space, updates, search, etc. So there's no single recipe. It's very probable, however, that multiple queries will bog down scrolling. Worse, on some devices only. A temporary table may or may not be OK depending on the overall size of your data. And you may want to keep this redundant table in sync with the main one, making simultaneous updates to both.
One of the simplest trade-offs could be adding a redundant TEXT/CLOB column with the tag data (XML, JSON, other markup/separated format) to TAB_ARTICLES and keeping it in sync with your detail data. By the way, you will really need the M:M schema only if your queries substantiate that. Otherwise, the single table would suffice.
Again, I'd list and evaluate all the criteria first and decide what dimensions really need to be scalable and simplify the rest.
I was wondering if it's possible (it should be) to query multiple tables simultaneously (several at once) in SQLite. Basically I have several tables that have the exact same columns, but the data in them is just organized by the table it's in. I need to be able to use SELECT to get data from the tables (I heard UNION could help), which matches a condition, then group the data by the table it's in.
In other words, would something like this be possible?
SELECT * FROM table1,table2,table3,table4,table5,table6 WHERE day=15 GROUP BY {table}
I'd rather not resort to having to query the tables individually as then I would have a bunch of Cursors that I'd have to manually go through and that would be difficult when I only have one SimpleCursorAdapter? Unless a SimpleCursorAdapter can have several Cursors?
Thanks.
EDIT: The structure of my tables:
Main Table - contains references to subtables in a column "tbls"
and meta-information about the data stored in the subtables
Subtable - contains reference to subsubtables in a column "tbls"
and meta-information about the data stored in the
subsubtables
Subsubtable - contains the actual entries
Basically these tables just make it easier to organize the hierarchical data structure. I suppose instead of having the subsubtables, I could keep the actual entries in the subtable but add a prefix, and have a separate table for the meta-information. It just seems it would be harder to delete/update the structure if I need to remove a level in this data set.
You can create view based on your tables, the query of your view is union of your tables.
create view test as select * from table1 union select * from table2
now you can filter data as you want
for more info check union & view
http://www.w3schools.com/sql/sql_union.asp
http://www.w3schools.com/sql/sql_view.asp
In the end, I decided to forgo having many subsubtables, and instead adding another column like Tim and Samuel suggested. It will probably be more efficient as well then chaining SELECTs with UNION.