Develop Reference

Storing/retrieving highly dynamic data from an Android app

I'm trying to develop an Android app to help me test ceramic glazes, and have run into a problem with storing data. The main feature I want is, within one glaze, the ability to modify its recipe, yet still keep the older versions of that recipe. Since a lot of my data is repetitive (finish, opacity, firing atmosphere, etc.), a database would be the best way to store that data. However, each glaze can have any number of versions, and the recipe within each version can have any number of ingredients (normally 1-10, but as much as 20). I cannot think of a way to store and retrieve this "two dimensional" dynamic data. I could make an individual database for each glaze, but I feel that defeats the purpose of a database. Any help implementing a way to store this highly dynamic data would be appreciated. Thank you.

I think you are looking to have a number of related tables.
I'd suggest a glaze base table, containing the base glazes, a table for recipe/versioned glazes. An ingredients/properties table. An then an extended link/associative table table this being used to link ingredients properties to recipes and also contain amounts.
The Glaze Table would be very simple just the name of the glaze and a unique identifier. e.g
CREATE TABLE IF NOT EXISTS glazes (_id INTEGER PRIMARY KEY, glazename TEXT)
The GlazeVersion Table is again relatively simple, a unique identifier, the version and a reference to it's parent/base glaze. e.g.
CREATE TABLE glazeversions (_id INTEGER PRIMARY KEY, version TEXT, glazebase_ref INTEGER)
The Indgredients table is yet again simple, a unique id and name e.g.
CREATE TABLE ingredients (_id INTEGER PRIMARY KEY, ingredient TEXT)
The Recipes table would have a reference to the glazeversion (and therefore implicitly the glaze), a reference to the ingredient and a field for the amount of the ingredient e.g.
CREATE TABLE IF NOT EXISTS recipes (glazeversion_ref INTEGER, ingredient_ref INTEGER, amount INTEGER, PRIMARY KEY(glazeversion_ref, ingredient_ref))
So lets assume we have 3 base glazes RED, GREEN and BLUE. There would be 3 entries in the glazes table (with for illustrative purposes id's 1 2 and 3).
Also lets assume that each has three versions named Version001, Version002 and Version003 (9 rows in the glazeversion table id's 1-9 for illustrative purposes).
The resultant glazeversions table could be :-
A query could be written to show the links/associations/references e.g.
SELECT * FROM glazeversions JOIN glazes WHERE glazebase_ref = glazes._id ORDER BY glazename
the result would be :-
For the Ingredients table lets assume that there are 5 ingredients, Ochre, Calcium, Atmosphere, Temperature (temp for short) and Duration (id's 1-5 respectively). e.g.
Now it's onto the Recipes table, let's initially assume 1 recipe for glaze Blue version Version001 and it has 3 ingredients Atmosphere, Temp and Ochre
The recipes table would have 3 rows all three referencing the glazeversion Version001 for glaze Blue (_id 2) but each referencing the respective ingredient (id's 1, 3 and 4).
We'll also assume two other glazes:-
Red Version001 with 4 ingredients Duration, Temp, Atmosphere and Ochre so 4 rows all referencing glazeversion Version001 for Red (_id 1), respective ingredients id's 1, 3, 4 and 5.
Green Version003 with 2 ingredients Ochre and Calcium. So both rows reference glazeversion Version003 for Blue (_id 9), repective ingredients are id's 1 and 2.
So the table looks like:-
The following query is an example of tying everything together:-
SELECT glazename, version, ingredient, amount FROM recipes JOIN ingredients ON ingredient_ref = ingredients._id JOIN glazeversions ON glazeversion_ref = glazeversions._id JOIN glazes ON glazebase_ref = glazes._id
This would result in :-
If you wanted just glaze Green Version003 then you could add a WHERE clause along the lines of:-
SELECT glazename, version, ingredient, amount FROM recipes JOIN ingredients ON ingredient_ref = ingredients._id JOIN glazeversions ON glazeversion_ref = glazeversions._id JOIN glazes ON glazebase_ref = glazes._id WHERE glazeversions._id = 9
which would result in :-

You should be able to add multiple glazes that have the same identifier (say name for example), and for each one add the relevant information as well as a date. Then when you choose to view a glaze, you pull down all the info for a glaze with that identifier, and sort by the date.
Really the only extra thing you would be adding is another date, id, and name. Otherwise the entry holds all the same data you would otherwise need.

Designing date orientated database sqlite

I'm new in sqlite. I've built database but based on query I was trying to solve for it (which was over-complicated), I was suggested to look into normalising database, which I did, but can't seem to find examples on database that would be orientated around dates like a diary, with lots of daily entries. I'm working on app that would help log in everyday what I've eaten, what exercise did I do, what activities I've done, what was my well-being, how many hours I've slept. I will be able to go back to any day in the past and see what I was up to, so it will have to look up all entries for that particular date.
So I understand I need separate tables for food type, exercise type, activities types, event types and I need main table "diary" which will log each time date and id referencing another table. So I'm wondering if in that diary table I can have date column, id column and lets say type column (which will differentiate which table does id column references) or should I rather have date column and column for each of the other tables ids, even though I will be logging only one type at the time?
Also, would indexing the date column be a good idea?
Or maybe there is a better way to design that database? Any suggestions will be appreciated.

So I understand I need separate tables for food type, exercise type,
activities types, event types
If normalising then perhaps consider a single table for all types with a column to indicate the type.
So I'm wondering if in that diary table I can have date column, id
column and lets say type column (which will differentiate which table
does id column references) or should I rather have date column and
column for each of the other tables ids, even though I will be logging
only one type at the time?
If you are logging and assuming human input (as opposed to automated) then it is highly likely that a timestamp would be sufficient to uniquely identify a row.
As such there would be little need for an id column(in theory).
Saying that SQLite, unless you specify WITHOUT ROWID (which you might consider, this may be of use in deciding:-Clustered Indexes and the WITHOUT ROWID Optimization ), automatically creates a unique row identifier column i.e ROWID.
If you code a column as columnname INTEGER PRIMARY KEY or columnname INTEGER PRIMARY KEY AUTOINCREMENT then columnname becomes an alias for ROWID, in which case the unique value will be provided, if you do not provide a value when inserting.
However, if you were to specify timestamp INTEGER PRIMARY KEY and provide the current date/time as a value for the column when inserting, the current date/time would be stored and would also be indexed (it would have to be unique (current date/time would very likely be)).
So I'd suggest that a log entry need only be something like CREATE TABLE log (timestamp INTEGER PRIMARY KEY, eventref INTEGER);, where eventref is a reference to the event type.
As _id is required at times e.g. for a CursorAdapter then you could specify the columns to be extracted as *, timestemp as _id (3 columns timestamp, eventref and _id (timestamp and _id would be identical))or timstemp as _id, * (3 columns but _id, timestamp and eventref) or timestamp as _id, eventref (2 columns _id and eventref).
So using this model as the basis would minimise columns and be indexed automatically.
An example
You may have the events table as :-
Along with log table as :-
A query such as SELECT * FROM log JOIN events WHERE eventref = _id would give:-
Note! made up timestamps for illustration purposes

How do I organize this information in SQLite?

I am creating an application and using SQLite for a database, but I never used a database class.
I'm trying to figure out how to organize the following information:
A User Class: Has a name, two integer fields, and a LIST of groups,
A Group class: Has a name, a double field, and a LIST of users
A user will also eventually have to have some sort of authentication
(username/password) field but I plan to deal with that later.
If you could help me think through how to organize this information in SQLite, that would be great.

Tables can be organised as follows,
Table User_tbl
--------------------------------------------------
_ID int primary key
--------------------------------------------------
name text name of user
--------------------------------------------------
field1 int int field 1
--------------------------------------------------
field2 int int field 2
--------------------------------------------------
Table Group_tbl
--------------------------------------------------
_ID int primary key
--------------------------------------------------
name text name of group
--------------------------------------------------
field double double field
--------------------------------------------------
As per #cricket_007s answer, you need another table that links both these fields,
Table User_groups_tbl
--------------------------------------------------
_ID int primary key
--------------------------------------------------
user_id int _ID value from User_tbl
--------------------------------------------------
group_id int _ID value from Group_tbl
--------------------------------------------------
A INNER JOIN query like,
SELECT User_tbl._ID, User_tbl.name, Group_tbl._ID, Group_tbl.name, User_groups_tbl.user_id, User_groups_tbl.group_id
FROM User_tbl, Group_tbl, User_groups_tbl
INNER JOIN User_tbl.User_tbl._ID ON User_groups_tbl.User_groups_tbl.user_id
INNER JOIN Group_tbl.Group_tbl._ID ON User_groups_tbl.User_groups_tbl.group_id
WHERE User_groups_tbl.user_id = user_id;
will yield all groups in the user is in.
Likewise an INNER JOIN query like,
SELECT User_tbl._ID, User_tbl.name, Group_tbl._ID, Group_tbl.name, User_groups_tbl.user_id, User_groups_tbl.group_id
FROM User_tbl, Group_tbl, User_groups_tbl
INNER JOIN User_tbl.User_tbl._ID ON User_groups_tbl.User_groups_tbl.user_id
INNER JOIN Group_tbl.Group_tbl._ID ON User_groups_tbl.User_groups_tbl.group_id
WHERE User_groups_tbl.group_id = group_id;
will yield all users in the a group.
Explanation for User_groups_tbl
This table is used to store the many-to-many relationship between the two tables User_tbl and Group_tbl. When a user is added to a new group or vice versa, add an entry with the _ID of the corresponding group and the _ID of the corresponding user in this table.

As far as i understand, the two tables form a many-many relationship. So, a "good" database design would have 3 tables.
Table 1(User): user_ID(primary key), name, field_1, field_2
Table 2(Group): group_ID(primary key), name, double_Field
Table 3(Relationship): user_ID(foreign key), group_ID(foreign key).

If I understood the question correctly , you need to deal with the many-many relationship between user and group. With SQLite its really difficult to handle that kind of relation.
I recommend you to use android ORM for that such as Realm Android which is faster than sqlite and perfect for your need. For many-many relationship, make your models as https://realm.io/docs/java/latest/#many-to-many

You could go to Codecademy and take the SQL course. Or you could use Realm or Cupboard or ActiveAndroid (there's several libraries to ease SQLite development). And, unless you are familiar with writing SQL code, then it's best to use one of those.
Basically, you'll have a User table, Group table, then a UserGroup lookup table (2 columns) that lists the ID values of which user ids belong to which group ids.
This can be seen as a many-to-many relationship.

Dynamic Tables in Android SQLite

My question involves databases - The scenario is this:
I have an app that tracks when people borrow items. I have an existing table which tracks who they are, what they have borrowed and so on. What I would like to do is create a separate table to track what the person has borrowed, their contact info, if they returned it, etc.
My idea to do this would be to create a Dynamic table in SQL that would hold the records for 1 person, ie
John Smith
DVD; July 12, 2012; Returned in good condition; etc
As I'm still learning, I wanted to know if:
This is possible, feasible or if there is a smarter way of going about it.

Your answer depends on your scenario;
If you are only interested with "who" borrowed "what" (currently) and not "when" except last occurance, and you are assuming there are always only 1 copy of an item, then you can use one to one relation as:
CREATE TABLE Person
(
PersonId int IDENTITY(1,1) NOT NULL,
Name nvarchar(30) NOT NULL,
Surname nvarchar(30) NOT NULL,
BorrowItemId int NULL FOREIGN KEY REFERENCES Item UNIQUE,
BorrowDate datetime NULL,
ReturnDate datetime NULL,
ReturnCondition nvarchar(50) NULL,
CONSTRAINT PK_Person PRIMARY KEY CLUSTERED (PersonId ASC),
)
CREATE TABLE Item
(
ItemId int IDENTITY(1,1) NOT NULL,
ItemDescription nvarchar(50) NOT NULL,
CONSTRAINT [PK_Item] PRIMARY KEY CLUSTERED (ItemId ASC)
)
If you have multiple copies of each item you should remove the UNIQUE key on BorrowItemId changing relation to one to many. In case;
To see the items borrowed and returned with person information:
SELECT PersonId, Name, Surname, ItemDescription, ReturnDate, ReturnCondition
FROM Person INNER JOIN Item
ON BorrowItemId = ItemId
WHERE BorrowItemId IS NOT NULL
AND ReturnDate IS NOT NULL
You can add PersonId filter in WHERE clause to query for specific person
This isn't a good design since you can insert records without date information or you can even have records with date info but no related BorrowItemId. I suggest using many to many and keep historic data (can be handy) or overwrite using update each time the person borrows a new item

Their contact information could be linked into the table which tracks who they are.
If you have not created a table yet for the returns then I suggest you reference the borrowing table's ID and set in the borrowing table a flag to say this item has been returned.
I am not too sure why you would want to create a new table to collate all the information. If you want to get all the information together then I suggest using the SQL keywrod JOIN when preparing statements. If you really want to store the information later on in a table you can but it will just be duplicates in your database.
A tutorial on the different types of joins you can do : http://www.w3schools.com/sql/sql_join.asp

It is definitely possible to do as you describe. It really isn't a very good strategy, though. Your new table is, exactly, equivalent to an additional column in the existing table that tags the row as belonging to a specific individual.

Automatically Create Insert Statements to Fill Junction Table in a Pre-Created DB

For a simple android app I'm creating as a teaching tool for myself (for using relational dbs/SQL among other things - pardon the simplicity of the question if you will). I'm pre-creating a sqlite db to ship with the application. I'm doing this based on the following SO question.
I've got two tables with a many to many relationship and a junction table to define those relationships as follows:
CREATE TABLE Names (_id INTEGER PRIMARY KEY,
name TEXT
);
CREATE TABLE Categories (_id INTEGER PRIMARY KEY,
category TEXT
);
CREATE TABLE Name_Category (name_id INTEGER,
category_id INTEGER,
PRIMARY KEY (name_id, category_id),
foreign key (name_id) references Names(_id),
foreign key (category_id) references Categories(_id)
);
I've got sets of insert statements to fill the Names and Categories tables. I'm now faced with the task of filling the junction table. I'm sure that I could create the insert statements by hand by looking up the ids of the names and categories that I want to match, but that seems a bit silly.
In order to automatically create the insert statements for the junction table, I imagine that I could create a script based on a set of name and category pairs that will search for the appropriate ids and dump an insert statement. (I came up with this as I was asking the question and will research it. Don't you love it when that happens?)
Does anybody have any suggestions for ways to do this?
EDIT I added the foreign keys because, as pointed out below, they'll help maintain integrity between the tables.
EDIT #2 To solve this, I created a simple Perl script that would take a text file with name - category pairs and dump them out into another file with the appropriate SQL statements.
The name - category text file has a format as follows:
'Name' 'Category'
The Perl script looks like this:
use strict;
use warnings;
open (my $name_category_pair_file, "<", "name_category.txt") or die "Can't open name_category.txt: $!";
open (my $output_sql_file, ">", "load_name_category_junction_table.sqlite") or die "Can't open load_name_category_junction_table.sqlite: $!";
while (<$name_category_pair_file>) {
if (/('[a-zA-Z ]*') ('[a-zA-Z ]*')/) {
my $sql_statement = "INSERT INTO Name_Category VALUES (
(SELECT _id FROM Names WHERE name = $1),
(SELECT _id FROM Categories WHERE category = $2))\;\n\n";
print $output_sql_file $sql_statement;
}
}
close $name_category_pair_file or die "$name_category_pair_file: $!";
close $output_sql_file or die "$output_sql_file: $!";

You can use this insert in your script or code (replacing the strings or using ?):
insert into Name_Category values(
(select _id from Categories where category='CAT1'),
(select _id from Names where name='NAME1'));
Also, you can alter the Name_Category table to constraint on the values that can be inserted and/or deleted:
CREATE TABLE Name_Category ( name_id INTEGER NOT NULL,
category_id INTEGER NOT NULL,
PRIMARY KEY (name_id, category_id),
foreign key (name_id) references Names(_id),
foreign key (category_id) references Categories(_id));

create two main tables first and then create a junction table in which primary key of both main tables will be available as foreign key.. Primary key of junction table will be union
of primary key of first and second main table.
Create trigger now to automatically insert into junction table...
Also don't forget to create table with cascade deletion and cascade updatation so that any value updated or deleted in main tables will be automatically reflected in junction table