Develop Reference

how to store Android database with variable number of attributes per row

For my Android app, I want to save data using sqlite with this format:
name, date, attr1, attr2, attr3,...
These are the requirements:
each date can only contain each name once
there can be a variable number of attributes(numbers) for each name
each specific name has the same number of attributes
The app will be used to track events throughout the day. Events can have zero or more numeric properties.
The questions are: is sqlite the best way to store things here? If so how do I design my database? What other ways are there to store this kind of data?

is sqlite the best way to store things here?
This will depend on a number of other factors, such as how the data will be queried and used, the volume of transactions, data growth and retention, etc. From what you've described, though, SQLite is a great option, offering functionality out-of-the-box that supports some of your requirements directly, and is commonly used in such cases.
If you don't have much experience with relational databases, implementing this functionality may seem difficult at first, but like learning a new language or framework, it will get easier with time.
If so how do I design my database?
Let's step through each of your enumerated requirements...
each date can only contain each name once
SQLite supports the UNIQUE constraint. For example, if your columns were named name and date, you could add the following to your CREATE TABLE statement:
UNIQUE(name, date)
(A more complete CREATE TABLE statement is in the next example below, and it includes this constraint.)
This constraint prevents the insertion of rows with name/date pairs that already exist. Using android.database.sqlite.SQLiteDatabase, if you attempt to insert a row into the table with a duplicate name/date pair, a SQLiteConstraintException will be thrown at runtime. You will need to handle this exception in your java code.
there can be a variable number of attributes(numbers) for each name
This is a textbook case for normalizing the database, putting your data into multiple tables. For example:
CREATE TABLE names (
name_id INTEGER PRIMARY KEY AUTOINCREMENT NOT NULL,
name TEXT NOT NULL,
date DATETIME,
UNIQUE(name, date));
CREATE TABLE attrs (
name_id INTEGER NOT NULL,
attr_value INTEGER NOT NULL,
FOREIGN KEY(attr_value) REFERENCES names(name_id));
Your queries that retrieve attribute data would then JOIN the two tables. Since you indicated that "Events can have zero or more numeric properties", a LEFT OUTER JOIN might be most appropriate, as it will return names and dates even if there are no attributes.
Here's an example query, by name:
SELECT n.name, n.date, a.attr_value
FROM names AS n
LEFT OUTER JOIN attrs AS a
ON n.name_id = a.name_id
WHERE n.name = 'SMITH'
ORDER BY n.name, n.date, a.attr_value;
This query would return results like the following:
name date attr_value
--------------- ---------- ------------
SMITH 2015-02-13 1027
SMITH 2015-02-13 4426
SMITH 2015-02-13 8390
SMITH 2015-02-20 4426
SMITH 2015-02-20 8152
SMITH 2015-02-20 9328
You can then iterate through and process these results in java. If your results include multiple names and/or dates, then in your loop you should keep track of the last used name and date. If the name/date in the current record is the same, the attribute belongs to the current one. If the name/date is different, then this is a new one.
Note that this approach to your database design is flexible, allowing you to query on the attributes, for instance, to see what name/date pairs are associated.
Also note that there is a FOREIGN KEY constraint on the attrs table, meaning that if you attempt to insert a record into that table with a name_id that does not exist in the names table, a SQLiteConstraintException will be thrown at runtime. You will need to handle this exception in your java code.
each specific name has the same number of attributes
You will need to accommodate this requirement in your java code, probably doing some checks in the database prior to performing an INSERT.
What other ways are there to store this kind of data?
Flat files, JSON, XML, third-party data stores (with their own libraries), to name a few.

I'm not sure but I think the best way to achieve your requirement is to use sqlite and to solve your problem you can have 3 columns only. One for the name and one for the date and the other contains a JSON array that represents the rest of the attributes.

SQL Database with two languages

I am not an expert in the SQL and database in general.
I am creating an android application with the feature of multi-languages for both the interfaces and the content itself.
my question is for the content language: is there any way to retrieve data from the database which is containing only one language (English),like translating the content to the other language option ??
because I think it is not efficient to store both languages in the database.
thanks in advance,

Text belonging to the client app is usually stored with that app, so you don't have to load "OK" and "Cancel" in the appropriate language from database only to show some buttons. (But you could store this in a database of course, if you wanted to.)
Content data on the other hand is usually stored in the database. It doesn't make much sense, to store part of it in the database and part of it somewhere else. If you store products in your database, then store the text necessary with it. It belongs there. As to accessing data, this is not slow, because you don't retrieve text in all languages, but only in the language desired.
Here is a typical table design:
create table product
(
id number(10),
buying_price number(6,2),
selling_price number(6,2),
id_vat number(1),
...
);
create table product_name
(
id_product number(10),
language_iso varchar(2), -- 'en', 'de', 'fr', ... or chose another code
text varchar(100)
);
create table product_description
(
id_product number(10),
language_iso varchar(2),
text varchar(2000)
);
You could also decide for a more generic design:
create table products
(
id number(10),
tranlation_number number(10),
buying_price number(6,2),
selling_price number(6,2),
id_vat number(1),
...
);
create table translations
(
tranlation_number number(10),
content_name varchar2(30), -- 'PRODUCT_NAME', 'PRODUCT_DESCRIPTION', ...
language_iso varchar(2), -- 'en', 'de', 'fr', ... or chose another code
text varchar(4000)
);
There are more possibilities. Whichever you chose, accessing data simply means a join and a constant for the language. There should be no problem with performance at all. And data is where it belongs.

What is the advantage of FTS over custom solution?

I have a biggish database ~32mb which has lots of text in 4 languages. Including Arabic and Urdu. I need to search this text in the most efficient way (speed & size).
I am considering FTS, and trying to find out how to implement it. Right now I am reading http://www.sqlite.org/fts3.html#section_1_2 about it.
It seems to me, an FTS table is just like a normal table used to index all the different words. So my questions are:
1) If to populate FTS I have to do all the inserts myself, then why not make my own indexed word table, what is the difference?
Answer : Yes there are many advantages, many built in functions that help. For example with ranking etc, searching of stems and the transparent nature of how it all works in android makes the FTS approach more appealing.
2) On the google docs I read its a virtual in memory table, now this would be massive right... but it doesnt mention this on the SQLite website. So which is it?
3) Is there an easy way to generate all the different words from my columns?
4) Will the FTS handle arabic words properly?

FTS allows for fast searching of words; normal indexes only allow to search for entire values or for the beginning of the value.
If you table has only one word in each field, using FTS does not make sense.
FTS is a virtual table, but not an in-memory table.
You can get individual terms from the full-text index with the fts4aux table.
The default tokenizer works only with ASCII text.
You have to test whether the ICU or UNICODE61 tokenizers work with your data.

1) If to populate FTS I have to do all the inserts myself, then why
not make my own indexed word table, what is the difference?
Using your own indexed word table, you would have parse words in sentences. You would then need a table for sentences and another to words. And you should do this efficiently.
2) On the google docs I read its a virtual in memory table, now this
would be massive right... but it doesnt mention this on the SQLite
website. So which is it?
Don't understand your question. Data is handled via virtual table extension, however back storage is done in database (FTS4 creates 5 tables for each virtual table). Check this:
sqlite> CREATE VIRTUAL TABLE docs USING fts4();
sqlite> .schema
CREATE VIRTUAL TABLE docs USING fts4();
CREATE TABLE 'docs_content'(docid INTEGER PRIMARY KEY, 'content');
CREATE TABLE 'docs_segments'(blockid INTEGER PRIMARY KEY, block BLOB);
CREATE TABLE 'docs_segdir'(level INTEGER,idx INTEGER,start_block INTEGER,leaves_
end_block INTEGER,end_block INTEGER,root BLOB,PRIMARY KEY(level, idx));
CREATE TABLE 'docs_docsize'(docid INTEGER PRIMARY KEY, size BLOB);
CREATE TABLE 'docs_stat'(id INTEGER PRIMARY KEY, value BLOB);
sqlite>
3) Is there an easy way to generate all the different words from my
columns?
For sure. But that's not easy. That's what FTS does.
4) Will the FTS handle arabic words properly?
I'm not sure. Does arabic languages uses ICU word boundaries? From Tokenizer:
The ICU tokenizer implementation is very simple. It splits the input
text according to the ICU rules for finding word boundaries and
discards any tokens that consist entirely of white-space. This may be
suitable for some applications in some locales, but not all. If more
complex processing is required, for example to implement stemming or
discard punctuation, this can be done by creating a tokenizer
implementation that uses the ICU tokenizer as part of its
implementation.

Multi-language database

I am developing an Android application that supports Arabic and English, but I am still confused about the database design:
The approach I am following is
creating a language table that holds two records (Arabic and English)
creating a table that holds normal data (not language specific)
creating a translation table that holds language specific content
For example to implement (Help) I created the following tables:
Language (int id, text Code)
Help (int id)
Help_translation(int id, int Help_id, int Language_id, text Question, text Answer)
I have read this solution so many time, but I still don't know why do we have a separate table for (Help) that does nothing!
Is there something wrong about my approach?

On the face of it, the Language table will appear once in the system. I suggest using the 'll_tt' notation for language and territory based on the ISO country codes (ISO 3166) and ISO language codes (ISO 639-2). Thus, you might have en_gb for British English, en_us for US English, and ar_sa for Saudi Arabian Arabic, and ar_eg for Egyptian Arabic. This may be more powerful than you currently need, but gives you a direction for the future.
Thus, the Language table might have columns:
id integer (autoincrement) Primary key.
code char(5) — Unique.
The Help table in your design is for 'documentation' purposes. It records the valid help numbers that the application can use. A fuller version of the table might include information about the message: where it is used in the application; which version it was introduced in; which version it became obsolete with (or maybe not — we have internationalized message files that have to be good for use with releases over a period of about 10 years); dates associated with the version — or dates instead of the version; and 'notes to the translator' (guidelines for how to translate the message, if any special guidelines are needed).
The minimalistic version of the Help table with just the Help ID number will do for the time being:
id integer (autoincrement). Primary key.
The Help_Translation table stores the strings that the application will display. The id column is of minimal value here; I would omit it (but you can keep it if you wish). The Help_ID column is a foreign key reference to the Help table; the Language_ID column is a foreign key reference to the Language table. For your chosen application, there appears to be both a question and an answer for each help item.
Therefore, the Help_Translation table has the columns:
id integer (autoincrement, optional, not sure when you'll use it).
Help_ID integer — foreign key references Help(ID).
Language_ID integer — foreign key references Language(ID).
Question text — the question in the appropriate language for the given Help_ID.
Answer text — the answer to the question in the appropriate language for the given Help_ID.
Primary key: Help_ID and Language_ID.
An alternative design would have simply a translation table with a 'Message ID' and a 'Language ID' and the translated string (primary key on Message ID and Language ID). There would be a table of Messages, identifying the valid Message IDs plus supporting data of the type outline previously. This might include the default message to be used (untranslated) when there is no translated version of the message for a particular language/territory (or you can make up more complex schemes for handling missing messages, so that when Arabic is requested but the translation for, say, Libya (LY) is incomplete, it falls back onto ar_sa and only then onto en_gb). Your Help table might then contain a Help ID, plus two Message ID values, one for the Question and one for the Answer. The advantage of this scheme is that all translated messages are in a single table.
There are undoubtedly other schemes that could be devised.

Best way to search sqlite database

In my application ,am work with a large database.Nearly 75000 records present in a table(totally 6 tables are there).i want to get a data from three different table at a time.i completed that.but the search process was slow.how can i optimise the searching process?

You might want to consider using the full-text search engine and issuing SELECT...MATCH queries instead. Note that you need to enable the FTS engine (it's disabled by default) and create virtual tables instead of regular tables. You can read more about it here.

Without being able to see the table structure (or query) the first thing I'd suggest is adding some indexes to the tables.
Lets say you have a few tables like:
Author
id
last_name
first_name
Subject
id
name
Book
id
title
author_id
subject_id
and you're wanting to get all the information about each of the books that an author with last_name="Smith" and first_name="John" wrote. Your query might look something like this:
SELECT * FROM Book b
LEFT JOIN Subject s
ON s.id=b.subject_id
LEFT JOIN Author a
ON a.id=b.author_id
WHERE a.last_name='Smith'
AND a.first_name='John';
There you'd want the last_name column in the Author table to have an index (and maybe first_name too).