I have 4 tables:
tasks table
(task_id , department_id , task_title , task_description , task_start_date , task_due_date , task_rating , task_is_completed)
employees table
(employee_id , department_id , employee_name , employee_salary , employee_hire_date)
departments table
(department_id , department_name)
employees_tasks join table
(employee_id , task_id)
Each table is an entity in room database.
I want to return 2 extra columns with the (select * from employees)
one is for calculating employee's rating (by getting average of task_rating column in tasks table the tasks must be completed) the other column is to show the number of tasks running for that employee (by getting count of rows in tasks with task_is_completed = 0 )
I don't know which table to join with which table. we managed to make two separate SQL statements that return those 2 columns by using union and left joins but they are pretty ugly and when combining them it doesn't work.
what we have tried
select employees.employee_name , employees.employee_id ,avg(tasks.task_rating) as Ratings from employees , tasks inner join employees_tasks on(employees.employee_id = employees_tasks.employee_id )AND tasks.task_id = employees_tasks.task_id where tasks.task_is_completed = 1 group by (employees.employee_name )
union select employees.employee_name, employees.employee_id, avg(0) as Ratings from employees where employees.employee_id not in (select employees.employee_id from employees , tasks inner join employees_tasks on(employees.employee_id = employees_tasks.employee_id ) AND tasks.task_id = employees_tasks.task_id where tasks.task_is_completed = 1 group by (employees.employee_name ) ) group by employees.employee_id order by employees.employee_id ;
select employees.employee_name , employees.employee_id ,count(tasks.task_title) as tasks_Running from employees , tasks inner join employees_tasks on(employees.employee_id = employees_tasks.employee_id )AND tasks.task_id = employees_tasks.task_id where tasks.task_is_completed = 0 group by (employees.employee_name )
union select employees.employee_name , employees.employee_id ,0 as tasks_Running from employees where (employees.employee_id not in (select employees.employee_id from employees , tasks inner join employees_tasks on(employees.employee_id = employees_tasks.employee_id )AND tasks.task_id = employees_tasks.task_id where tasks.task_is_completed = 0 group by (employees.employee_name )))group by (employees.employee_name) order by employees.employee_id ;
We want the output to be like this
(employee_id , department_id , employee_name , employee_salary , employee_hire_date , ratings , numTasksRunning)
I believe the following may suit :-
WITH
-- Common Table Expression 1 - Average of Completed Tasks per employee
employee_completedtask_info AS (
SELECT employees.employee_id,avg(tasks.task_rating) AS atr
FROM employees_tasks
JOIN tasks ON employees_tasks.task_id = tasks.task_id
JOIN employees ON employees_tasks.employee_id = employees.employee_id
WHERE tasks.task_is_completed > 0
GROUP BY employees.employee_id
),
-- Common Table Expression 2 - Incompleted Taks per employee
employee_notcompleted_info AS (
SELECT employees.employee_id,count() AS itc
FROM employees_tasks
JOIN tasks ON employees_tasks.task_id = tasks.task_id
JOIN employees ON employees_tasks.employee_id = employees.employee_id
WHERE tasks.task_is_completed = 0
GROUP BY employees.employee_id
),
-- Common Table Expression 3 - Total Tasks per Employee
employee_total_tasks AS (
SELECT employees.employee_id,count() AS ttc
FROM employees_tasks
JOIN tasks ON employees_tasks.task_id = tasks.task_id
JOIN employees ON employees_tasks.employee_id = employees.employee_id
GROUP BY employees.employee_id
)
SELECT employees.employee_name,
CASE WHEN atr IS NOT NULL THEN atr ELSE 0 END AS average_completed_task_rating,
CASE WHEN itc IS NOT NULL THEN itc ELSE 0 END AS incomplete_task_count,
CASE WHEN ttc IS NOT NULL THEN ttc ELSE 0 END AS total_task_count
FROM employees
LEFT JOIN employee_completedtask_info ON employees.employee_id = employee_completedtask_info.employee_id
LEFT JOIN employee_notcompleted_info ON employees.employee_id = employee_notcompleted_info.employee_id
LEFT JOIN employee_total_tasks ON employees.employee_id = employee_total_tasks.employee_id
;
based upon data generated as per the following :-
DROP TABLE IF EXISTS employees;
CREATE TABLE IF NOT EXISTS employees (employee_id INTEGER PRIMARY KEY, department_id INTEGER, employee_name TEXT, employee_salary REAL, employee_hire_date TEXT);
DROP TABLE IF EXISTS departments;
CREATE TABLE IF NOT EXISTS departments (department_id INTEGER PRIMARY KEY, department_name TEXT);
DROP TABLE IF EXISTS employees_tasks;
CREATE TABLE IF NOT EXISTS employees_tasks (employee_id INTEGER, task_id INTEGER, PRIMARY KEY(employee_id, task_id));
INSERT INTO departments VALUES
(null,'Maths'),(null,'English'),(null,'Craft')
;
INSERT INTO employees VALUES
(null,1,'Fred',55000,'2000-01-02'),
(null,2,'Mary',62000,'1996-03-20'),
(null,3,'Tom',52000,'2004-10-11'),
(null,3,'Susan',72000,'1999-06-14'),
(null,2,'Bert',66000,'2000-10-15'),
(null,1,'Jane',70000,'1992-04-02')
;
INSERT INTO tasks VALUES
(null,3,'Task 001 - Craft','Do the Craft thinggy','2018-01-01','2018-08-19',10,0),
(null,1,'Task 002 - Maths','Do the Maths thinggy','2018-03-14','2019-03-13',20,0),
(null,2,'Task 003 - English','Do the English thinggy','2018-02-14','2018-09-14',8,0),
(null,3,'Task 004 - Craft','Do the Craft job','2018-01-01','2018-08-19',10,1),
(null,1,'Task 005 - Maths','Do the Maths job','2018-03-14','2019-03-13',20,1),
(null,2,'Task 006 - English','Do the English job','2018-02-14','2018-09-14',8,1),
(null,3,'Task 007 - Craft','Craft thinggy','2018-03-03','2018-11-21',10,0),
(null,1,'Task 008 - Maths','Maths thinggy','2018-03-14','2019-03-13',20,0),
(null,2,'Task 009 - English','English thinggy','2018-02-14','2018-09-14',8,0)
;
INSERT INTO employees_tasks VALUES
(1,2),(1,5),(1,8),(1,6),
(2,2),
(3,1),(3,4),(3,7)
;
This results in :-
Note This converts null entries to 0's (i.e. in the above there are no tasks for Susan, Bert and Jane so nulls for their task counts/averages, which complicates matters a little hence the CASE WHEN ... THEN ... ELSE .... END AS clauses).
Note I've included the total tasks counts as this may be useful/wanted (the third CTE extracts this info)
Performance wise, which one is better (for a single table) ?:
Multiple select in single query or
Single select in multiple queries?
Example:
/*table schema*/
create table ACTIVITY_TABLE (
ID integer primary key autoincrement,
INTERVAL_ID integer not null,
ACTIVITY_TYPE text ,
WIFI_LIST text,
COUNT integer not null );
/*Multipe selects in single query*/
select * from
(select sum(COUNT)/60 as Physical from ACTIVITY_TABLE where INTERVAL_ID >=1 and INTERVAL_ID <=3 and (ACTIVITY_TYPE="Still" or ACTIVITY_TYPE = "Walking" or ACTIVITY_TYPE = "Running" ) ),
(select sum(COUNT)/60 as vehicular from ACTIVITY_TABLE where INTERVAL_ID >=1 and INTERVAL_ID <=3 and (ACTIVITY_TYPE="InVehicle" ) );
For your example, you can modify the query in the following way which will be much more efficient, I didn't tested the query.
select
sum(case when activitytype='walking' or activitytype='still' or activitytype='running' then COUNT else 0 end )/60 as physical,
sum(case when activitytype='InVehicle' then COUNT else 0 end)/60 as vehicular
from
activity_table
where interval_id>=1 and interval_id<=3
My database has 6384 records and I am using the below query:
SELECT T.t_name, S.s_code, S.s_name, R.s_code, R.s_name, M.arrival_time, L.arrival_time, M.dest_time, M.train_id, S.id, R.id
FROM TRAIN_SCHEDULE M,
TRAIN_SCHEDULE L,
TRAIN T,
STATION S,
STATION R
WHERE S.s_name = 'Versova'
AND R.s_name = 'Ghatkopar'
AND M.arrival_time > '00:00:00'
AND M.arrival_time < L.arrival_time
AND M.train_id = L.train_id
AND M.dest_time = L.dest_time
AND T.id = M.train_id
AND S.id = M.station_id
AND R.id = L.station_id
This query takes 8 second to fetch the data.
I have also indexed my tables, but fetching time is reduced to only 2 seconds.
Schema:
CREATE TABLE [STATION] (
[id] INTEGER NOT NULL PRIMARY KEY AUTOINCREMENT,
[s_code] VARCHAR(10) NOT NULL,
[s_name] VARCHAR(50) NOT NULL);
CREATE TABLE TRAIN_SCHEDULE(
id INT,
station_id INT,
train_id INT,
arrival_time NUM,
departure_time NUM,
dest_time NUM
);
CREATE TABLE TRAIN(id INT,t_name TEXT);
CREATE INDEX idx_arrival_time ON train_schedule (arrival_time);
CREATE INDEX idx_dest_time ON train_schedule (dest_time);
CREATE INDEX idx_id ON train (id);
How can I improve this?
You can check with EXPLAIN QUERY PLAN which indexes are being used.
In this query, the database needs to scan through the STATION table; an index on the name column would improve this (although not by much with such a small table):
CREATE INDEX Station_Name ON STATION(s_name);
Also, lookups on the TRAIN_SCHEDULE table are done over multiple columns.
The query optimizer cannot use more than one index per table instance, so you should create a multi-column index.
And a column with a non-equality comparison must come last (see the documentation):
CREATE INDEX Schedule_Station_Train_DestTime_ArrivalTime
ON TRAIN_SCHEDULE(station_id, train_id, dest_time, arrival_time);
Also execute ANALYZE once to help the optimizer pick the right index.
I am working with GTFS data on Android (SQlite). And I would like to improve performance when I do select queries in my database filled with GTFS data.
The query below select the stop times associated to a route at a stop:
The first sub query gets the daily stop times on thursday.
The second gets all the exception stop times which are not valid for TODAY (2013-07-25).
The third one gets all the exception stop time which are only valid for TODAY (2013-07-25).
Then I remove the non-valid one and add the valid one to the first sub query.
select distinct stop_times_arrival_time
from stop_times, trips, calendar
where stop_times_trip_id=trip_id
and calendar_service_id=trip_service_id
and trip_route_id='11821949021891616'
and stop_times_stop_id='3377699721872252'
and calendar_start_date<='20130725'
and calendar_end_date>='20130725'
and calendar_thursday=1
and stop_times_arrival_time>='07:40'
except
select stop_times_arrival_time
from stop_times, trips, calendar, calendar_dates
where stop_times_trip_id=trip_id
and calendar_service_id=trip_service_id
and calendar_dates_service_id = trip_service_id
and trip_route_id='11821949021891694'
and stop_times_stop_id='3377699720880977'
and calendar_thursday=1
and calendar_dates_exception_type=2
and stop_times_arrival_time > '07:40'
and calendar_dates_date = 20130725
union
select stop_times_arrival_time
from stop_times, trips, calendar, calendar_dates
where stop_times_trip_id=trip_id
and calendar_service_id=trip_service_id
and calendar_dates_service_id = trip_service_id
and trip_route_id='11821949021891694'
and stop_times_stop_id='3377699720880977'
and calendar_thursday=1
and calendar_dates_exception_type=1
and stop_times_arrival_time > '07:40'
and calendar_dates_date = 20130725;
It took about 15 seconds to compute (which is very long).
I am sure there is a better to do this query since I do 3 different queries (almost the same by the way) which take time.
Any idea how to improve it?
EDIT:
Here is the schema:
table|calendar|calendar|2|CREATE TABLE calendar (
calendar_service_id TEXT PRIMARY KEY,
calendar_monday INTEGER,
calendar_tuesday INTEGER,
calendar_wednesday INTEGER,
calendar_thursday INTEGER,
calendar_friday INTEGER,
calendar_saturday INTEGER,
calendar_sunday INTEGER,
calendar_start_date TEXT,
calendar_end_date TEXT
)
index|sqlite_autoindex_calendar_1|calendar|3|
table|calendar_dates|calendar_dates|4|CREATE TABLE calendar_dates (
calendar_dates_service_id TEXT,
calendar_dates_date TEXT,
calendar_dates_exception_type INTEGER
)
table|routes|routes|8|CREATE TABLE routes (
route_id TEXT PRIMARY KEY,
route_short_name TEXT,
route_long_name TEXT,
route_type INTEGER,
route_color TEXT
)
index|sqlite_autoindex_routes_1|routes|9|
table|stop_times|stop_times|12|CREATE TABLE stop_times (
stop_times_trip_id TEXT,
stop_times_stop_id TEXT,
stop_times_stop_sequence INTEGER,
stop_times_arrival_time TEXT,
stop_times_pickup_type INTEGER
)
table|stops|stops|13|CREATE TABLE stops (
stop_id TEXT PRIMARY KEY,
stop_name TEXT,
stop_lat REAL,
stop_lon REAL
)
index|sqlite_autoindex_stops_1|stops|14|
table|trips|trips|15|CREATE TABLE trips (
trip_id TEXT PRIMARY KEY,
trip_service_id TEXT,
trip_route_id TEXT,
trip_headsign TEXT,
trip_direction_id INTEGER,
trip_shape_id TEXT
)
index|sqlite_autoindex_trips_1|trips|16|
And here is the query plan:
2|0|0|SCAN TABLE stop_times (~33333 rows)
2|1|1|SEARCH TABLE trips USING INDEX sqlite_autoindex_trips_1 (trip_id=?) (~1 rows)
2|2|2|SEARCH TABLE calendar USING INDEX sqlite_autoindex_calendar_1 (calendar_service_id=?) (~1 rows)
3|0|3|SCAN TABLE calendar_dates (~10000 rows)
3|1|2|SEARCH TABLE calendar USING INDEX sqlite_autoindex_calendar_1 (calendar_service_id=?) (~1 rows)
3|2|0|SEARCH TABLE stop_times USING AUTOMATIC COVERING INDEX (stop_times_stop_id=?) (~7 rows)
3|3|1|SEARCH TABLE trips USING INDEX sqlite_autoindex_trips_1 (trip_id=?) (~1 rows)
1|0|0|COMPOUND SUBQUERIES 2 AND 3 USING TEMP B-TREE (EXCEPT)
4|0|3|SCAN TABLE calendar_dates (~10000 rows)
4|1|2|SEARCH TABLE calendar USING INDEX sqlite_autoindex_calendar_1 (calendar_service_id=?) (~1 rows)
4|2|0|SEARCH TABLE stop_times USING AUTOMATIC COVERING INDEX (stop_times_stop_id=?) (~7 rows)
4|3|1|SEARCH TABLE trips USING INDEX sqlite_autoindex_trips_1 (trip_id=?) (~1 rows)
0|0|0|COMPOUND SUBQUERIES 1 AND 4 USING TEMP B-TREE (UNION)
Columns that are used for lookups should be indexed, but for a single (sub)query, it is not possible to use more than one index per table.
For this particular query, the following additional indexes would help:
CREATE INDEX some_index ON stop_times(
stop_times_stop_id,
stop_times_arrival_time);
CREATE INDEX some_other_index ON calendar_dates(
calendar_dates_service_id,
calendar_dates_exception_type,
calendar_dates_date);
I have a query with a subquery that returns multiple rows.
I have a table with lists and a table with users. I created a many-to-many table between these two tables, called list_user.
LIST
id INTEGER
list_name TEXT
list_description TEXT
USER
id INTEGER
user_name TEXT
LIST_USER
id INTEGER
list_id INTEGER
user_id INTEGER
My query with subquery
SELECT * FROM user WHERE id = (SELECT user_id FROM list_user WHERE list_id = 0);
The subquery works (and I use it in code so the 0 is actually a variable) and it returns multiple rows. But the upper query only returns one row, which is pretty logical; I check if the id equals something and it only checks against the first row of the subquery.
How do I change my statement so I get multiple rows in the upper query?
I'm surprised the = works in SQLite. It would return an error in most databases. In any case, you want the in statement:
SELECT *
FROM list
WHERE id in (SELECT user_id FROM list_user WHERE list_id = 0);
For a better performance, use this query:
SELECT LIST.ID,
LIST.LIST_NAME,
LIST.LIST_DESCRIPTION
FROM LIST,
USER,
LIST_USER
WHERE LIST.ID = LIST_USER.USER_ID = USER.ID AND
LIST.LIST_ID = 0