So, this is a common problem in apps that track your location over a journey (a run or cycle workout, for example).
Clearly GPS navigators have less trouble, since they can assume you snap to a point on a road - however, if you're running in the park, snapping to some road grid is going to give you totally crazy numbers.
The problem as far as I see it is to combine the great-circle distances between the waypoints, but taking into account the errors (accuracy values) such that you don't veer off course too far for a low-accuracy point. The rough implementation in my head involves plotting some bezier curve (using the velocity/bearing at the point to add spline direction and weight) and integrating over it.
However, clearly this is something people have sovled before. Anyone know of the implementations, or are they all buried in proprietary software?
Bonus points for anyone who can also use the (mostly) less accurate cell tower points (which come with different/out-of-sync timestamps, and no velocity or bearing information).
The eventual implementation will be in javascript or python, whichever is faster (I'm using SL4A,) but I'm looking for general algorithms here.
To get everyone started, here is the naive algorithm, not using any velocity or bearing info.
The arc length s is calculable from the two (long, lat) pairs (the start and end waypoints) of the segment we'll start with, by the standard formula.
Assuming we've converted the value pairs into standard spherical coordinates phi and theta (here as arrays, so using phi[0] and phi[1] for locations 0 and 1) in radians, the arc length is just:
from math import sin, cos, arccos, sqrt
s = arccos(
sin(phi[0]) * sin(phi[1]) * cos(theta[0] - theta[1]) +
cos(phi[0]) * cos(phi[1])
)
However, since we've got a massive horrid function, we need to use the chain rule to work out the first order errors, and we get the following monster for delta_s:
delta_s = (1.0 / abs(sin(s))) * (
delta_phi[0] * abs(
sin(phi[0]) * cos(phi[1]) -
cos(phi[0]) * sin(phi[1]) * cos(theta[0] - theta[1])
) +
delta_phi[1] * abs(
sin(phi[1]) * cos(phi[0]) -
cos(phi[1]) * sin(phi[0]) * cos(theta[0] - theta[1])
) +
(delta_theta[0] + delta_theta[1]) * abs(
sin(phi[0]) * sin(phi[1]) * sin(theta[0] - theta[1])
)
)
We perform this operation on every pair of successive points in order, sum the ss, add the errors in quadrature as normal:
accumulator = 0.0
for error in errors:
accumulator += error * error
journey_error = sqrt(accumulator)
and thus, we know the uncertainty on our rubbish distance estimate. (we can even keep the accumulator around to speed up the calculation if we add a few points on the end - as we could in practise with real time data.)
However, this is going to give us huge errors, and only a very fuzzy idea of how far we've actually gone. This can't be how actual GPS equipment estimates distances, as it would never be accurate enough unless it had amazing signal all the time:
What we need is some more nuanced path approximation, which only bends the path out of position for the type of inaccurate points shown, rather than diverting it completely and massively increasing the distance estimate — in asking the question I was hoping to find out how all the existing implementations (probably) do it!
Related
For my app, as I get wrong results caused by rounding errors (for instance, the distance calculated between two points separated by 1853m at the surface of the earth is 0!), I need to compute sine and cosine with a double precision. Unfortunately, I can't find any suitable functions in the NDK.
How could I get more accurate results?
The formula I use is 60*1852*toDegrees(acos(sin(toRadians(lat1))*sin(toRadians(lat2))+cos(toRadians(lat1))*cos(toRadians(lat2))*cos(toRadians(long2-long1))))
with lat1=45.7729721; lat2=45.792984; long1=2.96383333; long2=2.96895313
Tackling this problem by trying to get higher precision seems like a bad idea to me. Your problem is horribly-conditioned and not very tractable with floating point numbers.
Let's take a look at the important expression (omitting the conversions):
acos(sin(lat1)*sin(lat2) + cos(lat1)*cos(lat2)*cos(long2-long1))
For your input data, lat1 and lat2 are extremely close to one another. Further, the difference between long2 and long1 is very small, so its cosine will be extremely close to one.
Given this, the expression inside the acos is almost exactly this identity:
sin(x)*sin(x) + cos(x)*cos(x) = 1
Hoping to take the acos from this expression will yield exactly the result you observed, since floating point numbers won't deal with this well (you have a lot of precision around zero but not around other integers).
You should look to use a different formula altogether. I am a bit rusty on my clever trigonometric identities, but if you have the geometry at hand from which you derived this, it shouldn't be hard to obtain a different definition that involves less catastrophic loss of precision near 1.
Ok, armed with https://en.wikipedia.org/wiki/List_of_trigonometric_identities, Consider this:
sin(lat1)*sin(lat2) = 0.5 (cos(lat1 - lat2) - cos(lat1 + lat2)) =: a
cos(lat1)*cos(lat2) = 0.5 (cos(lat1 - lat2) + cos(lat1 + lat2)) =: b
cos(long2-long1) =: c
The expression then becomes acos(a + b * c), or
0.5 (1 + c) cos(lat1 - lat2) - 0.5 (1 - c) cos(lat1 + lat2)
The terms that have dangerous cancellation here are (1-c) and cos(lat1 - lat2) (at least for your given inputs). For the former, you could try looking for polynomial approximations that calculate 1 - cos(x) accurately for x close to 0. Eliminating loss of precision in the latter is more difficult (but also not as crucial for your given inputs).
I working on an app that I need to calculate distance travelled from point A to point B (by car).
I asked Elm Electronics (chipset manufacturer) and they said there is no standard OBD-II PID to return mileage from odometer, although car manufacturers might provide a PID. Since this way is not standard then I found another way.
PID 0131 (Distance traveled since codes cleared), is returning something that I think might be helpful. IF i'm able to clear it at point A and read its value at point B then I'm done :)
I thought a lot to guess what does "codes cleared" mean but I couldn't realize what does it mean? and my another important question, how to clear code or reset this PID?
Any suggestion would be appreciated. Thanks.
Update
I just tested on two Cars.
On Benz car no OBD-II command works. I couldn't get any data :(
I got correct reply on Persona car (Local Malaysia) but 0x0131 PID was always returned 7F01 which is 16608KM even after passing few Kms. I tried to reset it by sending 04 command (as Eric suggested on his answer), However, nothing got clear :) and I still got 7F01 as response.
My Library can be used for anyone who is looking for OBD-II lib from here.
So, What I plan to do is, since I'm able to get speed (v) then I'm able to calculate distance based on d = v * t formula.
Elm Electronics are right. The resetting trouble codes solution is a possible, but maybe unwanted workaround though.
Mode 04 is for resetting the codes. Sending 04 will reset the MIL (Malfunction Indicator Light) and reset the codes you want to reset.
In the comments, Chris suggested to use the value, and than keep track of this value yourself. That way you don't need to misuse the Mode 04.
Th 0131 value overflows at 65535 km. But when you bring you car in for maintenance, they could reset this value, depending on who is maintaining your car ofcourse.
Source: Mode 04 - Wikipedia
There are two PIds: 0x0121 Distance travelled with malfunction indicator lamp (MIL) on which keeps the distance with MIL on and 0x0131 Distance travelled since codes cleared which keeps the distance after clearing the MIL by using mode 0x04. By saying code, it meant the Diagnostics Trouble Code (DTC). When one of them keeps counting the distance the other one is fixed and activation is done for them only based on MIL on or off.
For having the milage, afaik, you need to have current mileage from the odometer as the reference, in addition to those two PIDs. For example, if the current mileage on the odometer* is X and the first time readings for those two PIDs are Y and Z respectively, and x and y are real-time readings from those two PIDs, these two formulas can give you the mileage and trip distance:
Real-Time mileage** = X + (y - Y) + (z - Z)
Trip distance (MIL Off) = x(end) - x(start)
Trip distance (MIL On) = y(end) - y(start)
*The odometer is supposed to be available by PID 0x01A6 Odometer, but in almost all the vehicles, it's not supported.
**The overflow in readings from those two PIDs should be considered as well.
I think You can use the PID 0x011F (Run time since engine start) and PID 0x010D (Vehicle speed). If you save these values in an sd card then you can multiply these two values.
I have database with about 12 000 entries. Each entry has given latitude, longitude and empty distance. What I need to do is to find 25 nearest entries from current GPS position. My ORM is greenDao.
Have 2 problems:
I don't know distance between me and entries yet and I can't load all entries to RAM because when I do, heap goes up to 70MB and app crashes at OutOfMemoryException (so I need to use lazy loading).
I tried this approach:
Get iterator for given table
load entry, calculate its distance from my current position, save entry to ArrayList buffer (I flush buffer every 1000 entries back into db (it is just updateInTx(...)) and then clean it)
repeat point 2 until iterator.hasNext();
query from entries with limit(25).orderAsc()
result
This works, but from point 1-3 it is very very slow (takes about 25 sec on Nexus 7) . Rest takes about 1.5 sec.
I have to do this every time when user launches app or requests data refresh.
Any ideas how to solve it better way?
Thanks
EDIT:
This is function for calculating distance, so its hard to make this in SQL :(
double getDistance(GPSCoords myPos, Place place) {
double dlong = (place.getLongitude() - myPos.getLongitude()) * d2r;
double dlat = (place.getLatitude() - myPos.getLatitude()) * d2r;
double a = Math.pow(Math.sin(dlat / 2.0), 2) + Math.cos(myPos.getLatitude() * d2r)
* Math.cos(place.getLatitude() * d2r) * Math.pow(Math.sin(dlong / 2.0), 2);
double c = 2 * Math.atan2(Math.sqrt(a), Math.sqrt(1 - a));
double d = 6367 * c;
return d;
}
You should be able to let SQL do the work inside the database:
select ((x - ?)*(x - ?) + (y - ?)*(y - ?)) as distsq from entries
order by dist limit 20
Unfortunately sqlite doesn't provide exponentiation, so the duplicated terms are needed.
If this is still not fast enough, another approach would be to make bounding box queries centered on your location, adjusting the size of the bounding box by binary search until you have 30 or a few more entries. Indexes on each of the x and y dimension will speed these along.
Edit Since the OP says earth curvature is important, a bounding box technique is probably the best approach we can get with unextended sqlite. Here is a proposed algorithm:
Let P be the current position
Let Slat = lat0 be the bounding box latitude half-size initialized with a "best guess"
Let Slon = lon0 be the bounding box longitude half-size initialized with a "best guess"
// NB the best guesses should cover an approximately square area on the ground
loop
Let W = P.lon - Slon, E = P.lon + Slon, N = P.lat + Slat, S = P.lat - Slat
C = select count(*) from entries
where W <= lon and lon <= E and S <= lat and lat <= N
if C indicates the result is too big (e.g. for memory or read time),
Slat = 0.5 * Slat
Slon = 0.5 * Slon
else
Let R be the result of the same query for * instead of count(*)
Let D be the geometric distance from P to the nearest point on bounding box
Compute r.dist for all r in R (in memory)
Sort R by dist (in memory)
Throw away the tail elements of R where r.dist > D
// Can't use these because points outside bounding box might be closer!
If at least 20 remaining R elements,
return top 20
else
Slat = 2 * Slat
Slon = 2 * Slon
end if
end if
end loop
Note you need indices for lat and lon. I don't know how good the SQLite query optimizer is in this case. A good optimizer will pick either the lat or lon index based on statistics accumulated from past queries, use this to quickly find all points in the bounding box range for that dimension, then do a scan of this result to get the final. If the optimizer is not that clever, you want to index only the dimension likely to produce the smallest initial result: in the average case this is the one with greatest geometric extent (distance covered).
The r* tree index will make the bounding box queries much faster, but at least through Jelly Bean, you'd have to provide your own SQLite instance with this extension included. Perhaps later Android versions included it? I don't know.
Additionally, if you get to the point of including a custom SQLite with the app, it would be pretty easy to add the distance (with curvature) function as an extension.
I don't see why exactly you feel you need to lazy load your entries. The 70MB heap number sounds pretty suspicious with only 12k entries. Are you grabbing the entire row just to calculate distance? Try just grabbing the columns you need:
Latitude
Longitude
Primary Key
Assuming each is 8 bytes apiece, that's 24 * 12000 bytes, or roughly 280 kilobytes. Give it some overhead room for just being Java, but you're still looking at something very manageable.
Then you can do the calculations in code, and just have it spit out the primary key for each of the closest points. A second query can grab just those 25(the entire row this time), and you're done!
There are a lot of examples of distance calculation using different flavors of SQL out there. Loading every row from your DB and calculating how far it is, then sorting and taking the closest is going to be slow just from the back and forth to the database. Doing the calculation in SQL and only retrieving the ones you need is going to be much more performant.
You can try moving the distance calculation to the sql db.
you can also put some smarter code, that will run distance calculation until he finds 25 places that their distance from current location is less than x(you choose). or even less than 25 items (maybe you just need 7 to fill the screen) and than continue the calculation in background when user is already in the app.
It'll be a much better user experience.
I have an interesting question. I have a latitude value of 51.445376 and a longitude value of -0.190624 (as an example, say this was recieved by an androids location listener). This particular location (and it's values) is a point of interest and it is stored in my database. I am using a location listener on my android, and I will send the users current location to the server to detect if the user has come within a one kilometre radius of that particular point of interest. Does anyone know what algorithm to use to check if the users current longitude and latitude is within that one kilometre radius of the point of interests longitude and latitude? Or something I can see?
Perhaps someone could help me out, as my math is not the greatest in the world.
For geographic coordinates, use this function (public domain):
public static double distanceInKM(
double latStart,double lonStart,
double latEnd,double lonEnd
){
latStart=Math.toRadians(latStart);
latEnd=Math.toRadians(latEnd);
return 6370.997*Math.acos(
Math.cos(Math.toRadians(lonEnd - lonStart))*Math.cos(latStart)*
Math.cos(latEnd)+Math.sin(latStart)*Math.sin(latEnd));
}
Calculating the shortest geographic path is called the "inverse geodesic problem", and this is discussed in C.F.F. Karney's article "Algorithms for geodesics, 2012. The method above shows a distance function based on the spherical law of cosines, which is a less accurate way to compute this path, especially because it assumes Earth is a perfect sphere.
This post from 2008 might be helpful.
It links to a MovableType description of how to use the Haversine formula to calculate the distance between two points on a sphere. It's important to note that it's not 100% accurate since the Earth is not a perfect sphere (it's equatorial radius is some 20km different from its polar radius) but should be sufficient for your needs.
The MovableType link also describes how to use an equi-rectangular projection (translation of lat/long to rectangular coordinates) and the pythagorean theorem to determine a rougher (but faster) approximation of the distance (good for small distances of ~1-2km.)
A simple solution: Simply find the distance, by first finding dx and dy.
Say I have point 1 (p1) and point 2 (p2)
dx = p2.x - p1.x;
dy = p2.y - p1.y;
Now find the distance d as follows
d = sqrt(dx^2 + dy^2)
In a programming language like java, you can use the following lines:
double d = Math.sqrt(dx*dx + dy*dy);
Hope this helps as your starting point!
Note: x & y must be calculated accordingly for your lat&lon info. I was going to write a short code snippet but I saw someone else has a nice solution (I upvoted his response).
I have in my android application a database table with geo pointes (lat and lon are decimal degree values), about 1000 points. And I need to select 20 nearest point to some given geo point.
I've found at Stackoverflow the answer how to compute distance between two geo points and was very happy, till I tried to write my query. I've found out, that it's not possible to use trignometrical functions in built-in sqlite of android.
But then I've got an Idea. I don't really need to compute a distance. The near a point is to another one the smaller difference in their geo coordinates should be.
How could I use this fact? Would it be enough to order saved points by (lat_0 - lat_n)^2 + (lon0-lon_n)^2, where lat_0 and lon_0 are geo coordinates of a given point?
Thank you,
Mur
UPD
So, the best way to get an answer for my question was to test approach I describe above.
It works pretty well but not really exactly compared to exact distance.
So if you just need to compute a distance, this solution is ok, but in my case I also needed to order stations by distance and couldn't use this solution.
My thanks go on John at CashCommons and Philip. Thank you guys
If your points are separated within a city (more or less), that approximation will work fine. The approximation falls apart if you go worldwide, though.
EDIT: Based on Philip's comment below, you should scale one of the components. Germany is about 50 degrees north latitude, so multiplying the longitude by (cos 50 deg) will do better.
Yes. :-) The actual distance is sqrt( (lat_0 - lat_n)^2 + (lon0-lon_n)^2 ) but ordering by (lat_0 - lat_n)^2 + (lon0-lon_n)^2 is sufficient.
Hmm... I'm not sure how that ordering would work? Wouldn't you need a different order for each point to indicate it's neighbors.
The simplest solution is to just iterate through all the points and compute the geometrical distance between the points. For 1000 points this should happen fairly fast.
The most optimized solution (in terms of retrieval speed) is to compute the neighbors of each point when you insert them in the database. For example you can keep the list of ids as a comma separate string and insert in the database?. Then when you need someones neighbors you do directly to them. However this is going to become a pain if you need to insert new points. Basically you'll need to re-compute the neighbors.