I am trying to understand all this map formats for OpenStreetMap and I really got confused.
The OSM wiki has lots of information, but it looks like it is spread all over different places and i cannot get solid understanding of all the formats.
I am looking for something that can be used in Android for offline use. I know that there are lots of frameworks or even done apps that use different file formats, but as for me they file formats they use are all huge.
As I understand the most lightweight format supported by OSM is PBF-Binary, and it is raster format, right?
I have found that it's possible to convert it to *.map format that is vector, right?
The size then is about 40% less than PBF-Binary, but it has to be rendered and it will not be as fast as raster, right?
So another question is, what is the most compact OSM map format that can be used for android?
I know one app that i use a lot - MapsWithMe and it has small map files and they are very fast, but i don't know if it uses raster or vector, i know only they use OSM maps, but as i understand they have created their own format based on it or smth like that.
I have come across GeoJson and the map files are very small, not more than several megabytes. So now i'm getting confused why it is so, why then it's not used for mobile development, cause i googled 'geojson android' and no information about it.
Are there any comparison tables of these formats?
So if somebody has a link where i can learn about all this things, could you please give it to me?
Thanks
PBF format is a much smaller alternative to XML. It contains the same raw vector data. You can convert from one format into the other without loosing any information. PBF is smaller and faster because it is binary data whereas XML is plaintext. The OSM wiki has a short overview of common OSM file formats.
I don't know where you got the information that GeoJson is small. The size of a map depends on several attributes. Mainly coverage and detail. Usually you don't want to have an offline map covering the whole world on your device because it will be very large. Most of the time you just need a small area, like a country. And often you don't need every information OSM can offer. Roads, cities and important POIs are usually sufficient for routing and searching.
You didn't tell us what you want to do with the map. Just drawing it? Or do you also need a routing and search functionality? What map format would be the most useful for you depends on your use case.
There is already lots of software for Android using OSM, including various open source programs. You can take a look at them if you need inspiration for your software.
Related
I am currently working on an HDR application that requires the use of Camera2 to be able to customize HDR settings.
I have developed a customized algorithm to retrieve certain data from Raw DNG images and I would like to implement it on Android.
I am unfortunately not an expert in Java/Android, so I taught myself how to code. Using other formats, I have usually worked with bitmaps to retrieve pixel data. ( which was relatively an easy task concerning the existing methods )
Concerning DNG files, I have found no documentation showing me how to retrieve the pixels data. I thought of bufferizing the image, however the DNG file format contains many information other than pixels and I'm afraid I am unable to find an extraction strategy using bufferstream. (I just want to store the pixels inside an array)
Anyone has an idea ? Would highly appreciate some tips.
Best regards
Camera2 does not produce DNGs directly - it produces plain RAW buffers, which you can then save to a DNG via DngCreator.
Are you operating on the initial RAW buffers, or saving DNGs and then loading them back?
In general, DNGs are not full baked images, so quite a bit of code is needed to render them completely - see for example Adobe's DNG SDK.
I'm new at osmdroid and I'm trying to build simple GPS tracking application that must work offline. So first of all I know that osmdroid works with .osm But how to download osm file directly to my android device and more importantly how later I need to use it. I dont understand .osm file is simply just xml with lots of coordinates and other data. So how that .osm file can be transform to a graphical map? I know that is necessary to parse .osm file to get data, but where to download actual graphical map?
osmdroid does not currently present .osm data, mainly due to performance reasons. osmdroid does render raster images that someone can capture into a zip or database and then transfer to device. The problem really is space. It's a big planet and getting raster images can be huge! You'd have to prepare geographic regions into a database then host them somewhere and have your app download them on demand.
osmdroid as a wiki on offline maps here:
https://github.com/osmdroid/osmdroid/wiki/Offline-Map-Tiles
An alternative approach is something like Mapsforge, which can preconvert .osm data into their own binary format, also chunked up by geographic region. Same rules apply, however the file/download sizes are much smaller. Support for osmdroid using Mapsforge is in beta at the moment, however mapsforge can run on it's own and has it's own demo apps and whatnot.
To transform the .osm file into a graphical map, you need something like renderd and a slew of other tools, all on open street maps wiki. Again, you need a boat ton of storage for the planet to convert it into raster images. Then you'd still have to package them somehow and get them on device.
Or you can just use online maps.
I have found a number of resources but nothing that has quite helped me with what I am looking for. I am trying to understand the .png and.jpg file formats enough to be able to modify and/or read the exif or other meta data in the files or to create my own meta data if possible.
I want to do this in the context of an android application so we can keep if there but it is really not exclusive to that. I am trying to figure out how to do this using a simple imput stream byte array and go from there.
Android itself has to at least extract the RGB pixel information at some point when it creates a bmp image from the stream, I took a look in the BitMapFactory source to try and understand it but I got lost somewhere after delving into the Native files.
I assume the bmps are losing any exif/meta data in the files based on my research. So I guess I want to break the inputstreams down by byte arrays and remove meta data. In .pngs I know there is no 'standard' but based on this page it seems there is some organization of the meta data you can store.
With all that said, I wouldn't mind just leaving exif/png standards behind and trying to store my own information in some sort of standardized way, but I need to know more about how the image readers id the files as either jpg, png, ect. then determine where the pixel information is located.
So I guess my first question is, has anyone done something similar to this before so that they can file me in? If not, does anyone know of any good libraries that might be good for educational purposes into figuring out how to locate and extract this data?
Or even more basically, what is a good way to find meta data and/or the exif standard or even the rgb data programmatically using something like a byte array?
There are a lot of things to address in your question, but first I should clarify that when you say "Android itself has to at least extract the RGB pixel information," what you're referring to is the act of decompression, which is complicated in the case of JPEG, and non trivial even for PNG. I think it would be very useful for you to read through the wikipedias for JPEG and PNG before attempting to go any further (especially sections on header, syntax, file structure, etc).
That being said, you've got the right idea. It shouldn't be too difficult to read in the header of an image as a byte array/stream, make some changes, and replace the old file. A PNG file can be identified by the first 8 bytes, and there should be a similar way to identify a JPEG - I can't remember off the top of my head.
To modify PNG meta data, you'll have to understand "chunks" - types/names, ordering, format, CRC, etc. The libpng website has some good resources for this, here's general PNG info, as well as chunk specifications. Make sure you don't forget to recalculate the CRC if you change anything.
JPEG sections off a file using "markers," which are two bytes long and always start with FF. Exif is just a regular JPEG file with a more specific structure for meta data, and this seems like a reasonable introduction: Exit/TIFF
There are probably libraries for Android/Java that conveniently take care of this for you, but I've never used any myself. A quick google search turns up this, and I'm sure there are many other options if you don't want to take the time to write a parser yourself.
I am building the prototype for an Android app that should result in a versatile map that can deal with a large but static amount of environmental data combined with (user-generated) data that will be hosted on a separate platform.
My current approach is to fetch the large static data via Google Fusion Tables (in original a shape file - .shp, .dbf, .shx that I have exported as a .kml with QGisMaps - the "smaller" .kml I am using is already 66.4MB, the one intended for final use over 140 MB...) I know it is possible to slice down .kml but will this have an effect on loading times?
You can see a current example of the shape data on Google Maps here https://www.google.com/fusiontables/DataSource?snapid=S717313eWaJ
But now I am wondering if I am on the right path at all since this one layer of data is loading very slowly on my machine and I am not aware of the hassle to put this map into an Android app - and more importantly also fetching more data from a separate server and putting it as another (filterable) layer on top of it. This layer will be collected data from users, thus not "static" but regularly updated - integer values displayed in color per geodata.
Several people hinted me to use Open Street Maps but some advised that Android is very eloquent with Google Maps - in the middle of development I thought maybe someone can tell me whether I chose the right direction to go for or are most likely facing less problems with an alternative solution.
I would be really glad if someone could advise me on the issue with combining map data from two different sources layerd onto one map in Android whereas one bunch of data is huge but static and the other user-generated values.
Thanks so much for your time and looking over this
best
Birgit
EDIT #1: I am now switching to Open Street Maps, hoping to be able to set up a GeoServer myself hosting the data and querying the WMS with the App. It seems like the size of the data still has to be compressed in some way, I will keep you updated.
I'll not be able to address all of your concernes, but I can share some of my experience with maps and ovelays.
The most important concept you must be aware is the memory heap. Each android application has a maximum memory amount that can be used to allocate data objects, called memory heap. This limit is different from device to device and can go from 16MB in small older devices to 64MB in new tablets.
Having said that, you will never be able to load all your data at one shot if the data structure required to hold it exceeds the maximum heap available for the application.
Also, usaging large amounts of memory makes your application better candidate to be terminated by users or by the SO when additional memory is required by critical activities (like a phone call).
Now, going to the maps part...
I've used Google maps as well as mapsforge api for OpenStreet maps. Google maps requires internet connection, while mapsforge enables you to use a local map file or online connection. OpenStree maps don't provide the sattelite view.
Regarding performance, online connection tends to be faster, as they download already rendered map images, while offline map requires the images to be rendered as needed. Mapforges can use a local cache to improve this a little.
Displaying Overlays:
I've used both API's to display paths with arround 10.000 points over a map, together with additional overlays (compass, scale bar, pop-ups, etc.) and the code runs smoothly in a mid-rande device. However, my code have some optimizations to reduce recreation of paths (when user zoom or move map) by transforming the already existing path.
good luck.
So this is how I eventually converted a .shp file into a format that could be displayed by OSMdroid's XYTilesource.
I exported the layers separately as .kml projected WGS84 and imported them as layers to TileMill. After having defied the colors for each layer in CSS, I exported my map as a set of MBTiles. I uploaded them to Mapbox.com. From there I can fetch the tiles which are in the format a.tiles.mapbox.com/v3/myUsername.myTiles/{z}/{x}/{y}.png in my OSMdroid Mapview.
final myTileSource tileSource = new XYTileSource("myTiles", null, 10, 16, 256, ".png", "http://a.tiles.mapbox.com/v3/myUserName.myTiles/");
I would like to display rich maps contained in Autocad .dwg files in my Android app. I'd like to know if there is an existing java library to do so?
If not, is it doable to translate a dwg file to a model and draw that model shape by shape on the screen?
I can't use Google maps instead of these files. The whole point of the app is to display a map with a lot of data and all that data is contained in my autocad files library.
Besides, is there an alternative format to DWG that I could use to do exactly this? (UPDATE: would SVG do the trick?)
What I ended up doing is to convert all my files in KML. I've selected this format because it might be the most used format on Android to do GIS.
To make a custom map, I'm drawing overlays on a mapview using these hints. It's basically the most interesting solution. I don't need to pay attention to geolocation stuff, the cursor and the scale are manage by Google API itself.
If you're going to convert I'd probably suggest DXF. I haven't used it, but YCad is a java DXF library. There are also a number of DXF to SVG converters available but it's probably best to avoid two conversions.