I'm looking for the best way to find a random English word using an Android app. Is there a better way than saving a .txt file with all the words in your res/ folder, such as
http://www.stanford.edu/class/cs106l/assignments/dictionary.txt
then importing it into an ArrayList? The app I'm developing will search for words quite often, such as a random word starting with the letters 'A' or 'Be'. I'm worried about memory and runtime with this function.
Create a SQLite database containing all the words. You can then query that database using SQL to match your random word with various starting letters.
SQLite Details: http://developer.android.com/reference/android/database/sqlite/package-summary.html
Look for documentation on using LIKE in SQL for your text matching.
Example: http://www.techonthenet.com/sql/like.php
For randomness, find all words matching your criteria. Get the count of them (call it 'n'), then pick a random number 'x' between 1 and n, and take the xth record.
Use a prefix tree (or Trie) as described here.
It's extremly efficient, as all word-beginnings are only stored once.
Related
in my Android app I need to select data from a SQLite database. However, I have a search field in which users can type the name of a location. As they may typo this name, I need to be able to draw the relevant records from the database according to this.
For example:
Input: Ferris Whrrl
Actual: Ferris Wheel
It should locate the Ferris Wheel entry correctly despite the typo. I realise that editdist3 is not implemented in Android SQLite. I'd also rather not select the entire database into RAM and loop through all Strings and calculate Levenshtein distance manually as this would be super resource intensive.
Any help will be greatly appreciated!! :)
I think the best thing you can do is to implement an Auto Suggestion function just like all search engines nowadays do.
You could implement a dynamic algorithm that searches and caches values as needed when a new letter is entered. However, if you just operate on the bare metal without the help of any external service you will have resource problems too - depending on the size of your database and other context variables.
Having a proper index will help but I still think to load the all necessary values into an array and search is faster.
Here you need to create an AI system to guess the correct word. I think this is not relevant to SQL or Android.
Or you can use online AI system to guess the correct word.
I'm currently working on a project where I have a dictionary list of words of the english language. I use this list to figure out if a user has inputted a valid word. The whole list is rather lengthy, were talking about 100k of Strings, but I have partitioned the list into smaller bits. These files are stored as xml files and are accessed by the assetmanager. I use the parser as such to open the file:
document = documentBuilder.parse(xmlFile);
The document is a Document object using the library: org.w3c.dom.Document
So the problem is that this actually slows down the app A LOT, even with the smaller partitioned files (in fact initially I just had one file with all the 100k Strings and it crashed the app immediately). So should I use text files instead? Or instead of using the assetmanager should I store my data in resources? Or any other way to optimize speed for that matter? I'm open to any suggestions.
Why not store the word list in an sqlite table? You get indexed-based searches. If each entry is one word then a normal table is fine. If you are using multiple words per entry, then an fts table (full-text-search) solution would be best.
If speed is paramount (i.e. you don't care how much memory you use up -- which would be very unusual for an android app), then loading the words into a hashtable would probably give you the fastest per word lookup. It would take some time to load the list on startup, however.
So I was playing around with different approaches and found out that instead of using partitioned xml files, I used text files saved in the asset folder which greatly reduced execution time.
if you have a big traitement to deal with and you this my make your apps slow, i recommande you to use AsyncTask so that you can do what ever you want in backGround and meanwhile you can do another thing, take a look in this official documentation
I have implemented FTS in my app, but the size is too big. I would like to compress it somehow. I read this on the sqlite website:
-- Create an FTS4 table that stores data in compressed form. This
-- assumes that the scalar functions zip() and unzip() have been (or
-- will be) added to the database handle.
CREATE VIRTUAL TABLE papers USING fts4(author, document, compress=zip, uncompress=unzip);
But I am struggling to find an example of these scalar functions. Please if someone could provide an example of how I may implement such a function for compressing my text.
The text will be in english to start, but will eventually include arabic.
Update
Since it does not seem possible to compress the FTS tables. I would like to limit the actual tokens generated which would lead to smaller sizes.
For example the word "Book" will AFAIK be stored as Book Boo Bo B etc. However I only need it to be stored as complete words as that is all my users will search for.
How can I acheive that?
Even if you found a compression function example (which exit!), SQLite in Android doesn't support user defined functions.
Sample you found will not work in Android - at least in a standard Android device.
I am developing a glossary using the sample code Searchable Dictionary. Thanks to searching here, I have figured out how to update the database, which is a .txt file, and then get it to load by changing the version number in Dictionary.java.
My question is, how to do the following:
I would like to be able to insert illustrative images into the definitions.
I would also like to insert links to other entries in the dictionary (e.g. 'inventory' should have a link to 'product flow' and other related terms).
I would also like to know how to insert a carriage return.
My original glossary in spreadsheet format has several fields: 'term' 'definition' 'example' 'related terms'. I want to be able to put in links and images inside these fields and have a couple of carriage returns in between each field to differentiate them.
The dictionary code seems to take in everything as a string, so even if I try to put 'image.jpg', or '\n' for a new line, it simply prints that as part of the string. Is there a way around this?
Searching stackoverflow gave a few links to using SQLite. I am honestly a newbie at all this; the last time I programmed anything significant was ten years ago. Rewriting the code to directly access a SQLite database would be nontrivial for me. So I would like to know if that is really the route I should be taking. If it is, then could you point me to the most simple tutorials for constructing a dictionary that way? I downloaded SQLite data browser, but haven't figured out how to use to construct a new database. I know it should not be so hard; I just don't know what I am doing. :(
If there is an easy way to just do it inline, still using the Searchable Dictionary sample code as a base, that would really make my day. Otherwise, any specific suggestions/directions would be really appreciated.
Thank you!!
Update:
For clarification, below is an example of one entry in my glossary, as desired. There are carriage returns between sections, and links and images are inline with text:
Heijunka, or Load Leveling - An approach to smooth production flow when a mix of products is to be produced, by identifying for a selected time period, the smallest batch size at which to produce each specific product in the mix, before switching over to make another product in the mix.
Example:
Keeping a steady work flow, even if much slower than the original max, reduces waste (<-this is a link to the entry 'waste' in the glossary):
[image of line of balance graph with load leveling, and without]
Related Terms: work structure, demand leveling (<-These are links to respective entries)
Not sure if you saw this already, but Android has some developer lessons for saving Key-Value sets for simple data, and saving to SQLlite for more complex structures.
It sounds like your app needs a database called "Invetory" with the following fields: "ProductImage", "ProductTitle", "ProductLink". And you want to store the image as a BLOB. There's a good SO post on how to take an image from a URL and convert it to a byte array for storage: how to store Image as blob in Sqlite & how to retrieve it?
For the carriage return, i'm assuming you're using "\n"? If that's not working have you tried unescaping your string for TextView:
String s = unescape(stringFromDatabase)
Or for SQLlite:
DatabaseUtils.sqlEscapeString()
Key-value data: http://developer.android.com/training/basics/data-storage/shared-preferences.html
SQLlite data: http://developer.android.com/training/basics/data-storage/databases.html
Additional SQLite resources:
http://www.youtube.com/watch?v=G8ZRXdztESU
http://www.vogella.com/articles/AndroidSQLite/article.html
I want to know how exactly does sqllite works when you are dealing with database on android. I know that it writes everything on file with .db extension. But how does it read or write one particular table? Does it fetch the whole file or just the related part and how exactly does it do these operations? Can someone please suggest me some link? I tried google but the links I found just explain how to write queries.
For that you have to read basics of the databases .
all the db frameworks are almost same in terms of working so
you have to research on the basics of database (any).
here is some related information u can like
What does a database actually do to find out what matches a select statement?
To be blunt, it's a matter of brute force. Simply, it reads through each candidate record in the database and matches the expression to the fields. So, if you have "select * from table where name = 'fred'", it literally runs through each record, grabs the "name" field, and compares it to 'fred'.
Now, if the "table.name" field is indexed, then the database will (likely, but not necessarily) use the index first to locate the candidate records to apply the actual filter to.
This reduces the number of candidate records to apply the expression to, otherwise it will just do what we call a "table scan", i.e. read every row.
But fundamentally, however it locates the candidate records is separate from how it applies the actual filter expression, and, obviously, there are some clever optimizations that can be done.
How does a database interpret a join differently to a query with several "where key1 = key2" statements?
Well, a join is used to make a new "pseudo table", upon which the filter is applied. So, you have the filter criteria and the join criteria. The join criteria is used to build this "pseudo table" and then the filter is applied against that. Now, when interpreting the join, it's again the same issue as the filter -- brute force comparisons and index reads to build the subset for the "pseudo table".
How does the database store all its memory?
One of the keys to good database is how it manages its I/O buffers. But it basically matches RAM blocks to disk blocks. With the modern virtual memory managers, a simpler database can almost rely on the VM as its memory buffer manager. The high end DB'S do all this themselves.
How are indexes stored?
B+Trees typically, you should look it up. It's a straight forward technique that has been around for years. It's benefit is shared with most any balanced tree: consistent access to the nodes, plus all the leaf nodes are linked so you can easily traverse from node to node in key order. So, with an index, the rows can be considered "sorted" for specific fields in the database, and the database can leverage that information to it benefit for optimizations. This is distinct from, say, using a hash table for an index, which only lets you get to a specific record quickly. In a B-Tree you can quickly get not just to a specific record, but to a point within a sorted list.
The actual mechanics of storing and indexing rows in the database are really pretty straight forward and well understood. The game is managing buffers, and converting SQL in to efficient query paths to leverage these basic storage idioms.
Then, there's the whole multi-users, locking, logging, and transactions complexity on top of the storage idiom.
SQLite operation on Android is not any different from SQLite operation on any other platform.
Very short answer to your question: SQLite file is split into pages of fixed size.
Each database object (table, index, etc) occupies some number of pages. If objects needs to grow (like new rows are inserted into table) it may allocate more new pages either from free page list, or by growing database file in size. If rows are deleted or object dropped, reclaimed free space goes into free page list. During any operation, SQLite engine tries to NOT fetch whole file, however it maintains page cache for higher performance.
You can find much more detailed explanations on SQLite website in general, and about SQLite database file format in particular.