I have a list of 1000 words. I need to load an array with n randomly chosen words from that list (no repeats allowed). What is the best way of going about doing that?
My ideas:
1) Load the words into R.arrays to create a String array. Use collections.shuffle to shuffle the array, then pull the first n entries from it. Right now, I am having memory issues loading the initial array with the 1000 words using this method.
2) Load the words into a text file, read each word into a String array. Use same method to get first n entries.
3) Hard code the input of the words into a String array (I'd use a script to get that output of course). Use same method to get first n entries.
Is there a better way?
If you're mainly worried about memory usage and you're willing to give up computation speed, here's an algorithm that will get you there.
Keep your words in a text file, one word per line, with a fixed amount of characters per word, padding each word with spaces at the end to ensure a fixed word char size, call it s.
Create an array of max size n, call it w
Open a stream reader to the file containing the 1000 words
Get a random number between 1 and 1000, call it k
Seek to position k*s in the file stream and grab the next s characters
Add the word to w if it does not exist in the array yet
If the w array is full (ie. size=n), we're done, otherwise go back to step 3
Let us know how it goes. Happy coding!
Related
One string size is about 200 bytes,
and it stores 10~20 size in a daily array.
(Store 10~20 strings of 200bytes, as array type)
I have found a way to convert an array to a string
and store it in SQLite.
However, I do not know it's a good idea
because the size of the string is large.
1.
If large arrays of strings,
is it a good idea to store arrays as a string?
2.
or is there a better way?
I would like advice. Thank you.
You're actually placing your concern onto the wrong part of your database design.
For SQLite, the maximum length of a String is 1 billion bytes, so your worries about your 10-20 strings of 200 bytes each actually isn't considered that large.
There's really no harm in storing your array as a single long String in your database. Especially when it's nowhere close to the maximum limit of a String.
Your database query time won't become longer due to your String being long. The real concern here is the processing you'll be doing on that String to turn it back into an Array. Typically, if the String is extremely long, the real performance hit is when you're flattening the array into a String and when you're transforming that String back into an Array.
However, typically, this is something you'll show a loading indicator for to your users.
For storing an Array into a database, there's really only two ways to do so:
Flatten array into a single String and store the String as TEXT
Create a table meant to store the individual elements of the string, and include a column for a Foreign Key that allows you to associate those rows with the same array. Then you'll store each element of your String arrays as a row in this table.
Depending on what you need, one design is better than the other.
For example, you would normally prefer the second implementation if your app requires you to constantly edit individual elements of an array.
For such an example, it wouldn't make much sense to use the first solution, because this means every time you want to edit the contents of an array, you'll be fetching back the complete array in it's entirety. This is impractical when you only want to fetch or edit a particular portion of that String.
Therefore, in such an example, it is much more practical to store the individual elements of the arrays into individual rows of a Table meant for this type of data. You'll be querying only the row you want and updating only the row you want.
So to answer your questions, there's really only two ways to store your String array as a TEXT type in your SQLite database. Both ways work and the real concern is to consider which solution fits your needs best.
If your app only requires you to store and fetch the array in it's entirety each time, then the single String method might be more preferable.
But if your app requires you to work with individual elements of your array, then using the table method would be more convenient.
I'm new to C and programming in general. I'm stuck wondering why this thing is happening. Basically, I wrote this simple program to input a 6 character array from the user, and to print the same out. I'm using CPPDroid on my Android phone to compile and execute the code;
#include"stdio.h"
int main()
{
char c[6];
for(int i=0;i<=5;i++)
{
scanf("%c",&c[i]);
}
for(int j=0;j<=5;j++)
{
printf("%c",c[j]);
}
return 0;
}
For some reason, the first loop simply exits out before the rest of the elements are filled. I'd get an output like this (I entered a,b,s as the first 3 elements):
a
b
s
a
b
s
It just simply only takes 3 elements rather than 6, and prints them back. What's going on?
My apologies if this is a well known issue. I'm not familiar with terms used in programming much, so it's not easy for me to search for questions.
All the answers and comments mentioned it right. I will just add one more thing. Earlier the \n were also taken as input by the scanf. As a result
your loop ended and still the characters you desired were not read.
why the solution scanf("%c ",..) works?
Now, the trailing one is telling scanf() to skip any trailing
whitespace after the character input. It therefore keeps reading input
until it sees something that is not whitespace or the end of the
stream.
Also as pointed out, the leading white space would also let you achieve the same thing with the added benefit of having a smooth interactive input.
To give you an idea of what I mean I would give an excample:
int n,m;
scanf("%d ",&n);
printf("n is %d\n",n);
printf("Give 2nd number\n");
scanf("%d ",&m);
printf("m is %d\n",m);
So now you start giving input.
1
Enter
Now you expect so see the output n is 1. But it seems like it stopped.
You again type 2Enter
Now you see the output: n is 1. Then you see the output
n is 1
6<enter>
Give 2nd number
m is 2
That's what I meant when asked to avoid the trailing whitespace.
When you type in:
a
b
s
The Enter keystroke is counted as its own character (the newline character, '\n'), so you end up storing the following in c: ['a', '\n', 'b', '\n', 's', '\n'].
If you want to consume the newline, you can include it in the scanf() call; something like this:
scanf("\n%c",&c[i]);
I'm looking for a way to compare 2 strings partial. I need to clear this with an example.
The base string is "equality".
The string I need to check is spelled wrong: "equallaty". I want to conform this is partially correct so the input, even not right in a grammar way, is the same as the base string.
Now I can of course parse the string to an char array. Now I can check every single character, but if I check the first 4 characters they will be right, the rest will be wrong even if there are only 2 mistakes. So the check I want to use is that a minimum of 70 procent of the characters should match.
Is anyone able to help me get on the right track?
Compare the strings with an edit-distance metric like the Levenshtein distance. Such a metric basically counts the number of changes needed to make the strings equal. If the number of changes is small relative to the total size of the string, then you can consider the strings similar.
I am storing data on an interval basis (gps location) and I dont want the DB to swell up so I defined a MAX number of lines it can go up to and then it simply deletes the oldest line every time I insert a new one.
Now a database expert looked at my code and he says its way not efficient because deleting a row from the database is the most time/memory/procedures consuming action and I should avoid it at all cost.
He says I should instead, run-over the oldest line (update) after I reach MAX.
(so it goes top to bottom every time)
That means I need to save a separate "header" table to save my current pointer to the oldest line and update it on every insert (i don't want to lose it if the app crashes) ..
does it really more efficient ? any other ways to do this more efficiently?
Turning your database table into a ring buffer is silly.
If you really want to use this approach...
Don't use a database just use a data file and IO
In your data file each record will be a fixed size
[Time Stamp][Latitude][Longitude]
You can use string formatted or binary representations of your data it doesn't matter as long as they are fixed size.
----------gps.dat----------
[Ring Pointer]
[Time Stamp][Latitude][Longitude]
[Time Stamp][Latitude][Longitude]
...
[Time Stamp][Latitude][Longitude]
Ring Pointer is the binary representation of a long integer
When you first create the file you'll set its size to the size of an LONG_INTEGER_SIZE + (MAX_RECORDS * RECORD_SIZE)
When you want to add a record:
Read [Ring Pointer] from the beginning of the file
Write [Ring Pointer] + 1 to the beginning of the file (so people don't get confused keep the [Ring Pointer] variable the same just write the new value back to the file)
Go to location LONG_INTEGER_SIZE + (([Ring Pointer] % MAX_RECORDS) * RECORD_SIZE)
Write your new record at that location
I recently created a program that gets medi-large amounts of xml data and converts it into arrays of Strings, then displays the data.
The program works great, but it freezes when it is making the arrays (for around 16 seconds depending on the size).
Is there any way I can optimize my program (Alternatives to string arrays etc.)
3 optimizations that should help:
Threading
If the program freezes it most likely means that you're not using a separate thread to process the large XML file. This means that your app has to wait until this task finishes to respond again.
Instead, create a new thread to process the XML and notify the main thread via a Handler when it's done, or use AsyncTask. This is explained in more detail here.
Data storage
Additionally, a local SQLite database might be more appropriate to store large amounts of data, specially if you don't have to show it all at once. This can be achieved with cursors that are provided by the platform.
Configuration changes
Finally, make sure that your data doesn't have to be reconstructed when a configuration change occurs (such as an orientation change). A persistent SQLite database can help with that, and also these methods.
You can use SAX to process the stream of XML, rather than trying to parse the whole file and generating a DOM in memory.
If you find that you really are using too much memory, and you have a reason to keep the string in memory rather than caching them on disk, there are certainly ways you can reduce the memory requirements. It's a sad fact that Java strings use a lot of space. They require two objects (the string itself and an underlying char array) and use two bytes per char. If your data is mostly 7-bit ASCII, you may be better of leaving it as a UTF-8 encoded byte stream, using 1 byte per character in the typical case.
A very effective scheme is to maintain an array of 32k byte buffers, and append the UTF-8 representation of each new string onto the first empty space in one of those arrays. Your reference to the string becomes a simple integer: PTR = (buffer index * 32k) + (buffer offset). "PTR/32k" yields the index of the desired byte buffer, and "PTR % 32k" yields the location within the buffer. Use either an initial length byte or a null terminator to keep track of how long the string is. When you need to access one of the strings, don't allocate a new String object: unpack it into a mutable StringBuilder or work directly with the UTF-8 byte representation.
The above approach is obviously a lot more work, but can save you between a factor of 2 and 6 in memory usage (depending on the length of your strings). However, you should beware of premature optimization. If your problem is with the processing time to parse your input, or is somewhere else in your program, you could find that you've done a lot of work to fix something that isn't your bottleneck and thus get no improvement at all.