Develop Reference

Best way to compare IP addresses quickly

I'm parsing two CSV files which contains IP addresses.
The first is a source CSV, and the second is a "Blacklist".
Because of the size of the source file, I'm trying to optimize the speed at which I find IP addresses that match the blacklist.
EDIT: The blacklist consists of IP Address "Blocks". This means that each record in the blacklist has two IP addresses: A Start Block (ex. 216.254.128.0) and an End Block. (Ex. 216.254.223.255)
This means that direct lookups etc, will NOT work.
I'm wondering what's the best way to approach this. The brute strength method would be:
String[] parts = sourceIP.split("\\."); // String array, each element is text between dots
int hi = 255;
int lo = 0;
int mid = (hi - lo) / 2 ;
if (Integer.valueOf(parts[0]) > mid) {
mid = lo;
}
I could then repeat this for each part to decide whether or not the IP address is in the black list.
This seems pretty aggressive and with 4k+ records, this could take a very, very long time.
It could take 10+ iterations to decide each part and that would then have to be repeated to check the "High" part of the IP blocks in the blacklist. That's 80+ iterations per record.
I'm hoping to get some input here to see the best method for comparing IP addresses.
What are your thoughts?
Would it be possible to use a quick bitwise mask to compare values rapidly by serializing INetAddress?
FILE STRUCTURE CLARIFICATION:
Source IP File:
Contains a list of records from a database. (Aprox 4k). Each record contains names, addresses, emails, and IP Address.
Blacklist:
Contains 4.2k records. Each record is an IP Address "Block". This consists of two IP Addresses. 1. Start and 2. End.
If the record in the Source list has an IP address that's found in the blacklist, I need to save that record and add it to a new file.

I assume you're talking IPV4 addresses of the form xxx.xxx.xxx.xxx.
You can easily convert an IP address into an integer. Each segment (i.e. xxx) is 8 bits (i.e. one byte). So four of them together makes a 32-bit integer. So, given an IP address like "192.168.100.12", you can split it into its four parts, parse each one to a byte and create an integer. Say, for example, that you created a byte array of the segments:
ipBytes[0] = 192;
ipBytes[1] = 168;
ipBytes[2] = 100;
ipBytes[3] = 12;
You can turn that into an integer:
int ipAddress = ipBytes[0];
ipAddress = (ipAddress << 8) | ipBytes[1];
ipAddress = (ipAddress << 8) | ipBytes[2];
ipAddress = (ipAddress << 8) | ipBytes[3];
There are more efficient ways to do that, but you get the idea. Your language's runtime library might already have something that'll parse an IP address and give you the bytes to make it an integer.
You have a set of IP address ranges that you want to check your source addresses against. Load each of the ranges into a structure like this:
class IPRange
{
public int startIp;
public int stopIp;
}
And store those in an array or list. Then sort the list by starting IP address.
For each source IP address, convert it to an integer and do a binary search of the list, searching the starting IP address. The source address itself might not be (probably won't be) found, but when the binary search terminates the mid value will hold the index of the range whose starting IP address is less than or equal to the source address. You then just have to check the source address against that item's ending IP address to see if it's in the range.
Binary search is O(log n). If you're searching a list of 4,300 ranges, it's going to take at most 13 probes to find an address in the array. That should be plenty fast enough, even when doing 4,000 different searches. You're only talking on the order of 50,000 total probes of the range array.
A couple of notes:
First, as I said above, I assume you're talking about IPV4 addresses. If you're talking about IPV6 addresses, the same concepts still apply but you'll need a 64 bit integer. I don't know enough about IPv6 to say how you'd convert the address to 64 bit integer. Probably you should depend on you runtime library to get the address bytes.
Second: I assume that ranges don't overlap. That is, you won't have something like:
start range end range
192.168.1.1 192.168.2.255
192.168.2.1 192.168.3.255
If you have that, then an IP address could fall within either of those ranges. You could potentially construct overlapping ranges that would allow addresses to fall through the cracks. If you have overlapping ranges, the problem becomes a little bit more complicated.

Put both files in a String. Use split(",") to split the ip's in the first string. Loop through the obtained ips array. For every ip search for it in the second String like blacklist.indexOf("," + ip + ",") But first add a "," at start and end of blacklist string.

Brute force it.
Load everything into ram, no reason not to.
Split the ips into a 2d array.
{0:123,123,123,123}
Blacklist into a 3d array.
Now you can start searching for integers.
When you have a match compare the next section.
If source value higher then compare to the END block same section.
When you have a match push to a new array and write it to a file at the end.
If this takes more time to run then it took me to type this then close the porn you have open because your ram is full and its using your page file.

You could use a data structure called Bloom Filter. Which is rather efficient performance and storage wise. As for an example, there's a question here, Most Efficient way of implementing a BlackList that has an answer that recommends this.
As far as I know, also Google Chrome uses this technique, as also explained rather nicely at Matthials Vallentine's blog post A Garden Variety of Bloom Filters.
Yet more explanation succinctly can be had found at Adobe leaked credentials checker. Some excerpts
The original leak is about 9.3GB uncompressed, of which 3.3GB is email
addresses [...] This means the data can fit into 512MB (i.e. 232 bits)
of memory and allows us to perform lookups in constant time [...] An
optimal bloom filter which is allowed to occupy 840MB would have
practically no false positives at all.

It seems like the most direction solution would be to use an interval tree to store the blacklist. Then check if the IP intersects with any of the intervals.
You also might want to consider using a Trie/hashtable to get fast lookups where the interval is the same. IE: 216.254.128.0 to 216.254.223.255 can be merged to 216.254.(128.0, 223.255), where the () is the interval. Thus you'd end up with two hash-table lookups (one for 216 and one for 254) then a search in an interval tree, which is likely to only contain a small number of elements.
You can also merge overlapping intervals into a single interval, which can probably be done as you build the interval tree. Which ends up being more like a binary search tree in that case.

send pin number and pin value from android to arduino kit via bluetooth

what i want is sending 3 pins numbers and values from my android to arduino kit via bluetooth in a single write command
i tried to pass them separated by commas and parsing them ParseFloat() but i don;t know clearly how this function works
if i send (12,4.6),(13,3.2),(14,2)
x=Serial.parseFloat();
y=Serial.parseFloat();
z=Serial.parseFloat();
x=12 , y = 4.6 , z = 13 ;
that's Right?
i want to know the best format to send them to arduino and how to parse in arduino code
thanks

From http://arduino.cc/en/Serial/ParseFloat :
Serial.parseFloat() returns the first valid floating point number from the Serial buffer.
Meaning to the parser, for x, 12 is the first valid floating point number.
So the output is expected since floating point values will be in the form :
123.456
So to get back to your problem : You can use the parseFloat to get the pin number. However to get the value, you should first get the string that only contains the value. To do this you can use the function indexOf to find the location of the "," in your string and get the substring that starts after it with the substring function .

How to check given phone number is international or non-international number in android?

My requirement is to find out the message recipient number is an International number or National number.
Is there any android API or any third party library available to implement the same.
I want to use libphonenumber google API[android/externals/libphonenumber] but don't know how to check International and Non-international number.

I don't think telephonyManager.getSimCountryIso() would help you determine 'to which country the call is being made' as it will return your country's ISO.
Moreover, length of ISD codes vary across countries. For some countries it is 1, for some it's 2, for some it's 3 and for others it's 4. So you will need to extract/make 4 different keys of these lengths from the outgoing number as I have shown below:
Say the out going number is +91-XXX-XXX-XXXX. then you'll create 4 keys as:
9 (1 digit key)
91 (2 digit key)
91X (3 digit key)
91XX (4 digit key)
Now check if any of these 4 keys is present in this list: ISO List .
[EDIT: Alternative Solution]
Again, if you only need to determine if the call being made is international or not then you can simply check for below condition:
if(outgoing-number `startswith` "00" || outgoing-number does not `startswith` your "country's- ISD-code") {
//it's an international call;
} else {
//it's a domestic call;
}

Rather use the google library which is used internally in Android as well.
http://code.google.com/p/libphonenumber/

Get country code from numbers that are different length

We know that there are numbers with different length. In Europe we mostly have 9 digits numbers plus country code.
In North America we often find 10 digits numbers.
I am trying to get my head around an idea how to get a country code from a number that may be of different length.
Any ideas? Maybe you know some working libs that can do it?

The key facts:
The country code is always at the start of the number, so it is easy to find no matter the length of the number.
There is no overlap, as #Luis points out.
A (looks pretty) complete list of country codes is give here. If you sort them by length (shortest first) and run through the list comparing the first n digits with the list entries you will get the answer.
However, if you look at the list you wall see that there are various groups of codes. A more intelligent approach would note that:
All numbers beginning with 1 are US, Canada or other US related places in which case the next three digits tell you which.
7 is Khazakstan
Apart from 20, all country codes beginning with 2 are three digits.
and so on ...

Country codes are parsed left-to-right with deterministic endpoints similar to the idea of Huffman coding. ie, if you see a 1 first, stop, it's the US/Canada/related territories. If you see most other numbers besides 7 (Russia/Kazakhstan), keep going. Some of those numbers may terminate on the second value.
The list of country codes is here: http://www.howtocallabroad.com/codes.html
It should be trivial for you to take this and write your own string parser of a phone number in order to determine which country code is present.
(don't forget that if these are numbers from within a particular country, you also have to take that country's exit code into account, which is also on the page I linked)
Edit: Oh, I guess luis covered it. But Jakob is incorrect in his comment about Barbados. Barbados is the same country code as the US; the 246 is its local "area code" within the US/Canada's country code.

I assume that you are talking about phone number country codes. Country codes are defined by the ITU ( https://en.wikipedia.org/wiki/List_of_country_calling_codes ). The country codes can be 1, 2 or 3 digits. Your only alternative is to have a list of all country codes and parse it from there. Note that there is no overlap; for instance, +44 belongs to the UK, and no country starts with just 4.
UPDATE: The North American Area has 4 digit prefixes, not 1, composed of +1 and a NPA of 3 digit (http://en.wikipedia.org/wiki/North_American_Numbering_Plan). The same rule applies though, in that +1-NPA cannot be repeated. Barbados seems to be +1246, but no other country or region can start with +1246. You can get the list of all NPA from http://en.wikipedia.org/wiki/List_of_North_American_Numbering_Plan_area_codes

Checking if call being made is international

I'd like to check if a call being made is international or not. I have the SIM number which I obtain by using TelephonyManager.getLine1Number() and also the Country code ISO of the SIM Card which I obtain using TelephonyManager.getSimCountryIso().
Is there anyway I can find the country code of the number to which the call is being made?

I don't think telephonyManager.getSimCountryIso() would help you determine 'to which country the call is being made' as it will return your country's ISO.
Moreover, length of ISD codes vary across countries. For some countries it is 1, for some it's 2, for some it's 3 and for others it's 4. So you will need to extract/make 4 different keys of these lengths from the outgoing number as I have shown below:
Say the out going number is +91-XXX-XXX-XXXX.
then you'll create 4 keys as:
9 (1 digit key)
91 (2 digit key)
91X (3 digit key)
91XX (4 digit key)
Now check if any of these 4 keys is present in this list: ISO List.
[EDIT: Alternative Solution]
Again, if you only need to determine if the call being made is international or not then you can simply check for below condition:
if(outgoing-number `startswith` "00" || outgoing-number does not `startswith` your "country's-ISD-code") {
//it's an international call;
} else {
//it's a domestic call;
}