The Javadoc for this says:
Only the lower two bytes of the integer oneChar are written.
What effect, if any, does this have on writing non-utf8 encoded chars which have been cast to an int?
Update:
The code in question receives data from a socket and writes it to a file. (A lot of things happen between receiving and writing, so I can't just use the string I get using BufferedReader#readLine()). I was using Writer#write(char[]) but this meant I had to create a new char array each time. To get around creating an array everytime, I had a single char array which is filled with -1 (cast to a char).
I then use TextUtils#getChars to fill it, expanding the array if necessary. For writing, I loop through the array, writing to the Writer until char[i] == (char) -1 == true.
Internally, write(int) will just cast its parameter to char, so write(i) is equivalent to write((char)i).
Now in Java, internally char is just an integer type, with the range 0-65535 (i.e. 16 bit). The cast int -> char is a "narrowing primitive conversion" (Java Language spec, 5.1.3), and int is a signed integer, hence:
A narrowing conversion of a signed
integer to an integral type T simply
discards all but the n lowest order
bits, where n is the number of bits
used to represent type T. In addition
to a possible loss of information
about the magnitude of the numeric
value, this may cause the sign of the
resulting value to differ from the
sign of the input value.
That's why the Javadoc says that only the lower two bytes are written.
Now, what this means in terms of characters depends on how you want to interpret the int values. A char in Java represents a Unicode code point in UTF-16, that is the 16 bit number represented by the char is interpreted as the number of the Unicode code point. So if each of your int values is the number of a 16 bit code point, you're fine (actually, this is only true for characters in the BMP; if you use characters in the supplementary planes, each Unicode code point will be encoded into two chars). If it's anything else (including a code point with more than 16 bit, or a negative number, or something else entirely), you'll get garbage.
What effect, if any, does this have on
writing non-utf8 chars which have been
cast to an int?
There is no such thing as a "non-utf8 char". UTF-8 is an encoding, that is a way to represent a Unicode code point, so the question as posed is meaningless. Maybe you could explain what your code does?
Related
I am quite new to all things Android and Kotlin. I am currently working with an Android app from Punch Through:
(Blog: https://punchthrough.com/android-ble-guide/)
(GitHub: https://github.com/PunchThrough/ble-starter-android)
The app connects with a BLE peripheral and allows the user to enter text to send to the peripheral via UART.
I am struggling interpreting what the following code means / does:
with(hexField.text.toString()) {
if (isNotBlank() && isNotEmpty()) {
val bytes = hexToBytes()
ConnectionManager.writeCharacteristic(device, characteristic, bytes)
}
}
Where hexField.text.toString() is the text entered in the EditText field by the user,
and
where hexToBytes() is defined as:
private fun String.hexToBytes() =
this.chunked(2).map { it.toUpperCase(Locale.US).toInt(16).toByte() }.toByteArray()
I have tried this a few times, always entering “111” and have am using Timber() to output the result of bytes. This result varies every time, for example:
[B#2acf801
[B#476814a
[B#e9a70e5
[B#10172a0
So, I assume that only the first three characters are relevant, and somehow there is no end of line / string information.
So perhaps I am only interested in: [B#.......
B# = 0x 5B 42 40
Hex: 5B4240
Dec: 5980736
Bin: 10110110100001001000000
So then I try (and fail) to interpret / breakdown what this code might be doing.
The first thing I struggle with is understanding the order of operation.
Here's my guess....
Given EditText entry, in this case I entered "111"
First:
this.chunked(2)
would produce something like:
"11 and "01"
Second, for each of the two items ("11 and "01"):
it.toUpperCase(Locale.US).toInt(16).toByte()
would produce byte values:
17 and 1
Third:
.map .toByteArray()
Would produce something like:
[1,7,1]
or
[0x01, 0x07, 0x1]
or
[0x0x31, 0x37, 0x31]
So, as you can see, I am getting lost in this!
Can anyone help me deconstruct this code?
Thanks in advance
Garrett
I have tried this a few times, always entering “111” and have am using Timber() to output the result of bytes. This result varies every time
The output when you try to print a ByteArray (or any array on the JVM) doesn't show the contents of the array, but its type and address in memory. This is why you don't get the same result every time.
In order to print an array's contents, use theArray.contentToString() (instead of plain interpolation or .toString()).
Regarding the interpretation of the code, you almost got it right, but there are a few mistakes here and there.
this.chunked(2) on the string "111" would return a list of 2 strings: ["11", "1"] - there is no padding here, just the plain strings with max size of 2.
Then, map takes each of those elements individually and applies the transformation it.toUpperCase(Locale.US).toInt(16).toByte(). This one makes the string uppercase (doesn't change anything for the 1s), and then converts the string into an integer by interpreting it in base 16, and then truncates this integer to a single byte. This part you got right, it transforms "11" into 17 and "1" into 1, but the map {...} operation transforms the list ["11", "1"] into [17, 1], it doesn't take the digits of 17 individually.
Now toByteArray() just converts the List ([17, 1]) into a byte array of the same values, so it's still [17, 1].
android app needs to generate uuid with 13 chars. But that may increase the chance of clashing.
Come up with this function, idea was adding the uuid's most/least SignificantBits, and then get the string from the Long. and then figure out the 13 byte length part from the result. Test run seems not seeing clash on single machine (+100,000 uuids).
But not sure the clashing possibility across machines.
is there a better way which generates 13 chars uuid and reasonable low classing rate?
val random = Random()
fun generateUUID() {
val uuid: UUID = UUID.randomUUID()
val theLong = if (random.nextBoolean()) {
uuid.mostSignificantBits + uuid.leastSignificantBits
} else {
uuid.leastSignificantBits + uuid.mostSignificantBits
}
return java.lang.Long.toString(theLong, Character.MAX_RADIX)
}
It won't be an UUID in the strict sense anymore; UUID describes a very specific data structure. Using the low bits of a proper UUID is generally a bad idea; those were never meant to be unique. Single machine tests will be inconclusive.
EDIT: now that I think of it, what exactly is "char" in the question? A decimal digit? A hex digit? A byte? An ASCII character? A Unicode character? If the latter, you can stuff a full proper UUID there. Just represent it as binary, not as a hexadecimal string. A UUID is 128 bits long. A Unicode codepoint is 20 bits, ergo 13 of those would cover 260 bits, that's well enough.
The Java char datatype is, effectively, slightly less than 16 bits. If by "13 chars" you mean a Java string of length 13 (or an array of 13 chars), you can still stuff a UUID there, with some trickery to avoid reserved UTF-16 surrogate pair values.
All that said, for globally unique ID generation, they usually use a combination of current time, a random number, and some kind of device specific identifier, hashed together. That's how canonical UUIDs work. Depending on the exact nature of the size limit (which is vague in the question), a different hash algorithm would be advisable.
EDIT: about using the whole range of Unicode. First things first: you do realize that both "du3d2t5fdaib4" and "8efc9756-70ff-4a9f-bf45-4c693bde61a4" are hex strings, right? They only use 16 characters, 0-9 and a-f? The dashes in case of the second one can be safely omitted, they're there just for readability. Meanwhile, a single Java char can have one of 63488 possible values - any codepoint from 0 to 0xFFFF, except for the subrange 0xD800..0xDFFF, would do. The string with all those crazy characters won't be nice looking or even printable; it could look something like "芦№Π║ثЯ"; some of the characters might not display in Android because they're not in the system font, but it will be unique all right.
Is it a requirement that the unique string displays nicely?
If no, let's see. A UUID is two 64-bit Java longs. It's a signed datatype in Java; would've been easier if it was unsigned, but there's no such thing. We can, however, treat two longs as 4 ints, and make sure the ints are positive.
Now we have 4 positive ints to stuff into 13 characters. We also don't want to mess with arithmetic that straddles variable boundaries, so let's convert each integer into a 3 character chunk with no overlap. This wastes some bits, but oh well, we have some bits to spare. An int is 4 bytes long, while 3 Java characters are 6 bytes long.
When composing the chars, we would like to avoid the area between D800 and DFFF. Also, we would want to avoid the codepoints from 0 to 1F - those are control characters, unprintable by design. Also, let's avoid character 0x20 - that's space. Now, I don't know exactly how will the string be used; whether or not it will be used in a text format that doesn't allow for escaping and therefore if certain other characters should be avoided to make things simpler downstream.
A contiguous character range is easier to work with, so let's completely throw away the range upwards from 0xD800, too. That leaves us with 0xD7DF distinct codepoints, starting from 0x21. Three of those is plenty enough to cover a 32-bit int. The rule for converting an int into a character triple is straightforward: divide the int by 0xD7DF twice, take the remainders, add the remainders to the base codepoint (which is 0x21). This algorithm is your vanilla "convert an int to a string in base N", with the knowledge that there can be no more than three digits.
All things considered, here goes Java:
public static String uuidToWeirdString(UUID uuid)
{
//Description of our alphabet: from 021 to 0xD7FF
final int ALPHA_SIZE = 0xD7DF, ALPHA_BASE = 0x21;
//Convert the UUID to a pair of signed, potentially negative longs
long low = uuid.getLeastSignificantBits(),
high = uuid.getMostSignificantBits();
//Convert to positive 32-bit ints, represented as signed longs
long []parts = {
(high >> 32) & 0xffffffff,
high & 0xffffffff,
(low >> 32) & 0xffffffff,
low & 0xffffffff
};
//Convert ints to char triples
int nPart, pos = 0;
char []c = new char[12];
for(nPart=0;nPart<4;nPart++)
{
long part = parts[nPart];
c[pos++] = (char)(ALPHA_BASE + part / (ALPHA_SIZE*ALPHA_SIZE));
c[pos++] = (char)(ALPHA_BASE + (part / ALPHA_SIZE ) % ALPHA_SIZE);
c[pos++] = (char)(ALPHA_BASE + part % ALPHA_SIZE);
}
return new String(c);
}
Feast your eyes on the beauty of the Unicode.
A UUID is a 128-bit data type, commonly shown in a 36-character hexadecimal representation, or about 4 bits per character.
Your example is "du3d2t5fdaib4". That only uses lower case Latin letters and Arabic numerals, which gives you about 5 bits per character, or 13×5=65 bits. If you also allow upper case Latin letters, that gives you about 6 bits per character, or 13×6=78 bits.
You cannot fit a 128-bit value into a 65- or 78-bit data type without throwing away nearly half of the bits, which will radically increase the odds of collision—perhaps even guarantee it depending on how the UUIDs were generated and which bits you throw away.
I can easily read 2e15 as "two quadrillion" at a glance, but for 2000000000000000 I have to count the zeroes, which takes longer and can lead to errors.
Why can't I declare an int or long using a literal such as 2e9 or 1.3e6? I understand that a negative power of 10, such as 2e-3, or a power of 10 that is less than the number of decimal places, such as 1.0003e3, would produce a floating point number, but why doesn't Java allow such declarations, and simply truncate the floating-point part and issue a mild warning in cases where the resulting value is non-integral?
Is there a technical reason why this is a bad idea, or is this all about type-safety? Wouldn't it be trivial for the compiler to simply parse a statement like
long x = 2e12 as long x = 2000000000000 //OK for long
and int y = 2.1234e3 as int y = 2123.4 //warning: loss of precision
It's because when you use the scientific notation you create a floating point number (a double in your example). And you can't assign a floating point to an integer (that would be a narrowing primitive conversion, which is not a valid assignment conversion).
So this would not work either for example:
int y = 2d; //can't convert double to int
You have a few options:
explicitly cast the floating point to an integer: int y = (int) 2e6;
with Java 7+ use a thousand separator: int y = 2_000_000;
Because it's a shortcoming of Java.
(Specifically, there is clearly a set of literals represented by scientific notation that are exactly represented by ints and longs, and it is reasonable to desire a way to express those literals as ints and longs. But, in Java there isn't a way to do that because all scientific notation literals are necessarily floats because of Java's language definition.)
You are asking about the rules on writing a integer literals. See this reference:
http://docs.oracle.com/javase/tutorial/java/nutsandbolts/datatypes.html
The capability to use scientific notation as an integer literal might make things easier indeed but has not been implemented. I do not see any technical reason that would prevent such a feature from being implemented.
If this is a duplicate, first of all, I'm sorry I've been looking all around and I haven't found how to solve my problem (or my head is too messed up).
I think that is better if I throw my problem rather than explain it.
I have an integer value wich is int234. I convert this value to its hex string representation EA. Note that this is "EA", it is a string.
Then I use Integer.parseInt to convert it to 0xEA (which is an hex value, not hex string) and finally I need the ASCII (or character) representation of this value.
The problem is that when I decode it using Integer.toHexString I get C3AA instead of EA
This is the best explanation I've found. But still I don't know how to solve it..
I hope you can help me out! Thanks in advance!
EDIT
I'd like to transmit the int 234 in a UDP datagram that's why I use the conversions. (This UDP channel mainly sends ASCII ("234") but I want an exception to that and once in a while transmit it in ony 1byte) That's why I do all this conversions (maybe it is easier than this, but I think I'm too close the problem that I can't see it with clarity)
I can confortly do it in C but I need to do it in Java. And my head is going crazy.
So 234 toHexString becomes `hex="EA";
// HEX -> ASCII code
for(int i=0; i<hex.length()-1; i+=2 ){
String output = hex.substring(i, (i + 2));
int decimal = Integer.parseInt(output, 16);
result.append((char)decimal);
}
return result.toString();
As result I get a strange character (the one that #fadden said). Now a question: this value is 2bytes long?
I transmit this odd character and when I use str = Integer.toHexString(prevResult); I get C3AA.
Maybe I'm messing thing up, but what I want is just to transmit 0xEA(byte) and get the int 234 in the other side.
For Example, Let's say I'd like to transmit the word "COMMAND" and append the value 234. Normally I would send "COMMAND234" but now I need the command value to be only one byte long. So the hex representation of what I'd be transmitting would be:
0x67 0x79 0x77 0x77 0x65 0x78 0x68 0xEA
C O M M A N D 234
I am gonna use 256bit AES encrypted data on a mobile database.
My question is, is there a special factor to multiply the database field sizes?
Example:
Lets say I am allowing size 10 str, when its encrypted the size goes up about x2-x3 times.
So I should use at least size 25 str in the field.
Is there a special number I could multiply my normal field sizes to find out the minimum safe encrypted size?
Well say that the length of the plain text is P in bytes (say, text in UTF-8 encoding) and B is the block size of the underlying cipher and you would use CBC as mode of operation and PKCS#7 padding. In that case you would retrieve the following size of ciphertext in bytes (including a pre-pended IV):
C = B + P + B - P % B
now to convert those bytes to hexadecimal characters you would simply multiply by 2:
H = C * 2
So for a single UTF-8 character lower than value 7F you would get 64 hexadecimal characters.
This is probably not what you were expected. You can switch to counter mode encryption which does not require padding. You could use some other unique number to derive an IV. You could store the ciphertext in binary or base 64 (which only requires one extra character value per 3 bytes). All in all there are too many options to discuss here. If you need to think about this much, it is probably better to keep to build in cryptographic functions.
If you go really far you should look into format preserving encryption, and using alphabets with a size that are not precisely on bit boundaries.