I am gonna use 256bit AES encrypted data on a mobile database.
My question is, is there a special factor to multiply the database field sizes?
Example:
Lets say I am allowing size 10 str, when its encrypted the size goes up about x2-x3 times.
So I should use at least size 25 str in the field.
Is there a special number I could multiply my normal field sizes to find out the minimum safe encrypted size?
Well say that the length of the plain text is P in bytes (say, text in UTF-8 encoding) and B is the block size of the underlying cipher and you would use CBC as mode of operation and PKCS#7 padding. In that case you would retrieve the following size of ciphertext in bytes (including a pre-pended IV):
C = B + P + B - P % B
now to convert those bytes to hexadecimal characters you would simply multiply by 2:
H = C * 2
So for a single UTF-8 character lower than value 7F you would get 64 hexadecimal characters.
This is probably not what you were expected. You can switch to counter mode encryption which does not require padding. You could use some other unique number to derive an IV. You could store the ciphertext in binary or base 64 (which only requires one extra character value per 3 bytes). All in all there are too many options to discuss here. If you need to think about this much, it is probably better to keep to build in cryptographic functions.
If you go really far you should look into format preserving encryption, and using alphabets with a size that are not precisely on bit boundaries.
Related
I am quite new to all things Android and Kotlin. I am currently working with an Android app from Punch Through:
(Blog: https://punchthrough.com/android-ble-guide/)
(GitHub: https://github.com/PunchThrough/ble-starter-android)
The app connects with a BLE peripheral and allows the user to enter text to send to the peripheral via UART.
I am struggling interpreting what the following code means / does:
with(hexField.text.toString()) {
if (isNotBlank() && isNotEmpty()) {
val bytes = hexToBytes()
ConnectionManager.writeCharacteristic(device, characteristic, bytes)
}
}
Where hexField.text.toString() is the text entered in the EditText field by the user,
and
where hexToBytes() is defined as:
private fun String.hexToBytes() =
this.chunked(2).map { it.toUpperCase(Locale.US).toInt(16).toByte() }.toByteArray()
I have tried this a few times, always entering “111” and have am using Timber() to output the result of bytes. This result varies every time, for example:
[B#2acf801
[B#476814a
[B#e9a70e5
[B#10172a0
So, I assume that only the first three characters are relevant, and somehow there is no end of line / string information.
So perhaps I am only interested in: [B#.......
B# = 0x 5B 42 40
Hex: 5B4240
Dec: 5980736
Bin: 10110110100001001000000
So then I try (and fail) to interpret / breakdown what this code might be doing.
The first thing I struggle with is understanding the order of operation.
Here's my guess....
Given EditText entry, in this case I entered "111"
First:
this.chunked(2)
would produce something like:
"11 and "01"
Second, for each of the two items ("11 and "01"):
it.toUpperCase(Locale.US).toInt(16).toByte()
would produce byte values:
17 and 1
Third:
.map .toByteArray()
Would produce something like:
[1,7,1]
or
[0x01, 0x07, 0x1]
or
[0x0x31, 0x37, 0x31]
So, as you can see, I am getting lost in this!
Can anyone help me deconstruct this code?
Thanks in advance
Garrett
I have tried this a few times, always entering “111” and have am using Timber() to output the result of bytes. This result varies every time
The output when you try to print a ByteArray (or any array on the JVM) doesn't show the contents of the array, but its type and address in memory. This is why you don't get the same result every time.
In order to print an array's contents, use theArray.contentToString() (instead of plain interpolation or .toString()).
Regarding the interpretation of the code, you almost got it right, but there are a few mistakes here and there.
this.chunked(2) on the string "111" would return a list of 2 strings: ["11", "1"] - there is no padding here, just the plain strings with max size of 2.
Then, map takes each of those elements individually and applies the transformation it.toUpperCase(Locale.US).toInt(16).toByte(). This one makes the string uppercase (doesn't change anything for the 1s), and then converts the string into an integer by interpreting it in base 16, and then truncates this integer to a single byte. This part you got right, it transforms "11" into 17 and "1" into 1, but the map {...} operation transforms the list ["11", "1"] into [17, 1], it doesn't take the digits of 17 individually.
Now toByteArray() just converts the List ([17, 1]) into a byte array of the same values, so it's still [17, 1].
android app needs to generate uuid with 13 chars. But that may increase the chance of clashing.
Come up with this function, idea was adding the uuid's most/least SignificantBits, and then get the string from the Long. and then figure out the 13 byte length part from the result. Test run seems not seeing clash on single machine (+100,000 uuids).
But not sure the clashing possibility across machines.
is there a better way which generates 13 chars uuid and reasonable low classing rate?
val random = Random()
fun generateUUID() {
val uuid: UUID = UUID.randomUUID()
val theLong = if (random.nextBoolean()) {
uuid.mostSignificantBits + uuid.leastSignificantBits
} else {
uuid.leastSignificantBits + uuid.mostSignificantBits
}
return java.lang.Long.toString(theLong, Character.MAX_RADIX)
}
It won't be an UUID in the strict sense anymore; UUID describes a very specific data structure. Using the low bits of a proper UUID is generally a bad idea; those were never meant to be unique. Single machine tests will be inconclusive.
EDIT: now that I think of it, what exactly is "char" in the question? A decimal digit? A hex digit? A byte? An ASCII character? A Unicode character? If the latter, you can stuff a full proper UUID there. Just represent it as binary, not as a hexadecimal string. A UUID is 128 bits long. A Unicode codepoint is 20 bits, ergo 13 of those would cover 260 bits, that's well enough.
The Java char datatype is, effectively, slightly less than 16 bits. If by "13 chars" you mean a Java string of length 13 (or an array of 13 chars), you can still stuff a UUID there, with some trickery to avoid reserved UTF-16 surrogate pair values.
All that said, for globally unique ID generation, they usually use a combination of current time, a random number, and some kind of device specific identifier, hashed together. That's how canonical UUIDs work. Depending on the exact nature of the size limit (which is vague in the question), a different hash algorithm would be advisable.
EDIT: about using the whole range of Unicode. First things first: you do realize that both "du3d2t5fdaib4" and "8efc9756-70ff-4a9f-bf45-4c693bde61a4" are hex strings, right? They only use 16 characters, 0-9 and a-f? The dashes in case of the second one can be safely omitted, they're there just for readability. Meanwhile, a single Java char can have one of 63488 possible values - any codepoint from 0 to 0xFFFF, except for the subrange 0xD800..0xDFFF, would do. The string with all those crazy characters won't be nice looking or even printable; it could look something like "芦№Π║ثЯ"; some of the characters might not display in Android because they're not in the system font, but it will be unique all right.
Is it a requirement that the unique string displays nicely?
If no, let's see. A UUID is two 64-bit Java longs. It's a signed datatype in Java; would've been easier if it was unsigned, but there's no such thing. We can, however, treat two longs as 4 ints, and make sure the ints are positive.
Now we have 4 positive ints to stuff into 13 characters. We also don't want to mess with arithmetic that straddles variable boundaries, so let's convert each integer into a 3 character chunk with no overlap. This wastes some bits, but oh well, we have some bits to spare. An int is 4 bytes long, while 3 Java characters are 6 bytes long.
When composing the chars, we would like to avoid the area between D800 and DFFF. Also, we would want to avoid the codepoints from 0 to 1F - those are control characters, unprintable by design. Also, let's avoid character 0x20 - that's space. Now, I don't know exactly how will the string be used; whether or not it will be used in a text format that doesn't allow for escaping and therefore if certain other characters should be avoided to make things simpler downstream.
A contiguous character range is easier to work with, so let's completely throw away the range upwards from 0xD800, too. That leaves us with 0xD7DF distinct codepoints, starting from 0x21. Three of those is plenty enough to cover a 32-bit int. The rule for converting an int into a character triple is straightforward: divide the int by 0xD7DF twice, take the remainders, add the remainders to the base codepoint (which is 0x21). This algorithm is your vanilla "convert an int to a string in base N", with the knowledge that there can be no more than three digits.
All things considered, here goes Java:
public static String uuidToWeirdString(UUID uuid)
{
//Description of our alphabet: from 021 to 0xD7FF
final int ALPHA_SIZE = 0xD7DF, ALPHA_BASE = 0x21;
//Convert the UUID to a pair of signed, potentially negative longs
long low = uuid.getLeastSignificantBits(),
high = uuid.getMostSignificantBits();
//Convert to positive 32-bit ints, represented as signed longs
long []parts = {
(high >> 32) & 0xffffffff,
high & 0xffffffff,
(low >> 32) & 0xffffffff,
low & 0xffffffff
};
//Convert ints to char triples
int nPart, pos = 0;
char []c = new char[12];
for(nPart=0;nPart<4;nPart++)
{
long part = parts[nPart];
c[pos++] = (char)(ALPHA_BASE + part / (ALPHA_SIZE*ALPHA_SIZE));
c[pos++] = (char)(ALPHA_BASE + (part / ALPHA_SIZE ) % ALPHA_SIZE);
c[pos++] = (char)(ALPHA_BASE + part % ALPHA_SIZE);
}
return new String(c);
}
Feast your eyes on the beauty of the Unicode.
A UUID is a 128-bit data type, commonly shown in a 36-character hexadecimal representation, or about 4 bits per character.
Your example is "du3d2t5fdaib4". That only uses lower case Latin letters and Arabic numerals, which gives you about 5 bits per character, or 13×5=65 bits. If you also allow upper case Latin letters, that gives you about 6 bits per character, or 13×6=78 bits.
You cannot fit a 128-bit value into a 65- or 78-bit data type without throwing away nearly half of the bits, which will radically increase the odds of collision—perhaps even guarantee it depending on how the UUIDs were generated and which bits you throw away.
The AUDIO_FORMAT_PCM_32_BIT and AUDIO_FORMAT_PCM_8_24_BIT are two high definition audio formats in Android Lollipop.
Seems they are all in 32 bit depth.
Who know the exactly difference between them?
You can find that information in audio.h:
/* Audio format consists of a main format field (upper 8 bits) and a sub
format field (lower 24 bits).
AUDIO_FORMAT_PCM_32_BIT and AUDIO_FORMAT_PCM_8_24_BIT are defined as:
AUDIO_FORMAT_PCM_32_BIT = (AUDIO_FORMAT_PCM |
AUDIO_FORMAT_PCM_SUB_32_BIT),
AUDIO_FORMAT_PCM_8_24_BIT = (AUDIO_FORMAT_PCM |
AUDIO_FORMAT_PCM_SUB_8_24_BIT),
And if we look at the definitions of AUDIO_FORMAT_PCM_SUB_32_BIT and AUDIO_FORMAT_PCM_8_24_BIT we find some helpful comments:
AUDIO_FORMAT_PCM_SUB_32_BIT = 0x3, /* PCM signed .31 fixed point */
AUDIO_FORMAT_PCM_SUB_8_24_BIT = 0x4, /* PCM signed 7.24 fixed point */
In response to Michael's comment:
signed .31 means 1 bit for sign, 0 bits for the whole part, and 31 bits for the fractional part. signed 7.24 means 1 bit for sign, 7 bits for the whole part, and 24 bits for the fractional part. Read up on fixed-point arithmetic if you want to know more about how it's used.
AUDIO_FORMAT_PCM_8_24_BIT most likely refers to a padded 8 bits of zeros as the 7.24 fixed point doesn't make sense for PCM data. This is because PCM data ranges from [1.0 .. -1.0]. (it technically should be 8.23, otherwise 7.24 == 25-bits!). So the use of a "whole" [number] part does not make sense.
A single sample of AUDIO_FORMAT_PCM_8_24_BIT will contain 4 bytes, where only 3 bytes will hold any meaningful data and the remaining single byte will be all zeros.
The alternative is AUDIO_FORMAT_PCM_24_BIT_PACKED that only contains 3 bytes per sample and no padding. 24-bit audio has a strange format, and it doesn't fit well in the powers of 2 of digital audio. It is typically easier to handle a 24-bit sample as if it was 32-bit.
I am looking to transfer pixel data from a server to an android program. On the server, the pixel data is in form RGBA, with one byte per color / transparency. Unfortunately on android the the corresponding pixel format is ARGB, meaning the alpha channel comes before the color data, instead of after, like it does on the server. I am worried that shuffling the RGBA data to ARGB format on the server will be too slow, and so I was hoping to find another way around that. The server is written in python by the way. I am capturing the screen data using the function presented here: Image.frombuffer with 16-bit image data. If there is a way to grab screen capture using this method (or some other) in ARGB format or even RGB_565 I would love to hear about that as well.
One trick I thought of to solve this problem was to use the isPreMultiplied flag on canvas.drawbitmap(int[], ...) and then send only the RGB bytes from the server. Then I could recompose the RGB bits into ints on the android device and send that to drawbitmap, ignoring the alpha channel entirely.
However, this leaves me with another problem. Ints are comprised of 4 bytes, and I have a sequence of 3 bytes in my byte[] array (the RGB values). I was using some of the solutions proposed here: byte array to Int Array to convert my byte[] to an int[] when I was transferring RGBA data. But now that it is just 3 byte sequences, I'm not sure how to quickly convert it to ints. I am hoping for close to real time image updating so I need a way to do this quickly. Any ideas?
int rgbInt = byteArray[0] << 16 + byteArray[1] << 8 + byteArray[2];
// not sure these are in the correct order, you may have to swap the indexes around.
You might also need to include
+ 0xFF << 24
to set the alpha value to opaque.
The Javadoc for this says:
Only the lower two bytes of the integer oneChar are written.
What effect, if any, does this have on writing non-utf8 encoded chars which have been cast to an int?
Update:
The code in question receives data from a socket and writes it to a file. (A lot of things happen between receiving and writing, so I can't just use the string I get using BufferedReader#readLine()). I was using Writer#write(char[]) but this meant I had to create a new char array each time. To get around creating an array everytime, I had a single char array which is filled with -1 (cast to a char).
I then use TextUtils#getChars to fill it, expanding the array if necessary. For writing, I loop through the array, writing to the Writer until char[i] == (char) -1 == true.
Internally, write(int) will just cast its parameter to char, so write(i) is equivalent to write((char)i).
Now in Java, internally char is just an integer type, with the range 0-65535 (i.e. 16 bit). The cast int -> char is a "narrowing primitive conversion" (Java Language spec, 5.1.3), and int is a signed integer, hence:
A narrowing conversion of a signed
integer to an integral type T simply
discards all but the n lowest order
bits, where n is the number of bits
used to represent type T. In addition
to a possible loss of information
about the magnitude of the numeric
value, this may cause the sign of the
resulting value to differ from the
sign of the input value.
That's why the Javadoc says that only the lower two bytes are written.
Now, what this means in terms of characters depends on how you want to interpret the int values. A char in Java represents a Unicode code point in UTF-16, that is the 16 bit number represented by the char is interpreted as the number of the Unicode code point. So if each of your int values is the number of a 16 bit code point, you're fine (actually, this is only true for characters in the BMP; if you use characters in the supplementary planes, each Unicode code point will be encoded into two chars). If it's anything else (including a code point with more than 16 bit, or a negative number, or something else entirely), you'll get garbage.
What effect, if any, does this have on
writing non-utf8 chars which have been
cast to an int?
There is no such thing as a "non-utf8 char". UTF-8 is an encoding, that is a way to represent a Unicode code point, so the question as posed is meaningless. Maybe you could explain what your code does?