Im investigating the use of com.google.crypto.tink:tink-android:1.6.1 in my current Android project.
The data I am encrypting includes the OAuth2 Access Token/Refresh Token I employ for my remote API calls, e.g. Access Token is my Bearer token for the Authorisation HTTP header.
Im concerned I have made an error in my encryption/decryption logic as I am experiencing an intermittent problem where I cannot Refresh the Token. The error from the server
{"error_description":"unknown, invalid, or expired refresh token","error":"invalid_grant"}
The refresh token cannot be expired as it lasts 24 hours.
My code that initialises Tink resembles this:-
private fun manageTink() {
try {
AeadConfig.register()
authenticatedEncryption = generateNewKeysetHandle().getPrimitive(Aead::class.java)
} catch (e: GeneralSecurityException) {
throw RuntimeException(e)
} catch (e: IOException) {
throw RuntimeException(e)
}
}
#Throws(IOException::class, GeneralSecurityException::class)
private fun generateNewKeysetHandle(): KeysetHandle =
AndroidKeysetManager
.Builder()
.withSharedPref(this, TINK_KEYSET_NAME, PREF_FILE_NAME)
.withKeyTemplate(KeyTemplates.get("AES256_GCM"))
.withMasterKeyUri(MASTER_KEY_URI)
.build()
.keysetHandle
Here is my code for encryption/decryption
import android.util.Base64
import com.google.crypto.tink.Aead
import javax.inject.Inject
const val BASE64_ENCODE_SETTINGS = Base64.NO_WRAP or Base64.NO_PADDING
data class Security #Inject constructor(private val authenticatedEncryption: Aead) {
fun conceal(plainText: String, associatedData: String): String {
val plain64 = Base64.encode(plainText.encodeToByteArray(), BASE64_ENCODE_SETTINGS)
val associated64 = Base64.encode(associatedData.encodeToByteArray(), BASE64_ENCODE_SETTINGS)
val encrypted: ByteArray? = authenticatedEncryption.encrypt(plain64, associated64)
return Base64.encodeToString(encrypted, BASE64_ENCODE_SETTINGS)
}
fun reveal(encrypted64: String, associatedData: String): String {
val encrypted = Base64.decode(encrypted64.encodeToByteArray(), BASE64_ENCODE_SETTINGS)
val associated64 = Base64.encode(associatedData.encodeToByteArray(), BASE64_ENCODE_SETTINGS)
val decrypted: ByteArray? = authenticatedEncryption.decrypt(encrypted, associated64)
return String(Base64.decode(decrypted, BASE64_ENCODE_SETTINGS))
}
}
Could the use of Base64 encode/decode be the issue?
Where is my mistake?
If Tink can decrypt your token, the issue should be with your token/the encoding and not with Tink (It is using authenticated encryption, so you are guaranteed that the bytes that you encrypted are going to be the bytes that you decrypt, or you get an error).
To also address the concern in the comment: Tink will happily encrypt and decrypt any byte strings (with the associated data also being allowed to be any byte string), and not be concerned with whether they are printable or even valid Unicode. You will have to base64 encode the output of Tink if you want to use it as a string, though, as it will be uniformly random distributed bytes, i.e. contain invalid Unicode and non printable characters with high probability.
Related
I am currently retrieving data from firestore and saving the retrieved data to a list, the list is used to create a cache file in the internal storage. When the app is started I check for network, and if there is no network I use the data from the cache to populate a recyclerview. This works fine, however I would like to perform CRUD operations while offline and I struggle to find a way to accomplish this.
This is the code when retrieving data from firestore and calling to create a cache file.
override fun getFromFirestore(context: Context, callback: (MutableList<PostFirestore>) -> Unit) {
db.firestoreSettings = settings
val notesList = mutableListOf<PostFirestore>()
try {
db.collection("DiaryInputs")
.addSnapshotListener { snapshot, e ->
notesList.clear()
if (snapshot != null && !snapshot.isEmpty) {
for (doc in snapshot.documents) {
val note = doc.toObject(PostFirestore::class.java)
notesList.add(note!!)
}
cacheHelper.createCachedFile(context, notesList)
callback(notesList)
} else {
//Refreshing the RV and cache if firestore is empty.
cacheHelper.deleteCachedFile(context)
callback(notesList)
}
}
} catch (e: Exception){
Log.d(TAG, "Failure", e)
}
}
This is how the creation of the cache file is made:
#Throws(IOException::class)
override fun createCachedFile(context: Context, notesList: MutableList<PostFirestore>) {
val outputFile = File(context.cacheDir, "cache").toString() + ".tmp"
val out: ObjectOutput = ObjectOutputStream(
FileOutputStream(
File(outputFile)
)
)
out.writeObject(notesList)
out.close()
}
This is the dataclass PostFirestore
class PostFirestore: Serializable {
var id: String? = null
var diaryInput: String? = null
var temp: String? = null
var creationDate: String? = null
constructor() {}
constructor(id: String, diaryInput: String, temp: String, creationDate: String) {
this.id = id
this.diaryInput = diaryInput
this.temp = temp
this.creationDate = creationDate
}
}
What should I do to be able to add something to the cached file "cache.tmp" instead of overwriting it ? And possibly how to edit/delete something in the list that is stored in the cached file?
Firestore handles offline read and writes automatically. See its documentation on handling read and writes while offline.
If you don't want to depend on that, you'll have to replicate it in your own persistence layer. This includes (but may not be limited to):
Keeping a list of pending writes, that you send to the server-side database when the connection is restored.
Return the up-to-date data for local reads, meaning you merge the data from the local cache with any pending writes of that same data.
Resolve any conflicts that may happen when the connection is restored. So you'll either update your local cache at this point, or not, and remove the item from the list of pending writes.
There are probably many more steps, depending on your scenario. If you get stuck on a specific step, I recommend posting back with a new question with concrete details about that step.
I am calling a post API with some parameters. one the parameters is
activatecode="aaaaaa$rr"
when the API call is made, it is sent as
activatecode=aaaaaa%24rr
The $ is encoded as %24. How to avoid this and send the special character as it is?
I am using Retrofit 2.9.0.
I have this service :
interface WordpressService {
#GET("wp-json/wp/v2/posts")
suspend fun getPosts(
#Query("page") page: Int,
#Query("per_page") limit: Int,
#Query("_fields", encoded = true) fields: String = "date,link,title,content,excerpt,author"
): List<Post>
}
Without putting encode = true, I end up with this request :
GET http://example.org/wp-json/wp/v2/posts?page=1&per_page=10&_fields=date%2Clink%2Ctitle%2Ccontent%2Cexcerpt%2Cauthor
With encode = true, I get :
GET http://motspirituel.org/wp-json/wp/v2/posts?page=1&per_page=10&_fields=date,link,title,content,excerpt,author
So in my case, adding encode = true in annotation solved my problem.
I have a simple Spring Boot WebFlux api and I am trying to use Ktor to call it.
Here is my controller
#RestController
#RequestMapping("api/v1")
class TestController {
#RequestMapping(method = [RequestMethod.GET], value = ["/values"], produces = [MediaType.TEXT_EVENT_STREAM_VALUE])
fun sayHello(#RequestParam(value = "name") name:String): Flux<Example> {
return Flux.interval(Duration.ofSeconds(1))
.map { Example(number = generateNumber()) }
.share()
}
fun generateNumber(): Int{
return ThreadLocalRandom.current().nextInt(100)
}
}
It just returns an object with a number every second
Now on the client side is where I want to use Ktor to get the data but I am unsure what to do. Reading the docs it looks like Scoped streaming is what I need but I am not sure how to get it to work with my controller
Here is the client code I have so far.
suspend fun getData(): Flow<Example> {
return flow {
val client = HttpClient()
client.get<HttpStatement>("http://192.168.7.24:8080/api/v1/values?name=hello").execute{
val example = it.receive<Example>()
emit(example)
}
}
}
When I try to make the call on the Android client I get this error
NoTransformationFoundException: No transformation found: class
io.ktor.utils.io.ByteBufferChannel (Kotlin reflection is not
available)
So it looks like I cant just serialize the stream into my object so how do I serialize ByteReadChannel to an object?
This is my first time trying out spring and ktor
To be able to serialize the data into an object you need to do t manually by reading the data into a byte array
client.get<HttpStatement>("http://192.168.7.24:8080/api/v1/values?name=hello").execute{
val channel = it.receive<ByteReadChannel>()
val byteArray = ByteArray(it.content.availableForRead)
channel.readAvailable(byteArray,0,byteArray.size)
val example:Example = Json.parse<Example>(stringFromUtf8Bytes(byteArray))
}
You can install a plugin for JSON serialization/deserialization, which will allow you to use data classes as generic parameters
https://ktor.io/docs/json.html#jackson
The same code works in runtime and doesnt work in test
There is such code
private fun generatePrivateKeyFromText(key: String): Key {
val kf = KeyFactory.getInstance("RSA")
val keySpecPKCS8 = PKCS8EncodedKeySpec(Base64.decodeBase64(key))
return kf.generatePrivate(keySpecPKCS8)
}
When I run or debug app it works ok, but this code fails on generatePrivate while testing
java.security.spec.InvalidKeySpecException: java.security.InvalidKeyException: IOException : algid parse error, not a sequence
#Test
fun decrypt() {
val encrypt = "MoRxCpLJNqxfXGVeU73zZFi+X2j2TLUTyIn1XRqCoEfeN8rNBR/YrEtumAz+8/0AaEsvx0+qTilfbw+edZd8Wfum4McWQ8oWXifvWLgoXybhxWUmCdi2fwA9Gw0parY6CSNYUDA2UuLrLLaDGMz/Jj4s4XmXKp5zuec1zXVdrPM="
val prkey = "MIICXAIBAAKBgQCAaTCQOxAZPfTXfgel2MMPUCLAf32TxfsXu71/c3mVFGtDPB+7IpmxCmEvAv6nlq1ymX1zRR5gIzNt4DZoM0RhE58eoksUXcmFcRnMX5V4bnI8DitHLdB2EZzdvnPX0Umzs+tE7I1ouDIocNQRsEoQeDcNPfz5av2zMPsg0Xl/yQIDAQABAoGAV8UOX5cvQsGZZ+2J1q8ZbI8OodrCf83z+V3mgYXxVZe2VSd0XNmiiWMZ2CNI4k3YUhtdpvtYbsfAsFpvdbuNAXMW82Zwsd0oluPzzoLELGkFvaUJlh2YGmizrBnEwvF0usJYwjsjUbXw3o1xKX8ILk5FBfdr2+L65YIIZ0UhqoECQQD/B0P8iZhoOTx7myhwlFCuVeSadwaOMGy2CgXRLvTFAtstz8YVO+D+yPKsEpAvMlFgEnkTt7tl8DRxMpKnfmO5AkEAgOZudjUD72xWfSJpCEcX76WGSASWE+fLCZ8C/HumIZ+0trW5/bsmBrI/6SldEJcy4b2bHh2nOggC/6R5rEUkkQJAAg779IDj0wuLOnAxLl90G0QkOT72tZUce4evLlYTsbdpL4B619cI5OWYV906frcIQx9DDO6xu4vp0HQZDPMPOQJAOVbH8ntY2ctmmdmRwWXmpusJ1cV8gTROJGSArpHOcAycFd628sCqhLYMKgsFZBjuQG7YrsfgGLdxpgijO1eykQJBAOE8+BrQwFWyOcgnUShPHo8mDOBkeplGr9VZdnWktac2aBr1Biovy+pipUyjSCAaFgOsSU0FDcK0I5ulTOpgMRg="
val decrypt = CryptoService.decrypt(encrypt, prkey)
assertEquals("Pika-pika", decrypt)
}
fun decrypt(ciphertext: String, key: String): String {
var decodedBytes: ByteArray? = null
try {
val c = Cipher.getInstance("RSA")
c.init(Cipher.DECRYPT_MODE, generatePrivateKeyFromText(key))
decodedBytes = c.doFinal(Base64.decodeBase64(ciphertext))
} catch (e: Exception) {
Log.e("Crypto", "RSA decryption error: $e")
}
return String(decodedBytes ?: ByteArray(0))
}
Working function is in Fragment
private fun testCrypto() {
val encrypt = "MoRxCpLJNqxfXGVeU73zZFi+X2j2TLUTyIn1XRqCoEfeN8rNBR/YrEtumAz+8/0AaEsvx0+qTilfbw+edZd8Wfum4McWQ8oWXifvWLgoXybhxWUmCdi2fwA9Gw0parY6CSNYUDA2UuLrLLaDGMz/Jj4s4XmXKp5zuec1zXVdrPM="
val prkey = "MIICXAIBAAKBgQCAaTCQOxAZPfTXfgel2MMPUCLAf32TxfsXu71/c3mVFGtDPB+7IpmxCmEvAv6nlq1ymX1zRR5gIzNt4DZoM0RhE58eoksUXcmFcRnMX5V4bnI8DitHLdB2EZzdvnPX0Umzs+tE7I1ouDIocNQRsEoQeDcNPfz5av2zMPsg0Xl/yQIDAQABAoGAV8UOX5cvQsGZZ+2J1q8ZbI8OodrCf83z+V3mgYXxVZe2VSd0XNmiiWMZ2CNI4k3YUhtdpvtYbsfAsFpvdbuNAXMW82Zwsd0oluPzzoLELGkFvaUJlh2YGmizrBnEwvF0usJYwjsjUbXw3o1xKX8ILk5FBfdr2+L65YIIZ0UhqoECQQD/B0P8iZhoOTx7myhwlFCuVeSadwaOMGy2CgXRLvTFAtstz8YVO+D+yPKsEpAvMlFgEnkTt7tl8DRxMpKnfmO5AkEAgOZudjUD72xWfSJpCEcX76WGSASWE+fLCZ8C/HumIZ+0trW5/bsmBrI/6SldEJcy4b2bHh2nOggC/6R5rEUkkQJAAg779IDj0wuLOnAxLl90G0QkOT72tZUce4evLlYTsbdpL4B619cI5OWYV906frcIQx9DDO6xu4vp0HQZDPMPOQJAOVbH8ntY2ctmmdmRwWXmpusJ1cV8gTROJGSArpHOcAycFd628sCqhLYMKgsFZBjuQG7YrsfgGLdxpgijO1eykQJBAOE8+BrQwFWyOcgnUShPHo8mDOBkeplGr9VZdnWktac2aBr1Biovy+pipUyjSCAaFgOsSU0FDcK0I5ulTOpgMRg="
val decrypt = CryptoService.decrypt(encrypt, prkey)
println(decrypt) // "Pika-pika"
}
I call it on onViewCreated
Updated:
I added BC provider (thanks, #JamesKPolk)
private fun generatePrivateKeyFromText(key: String): Key {
Security.addProvider(BouncyCastleProvider())
val kf = KeyFactory.getInstance(algorithm)
val keySpecPKCS8 = PKCS8EncodedKeySpec(Base64.decodeBase64(key))
return kf.generatePrivate(keySpecPKCS8)
}
But it is still ok in runtime and not while testing
javax.crypto.BadPaddingException: Decryption error
So problem for different running code didnt go.
What the difference between runtime and test which crashes code?
The issue is that the private key is not a PKCS8EncodedKeySpec, but rather an RSAPrivateKey object from PKCS#1. The BC provider, however, will still decode this mistake without complaint. However, other providers will rightfully complain. My guess is that the runtime is using an older version of Android where the default provider is BC, but your test is using a newer version where that isn't the case.
The fix is to make your private key a proper PKCS8EncodedKeySpec. Alternatively, you can explicitly request the "BC" provider. To do so, you need to specify "BC" in the getInstance() call: val keyFactory = KeyFactory.getInstance("RSA", "BC")
However, note that it appears that BC provider support is on its way out.
To convert a private key in the PKCS#1 format, either wrap a 'BEGIN RSA PRIVATE KEY'-style header and footer around the base64 blob or decode the base64 blob and place that in a file, then run:
openssl pkcs8 -topk8 -in privkey.pem -outform der -nocrypt | openssl base64 -A
or
openssl pkcs8 -topk8 -in privkey.der -inform der -nocrypt | openssl base64 -A
A second issue comes from relying on defaults. Instead of doing
val c = Cipher.getInstance("RSA")
which gets you defaults for mode and padding and thus is non-portable, always specify the full "algorithm/mode/padding" transformation string to Cipher.getInstance(). In you case, it appears the data is not padded (an insecure mode) you'd need something like
val c = Cipher.getInstance("RSA/ECB/NoPadding")
However, you really should use proper randomized padding, and currently that is OAEP padding.
Summary
The runtime environment is Android, but I think the test environment is Oracle Java (or maybe openjdk). There are evidently two critical differences in those environments:
Android uses the BC provider for KeyFactory which will handle private keys encoded in PKCS#1 RSAPrivateKey format. Oracle Java only supports PKCS8 encoded keys.
In Android, Cipher.getInstance("RSA") defaults to Cipher.getInstance("RSA/ECB/NoPadding"), but Oracle Java defaults to Cipher.getInstance("RSA/ECB/PKCS1Padding")
I am trying to make use of the twitter api, and am setting up a handler to deal with twitter api requests.
To do this I am using an HTTPUrlConnection and following the Twitter api docs
I've managed to get authenticated using the 3-legged OAuth process, but am stuck when actually trying to make requests with the oauth token.
Here is an example of what my auth headers look like:
Accept=*/*,
Connection=close,
User-Agent=OAuth gem v0.4.4,
Content-Type=application/x-www-form-urlencoded,
Authorization=
OAuth oauth_consumer_key=****&
oauth_nonce=bbmthpoiuq&
oauth_signature=*****%3D&
oauth_signature_method=HMAC-SHA1&
oauth_timestamp=1570586135&
oauth_token=*****&
oauth_version=1.0,
Host=api.twitter.com
for each header in the auth header I add it to my HTTP GET call like this:
urlConnection.setRequestProperty(header.key, header.value)
Note that Authorization points to one string
OAuth oauth_consumer_key=****&oauth_nonce=bbmthpoiuq&oauth_signature=*****%3Doauth_signature_method=HMAC-SHA1oauth_timestamp=1570586135&oauth_token=*****&oauth_version=1.0,Host=api.twitter.com
The following params are collected as follows:
oauth_consumer_key is my application API key
oauth_signature is computed by the hmacSign function below
oauth_token is the "oauth_token" received in the response from /oauth/access_token
The hmacSign function:
private fun hmacSign(requestType: RequestType, url: String, params: Map<String, String>): String {
val type = "HmacSHA1"
val key = "$API_SECRET&$tokenSecret"
val value = makeURLSafe("${requestType.string}&$url${getURLString(params.toList().sortedBy { it.first }.toMap())}")
val mac = javax.crypto.Mac.getInstance(type)
val secret = javax.crypto.spec.SecretKeySpec(key.toByteArray(), type)
mac.init(secret)
val digest = mac.doFinal(value.toByteArray())
return makeURLSafe(Base64.encodeToString(digest, NO_WRAP))
}
private fun makeURLSafe(url: String) : String {
return url.replace("/", "%2F")
.replace(",", "%2C")
.replace("=", "%3D")
.replace(":", "%3A")
.replace(" ", "%2520")
}
protected fun getURLString(params: Map<String, Any>) : String {
if (params.isEmpty()) return ""
return params.toList().fold("?") { total, current ->
var updated = total
updated += if (total == "?")
"${current.first}=${current.second}"
else
"&${current.first}=${current.second}"
updated
}
}
In the GET call I'm referring to, tokenSecret would be the oauth secret received from /oauth/access_token
After i make the call I get a 400: Bad Request
Is there anything obvious I'm doing wrong?
Update: By putting the params at the end of the url like ?key=value&key2=value2... instead of a 400 I get a 401. So I'm not sure which is worse, or which is the right way to do it.
Okay, finally got it working
So using the suggestion in the comments, I downloaded postman and copied all my info into postman - when i made the request there, I got a 200!
So then i looked back and tried to figure out what was different and there were a few things:
First the hmac sign function was missing an &
New function (added another & after the url):
private fun hmacSign(requestType: RequestType, url: String, params: Map<String, String>): String {
val type = "HmacSHA1"
val key = "$API_SECRET&$tokenSecret"
val value = makeURLSafe("${requestType.string}&$url&${getURLString(params.toList().sortedBy { it.first }.toMap())}")
val mac = javax.crypto.Mac.getInstance(type)
val secret = javax.crypto.spec.SecretKeySpec(key.toByteArray(), type)
mac.init(secret)
val digest = mac.doFinal(value.toByteArray())
return makeURLSafe(Base64.encodeToString(digest, NO_WRAP))
}
Next I noticed my auth header had its params seperated with & but they all should've been replaced with , - i also needed to surround each of my values in ""
So it looked like:
OAuth oauth_consumer_key="****",oauth_nonce="bbmthpoiuq",oauth_signature="*****%3D",oauth_signature_method="HMAC-SHA1",oauth_timestamp="1570586135",oauth_token="*****",oauth_version="1.0",Host="api.twitter.com"
After these changes i started getting 200!
Hopefully this helps someone else (though im sure its unlikely considering how specific these issues were)