Develop Reference

Google Service Account's Private Key problems with .NET client tool used to publish Android app to Google Play Console

I am creating an open source tool called google-play-publisher developed as a .NET 7 REST API which delegates on a Google API .NET Client.
The idea is to run it as a container (i.e: service) in my GitLab CI/CD pipeline in order to upload Android apps (i.e: aab files) to Google Play Developer, as it exposes a simple endpoint that I can invoke with curl like this:
curl -XPOST --data-binary #your_file.aab http://localhost:5000/api/apps/<your_package_name>/tracks/<track_name>
to make the continuous delivery convenient.
I have all necessary to make it work, including a Google Service Account with proper roles and credentials, which is known by my Google Play Console.
I download the keys as a JSON format, and it has a format like this (fake values):
{
"type": "service_account",
"project_id": "pc-api-...",
"private_key_id": "5342dce..",
"private_key": "-----BEGIN PRIVATE KEY-----\nbunchofcharacters\nandmorecharactes=\n-----END PRIVATE KEY-----\n",
"client_email": "publisher#pc-api-foo-iam.gserviceaccount.com",
"client_id": "123456",
"auth_uri": "https://accounts.google.com/o/oauth2/auth",
"token_uri": "https://oauth2.googleapis.com/token",
"auth_provider_x509_cert_url": "https://www.googleapis.com/oauth2/v1/certs",
"client_x509_cert_url": "https://www.googleapis.com/robot/v1/metadata/x509/publisher#pc-api-foo-iam.gserviceaccount.com"
}
NOTE: The -----BEGIN PRIVATE KEY----- and -----END PRIVATE KEY-----\n are very necessary. The Google API .NET client library fails if removed.
From that, I just need 6 things:
type
project_id
private_key_id
private_key
client_email
client_id
in order to be authorized by the Google Publisher v3 API and upload AAB for a specific package into a specific track.
The application needs only those 6 settings to be injected when starting, either by configuring them at appsettings.json or as environment variables injeted when spinning up the container. Pretty standard so far.
My appsettings.json would look like this (with the proper values from the credentials json)
"ServiceAccount": {
"PrivateKey": "***",
"ClientEmail": "***",
"ProjectId": "***",
"Type": "***",
"PrivateKeyId": "***",
"ClientId": "***"
},
When I configure this appsettings.json with the proper values, everything works well.
But I don't want to use appsettings.json for secrets. I need to use environment variables so that I can use the tool with docker and inject the variables when spinning up the container/service.
But when running the tool as a docker container or as a service inside GitLab CI/CD pipeline, I have some problems.
PROBLEM 1 - Docker run doesn't like the private key format
When I run the tool (version 0.1.8) as a docker container like this
docker run \
--name google-play-publisher \
-p 5000:80 \
-e ServiceAccount__PrivateKey="-----BEGIN PRIVATE KEY-----\nbunchofcharacters\nandmorecharactes=\n-----END PRIVATE KEY-----\n" \
-e ServiceAccount__ClientEmail="publisher#pc-api-foo-iam.gserviceaccount.com" \
-e ServiceAccount__ProjectId="pc-api-..."\
-e ServiceAccount__Type="service_account" \
-e ServiceAccount__PrivateKeyId="5342dce.." \
-e ServiceAccount__ClientId="123456" \
registry.gitlab.com/roundev/devops/google-play-publisher:0.1.8
and then I POST an aab file for a package
curl -XPOST --data-binary #my_file.aab http://localhost:5000/api/apps/com.foo.app/tracks/production
The authentication fails
System.ArgumentException: PKCS8 data must be contained within '-----BEGIN PRIVATE KEY-----' and '-----END PRIVATE KEY-----'. (Parameter 'pkcs8PrivateKey')
because docker run doesn't seem to like passing special environment variables with \n characters, etc.
PROBLEM 2 - GitLab CI/CD Variables doesn't like that format either
It cannot mask it, which is a problem, and then something doesn't work but I cannot see yet the error (no error shown, the pipeline passes but the AAB is not uploaded, so I need to figure out a way to read GitLab CI/CD service logs, which is out of scope for this)
SOLUTION ATTEMPT 1 - Convert private key value to base64 and decode it
I thought of encoding the private key in base64 (not sure if there is another alternative) in order to gain the following advantages:
GitLab CI/CD variable could be masked
Docker run would not behave strange when passing base64 string as environment variable
But this shows a different problem.
Using an online base 64 encoder and decoding it in the code to generate google auth credentials doesn't work. The \n characters seem to mess up things.
I have tried different variants, but I always get some invalid_token problems later on when using Google Publisher API.
// Functions used for encoding and decoding
string EncodeBase64(string text)
{
var textBytes = Encoding.UTF8.GetBytes(text);
var result = Convert.ToBase64String(textBytes);
return result;
}
string DecodeBase64(string base64Text)
{
var base64EncodedBytes = Convert.FromBase64String(base64Text);
var result = Encoding.UTF8.GetString(base64EncodedBytes);
return result;
}
I haven't yet found a good solution that allows me to:
Make my tool work with docker run by passing private key as an environment variable which doesn't cause problems, so that it could work the same as when running it with appsetting.json
Configure the environment variable in GitLab CI/CD as a variable in a secure way (so that it can be masked).
There must be a good solution using some kind of encoding/decoding private key. But how?

I've resolved it with Hex approach.
Basically the solutions is to grab the Json value for the private_key as it is, and transform it to Hex, for example using this online tool https://codebeautify.org/string-hex-converter so the private key that looked like this "-----BEGIN PRIVATE KEY-----\nfoo...\nbar\n-----END PRIVATE KEY-----\n" now looks like this 2d2d2d2d2d424...b45592d2d2d2d2d5c6e
This string has the following characteristics:
It can be protected as an environment variable in GitLab CI/CD
It can be passed as a docker environment variable
It can be easily read and transformed into the original private_key with this:
private class JsonCredentials
{
public string Type { get; init; } = string.Empty;
public string PrivateKeyId { get; init; } = string.Empty;
public string PrivateKeyHex { get; init; } = string.Empty;
public string PrivateKey => ToJsonValue(PrivateKeyHex);
public string ClientEmail { get; init; } = string.Empty;
public string ProjectId { get; init; } = string.Empty;
public string ClientId { get; init; } = string.Empty;
private string ToJsonValue(string hexadecimalValue)
{
var hexadecimalValueBytes = Convert.FromHexString(hexadecimalValue);
var result = Encoding.UTF8.GetString(hexadecimalValueBytes);
var unescapedResult = Regex.Unescape(result);
return unescapedResult;
}
}
See my open source repo for more details: https://gitlab.com/roundev/devops/google-play-publisher and to give it a try.

I believe your problem is the escaped new lines (/n) in your key, specially with the base 64 approach.
Try generating the base 64 string by pasting your key to an online base 64 encoder, like this:
-----BEGIN PRIVATE KEY-----
Whatever
-----END PRIVATE KEY-----
Rather than:
-----BEGIN PRIVATE KEY-----/nWhatever/n-----END PRIVATE KEY-----/n
Same for Gitlab. Paste your key with proper line breaks rather than /n
With regards to docker run, apparently /ns are not parsed as new lines and taken literally, so you'd need to investigate how to work around that. But basically either replace them with new lines in your code or go down the base 64 string road.

Can generate a Certificate for an RSA public key

I'm trying to get a Certificate for an RSA public key to store it in a KeyStore. For this, I use the code below :
certificates=new Certificate[1];
myEncodedPublicKey=myKey.getPublic().getEncoded();
byteStream=new ByteArrayInputStream(myEncodedPublicKey);
certificates[0] = myCertificateFactory.generateCertificate(byteStream);
Despite a non empty encoded public key, I get the following error : Unable to initialize, java.io.IOException: Short read of DER length.
Does anyone know what could be the origin of this problem?

Google Pay "quick-start-demo" not displaying card list if I choose "DIRECT" method for Paymenttokenization option

I have followed the tutorial provided HERE. Quick start demo can be found HERE.
If I choose in "tokenizationSpecification" param below method,
private static JSONObject getGatewayTokenizationSpecification() throws JSONException {
return new JSONObject(){{
put("type", "PAYMENT_GATEWAY");
put("parameters", new JSONObject(){{
put("gateway", "example");
put("gatewayMerchantId", "exampleGatewayMerchantId");
}
});
}};
}
It works fine but if I Choose below method,
private static JSONObject getDirectTokenizationSpecification()
throws JSONException, RuntimeException {
if (Constants.DIRECT_TOKENIZATION_PARAMETERS.isEmpty()
|| Constants.DIRECT_TOKENIZATION_PUBLIC_KEY.isEmpty()
|| Constants.DIRECT_TOKENIZATION_PUBLIC_KEY == null
|| Constants.DIRECT_TOKENIZATION_PUBLIC_KEY == "REPLACE_ME") {
throw new RuntimeException(
"Please edit the Constants.java file to add protocol version & public key.");
}
JSONObject tokenizationSpecification = new JSONObject();
tokenizationSpecification.put("type", "DIRECT");
JSONObject parameters = new JSONObject(Constants.DIRECT_TOKENIZATION_PARAMETERS);
tokenizationSpecification.put("parameters", parameters);
return tokenizationSpecification;
}
Its not working.[Not displaying my list of cards.]
Note:- I have performed below method to generate public key and replaced it in constants file as well.
# generate private key
openssl ecparam -name prime256v1 -genkey -noout -out key.pem
# generate a base64-encoded public key
openssl ec -in key.pem -pubout -text -noout 2> /dev/null | grep "pub:" -A5 | sed 1d | xxd -r -p | base64 | paste -sd "\0" -
Question:-
-Do I must have developer profile in google pay to run the demo ?

Country may be a factor in this case.
To eliminate this, can you try and create a new Google account for testing purposes? Create this Google account in the US and add a payment method/credit card to that account.
Test it out with the following example: https://jsfiddle.net/pxsb4jhn/ (it works for me, I am located in the US)
const allowedCardNetworks = ["AMEX", "DISCOVER", "INTERAC", "JCB", "MASTERCARD", "VISA"];
const tokenizationSpecification = {
"type": "DIRECT",
"parameters": {
"protocolVersion": "ECv2",
"publicKey": "BOdoXP+9Aq473SnGwg3JU1aiNpsd9vH2ognq4PtDtlLGa3Kj8TPf+jaQNPyDSkh3JUhiS0KyrrlWhAgNZKHYF2Y="
}
};
If it works and displays the card, it is likely a problem with DIRECT integration now being available in the country of the user you were using.
It it still doesn't work, it is potentially an issue the card not supporting DIRECT integration, in which case, try with a different card (ideally from another country like the US - this may be difficult if you don't have one available).

There are no accepted cards available for use with this merchant.
This message usually comes when the current Google user doesn't have any cards that are compatible with the payment options provided by the merchant. Specifically allowedCardNetworks and allowedAuthMethods.

Listener fo KeyChain Android

I have used
KeyChain.choosePrivateKeyAlias
I have successfully extracted the Private Key as well as the Public Key.
I want to handle the Click Listener of KeyChain, whether the user has allowed or deny the installation of certificates.
I couldnt find anything in the developers documentation.
Thank you

On Android ICS phone I have imported the PKCS#12 file containing private key and certificate. Then i run
KeyChain.choosePrivateKeyAlias(this, this, new String[] { "RSA" }, null, null, -1, null);
In the certificate selection dialog i choose the one just installed.
In the 'alias' callback i do the following:
public void alias(final String alias) {
...
protected Boolean[] doInBackground(Void... arg0) {
...
PrivateKey pk = KeyChain.getPrivateKey(ctx, alias);
Log.d(TAG, "EncodedPrivateKey: " + pk.toString());
And it gives me the full content of the private key.
Does it mean that any application, once allowed by user (in the cert. selection dialog), can read any private key installed from .pfx file?
Is the following scenario possible by standard Android means - "administrator" installing .pfx file with the cert.+private key and the permissions to read it are limited to the one specific app?

How to generate a RSA key in Android similar to an OpenSSL PEM_write_PUBKEY generated key

I have a C application, within this application I generate a RSA key pair with the following code (error checking left due to readability):
void generateKeyPair(char* pass) {
EVP_PKEY *pkey = NULL;
RSA* r;
OpenSSL_add_all_algorithms();
RAND_load_file("/dev/urandom", 1024);
r = RSA_generate_key(KEY_LENGTH, RSA_F4, NULL, NULL);
pkey = EVP_PKEY_new();
EVP_PKEY_assign_RSA(pkey, r);
FILE* fp = fopen("private.key", "w");
PEM_write_PrivateKey(fp, pkey, EVP_aes_256_cbc(), NULL, 0, NULL, pass);
fclose(fp);
fp = fopen("public.key", "w");
PEM_write_PUBKEY(fp, pkey);
fclose(fp);
}
I synchronize the public PEM keys through a server between the devices. Now I have to write a compatible Android application though I have to generate the public key in the same format then PEM_write_PUBKEY does.
I know that I should have done the synchronisation in DER format or something but now I can't change the design any more. There is no way around generating the keys in the similar format.
I guess that it is a #PCKS1 base64 encoded key but I am not sure and I don't know how to generate a similar one in Android - Androids standard is #PCKS8. However I would prefer not to include spongy castle in my project if there is a way around.

You are in luck; it seems you only have to PEM encode your public key as both Java and OpenSSL use the same SubjectPublicKey structure used for X5.09 certificates. You can get to this by running RSAPublicKey.encode().
Unfortunately I don't know any other library that performs PEM encoding, but I'm sure you can strip out the code from Bouncy or Spongy if required; the Bouncy Castle libraries have a very liberal licensing structure.