Develop Reference

APDU commands to get data from NFC tag using ISODEP

I am trying to read the information stored on my German Sparkasse Girocard. My app successfully recognizes the (ISODEP) tag. To read the stored information, I need to send a sequence of APDU commands, but I am not sure which.
From my understanding I need to send a SELECT command first:
byte[] SELECT = {
(byte) 0x00, // CLA Class
(byte) 0xA4, // INS Instruction
(byte) 0x04, // P1 Parameter 1
(byte) 0x00, // P2 Parameter 2
(byte) 0x09, // Lc
(byte) 0xD2,0x76,0x00,0x00,0x25,0x47,0x41,0x01,0x00, // AID
(byte) 0x00 //Le
};
Tag tagFromIntent = intent.getParcelableExtra(NfcAdapter.EXTRA_TAG);
IsoDep tag = IsoDep.get(tagFromIntent);
tag.connect();
byte[] result = tag.transceive(SELECT);
text.setText(Integer.toHexString(result[0]) + ", " + Integer.toHexString(result[1]));
The status response should be 9000 if it works. I am getting 6F44 which indicates that there was some sort of error (I think). I am also not quite sure if I am using the correct AID, but it has also not worked using others, which I thought could be correct.
What is my error and which commands do I have to send to retrieve the data?
Thanks in advance.

Getting data from an EMV card (e.g. Girocard, Mastercard, Visacard) is more a "question and answer" puzzle - you are asking the card, get a response, analyze the data and ask the next question.
The analyzing part is done here by using the "TLV Utilities" from emvlab.org (https://emvlab.org/tlvutils/).
To get more information about the Application Identifier ("AID"s) see the complete list at: https://www.eftlab.com/knowledge-base/211-emv-aid-rid-pix/.
Here is an example of reading a German Girocard (mine is from a "Volksbank"):
Step 1: ask the card which applications are on the card using the "select PPSE" command. The 2 bytes at the end "90 00" say that the answer is successfull:
selectPpseCommand: 00 A4 04 00 0E 32 50 41 59 2E 53 59 53 2E 44 44 46 30 31 00
selectPpseResponse: 6F 67 84 0E 32 50 41 59 2E 53 59 53 2E 44 44 46 30 31 A5 55 BF 0C 52 61 19 4F 09 A0 00 00 00 59 45 43 01 00 87 01 01 9F 0A 08 00 01 05 01 00 00 00 00 61 1A 4F 0A A0 00 00 03 59 10 10 02 80 01 87 01 01 9F 0A 08 00 01 05 01 00 00 00 00 61 19 4F 09 D2 76 00 00 25 47 41 01 00 87 01 01 9F 0A 08 00 01 05 01 00 00 00 00 90 00
Parsed response:
6F File Control Information (FCI) Template
84 Dedicated File (DF) Name
325041592E5359532E4444463031
A5 File Control Information (FCI) Proprietary Template
BF0C File Control Information (FCI) Issuer Discretionary Data
61 Application Template
4F Application Identifier (AID) – card
A00000005945430100
87 Application Priority Indicator
01
9F0A Unknown tag
0001050100000000
61 Application Template
4F Application Identifier (AID) – card
A0000003591010028001
87 Application Priority Indicator
01
9F0A Unknown tag
0001050100000000
61 Application Template
4F Application Identifier (AID) – card
D27600002547410100
87 Application Priority Indicator
01
9F0A Unknown tag
0001050100000000
There are 3 applications with 3 different AIDs available on the card:
A00000005945430100: Zentraler Kreditausschuss (ZKA) Germany Girocard Electronic Cash
A0000003591010028001: Euro Alliance of Payment Schemes s.c.r.l. – EAPS Belgium Girocard EAPS ZKA (Germany)
D27600002547410100: ZKA Germany Girocard ATM
Step 2: Now I'm reading the first AID using the "Select AID" command - the AID is 18 characters = 9 bytes long:
selectAidCommand: 00 A4 04 00 09 A0 00 00 00 59 45 43 01 00 00
selectAidResponse: 6F 47 84 09 A0 00 00 00 59 45 43 01 00 A5 3A 50 08 67 69 72 6F 63 61 72 64 87 01 01 9F 38 06 9F 02 06 9F 1D 02 5F 2D 04 64 65 65 6E BF 0C 1A 9F 4D 02 19 0A 9F 6E 07 02 80 00 00 30 30 00 9F 0A 08 00 01 05 01 00 00 00 00 90 00
Parsed response:
6F File Control Information (FCI) Template
84 Dedicated File (DF) Name
A00000005945430100
A5 File Control Information (FCI) Proprietary Template
50 Application Label
g i r o c a r d
87 Application Priority Indicator
01
9F38 Processing Options Data Object List (PDOL)
9F02 06
9F1D 02
5F2D Language Preference
d e e n
BF0C File Control Information (FCI) Issuer Discretionary Data
9F4D Log Entry
190A
9F6E Unknown tag
02800000303000
9F0A Unknown tag
0001050100000000
Step 3: get the processing options to read the card details
This is the first "tricky" point as it is not easy how to explain to get the right data for your card/AID. In the above response there is a section for the Processing Options Data Object List (PDOL) the card is requesting and the length of the fields - here we do have 2 fields with a length of 6 and 2 bytes, in total 8 bytes. The 8 bytes are just 8 "x00"s with the "header" 83 08, so the complete length is 10 bytes = x0A:
9F38 Processing Options Data Object List (PDOL)
9F02 06
9F1D 02
A more detailed explanation can be found here: https://stackoverflow.com/a/20810855/8166854
getProcessingOptionsCommand: 80 A8 00 00 0A 83 08 00 00 00 00 00 01 00 00 00
getProcessingOptionsResponse: 77 1E 82 02 19 80 94 18 18 01 01 00 20 01 01 00 20 04 04 00 08 05 05 01 08 07 07 01 08 03 03 01
Parsed response:
77 Response Message Template Format 2
82 Application Interchange Profile
1980
94 Application File Locator (AFL)
18010100 20010100 20040400 08050501 08070701 08030301
The most important part for the next step is the "Application File Locator (AFL)" - we need to read the file system with the data that are coded in these 4 byte blocks.
Step 4: read the records from the card
This is the part where I get lost when trying to read the card. You need to get the SFI and RECORD from the first 3 bytes of an AFL block and run a read record command. The following command reads the 4th sector of the AFL list - the command may work or not with your card and if it works you may get different data from your card.
WARNING: providing the response data to an internet form may reveal data like account number or
credit card number - my response is masked so the account number is not 1111111111:
readRecordCommand: 00 B2 05 0C 00
readRecordResponse: 70 38 5F 24 03 21 12 31 5A 0A 67 26 42 89 11 11 11 11 11 7F 5F 34 01 02 5F 28 02 02 80 9F 07 02 FF C0 9F 0D 05 FC 40 A4 80 00 9F 0E 05 00 10 18 00 00 9F 0F 05 FC 40 A4 98 00 90 00
Parsed response:
70 EMV Proprietary Template
5F24 Application Expiration Date
211231
5A Application Primary Account Number (PAN)
6726428911111111117F
5F34 Application Primary Account Number (PAN) Sequence Number
02
5F28 Issuer Country Code
0280
9F07 Application Usage Control
FFC0
9F0D Issuer Action Code – Default
FC40A48000
9F0E Issuer Action Code – Denial
0010180000
9F0F Issuer Action Code – Online
FC40A49800
I strongly recommend that you use a library for the steps 3 and 4; I'm using
https://github.com/devnied/EMV-NFC-Paycard-Enrollment
for this.
This is just a basic explanation for the first steps but now you get some useful responds from your card - good luck for your next steps.
A complete Android example app for the above mentioned library is here (disclaimer: I'm the author):
https://github.com/AndroidCrypto/Android-EMV-NFC-Paycard-Example

How to properly overwrite content of file using android storage access framework

>> Background
I want to use the SAF (Storage Access Frameword) to save data files of my app to the user's desired location on storage media. I first create the file at app dedicated folder and then copy it to the file that users has selected from file chooser dialog (the codes are comming later).
this procedure works perfectly for new files but for existing files though the file chooser warns for overwriting the file, the final file will not erased before writing.
By counting the bytes written and investigating the files using hex editor, the code writes correct bytes to the output stream but: if the existing file has more bytes than bytes to be written, the final overwritten file is corrupted (Not actually corrupted, please see the next sections for clarify) and if the existing has less bytes than bytes to be written, the final overwritten file is correct.
>> More Detail and codes
I use the codes bellow to show the issue (the jpg is as sample):
and i will try to work with two files:
file1.jpg 166,907 bytes
file2.jpg 1,323,647 bytes
file3.jpg The final file with variable size
First i will copy the file1 it to user selected folder with the name file3 (the destination file), then overwrite it with file2 and finaly i will overwrite it again with file1. see what are the codes and what happens:
Code for call the file chooser:
val intent = Intent(Intent.ACTION_CREATE_DOCUMENT).apply {
addCategory(Intent.CATEGORY_OPENABLE)
type = "image/jpeg"
}
startActivityForResult(intent, request)
Now at onActivityResult() i process the data as bellow:
contentResolver.openOutputStream(fileUri)?.use {output->
val input = FileInputStream(File(getExternalFilesDir(null),"Pictures/file1.jpg"))
// file1.jpg for first run, file2.jpg for 2nd run and file1.jpg again for 3rd run
copyStream(input, output)
}
And the code for copying the stream:
#Throws(IOException::class)
fun copyStream(input: InputStream, output: OutputStream) {
val buffer = ByteArray(1024)
var bytesRead = input.read(buffer)
while (bytesRead > 0) {
output.write(buffer, 0, bytesRead)
bytesRead = input.read(buffer)
}
input.close()
//The output will be closes by kotlin standard function "use" at previous code
}
Now at first run the file3.jpg is exactly the same as file1.jpg. also the file3.jpg is same as file2.jpg at the second run. but at third run which overwrites file3.jpg with content of file1.jpg (Which has kess bytes than file3.jpg) the size of file3.jpg remains 1,323,647 bytes and the first 166,907 bytes are same as file1.jpg and the remaining bytes until 1,323,647 are same as file2.jpg that was written at 2nd run.
this is content of the files in hex:
file1.jpg
0000:0000 | FF D8 FF E1 09 49 45 78 69 66 00 00 49 49 2A 00 | ÿØÿá.IExif..II*.
0000:0010 | 08 00 00 00 09 00 0F 01 02 00 06 00 00 00 7A 00 | ..............z.
...
0002:8BE0 | 56 5E 2A EC C7 36 6D B1 57 1C D5 CD 95 8A BB 2F | V^*ìÇ6m±W.ÕÍ..»/
0002:8BF0*| 36 6C 55 AD F2 F3 65 60 43 FF D9* | 6lU.òóe`CÿÙ
file2.jpg
0000:0000 | FF D8 FF E0 00 10 4A 46 49 46 00 01 01 00 00 01 | ÿØÿà..JFIF......
0000:0010 | 00 01 00 00 FF E1 01 48 45 78 69 66 00 00 49 49 | ....ÿá.HExif..II
...
0002:8BC0 | F2 07 23 D4 57 CA 7E 13 FD A9 23 B5 86 2D 3E 4D | ò.#ÔWÊ~.ý©#µ.->M
0002:8BD0 | 66 7B 58 D1 42 A3 4D 6A 57 80 38 C9 CF EB 5E 93 | f{XÑB£MjW.8ÉÏë^.
0002:8BE0 | E1 3F DA 36 CA EA 10 2E 7C 49 0B C4 E3 21 F6 8C | á?Ú6Êê..|I.Äã!ö.
0002:8BF0*| 9F D6 BB 63 8B A3 86 D5 34 B5 D9*E8 D2 E9 D7 AE | .Ö»c.£.Õ4µÙèÒé×®
0002:8C00 | B7 34 9F B5 85 18 C6 B5 DF 2E FA 6B AD B6 5D BC | ·4.µ..Ƶß.úk.¶]¼
0002:8C10 | F7 3D 6E F3 C3 50 6B 56 32 D9 CC 14 AB AE 30 C3 | ÷=nóÃPkV2ÙÌ.«®0Ã
...
0014:3260 | E8 8B 0A CE 4E 47 AD 4A 92 B2 E4 E6 8B 3B 7F 34 | è..ÎNG.J.²äæ.;.4
0014:3270 | 1C 55 D8 6C 14 83 BA 88 AB 98 46 4D 33 FF D9 | .UØl..º.«.FM3ÿÙ
file3.jpg (After the 3rd run)
0000:0000 | FF D8 FF E1 09 49 45 78 69 66 00 00 49 49 2A 00 | ÿØÿá.IExif..II*.
0000:0010 | 08 00 00 00 09 00 0F 01 02 00 06 00 00 00 7A 00 | ..............z.
...
0002:8BD0 | D9 B1 43 BA E6 39 B7 CD 8A B5 97 9B 36 29 76 5E | Ù±Cºæ9·Í.µ..6)v^
0002:8BE0 | 56 5E 2A EC C7 36 6D B1 57 1C D5 CD 95 8A BB 2F | V^*ìÇ6m±W.ÕÍ..»/
//content of file1 continues with content of file2 (Next line)
0002:8BF0*| 36 6C 55 AD F2 F3 65 60 43 FF D9*E8 D2 E9 D7 AE | 6lU.òóe`CÿÙèÒé×®
0002:8C00 | B7 34 9F B5 85 18 C6 B5 DF 2E FA 6B AD B6 5D BC | ·4.µ..Ƶß.úk.¶]¼
0002:8C10 | F7 3D 6E F3 C3 50 6B 56 32 D9 CC 14 AB AE 30 C3 | ÷=nóÃPkV2ÙÌ.«®0Ã
0002:8C20 | 8C F3 83 5E 55 3D 86 A1 F0 EB C5 72 E9 C6 62 E2 | .ó.^U=.¡ðëÅréÆbâ
...
0014:3260 | E8 8B 0A CE 4E 47 AD 4A 92 B2 E4 E6 8B 3B 7F 34 | è..ÎNG.J.²äæ.;.4
0014:3270 | 1C 55 D8 6C 14 83 BA 88 AB 98 46 4D 33 FF D9 | .UØl..º.«.FM3ÿÙ
As you can see the file3 starts with content of file1 and after the final bytes of file1 (FF D9) at third group at line 0002:8BF0 it continues with content of file2 (E8 D2) (Star points)
I tested the process of copying same files directly inside app's dedicated folder but the results were correct with correct file3 for all three runs. the problem was just for SAF.

I Found the answer after three days of searching and a day after asking here. I didn't remove the question as others may encounter the same problem.
The nature of the problem was taking me to the wrong way. It does not just arise from copying streams but also when writing e.g. 4 bytes (bbbb) to overwrite file with 8 bytes (aaaaaaaa). it creates a file with first 4 new bytes and then 4 old bytes! (bbbbaaaa).
So the answer is in FileOutputStream().
have the size of bytes that is written to the file (input.channel.size()) or (output.cannel.position()) and truncate remaining bytes (output.channel.truncate(size)).
as the code in question, i changed to:
contentResolver.openOutputStream(fileUri)?.use {output->
output as FileOutputStream
FileInputStream(File(getExternalFilesDir(null),"Pictures/file1.jpg")).use{input->
copyStream(input, output)
// this new line removes bytes beyond the input file size
output.channel.truncate(input.channel.size())
// or
// output.channel.truncate(output.channel.position())
}
}
UPDATE 2019/09/15
Thanks to the first comment from #mjanssen you can also get the same result by putting the output.channel.truncate(0) just before copying the file, copyStream(input, output):
contentResolver.openOutputStream(fileUri)?.use {
output-> output as FileOutputStream
FileInputStream(File(getExternalFilesDir(null),"Pictures/file1.jpg")).use{input->
output.channel.truncate(0)
copyStream(input, output)
}
}
That's it

Having wasted several days trying to understand why I was getting corrupt data at the end of the file when I was overwriting it, I thankfully found this post. My testing showed the same results as the OP. When writing new contents that are shorter than the existing file contents, residual data is left at the end of the file - leading to a 'corrupt' file. My, incorrect, assumption was that writing to the file would completely remove/overwrite the existing contents.
Using the Android Developers example on 'Edit A Document' ...(https://developer.android.com/training/data-storage/shared/documents-files#edit) highlighted the problem, since using their code example only overwrites the first record. IE a 100 line text file (for example), is still a 100 line text file if you use their code example, only the first record is changed. Useful if you want to just modify specific records 'in situ', but not if you want to completely replace the file contents !
Here is a modified, java version, of the sample code that ensures that the output is 'emptied' first before being overwritten (employing the approach described by the OP above)
private void overwriteDocument(Uri uri) {
try {
ParcelFileDescriptor pfd = getActivity().getContentResolver().
openFileDescriptor(uri, "w");
FileOutputStream fileOutputStream =
new FileOutputStream(pfd.getFileDescriptor());
// Use this code to ensure that the file is 'emptied'
FileChannel fChan=fileOutputStream.getChannel();
fChan.truncate(0);
fileOutputStream.write(("Overwritten at " + System.currentTimeMillis() +
"\n").getBytes());
// Let the document provider know you're done by closing the stream.
fileOutputStream.close();
pfd.close();
} catch (FileNotFoundException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
}
}

The easiest way to achieve the desired result is to open the file in truncate mode.
contentResolver.openOutputStream(uri, "wt")?.use { outputStream ->
// Write to outputStream
}
Relevant documentation:
ContentResolver.openOutputStream(Uri, String)
file modes

Reading Mifare Classic returns strange characters

When reading a MIFARE card with Android and converting the data to UTF-8 I get strange characters like �.
I'm trying to build an application that can read some kind of ID card we're using. The problem now is that I get weird characters between words and some words are split between blocks so how can I safely get a word I'm looking for?
For instance my readings is something like this:
43224���19032019��
at block 2 sektor 2 bindex :8
and with splitting where rest of the number starting with 19 is at a new block:
�me Name���M���19
at block 1 sektor 1 bindex :4
930402���NO934951
at block 2 sektor 1 bindex :4
c5 42 4e 49 44 00 07 4f 4f 4f 4f 4f 4f 00 4b 42 "Åbnid" "OOOOOO" "KB"
44 44 44 20 44 44 44 44 44 00 82 4d 00 c9 31 39 "DDD DDDDD" "M" "19"
39 34 34 33 34 32 00 d0 4e 4f 39 36 36 36 35 31 "944342" "NO966651"
00 00 00 00 00 00 70 f7 88 00 00 00 00 00 00 00
30 32 32 20 20 41 53 00 d3 54 4f 54 41 4c 20 4b "022" "AS" "Total k"
4f 4e 54 52 4f 4c 4c 20 41 53 20 00 c9 30 32 38 "ONTROLL AS" "028"
37 30 34 33 33 00 c9 32 30 32 31 30 32 31 31 00 "70433" "20210211"
00 00 00 00 00 00 70 f7 88 00 00 00 00 00 00 00
This is how I read from the card:
Tag tagFromIntent = intent.getParcelableExtra(NfcAdapter.EXTRA_TAG);
MifareClassic mfc = MifareClassic.get(tagFromIntent);
Here is my code I use for reading inside a for loop:
data = mfc.readBlock(bIndex + block);
and then for converting data to UTF8 for printing I use:
public String convertByteArrayToUTF8(byte[] bytes){
String encoded = null;
try {
encoded = new String(bytes, StandardCharsets.UTF_8);
}
catch (Exception e){
encoded = new String(bytes, Charset.defaultCharset());
}
return encoded;
}
I've tried with ASCII, UTF-16 etc with no luck.

First of all LOL for the question heading. I was in the same situation when I was a newbie. There is no tutorial online that provides you the exact code to read data from a Mifare classic card.
First understand the memory structure of the Mifare cards.
The memory of Mifare Classic divided into sectors, which are also divided into blocks of 16 bytes.
The MIFARE Classic 1K card has 16 sectors, each of which are divided into four blocks. If we do the math, we can figure out how the memory structure would be like: 16 bytes (1 block) * 4 blocks * 16 sectors = 1024 bytes.
The MIFARE Classic 4K card has 40 sectors, 32 of which are divided into four blocks and the remaining 8 are divided into 16 blocks. 16 bytes (1 block) * 4 blocks * 32 sectors + 16 bytes (1 block) * 16 blocks * 8 sectors = 4096 bytes. The memory structure is as follows:
The number on the blocks indicates its index. Each sector is protected by the site key written in the last block of the sector. For example, block 3 contains the site key for sector 1 and block 7 for sector 2. The last block in each sector also contains access conditions information such as “write”, “read” and “read & write”. The following figure demonstrates how the last block consists of:
Moreover, the data written in the card is binary i.e; 0 & 1.
Now, the steps you need to follow to read the data are:
step1: check whether the device support NFC or not.
step2: check if the device has NXP chip (especially for reading Mifare classic cards).
step3: instantiate the NFC manager and NFC adpater & define the techlist of card that you want to read.
step4: ask permission to access device NFC.
step5: create a intent to detect card and specify the MIME type you want to read(in most cases it is all MIME types).
step6: enable and disable foreground dispatch of adapter in onResume() and onPause() so that your app gets the priority to read the card when your activity is in foreground.
step7: When card comes in contact to device, you can get the tag information from intent.getParcelableExtra(NfcAdapter.EXTRA_TAG);
step8: read the card information i.e; card type, tech list etc..
step9: to read the data in the card you need to connect to the card via tag info retrieved above.
step10: iterate through all the sectors. Authenticate each sector with the default key
//https://developer.android.com/reference/android/nfc/tech/MifareClassic.html#authenticateSectorWithKeyA(int,%20byte[])
step11: on successful authentication read the binary data in the blocks of each sector.
step12: convert the binary data to string data so that we can read it.
step13: That's all, do whatever you want to do with the data.
surprise! get the complete working code at my github repositiory here:
https://github.com/codes29/RFIDReader
Note: I empathised how you feeling as a newbie and got this task where there is no proper tutorial for the same. So I updated my code that I wrote after a lot of struggle for days.
Here's the sample that you'll get after successful authentication and reading the data. The card that I scanned is empty as per now. But if there is data here then it'll surely be here instead of 0's.
Cheers! Happy coding bro!

So the data on your tag (excluding the sector trailers looks somewhat like that:
C5 42 4E 49 44 00 07 4F 4F 4F 4F 4F 4F 00 4B 42 ÅBNID..OOOOOO.KB
44 44 44 20 44 44 44 44 44 00 82 4D 00 C9 31 39 DDD DDDDD.‚M.É19
39 34 34 33 34 32 00 D0 4E 4F 39 36 36 36 35 31 944342.ÐNO966651
30 32 32 20 20 41 53 00 D3 54 4F 54 41 4C 20 4B 022 AS.ÓTOTAL K
4F 4E 54 52 4F 4C 4C 20 41 53 20 00 C9 30 32 38 ONTROLL AS .É028
37 30 34 33 33 00 C9 32 30 32 31 30 32 31 31 00 70433.É20210211.
This seems to be some form of structured data. Simply converting the whole binary blob into a UTF-8 (or ASCII) encoded string doesn't make much sense. Instead, you will need to reverse engineer the way that the data is structured (or, even better, you try to obtain the specification from the system manufacturer).
From what I can see, it looks as if that data consisted of multiple null-terminated strings embedded into some compact (Tag)-Length-Value format. The first byte seems to be the tag(?) + length, so we have
C5 Length = 5
42 4E 49 44 00 "BNID"
07 Length = 7
4F 4F 4F 4F 4F 4F 00 "OOOOOO"
4B Length = 11
42 44 44 44 20 44 44 44 44 44 00 "KBDDD DDDDD"
82 Length = 2
4D 00 "M"
C9 Length = 9
31 39 39 34 34 33 34 32 00 "19944342"
D0 Length = 16
4E 4F 39 36 36 36 35 31 30 32 32 20 20 41 53 00 "NO966651022 AS"
D3 Length = 19
54 4F 54 41 4C 20 4B 4F 4E 54 52 4F 4C 4C 20 41 53 20 00 "TOTAL KONTROLL AS "
C9 Length = 9
30 32 38 37 30 34 33 33 00 "02870433"
C9 Length = 9
32 30 32 31 30 32 31 31 00 "20210211"
The first byte could, for instance, be split into tag and length like this: TTTL LLLL (upper 3 bits encode the tag, lower 5 bits encode the length of the following value). This would give the following tags
0x6 for "BNID", "19944342", "NO966651022 AS", "TOTAL KONTROLL AS ", "02870433", and "20210211"
0x0 for "OOOOOO"
0x2 for "KBDDD DDDDD"
0x4 for "M"
Hence, the split between tag and length might also be TTLL LLLL (upper 2 bits encode the tag, lower 6 bits encode the length of the following value).
Unfortunately, the format doesn't resemble any of the popular formats that I'm aware of. So you could just continue your reverse engineering by comparing multiple different cards and by deriving meaning from the values.
So far, in order to decode the above, you would start by reading the first byte, extract the length from that byte, cut that amount of follow-up bytes and convert them into a string (based on the sample that you provided, ASCII encoding should do). You can then continue with the next byte, extract the length information from it, ...

Stream h264 to javafx possibly using javacv/ffmpeg

I'm really stuck on getting a video stream to play on a java fx project.
-- Short version:
I'm streaming h264/avcc flavor video from an android phone to a desktop computer. However javafx doesn't have an easy solution for displaying stream. I'm attempting to use javacv / ffmpeg in an attempt to make this work. However I am getting errors from ffmpeg.
1) Is there a better way to display streaming video on javafx?
2) Do you have a sample project or good tutorial for javacv ffmpegframegrabber?
3) I think I may be missing some small detail in mycode but Im not sure what i would be.
-- Longer Version:
1) On the android end Im getting video using mediarecorder. In order to get the sps/pps info I record and save a small movie to the device and then parse the sps and pps data.
2) Next, on the android, I split up the nalus to meet MTU req and send them over a udp connection to my desktop
3)On my desktop I reassmble the nalus( or trash them if they loose data) and feed those to an input stream that I gave to the framegreabber constructor.
-- The Code and Logs:
The errors are long and numerous depending on the flavor I feed it. Here are two separate examples which are usually repeated at great length
[h264 # 0000020225907a40] non-existing PPS 0 referenced
[h264 # 0000020225907a40] decode_slice_header error
[h264 # 0000020225907a40] no frame!
[h264 # 00000163d8637a40] illegal aspect ratio
[h264 # 00000163d8637a40] pps_id 3412 out of range
[AVBSFContext # 00000163e28a0e00] Invalid NAL unit 0, skipping.
!! One big caveat that I am aware off is that I have not implemented timestamps
which I created on the android device when feeding ffmpeg. I think it should still show distorted images without this though
Because I have spent all day guessing and trying I have several "flavors" of data I have shoved through. I am only showing the first section of each nal which I believe if correct would at least show a garbage image as long as my sps and pps are right
sps: 67 80 80 1E E9 01 68 22 FD C0 36 85 09 A8
pps: 68 06 06 E2
Below is annex B style.
These were each prefixed with either 00 00 01 and 00 00 00 01
Debug transfer 65 B8 40 0B E5 B8 7B 80 5B 85
Debug transfer 41 E2 20 7A 74 34 3B D6 BE FA
Debug transfer 41 E4 40 2F 01 E0 0C 06 EE 91
Debug transfer 41 E6 60 3E A1 20 5A 02 3C 6D
Debug transfer 41 E8 80 13 B0 B9 82 C3 03 F4
Debug transfer 41 EC C0 1B A3 0C 28 F1 B0 C8
Debug transfer 41 EE E0 1F CE 07 30 EE 05 06
Debug transfer 41 F1 00 08 ED 80 9C 20 09 73
Debug transfer 41 F3 20 09 E9 00 86 60 21 C3
VideoDecoderaddPacket type: 24
Debug transfer 67 80 80 1E E9 01 68 22 FD C0
Debug transfer 68 06 06 E2
Debug transfer 65 B8 20 00 9F 80 78 00 12 8A
Debug transfer 41 E2 20 09 F0 1E 40 7B 0C E0
Debug transfer 41 E4 40 09 F0 29 30 D6 00 AE
Debug transfer 41 E6 60 09 F1 48 31 80 99 40
[h264 # 000001c771617a40] non-existing PPS 0 referenced
Here I tried Avcc style. You can see the first line is the combination of the sps pps followed by idr and then repeated non idr
Debug transfer 18 00 0E 67 80 80 1E E9 01 68
Debug transfer 00 02 4A 8F 65 B8 20 00 9F C5
Debug transfer 00 02 2F DA 41 E2 20 09 E8 0F
Debug transfer 00 02 2C 34 41 E4 40 09 F4 20
Debug transfer 00 02 4D 92 41 E6 60 09 FC 2B
Debug transfer 00 02 47 02 41 E8 80 09 F0 72
Debug transfer 00 02 52 50 41 EA A0 09 EC 0F
Debug transfer 00 02 58 8A 41 EC C0 09 FC 6F
Debug transfer 00 02 55 F9 41 EE E0 09 FC 6E
Debug transfer 00 02 4D 79 41 F1 00 09 F0 3E
Debug transfer 00 02 4D B6 41 F3 20 09 E8 64
The following class is where I try to get javacv/ffmpeg to show the video. I dont think its an ideal solution and am researching canvasfram as a replacement to the image view.
public class ImageDecoder {
private final static String TAG = "ImageDecoder ";
private ImageDecoder(){
}
public static void streamImageToImageView(
final ImageView view,
final InputStream inputStream,
final String format,
final int frameRate,
final int bitrate,
final String preset,
final int numBuffers
)
{
System.out.println("Image Decoder Starting...");
try( final FrameGrabber grabber = new
FFmpegFrameGrabber(inputStream))
{
final Java2DFrameConverter converter = new Java2DFrameConverter();
grabber.setFrameNumber(frameRate);
grabber.setFormat(format);
grabber.setVideoBitrate(bitrate);
grabber.setVideoOption("preset", preset);
grabber.setNumBuffers(numBuffers);
System.out.println("Image Decoder waiting on grabber.start...");
grabber.start(); //---- this call is blocking the loop
System.out.println("Image Decoder Looping---------------------------
-------- hit stop");
while(!Thread.interrupted()){
//System.out.println("Image Decoder Looping");
final Frame frame = grabber.grab();
if (frame != null){
final BufferedImage bufferedImage =
converter.convert(frame);
if (bufferedImage != null){
Platform.runLater(() ->
view.setImage(SwingFXUtils.toFXImage(bufferedImage, null)));
}else{
System.out.println("no buf im");
}
}else{
System.out.println("no fr");
Thread.currentThread().interrupt();
}
}
}catch (Exception e){
System.out.print(TAG + e);
}
}
}
Any help is greatly appreciated.

So I had two problems.
The first was that my sps pps parsing method had a mistake. Notice the 2nd and 3rd bytes are same
The second was I accidentally over sized a buffer and created large 0x00 padded areas which emulated start codes.
This was a big project for me and I want to help other. Please visit my website where I wrote a lengthy multi-part discussion about streaming h264

Android isoDep.transceive on Generate AC command always returns 6D00 unsupported

I'm trying to emulate a PoS, point of sale :), system and complete a transaction with Google wallet running on a 2013 Nexus 7 (no secure element) v4.4.2.
My PoS prototype is also running on a 2013 Nexus 7 v4.4.2.
I'm able to get NFC responses from the 2PAY_SYS_DDF01 request.
I'm able to select the MasterCard application ID.
I'm able to get Processing Options.
When I Read Records it doesn't look like Google wallet is returning all the mandatory EMV fields.
And finally when I request the Generate AC command it always returns 6D00 unsupported.
Area of the code that is a problem:
//set P1 to '40', to request an Transaction Certificate (offline transaction)
//and execute the Generate AC command.
byte[] generateAC = new byte[] {(byte)0x80, (byte)0xAE, 0x40, 0x00,
0x1D, //data length
0x00, 0x00, 0x00, 0x00, 0x00, 0x01, //amount1
0x00, 0x00, 0x00, 0x00, 0x00, 0x00, //amount2
0x08, 0x40, //term country
0x00, 0x00, 0x00, 0x00, 0x00, //tvr
0x09, 0x62, //tx curr
0x14, 0x04, 0x1B, //tx date
0x00, //tx type
0x12, 0x34, 0x56, 0x78, //random
0x00}; //le
response = isoDep.transceive(generateAC);
I was following Tim Beckers video
https://www.youtube.com/watch?v=qqobg1-HrfY
approx. 46 minutes in.
and Tim Beckers code sample:
https://github.com/a2800276/29c3/blob/master/smartshell.rb
UPDATE:
The response to the GPO command (80 a8 00 00 02 83 00 00) is
770a820200009404080101009000
77 Response Message Template Format 2
82 Application Interchange Profile
0000
94 Application File Locator (AFL)
08010100
UPDATE:
I tried:
byte[] computeCC = new byte[] {
(byte)0x80, // CLA = proprietary
(byte)0x2A, // INS = COMPUTE CRYPTOGRAPHIC CHECKSUM
(byte)0x8E, // P1
(byte)0x80, // P2
(byte)0x04, // Lc
(byte)0x00, 0x00, (byte)0x00, (byte)0x99, // Unpredicatable Number (numeric)
(byte)0x00, // Le
};
response = isoDep.transceive(computeCC);
hoping to at least get an error that indicated the wrong unpredictable number but I got 6700 incorrect length returned.
Earlier I got a response to Read Record 00 B2 01 0C 00
Part of that response is
Card Authentication Related Data [9F69]:
Data (Binary): XX XX XX XX XX XX XX XX XX XX XX XX XX XX XX
fDDA Version Number: 9F
(Card) Unpredictable Number: 6A 04 9F 7E
So I tried that Unpredictable number
byte[] computeCC = new byte[] {
(byte)0x80, // CLA = proprietary
(byte)0x2A, // INS = COMPUTE CRYPTOGRAPHIC CHECKSUM
(byte)0x8E, // P1
(byte)0x80, // P2
(byte)0x04, // Lc
(byte)0x6A, 0x04, (byte)0x9F, (byte)0x7E, // Unpredicatable Number (numeric)
(byte)0x00, // Le
};
response = isoDep.transceive(computeCC);
But I still get 6700 wrong length
UPDATE:
The response to READ RECORD (Record = 1, SFI = 1) command (00 B2 01 0C 00)
is:
70 7c 9f 6c 02
00 01 9f 62 06
00 00 00 00 00
38 9f 63 06 00
00 00 00 03 c6
56 29 42 35 33
39 36 XX XX XX
XX 31 XX XX 39
XX XX XX XX 5e
20 2f 5e 31 34
30 37 31 30 31
34 30 31 30 30
30 30 30 30 30
30 30 30 9f 64
01 04 9f 65 02
00 38 9f 66 02
03 c6 9f 6b 13
53 96 XX XX 1X
X9 XX XX d1 40
71 01 40 10 00
00 00 00 0f 9f
67 01 04 9f 69
0f 9f 6a 04 9f
7e 01 9f 02 06
5f 2a 02 9f 1a
02 90 00
--------------------------TLV-------------------------------
Output:
READ RECORD Response Message Template [70]:
Card Transaction Qualifiers (CTQ) [9F6C]:
Data (Binary): 00 01
Bit flags set:
1Bxb8: 0 - Online PIN not Required
1Bxb7: 0 - Signature Not Required
1Bxb6: 0 - Not applicable: Go Online if Offline Data Authentication Fails and Reader is online capable.
1Bxb5: 0 - Not applicable: Switch Interface if Offline Data Authentication fails and Reader supports VIS.
1Bxb4: 0 - Not applicable: Go Online if Application Expired
1Bxb3: 0 - Not applicable: Switch Interface for Cash Transactions
1Bxb2: 0 - Not applicable: Switch Interface for Cashback Transactions
1Bxb1: 0 - RFU
2Bxb8: 0 - Consumer Device CVM not Performed
2Bxb7: 0 - Card doesn't support Issuer Update Processing at the POS
2Bxb6: 0 - RFU
2Bxb5: 0 - RFU
2Bxb4: 0 - RFU
2Bxb3: 0 - RFU
2Bxb2: 0 - RFU
2Bxb1: 1 - RFU
PCVC3 (Track1) [9F62]:
Data (Binary): 00 00 00 00 00 38
Offline Counter Initial Value [9F63]:
Data (Binary): 00 00 00 00 03 C6
Track 1 Data [56]:
Data (ASCII): B5396XXXXXXXXXXXX^ /^14071014010000000000
NATC (Track1) [9F64]:
Data (Binary): 04
PCVC3 (Track2) [9F65]:
Data (Binary): 00 38
Terminal Transaction Qualifiers (TTQ) [9F66]:
Data (Binary): 03 C6
Card CVM Limit [9F6B]:
Data (Binary): 53 96 48 50 17 69 62 32 D1 40 71 01 40 10 00 00 00 00 0F
MSD Offset [9F67]:
Data (Binary): 04
Card Authentication Related Data [9F69]:
Data (Binary): 9F 6A 04 9F 7E 01 9F 02 06 5F 2A 02 9F 1A 02
fDDA Version Number: 9F
(Card) Unpredictable Number: 6A 04 9F 7E
Card Transaction Qualifiers: 01 9F
----------------------------------------

The response to the GET PROCESSING OPTIONS command indicates the following Application Interchange Profile (AIP):
82 Application Interchange Profile
0000
Google Wallet is basically a MasterCard (EMV contactless kernel 2), so decoding the AIP according to the rules of Kernel 2 results in the following:
Byte 1, b7 = 0: no SDA supported
b6 = 0: no DDA supported
b5 = 0: no cardholder verification supported
b4 = 0: no terminal risk management to be performed
b3 = 0: no issuer authentication supported
b2 = 0: no on-device cardholder verification supported
b1 = 0: no CDA supported
Byte 2, b8 = 0: no EMV mode supported
The important part is byte 2, bit 8: It indicates that your card does not support EMV mode. Hence, your card/Google Wallet is a PayPass card that supports only Mag-Stripe mode. Therefore, you cannot authenticate transactions using GENERATE AC. Instead, you can only let the card generate dynamic card verification codes (CVC3) using COMPUTE CRYPTOGRAPHIC CHECKSUM:
byte[] computeCC = new byte[] {
(byte)0x80, // CLA = proprietary
(byte)0x2A, // INS = COMPUTE CRYPTOGRAPHIC CHECKSUM
(byte)0x8E, // P1
(byte)0x80, // P2
(byte)0x04, // Lc
(byte)0xWW, (byte)0xXX, (byte)0xYY, (byte)0xZZ, // Unpredicatable Number (numeric)
(byte)0x00, // Le
};
response = isoDep.transceive(computeCC);
Note that the data field of the COMPUTE CRYPTOGRAPHIC CHECKSUM command must be filled with values according to the UDOL (in case there is no UDOL, the default UDOL is 9F6A04, indicating the unpredictable number, numeric).
The unpredictable number (numeric) is a BCD coded number in the range that is defined by the mag-stripe data file (see the AFL). In the past, for Google Wallet, this was a value between 0 and 99 (i.e. WW='00', XX='00', YY='00', ZZ='00'..'99').
UPDATE:
The data read from the card decodes as follows:
70 7c
9f6c 02 Mag-stripe application version number = Version 1
00 01
9f62 06 Track 1 bit map for CVC3
00 00 00 00 00 38
9f63 06 Track 1 bit map for UN and ATC
00 00 00 00 03 c6
56 29 Track 1 data
42 ISO/IEC 7813 structure "B" format
35333936 XXXXXXXX 31XXXX39 XXXXXXXX PAN (ASCII)
5e Field separator "^"
202f Cardholder name " /" (empty, see MC requirements)
5e Field separator "^"
31343037 Expiry date "14"/"07"
313031 Service code "101"
34303130303030303030303030 Track 1 discretionary data
9f64 01 Track 1 number of ATC digits
04
9f65 02 Track 2 bit map for CVC3
00 38
9f66 02 Track 2 bit map for UN and ATC
03 c6
9f6b 13 Track 2 data
5396 XXXX 1XX9 XXXX PAN (BCD)
d Field separator
1407 Expiry date
101 Service code
4010000000000 Track 2 discretionary data
f Padding
9f67 01 Track 2 number of ATC digits
04
9f69 0f UDOL
9f6a 04 Unpredictable number (numeric)
9f7e 01 Mobile support indicator
9f02 06 Amount authorized (numeric)
5f2a 02 Transaction currency code
9f1a 02 Terminal country code
So the card does provide a UDOL. Therefore, the COMPUTE CRYPTOGRAPHIC CHECKSUM command has to be adapted accordingly:
byte[] computeCC = new byte[] {
(byte)0x80, // CLA = proprietary
(byte)0x2A, // INS = COMPUTE CRYPTOGRAPHIC CHECKSUM
(byte)0x8E, // P1
(byte)0x80, // P2
(byte)0x0F, // Lc
// 9f6a 04 Unpredictable number (numeric)
(byte)0x00, (byte)0x00, (byte)0x00, (byte)0x12, // two digits according to UN/ATC bit map and number of ATC digits: 6 - 4 = 2
// 9f7e 01 Mobile support indicator
(byte)0x00, // no offline PIN required, no mobile support
// 9f02 06 Amount authorized (numeric)
(byte)0x00, (byte)0x00, (byte)0x00, (byte)0x00, (byte)0x01, (byte)0x00, // 1.00
// 5f2a 02 Transaction currency code
(byte)0x09, (byte)0x78, // Euro
// 9f1a 02 Terminal country code
(byte)0x00, (byte)0x40, // Austria
(byte)0x00, // Le
};
response = isoDep.transceive(computeCC);