Develop Reference

ESC/POS Termal printer UTF-8 charset set up

I am trying to print some currency with Citaq v8 (it has a termal printer), but it prints ?, because of the printer character set.
I have byte array (UTF-8) data to print. But the printer has default some character set. How can I change the default character set programatically(like using byte array)? I need to change printer charset settings.
Device: Citaq v8 (the device has a termal printer)
PS: I could not find any developer docs.

Perhaps UTF-8 is not supported as a character set/code table.
Even EPSON is supported only on some models.
FS ( C <Function 48>
Select character encode system
ASCII FS ( C pL pH fn m
Hex 1C 28 43 02 00 30 m
Decimal 28 40 67 2 0 48 m
TM-P20
1, 49 ASCII (ISCII) ISCII: Indian Script Code for Information Interchange
2, 50 UTF-8 Unicode encoding system
TM-m30II, TM-m30II-H, TM-m30II-NT
1, 49 1-byte character encoding Non-Unicode encoding method (ASCII [extended], Shift JIS, Big5, GB2312, GB18030, KS C, etc.)
2, 50 UTF-8 Unicode encoding system
You need to set the printer code page using the following two ESC/POS commands, and the character string to be requested for printing must be encoded and converted according to the code page before sending.
ESC t
Select character code table
ASCII ESC t n
Hex 1B 74 n
Decimal 27 116 n
ESC R
Select an international character set
ASCII ESC R n
Hex 1B 52 n
Decimal 27 82 n
Alternatively, you can create all the page contents to be printed with a Bitmap image and print it with one of the ESC/POS commands related to image printing.
However, it will be slow.

Print strings on Apex3 using printer command language

How to print strings on Apex3 using printer command language or SDK?
I created a library for android application to print Bitmap images on mobile thermal printers:
Apex3, 3nStar, PR3, Bixolon, Sewoo LK P30 using appropriate SDKs.
It works fine but pretty slow, every ticket of 30 cm length takes 20-40 secs to print (it depends on type of printer).
For 3nStar and Apex3 I started to do improvements to speed up a printing.
For 3nStar I developed a class which prints a header logo and formatted text with spanish characters.
(different alignments and different fonts for different parts of a ticket)
so, the ticket looks very similar as Bitmap image but a printing time is only 6 secs.
Now I have to do the same but with Apex3. And here I got stuck.
How I do it on 3nStar for strings:
I send in outputStream bytes which are commands for printer what to do.
outputStream.write(some_bytes)
First command always is
{0x1b, 0x74, 40} //Esc t ( -- [ISO8859-15 (Latin9)]
to print spanish characters.
Then, in a loop, for n strings:
I choose a font
{0x1B, 0x21, 0x00}//Esc ! 0 -- 0 is normal, 8 is bold etc.
where changing last byte I print different fonts: normal, bold, two height, two width and combined fonts.
Next I choose an alignment
{0x1b, 0x61, 48} //Esc a 48 for left, 49 for center, 50 for right
Then I convert a string in bytes using ISO_8859_1 to print spanish characters and also write in outputStream.
outputStream.write(messageString.getBytes(StandardCharsets.ISO_8859_1))
And last byte to send is
{0x0a} // Move on next line
And the above approach doesn't work with Apex3, also I failed using
http://www.old.adtech.pl/upload/Extech_Printer_Command_Language%20Rev_H_05062009.pdf
even though on page 1 of that book is written that is fit for Apex3.
I think I miss something, I start to see how to do it using some SDK feature of Android_SDK_ESC_V1.01.17.01PRO.jar
but I would prefer to do that using direct writing of bytes.

Answers I found from this
manual
Shortly differences with the approach that I described for 3nStar are:
1)Before printing set a char set
ESC F 1 //Esc t - for 3nStar
2)Set a font for text, for example
ESC K 1 3 CR //ESC F 1 - for 3nStar
3)Send a line of text with alignment
For 3nStar I can use an alignment command before sending a text, like
ESC 1 49 //Centering
But for Apex3 I have to know a line length which depends on type of font, also a length of printing string,
then I get
freeSpace = (lineLength - printingString)
and set spaces at the begining of a line (right alignment),
at the end (left alignment) or devide them (centering).
So, for both types of printers I use the same logic which differs only in 3 places.
It is simplified explanation as a real code includes several classes with hundreds lines of code.

Parse only a specific part of image with Tesseract

I am trying to use Tesseract OCR on Android to read the state of a gas meter when you take a picture of it:
This is the output when I parse this image:
vb"
22% BK-G4T ||||||||I||||I|||ii\|||\
’ 64 2007
22?: 06.0"! 'm'lm Mm. 23212274 ,
v 2,0 dm’ 1
pmn 0_5 bar tm ~25°C v‘40"(1 I
1amp é 0_o1m’ sb15°cl :Sp 20°c l
'I ELSTEQ~I¢¢>>InstrogwnSs HB Z _ 18 _ 1013 . ‘
a, 069373593435- 3 I
i'23212214 Y _ w w V'
g
The idea is to extract the first 5 digits of the state of the gas meter ( 06937 on this image ).
My question is, is there a way to train Tesseract to only parse this part of the image? Absolute coordinates are not an option since every picture would be different. I am guessing the best logic would be something like: parse only white numbers on black background.

By changing the page segmentation mode (psm), tesseract 4.00.00 alpha is able to read the meter line characters correctly as 06937598-m3 apart from other characters.
The command used is:
tesseract meter.jpg output --psm 11 -l eng
--psm 11 means to recognize "Sparse text. Find as much text as possible in no particular order".
Here is the output file with showing all the ASCII control characters.
If --psm 11 works on other meter images, then you could just need to search -m3 at the end of the line to extract the who meter line characters. With that, you can get the first 5 digits right away.
Hope this help.

Improve accuracy of Tesseract

I am using tess-two (Android port of Tesseract OCR engine). I am OCR'ing bills and receipts. I am using OpenCV 3.0 to preprocess the image. I have successfully made a bill into a binary image along with adaptive thresholding.
Original image:
Binary threshold of original:
Tesseract output.
PH:26051246/26145398
TIN=276302417620
CQSH/BILL
NO 009762 0 m 0 09-07-2015
DESCRIPTION on mm mu
_.__________
m 3.990 75.18 zoo-00
_____\
CASH 300-00
m: YOU----UISIT 9691!!
w TIN:276302417620
-\ c1 09:27:58 we no. 0
At present I have trained 3 dot-matrix fonts and two Merchant copy fonts. I have turned off the dictionary and added user-words for all five fonts. They are commonly used terms in Bills and Receipts. Strangely these changes don't seem to make any difference in the output.
I resized the image so that the font size is at least 12 pt. How do I improve the accuracy further? Can anyone specify a font or should I retrain the fonts.

How are the emoji images encoded in AndroidEmoji-htc.ttf file?

What file type is being used to embed the images in AndroidEmoji-htc.ttf? direct download: AndroidEmoji-htc.ttf
Images can be extracted from AppleColorEmoji.ttf easily because the PNG headers can be found using a hex editor. This ruby script can extract them. The algorithm is described here.
Sample of file in hex editor:
0706 2627 2626 2726 2627 2626 2727 2727 ..&'&&'&&'&&''''
2726 2627 2626 2726 2627 2636 3736 3637 '&&'&&'&&'&67667
3636 3736 3637 3737 3737 3636 3736 3637 6676677777667667
3636 0131 3636 3736 3635 3527 2626 3132 66.16676655'&&12
2627 2626 2726 2627 2721 2115 1533 3232 &'&&'&&''!!..322
1716 1617 1634 1110 0607 0606 0706 0623 .....4.........#
2315 1533 3335 3523 2226 2726 2627 3434 #..3355#"&'&&'44
3535 3332 3233 1616 1716 1617 1616 1716 553223..........
1617 1616 1716 1617 1616 1716 3237 3236 ............2726
3737 3736 3637 3636 3737 2323 0706 0607 7776676677##....
0606 0706 0627 2226 2726 2627 2626 2726 .....'"&'&&'&&'&
2627 2626 3130 2627 2626 2726 2627 2626 &'&&10&'&&'&&'&&
2727 3535 3736 3637 3636 0131 3232 3332 ''55766766.12232
1617 1616 1716 3017 1616 1516 0607 0606 ......0.........
0706 0607 0606 2323 3534 3437 3636 3736 ......##54476676
3608 bf05 020b 1c2a 0d09 0b0d 66a4 b72c 6......*....f..,
0202 0233 8a8d 9c9c 8d88 3302 0202 022a ...3......3....*
9a8d 631a 3c05 090b 3e1a 1a3e 0b09 0b0d ..c.<...>..>....
65a5 b918 1616 1d67 6572 4028 2a0f 251f e......ger#(*.%.
7365 691b 1814 1a98 8d65 183b 0409 0d2c sei......e.;...,
1a0b 0702 fb8b 1e1d 423e 3f44 2a0d 3a3f ........B>?D*.:?
034f 4f42 4435 3203 1e1b 040d 140d 0704 .OOBD52.........
0303 040d 1b0d 041b 1e03 3235 4442 4f4f ..........25DBOO
033f 3a0d 2a55 5265 1f21 3d40 4044 2a0d .?:.*URe.!=##D*.
3940 024d 4f44 4235 3502 1d1c 050d 1a0d 9#.MODB55.......
0502 0207 040d 140d 051c 1d02 3535 4244 ............55BD
4f4d 0240 390d 2a56 5301 c806 0d0b 2611 OM.#9.*VS.....&.
0b06 0503 120b 122e 1835 0837 33fe d9fe .........5.73...
dc2e 2c02 0b12 0407 0b02 0306 071a 1006 ..,.............
2a23 f8fb 2a30 070f 1d0d 022a 1818 0914 *#..*0.....*....
Update 6/18/2014:
At #naXa 's suggestion, opening the file in FontForge version 20120731-ML (current newest version) gave this error:
The following table(s) in the font have been ignored by FontForge
Ignoring 'dcmj'
In GID1 the advance width (2252) is greater than the stated maximum (2048)
Subsequent errors will not be reported.
Bad lookup table: format=6, first=65535 total glyphs in font=894
Somewhat expected because emojis in TTFs to this day are encoded proprietarily. The fact that I even see black and white emoji images using FontForge is a huge success because it means the TTF is standard for the most part. TTFs are not supposed to store color information I don't think.
They key is probably accessing the data in the dcmj table or wherever it is pointing to. Researching FontForge I found that BMP is a common image format for TTFs so I'm going to try and modify the ruby script using those assumptions and report back!
Update: 6/18/14
I found what appear to be BMP headers source1 source2, starting with 424D using a hex editor but the header doesn't seem valid.
Next I would try:
Parsing the TTF look at the data in each "glyph" to see if I can find more patterns. I imagine the ttf will say the start end end of the image data.
Look into the htc android apk to see how they are pulling and displaying emoji from the ttf.
I've run out of time on this for now, if anyone has any other suggestions I'm very interested.
UPDATE 6/20/2014
Double clicking on the glyph using #naXa's suggestion and exporting as any of the formats will give me non color icons of any size but still does not reveal the color bitmap emojis I was looking for.
I went down to the store to look at an HTC phone finally and saw, to my surprise, they are using Apple's emoji font seen through the messaging app:
I am almost certain these are stored in the HTC font provided above, but this conclusion has left extracting these images far less desirable.
Howver, it would still be cool to know, as a proof of concept, how to extract the color emojis. :)
EDIT: As Jasper pointed out, HTC does in fact have a custom emoji set as linked in his answer. The picture above was from a non updated phone. Still need to figure out how to extract these emojis!!

Unfortunately, I don't have an account, so let me start by apologizing for posting this as an answer instead of as a comment.
The picture you posted showing the Apple Color Emojis appear to come from a phone running an older version of Sense/Android, while the file you are referencing almost definitely comes from Sense 5-6/Android 4.3-4.4 If you look at the grayscale emojis you were able to extract from the file, you'll notice that they don't actually match up with the picture you provided. They do, however, match up with this: http://assets.hardwarezone.com/img/2013/10/HTC_One_Max_Emoticons_Keyboard_jpg.jpg
This leads me to conclude that it could be entirely possible that there are no conventional bitmaps stored in the TTF, rather there's some proprietary format that they use to assign colors to different parts of each emoji.
EDIT: Tried directly copying the file over to my phone to see what would happen (tried both replacing NotoColorFont.ttf as well as just directly copying and referencing it in fallback_fonts.xml, there doesn't seem to be any difference). Screenshot here: http://imgur.com/OGyq6T2
As you can see, they show up without color, yet we already know that both the default Android emoji and Apple Color Emoji both show up fine on Android devices, meaning that HTC doesn't follow whatever standard that Android and iOS use.
Tested on a Galaxy SII (i9100) running CyanogenMod 11 Milestone 8.

The emoji images in the AndroidEmoji-htc.ttf file are probably (since I don't have the font to test) stored in the same format as the standard Android emoji in Google's CBLC+CBDT OpenType tables.
You can disassemble/reassemble the font using ttx from FontTools(pypi, github) to confirm.
The direct answer to your question "What is the format?" is two options:
Uncompressed Color Bitmaps
The value ‘32’ of the bitDepth field of bitmapSizeTable struct defined in the CBLC table, to identify color bitmaps with 8-bit blue/green/red/alpha channels per pixel, encoded in that order for each pixel (referred to as BGRA from hereon). The color channels represent pre-multiplied color and are encode colors in the sRGB colorspace. For example, the color “full-green with half translucency” is encoded as \x00\x80\x00\x80, and not \x00\xFF\x00\x80.
All imageFormat values defined in the EBDT / EBLC tables are valid for use with the CBDT / CBLC tables.
Compressed Color Bitmaps
Images for each individual glyph are stored as straight PNG data. Only the following chunks are allowed in such PNG data: IHDR, PLTE, tRNS, sRGB, IDAT, and IEND. If other chunks are present, the behavior is undefined. The image data shall be in the sRGB colorspace, regardless of color information that may be present in other chunks in the PNG data. The individual images must have the same size as expected by the table in the bitmap metrics.

As I know, Emojis are stored in two different placfes - in .ttf - to display in text-only fields (for example, quick previews of message) and in images. Maybe you should dig into that way?

Haven't looked at Android emoji but I managed to extract out the iOS emoji
by jacking a few tools to do it as nothing on the net seems to do it 100%.
Hex Editor is key its all I used...
iOS 5.0 used uint8 type RGBA data stored as tuples
iOS 5.1 changed to pngs and these are written contiguously
iOS 6 combined both the iOS 5.0 & 5.1 format. Set 1 & 2 were uint8 type data & Set 3 (ipad # 96x96px) were optimised png format that Apple adopt e.g switching RGBA to BGRA...byte blitting apparently...
iOS 7 stayed the same as did iOS 8 to 8.2.
Hope that helps...