Internationalization and using unicode fonts for Asian languages - martinberlin/epaper-weather-station GitHub Wiki

As you know not all the world is using western characters to write. Chinese use ideograms which are divided in simplified Chinese and traditional. In Hong Kong and other areas they speak Cantonese and in Saudi Arabia and other countries they use Arabic ideograms for writing.

To solve that we decided to use a great component called: Open Font Render

That has the marvelous ability to read a binary TTF font and draw the font, using your selected library as in our case Lovyan GFX or EPDiy, placing the pixels like a pencil over your display. Is a bit slower than having the fonts per-rendered in C on a certain size but is an amazing solution to the problem of writing non-western characters on your screen.

Because how you could do it with Adafruit graphics library? Well you can't. Adafruit GFX has no support for unicode characters. And also not many other GFX solutions that are thought to write with the classic alphabet. And here it comes Open Font Render to the rescue.

But there is always a caveat

But of course that is not so easy. Because they are hundreds of ideograms in a language like Chinese and you are going to use only a few of them in your display.

So what you need to do as a first step is to create a subset TTF font with a font editor. And because I use Linux and will never downgrade myself to use a Windows App just for a task I found this amazing program: Font Forge

Which can do much more than open a TTF and let you open a new truetype to copy only what you need. It's a program where you can also design your own Font or Glyph that you want to use. Take that! You can design your own vector font, and have it printed, on the size you select on any display.

Well let's not go through the branches like politicians, I will come back to my goal that is to explain how to make a subset of fonts that we want to use in our display. I simply started doing this 5 minutes after opening the Font Forge program: I opened one big window to the right and a smaller with a new TTF to the left.

In the big one I had Google Noto font open with hundreds of ideograms. In the left my own small font that I want to use for my goal of displaying the information I want on my panel. And now comes the question: How are going to map this and what are you going to leave? For example I wanted to leave 0->9 and A to Z along with ° degree sign, the dot . and other characters I want to display with the same font. But small a-z I won't use, for that I can use a western font, that at this time was much less appealing than Chinese ;) But still make yourself aware that this is a slower way of writing that looks cool, but will also consume some time more battery, so you won't do it long if you care about consumption.

So there comes this critical point, when you have the right window, at almost full screen and you need to search for 10 ideograms. HOW TO?

Unicode comes to the rescue

And here comes the thing. Font Forge, won't let you search by the character itself, let's say 日 which is sunday... How the hell I can search for that in 20 pages that have hundreds of drawings which all look similar, but they are really not? Well it comes that Font Forge has a tool to find a Glyph by their Unicode number. Which you will need to get online or with whatever means you have at hand. For example:

 日 U+65E5

sunday is U+65E5 (Which corresponds also with CSS notation if you do HTML)

so you can just go to the top menu: Edit -> Select -> Select by Wildcard

And you will enter the U+65E5 above. And this is not a nice part. The select works but it does not have automatic scroll, I wanted to ask for it, but I should first collaborate with the program since it has 900+ Issues open on Github. And you need to be kind when someone gives you such a monumental tool for free. So you need to scroll and find out where the select is.

For that it comes really handy to figure out what Glyphs you need to use and make a table with their unicode numbers assigned:

Then you have to repeat this search and lookup assigning your fonts, also copying from master font, to your subset on the left the ones you want. BUT Where? And here comes the trick: To the known western alphabet you won't use. Maybe there is a trick to do it in the actual unicode, who knows, but what I know well is that no MCU unless you use a full legged Linux will hold the 24 Megabytes that is Noto font converted to binary. So YES, I'm sorry but need to extract a subset of the font if you want to use it in your small RAM MCU chip.

Doing it like this, in my case won't use a-z and having a german keyboard also ÄÖÜ could be used meaning you can just copy that selected fonts for the days of the week, month names, and others want in their places (Adapt this to your own keyboard). And then you can just use that in your translation C file, check this:

When the program reads one of those, like for example [ it will replace it for 星 and so on, making it render the chinese font that I selected. But for that first we need to add the Open Font Render as a component, which I did in Espressif IDF, making a fork of the original Open Font Render repository and adding it as a git submodule:

git submodule add https://github.com/martinberlin/OpenFontRender-idf.git components/OpenFontRender/upstream

Note I added that on a subdirectory since I want to add my own CMakeFiles to include all buildable files in the upstream directory. You don't need to do this if in the next version there is a CMakeFiles already like I added on my fork.

And then you will have it available if you REQUIRE it in your CMakeFiles for your program. Note that I barely touched Takkao files I limited myself just to add the required CMakeFiles with his C files and pointing where the Includes are. Please check the Espressif build system for more details since that highly exceeds what I want to address in this short document.

Then you have already OpenFontRender available as an IDF Component. Now we should also pass the TTF file into an Open Font Render readable C binary. And for that there is tool ready to use, just browse in the newly installed component to the folder:

components/OpenFontRender/upstream/tools/ttf2bin

And there copy that subset TTF font and name it yourFont.ttf So now we are going to use Takkao tool to convert it on a readable by C file:

python ttf2bin.py yourFont.ttf

And that will generate a binaryttf.h file that in this case is 85 Kb and not 24 megas since there is just a few fonts to process. Of course you need to copy this file wherever CMake reaches it to include it on your program. I use also a components/big-fonts where I place all my fonts that I want to use, so I leave it there along with the translations file so you have it separate of your program.

That's it, now we have the component installed, and also our Font generated which we mapped somewhere in a text file where a is equal X and so on, let's draw it for gods sake. So like any other component or library we will include it and instantiate it:

#include <chinese/binaryttf.h>
#include "OpenFontRender.h"

OpenFontRender render;

// Our GFX selected library to draw using IT8951:
#define LGFX_USE_V1
#include <LovyanGFX.hpp>
LGFX display;

And then whenever we want to write something we need to tell the component to load our generated font and with what we are drawing:

if (render.loadFont(binaryttf, sizeof(binaryttf))) {
  ESP_LOGE(TAG, "Render initialize error");
  return;
}
// Note I'm using Lovyan GFX instantiated with display
// So actually we are saying Open Font Render please use Lovyan drawPixel for your pleasures
render.setDrawer(display);

// For this render to work correctly we need to set both foreground and background color:
//                   FG RGB, Background RGB
render.setFontColor(0,0,0,   255,255,255);

And that's the basic setup. Note that if I didn't add the foreground (FG) and background color it draw always a black background. So I need to add white background and black tint expressly to generate the image below.

To draw you simply use it like any other GFX:

render.setFontSize(120); // Yes font size is generated on-the-fly
render.setCursor(x, y);
render.printf(text_buffer); // char text_buffer[] = "My text";

This is the result of the 1200*825 epaper display printed with chinese day of the week, month and temperature levels.

Short bad quality twitter video And just to make my point: This is no magic, it takes time to draw like this, and it will spend a bit more battery. But it solves a great problem with a doable solution, using vector fonts and free tools. Of course you can hide the drawing and show it all together, that's doable, but still will take it's time.

I hope you enjoyed this explanation and you add a ★ to our repository. This is open source, both the PCB controller and the code, so you can use it whenever you want in any project you desire.

Credits: TakkaO https://github.com/takkaO/openFontRender