EOL protobuf version is required

[bdraco]

This library requires protobuf < 4, however its reached EOL https://protobuf.dev/support/version-support/#python which prevents it from being used in newer projects which use protobuf 4+ and is blocking esphome/esphome#8105

[bdraco]

@moyogo Maybe I ask if there is a plan to move to protobuf 4?

Thank you!

[simoncozens]

I'm going to take a look at this (probably after the next release) - but wow, dependencies are weird. A library to interact with ESP devices needs to know the glyphs required in Google Fonts projects? I feel like something else has gone wrong here.

[bdraco]

I'm going to take a look at this (probably after the next release) - but wow, dependencies are weird. A library to interact with ESP devices needs to know the glyphs required in Google Fonts projects? I feel like something else has gone wrong here.

Thanks. Yeah it's always surprising what people want to with their ESP devices.

[paravoid]

I'm going to take a look at this (probably after the next release) - but wow, dependencies are weird. A library to interact with ESP devices needs to know the glyphs required in Google Fonts projects? I feel like something else has gone wrong here.

I'm a drive-by contributor to ESPHome and the original author of the ESPHome glyphset code. I acknowledge this is a bit weird, and I'm happy to take any criticism and take suggestions from subject-matter experts such as yourself. This is wildly off-topic for this issue tracker of course, but since you mentioned it perhaps I can explain :)

(Note that @bdraco in the meantime created a simplified https://github.com/esphome/esphome-glyphsets/ library to avoid this long dependency chain, embedding the glyphset data from glyphsets using a git submodule).

First of all, ESPHome is not a library but per se rather a project that builds firmware for embedded (ESP, but not only ESP) devices based on a YAML configuration. Think of Arduino, but instead of hand-rolling your own code, you write a few lines of YAML config to choose components and your particular set of configuration, and ESPHome assembles the right set of libraries (like a package manager), configures them, builds a C++ codebase, compiles it, and spits out a .bin file for you to flash to your device, or even flashes it directly for you! It is a project under the auspices of the Open Home Foundation and interfaces well with Home Assistant through an HTTP API (which is where protobufs come into play). It's a pretty cool project!

One of its relatively new features is the ability to also write text on LCDs or even write complicated full-fledged UIs to run on touchscreens (think e.g. Nest thermostat type of thing). These UIs will likely include text, either static (like e.g. a button label), or arbitrary text received over the network (e.g. imagine a home dashboard that also pops up new email notifications).

Since this is an embedded device with limited flash/RAM, it's not feasible to embed multi-megabyte fonts. At the same time, most users don't need the entirety of Unicode in their embedded device. So ESPHome has a YAML option for users to manually specify a list of the glyphs they were interested in. This was not very practical IMHO, and especially if you take into account having to define all permutations of precomposed glyphs (there was/is no support for NFD/NFC etc. as far as I could tell). I am Greek and it took me a while to manually handroll a list of glyphs for my language, and I still missed a few glyphs.

So I addressed that by interfacing with the glyphsets library, allowing users to define a YAML key with the (GF) sets they are interested in, such as for example:

  - file:
      type: gfonts
      family: Roboto
    id: roboto_european_core
    size: 16
    glyphsets:
      - GF_Latin_Core
      - GF_Greek_Core
      - GF_Cyrillic_Core

I considered a few other alternatives, like Unicode blocks (with the usual limitations w.r.t mapping them to languages), or defining a static set of glyphs in the ESPHome itself. Based on my research a considerable amount of work and debate has gone into defining which glyphs each GF glyphset should have, so reusing that (over e.g. having random non-font-experts contributors such as myself define them in ESPHome itself) felt like the superior option.

Hope at least this use case makes sense to you. I'd love to hear your thoughts and especially if you can think of better/easier/more straightforward/less "wrong" ways to achieve this 😄

[simoncozens]

OK, that's weird but also brilliant, and it does make some sense now! :-) However, I don't think a glyph-based approach is the right answer to the problem you're facing. This is because different font creators may have different glyph naming schemes, or even different approaches to how to handle precomposed, decomposed or unencoded glyphs. gfglyphsets is just the set of names of glyphs that Google Fonts asks of designers when commissioning a font for a particular level of language support; it's quite specific to the GF workflow, and I don't think it makes sense in a general-purpose solution.

The usual way to subset a font to a particular language or region set is to identify the Unicode codepoints needed, and then use something like hb-subset to trim down the font. hb-subset has a "layout closure" system which, after mapping the Unicodes to glyphs, also traces all possible unencoded glyphs reached by the substitution rules in the font and adds those to the active set. So you don't need to know the exact glyph names, because they get identified as part of layout closure.

Of course you then still need some way to map a set of languages to a set of Unicode codepoints, which is kind of what this library does, so we still need to solve the protobuf issue for you. :-)

[yanone]

But gfglyphsets does have Unicode-only representations of the glyphsets as part of the .nam files.

Since (particularly those three) glyphsets are already language-based, you would just go in a circle and recreate gfglyphsets from scratch.

Additionally, it has the print-unicodes command that you can use to assemble combinations of glyphsets for subsetting: glyphsets print-unicodes GF_Latin_Core GF_Greek_Core GF_Cyrillic_Core. You can also set separator and decorators to match your data consumption, see glyphsets print-unicodes -h.

The idea was that this gets called directly in command line calls such as pyftsubset unicodes=$(glyphsets print-unicodes GF_Latin_Core GF_Greek_Core GF_Cyrillic_Core)

(I still need to document this in the README, for now it just shows in the command line help.)