Bug#928969: stterm: incorrect rendering of cyrillic letters with Terminus

classic Classic list List threaded Threaded
9 messages Options
Reply | Threaded
Open this post in threaded view
|

Bug#928969: stterm: incorrect rendering of cyrillic letters with Terminus

Dmitry Bogatov-3

Package: stterm
Version: 0.8.2-1
Severity: normal

Dear Maintainer,

cyrillic letters in stterm with Terminus font are rendered as more wide
and overlapping than latin letters. On both "xterm" and under tty
cyrillic letters are rendered properly.

This issue is only with Terminus font: no other monospace font I checked
have such problems.

Please, try to reproduce it yourself. You will need:

 * xfonts-terminus
 * stterm
 * cyrillic text "Привет"

Start stterm:

        $ stterm -f Terminus

and view cyrillic text in that terminal. If you want, I can make
screenshots.

attachment0 (849 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Bug#928969: stterm: incorrect rendering of cyrillic letters with Terminus

Paride Legovini-2
Dmitry Bogatov wrote on 14/05/2019:

>
> Package: stterm
> Version: 0.8.2-1
> Severity: normal
>
> Dear Maintainer,
>
> cyrillic letters in stterm with Terminus font are rendered as more wide
> and overlapping than latin letters. On both "xterm" and under tty
> cyrillic letters are rendered properly.
>
> This issue is only with Terminus font: no other monospace font I checked
> have such problems.
>
> Please, try to reproduce it yourself. You will need:
>
>  * xfonts-terminus
>  * stterm
>  * cyrillic text "Привет"
>
> Start stterm:
>
> $ stterm -f Terminus
>
> and view cyrillic text in that terminal. If you want, I can make
> screenshots.
Thanks for your report Dmitry. It seems that I can't reproduce the issue
on my system (an up-to-date Debian unstable), see the attached
screenshot. I'm using:

stterm 0.8.2-1
xfonts-terminus 4.40-2

Is there anything else I could try?

Paride

2019-05-15-115112_567x314_scrot.png (11K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Bug#928969: stterm: incorrect rendering of cyrillic letters with Terminus

Dmitry Bogatov-3

[2019-05-15 12:02] Paride Legovini <[hidden email]>

> Dmitry Bogatov wrote on 14/05/2019:
> > cyrillic letters in stterm with Terminus font are rendered as more wide
> > and overlapping than latin letters. On both "xterm" and under tty
> > cyrillic letters are rendered properly.
> >
> > This issue is only with Terminus font: no other monospace font I checked
> > have such problems.
> >
> > Please, try to reproduce it yourself. You will need:
> >
> >  * xfonts-terminus
> >  * stterm
> >  * cyrillic text "Привет"
> >
> > Start stterm:
> >
> > $ stterm -f Terminus
> >
> > and view cyrillic text in that terminal. If you want, I can make
> > screenshots.
>
> Thanks for your report Dmitry. It seems that I can't reproduce the issue
> on my system (an up-to-date Debian unstable), see the attached
> screenshot.
On your screen shot everything is fine.

> I'm using:
>
> stterm 0.8.2-1
> xfonts-terminus 4.40-2

So do I. Strange. Please, try comparing it with rendering in different
terminal emulators. Here are two screenshots, one with 'stterm' (bad.png)
and another with 'terminology' (good.png) and another one with 'sakura'
(good-2.png).

Not sure, that it is relevant, but in xterm(1) I found following:

  The reason for this resource is to help with certain
  quasi-automatically generated fonts (such as the ISO-10646-1 encoding
  of Terminus) which have incorrect font-metrics.

Maybe there are known issues with terminus font, and other terminal
emulators provide special care for it?
--
        Note, that I send and fetch email in batch, once every 24 hours.
                 If matter is urgent, try https://t.me/kaction
                                                                             --

bad.png (29K) Download Attachment
good.png (327K) Download Attachment
good-2.png (23K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Bug#928969: stterm: incorrect rendering of cyrillic letters with Terminus

Paride Legovini-2
Dmitry Bogatov wrote on 16/05/2019:

>
> [2019-05-15 12:02] Paride Legovini <[hidden email]>
>> Dmitry Bogatov wrote on 14/05/2019:
>>> cyrillic letters in stterm with Terminus font are rendered as more wide
>>> and overlapping than latin letters. On both "xterm" and under tty
>>> cyrillic letters are rendered properly.
>>>
>>> This issue is only with Terminus font: no other monospace font I checked
>>> have such problems.
>>>
>>> Please, try to reproduce it yourself. You will need:
>>>
>>>  * xfonts-terminus
>>>  * stterm
>>>  * cyrillic text "Привет"
>>>
>>> Start stterm:
>>>
>>> $ stterm -f Terminus
>>>
>>> and view cyrillic text in that terminal. If you want, I can make
>>> screenshots.
>>
>> Thanks for your report Dmitry. It seems that I can't reproduce the issue
>> on my system (an up-to-date Debian unstable), see the attached
>> screenshot.
>
> On your screen shot everything is fine.
>
>> I'm using:
>>
>> stterm 0.8.2-1
>> xfonts-terminus 4.40-2
>
> So do I. Strange. Please, try comparing it with rendering in different
> terminal emulators. Here are two screenshots, one with 'stterm' (bad.png)
> and another with 'terminology' (good.png) and another one with 'sakura'
> (good-2.png).

I tried with xterm and sakura, the Cyrillic string renders exactly as in
stterm, with the correct character width and spacing.

> Not sure, that it is relevant, but in xterm(1) I found following:
>
>   The reason for this resource is to help with certain
>   quasi-automatically generated fonts (such as the ISO-10646-1 encoding
>   of Terminus) which have incorrect font-metrics.>
> Maybe there are known issues with terminus font, and other terminal
> emulators provide special care for it?

Could be, I don't know. The font's homepage [0] says that:

  Version 4.47 contains 1326 characters, covers about 120 language
  sets and supports ISO8859-1/2/5/7/9/13/15/16 [...]

and ISO5589-5 is Latin/Cyrillic, so I don't think there are
auto-generated fonts. It also says that Terminus is a "fixed width
bitmap font". On my system I have disabled the bitmap fonts: I have this
symbolic link in place:

/etc/fonts/conf.d/70-no-bitmaps.conf ->
/usr/share/fontconfig/conf.avail/70-no-bitmaps.conf

so maybe even if I explicitly ask for Terminus the terminals lie to me,
and another fallback font is actually used. Did you disable the bitmap
fonts?

Another question. You said xterm renders Terminus well. From how it
looks, can you tell if it is actually Terminus or another font?

Paride

[0] http://terminus-font.sourceforge.net/

Reply | Threaded
Open this post in threaded view
|

Bug#928969: stterm: incorrect rendering of cyrillic letters with Terminus

Dmitry Bogatov-3

[2019-05-17 18:43] Paride Legovini <[hidden email]>
> I tried with xterm and sakura, the Cyrillic string renders exactly as
> in stterm, with the correct character width and spacing.

I have found workaround. My regular locale is eo.utf-8, but many X
applications (stterm included) fail to start with following error:

        XOpenIM failed. Could not open input devices.

so I start them with LC_ALL=C.UTF-8. Funny thing, I just tried to start
"LC_ALL=ru_RU.utf8 stterm -f Terminus", and it rendered cyrillic text
properly.

But I still believe, that different rendring on different utf-8 locales
is a bug. Can you re-do checks with LC_ALL=C.UTF-8?

> > Not sure, that it is relevant, but in xterm(1) I found following:
> >
> >   The reason for this resource is to help with certain
> >   quasi-automatically generated fonts (such as the ISO-10646-1 encoding
> >   of Terminus) which have incorrect font-metrics.>
> > Maybe there are known issues with terminus font, and other terminal
> > emulators provide special care for it?
>
> Could be, I don't know. The font's homepage [0] says that:
>
>   Version 4.47 contains 1326 characters, covers about 120 language
>   sets and supports ISO8859-1/2/5/7/9/13/15/16 [...]
>
> and ISO5589-5 is Latin/Cyrillic, so I don't think there are
> auto-generated fonts. It also says that Terminus is a "fixed width
> bitmap font". On my system I have disabled the bitmap fonts: I have this
> symbolic link in place:
>
> /etc/fonts/conf.d/70-no-bitmaps.conf ->
> /usr/share/fontconfig/conf.avail/70-no-bitmaps.conf

No, I did no configuration in /etc/fonts/conf.d. I wasn't aware of it.

> so maybe even if I explicitly ask for Terminus the terminals lie to me,
> and another fallback font is actually used. Did you disable the bitmap
> fonts?

> Another question. You said xterm renders Terminus well. From how it
> looks, can you tell if it is actually Terminus or another font?

I did not checked that xterm(1) renders well:

        $ xterm -f Terminus

did not work. I checked sakura(1) and terminology(1). If they are lying
to me, they do it very well -- I can't tell the difference.
--
        Note, that I send and fetch email in batch, once every 24 hours.
                 If matter is urgent, try https://t.me/kaction
                                                                             --

Reply | Threaded
Open this post in threaded view
|

Bug#928969: stterm: incorrect rendering of cyrillic letters with Terminus

Paride Legovini-2
Dmitry Bogatov wrote on 19/05/2019:

>
> [2019-05-17 18:43] Paride Legovini <[hidden email]>
>> I tried with xterm and sakura, the Cyrillic string renders exactly as
>> in stterm, with the correct character width and spacing.
>
> I have found workaround. My regular locale is eo.utf-8, but many X
> applications (stterm included) fail to start with following error:
>
> XOpenIM failed. Could not open input devices.
>
> so I start them with LC_ALL=C.UTF-8. Funny thing, I just tried to start
> "LC_ALL=ru_RU.utf8 stterm -f Terminus", and it rendered cyrillic text
> properly.
>
> But I still believe, that different rendring on different utf-8 locales
> is a bug. Can you re-do checks with LC_ALL=C.UTF-8?

This is indeed strange. I don't doubt your report, but I still can't
reproduce the problem. This is what I did:

1. Start the terminal: LC_ALL=C.UTF-8 stterm -f Terminus
2. Run locale(1) from within the terminal to confirm that the locale
   settings get applied correctly; they do
3. Run: echo Привет
4. The Cyrillic letters get printed and echoed normally, no extra space.

>> On my system I have disabled the bitmap fonts: I have this
>> symbolic link in place:
>>
>> /etc/fonts/conf.d/70-no-bitmaps.conf ->
>> /usr/share/fontconfig/conf.avail/70-no-bitmaps.conf
>
> No, I did no configuration in /etc/fonts/conf.d. I wasn't aware of it.

The link is created (or deleted) when configuring fontconfig-config. If
you are not getting asked about bitmap fonts try with dpkg-reconfigure
-p low. I tried to re-enable the bitmap fonts in this way but nothing
changed.

If you compile st directly from the upstream source does the problem
show up? What about the older versions? Compiling st is quick and easy,
so this might be worth trying.

Let me know if there is anything else I can try.

Paride

Reply | Threaded
Open this post in threaded view
|

Bug#928969: stterm: incorrect rendering of cyrillic letters with Terminus

Dmitry Bogatov-3

[2019-05-19 23:49] Paride Legovini <[hidden email]>
> I tried with xterm and sakura, the Cyrillic string renders exactly as
> in stterm, with the correct character width and spacing.

> Let me know if there is anything else I can try.

Okay. I prepared kvm virtual machine image, that you can download
from "people.debian.org/~kaction/stterm-bug.img.xz".

After it boots, it shows Xdm prompt. Log is as "user:useruser". You will
have default "dwm" instance. Press Alt-Shift-Enter and some terminal
will appear.

There is ~user/I-Will-Show-You.sh script, that starts stterm to show
difference between cyrillic "Привет, мир!" and ascii "Hello, world!".

Hope, you will see same thing as me. After that, you can get root
permission with su and "root:rootroot".
--
        Note, that I send and fetch email in batch, once every 24 hours.
                 If matter is urgent, try https://t.me/kaction
                                                                             --

Reply | Threaded
Open this post in threaded view
|

Bug#928969: stterm: incorrect rendering of cyrillic letters with Terminus

Paride Legovini-2
Dmitry Bogatov wrote on 22/05/2019:

>
> [2019-05-19 23:49] Paride Legovini <[hidden email]>
>> I tried with xterm and sakura, the Cyrillic string renders exactly as
>> in stterm, with the correct character width and spacing.
>
>> Let me know if there is anything else I can try.
>
> Okay. I prepared kvm virtual machine image, that you can download
> from "people.debian.org/~kaction/stterm-bug.img.xz".
>
> After it boots, it shows Xdm prompt. Log is as "user:useruser". You will
> have default "dwm" instance. Press Alt-Shift-Enter and some terminal
> will appear.
>
> There is ~user/I-Will-Show-You.sh script, that starts stterm to show
> difference between cyrillic "Привет, мир!" and ascii "Hello, world!".
>
> Hope, you will see same thing as me. After that, you can get root
> permission with su and "root:rootroot".

Hi,

Using the VM I can reproduce the issue (well, obviously). My impression
is that stterm is *not* using Terminus to render the Cyrillic letters,
even if Terminus do have the needed glyphs. This is very clear when
comparing stterm and urxvt (from rxvt-unicode):

stterm -f Terminus-24 -e tail -f ~/test.txt
urxvt -fn Terminus-24 -e tail -f ~/test.txt

Forget the size and overlap problem and look at the Cyrillic "M" letter:
the "wedge" of the letter is way deeper in stterm. The Cyrillic letters
in stterm are from different, fallback font. The same font is used when
a nonexisting font is specified, e.g.

stterm -f NonExisting -e tail -f ~/test.txt

and this reinforces the idea of a fallback mechanism being in place.
This also somehow explains why I can't reproduce the problem locally: a
different fallback font is probably used on my system as I have a
different set of font packages installed. However I do see a difference
between stterm and urxvt, and stterm is clearly using the wrong font for
Cyrillic even on my system, while urxvt uses the correct font (Terminus
is quite recognizable).

I tried to reproduce by compiling st directly from the upstream source
tree and I am able to reproduce the issue (as expected, as there are
currently no patches in the Debian package). My conclusions are:

- This is an actual bug in st, not a configuration issue;
- The comparison with urxvt makes it apparent;
- The bug belongs to upstream.

Do you think you can follow-up to [hidden email]?

Paride

Reply | Threaded
Open this post in threaded view
|

Bug#928969: stterm: incorrect rendering of cyrillic letters with Terminus

Dmitry Bogatov-3

[2019-06-06 12:19] Paride Legovini <[hidden email]>

> [...]
> I tried to reproduce by compiling st directly from the upstream source
> tree and I am able to reproduce the issue (as expected, as there are
> currently no patches in the Debian package). My conclusions are:
>
> - This is an actual bug in st, not a configuration issue;
> - The comparison with urxvt makes it apparent;
> - The bug belongs to upstream.
>
> Do you think you can follow-up to [hidden email]?

What do you mean to "follow-up"? Add me into CC as required, I will
reply, but I'd rather not subscribe to mailing list.

Actually, I doubt I can be of any use now. Virtual machine image is
more eloquent, then hundred words :)

Thank you for your effort maintaining stterm.
--
Note, that I send and fetch email in batch, once in a few days.