Chinese characters will be replaced by # when exporting a page to PDF

classic Classic list List threaded Threaded
7 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Chinese characters will be replaced by # when exporting a page to PDF

Billtec
This post was updated on .
Hi all,

I'm using the xwiki v9.1.2 and I found that when exporting a page to PDF, the non-English characters will be replaced by "#".

I spent a lot of time searching in jira.xwiki.org and found that there's similar issues like XWIKI-4724 and XWIKI-1609. However, non of the solutions work on me.

Finally I found this commit log in github - https://github.com/xwiki/xwiki-platform/commit/437f28bee5545670b2478c93498f2f261a021aa5 and I manually create a fop-config.xml file in xwiki\WEB-INF\classes folder, with the "<auto-detect/>" not commented out. Then the problem solved.

The question is that why in 2015 the dev team want to comment out the auto-detect propert and it seems that it results on the issue I mentioned above. Will it be fixed in next release?

Regards,
Bill
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Chinese characters will be replaced by # when exporting a page to PDF

Thomas Mortagne
Administrator
As indicated by the comment <auto-detect/> is here to load the system
fonts which we are not supposed to need by default since we embed GNU
FreeFont.

It's not all "non-English" characters since it works fine for many of
them from what I see but I do reproduce the issue with Chinese
characters.

And indeed according to
https://www.gnu.org/software/freefont/FAQ.html, Freefont does not
support Chinese, Japanese, or Korean characters which is quite a
limitation by default. I'm not a big fan of enabling back
<auto-detect/> since it was taking a lot of memory when loading system
fonts (causing https://jira.xwiki.org/browse/XWIKI-13307) which where
totally useless for most of them. Ideally we should replace freefont
with a more complete font pack if anything like this exist in a
compatible license.

In any case don't hesitate to create an issue on http://jira.xwiki.org
about that.

On Wed, Mar 8, 2017 at 3:48 PM, Billtec <[hidden email]> wrote:

> Hi all,I'm using the xwiki v9.1.2 and I found that when exporting a page to
> PDF, the non-English characters will be replaced by "#".I spent a lot of
> time searching in jira.xwiki.org and found that there's similar issues like
> XWIKI-4724 and XWIKI-1609. However, non of the solutions work on me.Finally
> I found this commit log in github -
> https://github.com/xwiki/xwiki-platform/commit/437f28bee5545670b2478c93498f2f261a021aa5
> and I manually create a fop-config.xml file in xwiki\WEB-INF\classes folder,
> with the "" not commented out. Then the problem solved.The question is that
> why in 2015 the dev team want to comment out the auto-detect propert and it
> seems that it results on the issue I mentioned above. Will it be fixed in
> next release?Regards,Bill
>
>
>
> --
> View this message in context: http://xwiki.475771.n2.nabble.com/Chinese-characters-will-be-replaced-by-when-exporting-a-page-to-PDF-tp7603012.html
> Sent from the XWiki- Users mailing list archive at Nabble.com.



--
Thomas Mortagne
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Chinese characters will be replaced by # when exporting a page to PDF

Thomas Mortagne
Administrator
https://www.google.com/get/noto/ looks like a pretty complete
alternative (a bit too complete, we would need to select a subset of
those probably).

On Wed, Mar 8, 2017 at 6:01 PM, Thomas Mortagne
<[hidden email]> wrote:

> As indicated by the comment <auto-detect/> is here to load the system
> fonts which we are not supposed to need by default since we embed GNU
> FreeFont.
>
> It's not all "non-English" characters since it works fine for many of
> them from what I see but I do reproduce the issue with Chinese
> characters.
>
> And indeed according to
> https://www.gnu.org/software/freefont/FAQ.html, Freefont does not
> support Chinese, Japanese, or Korean characters which is quite a
> limitation by default. I'm not a big fan of enabling back
> <auto-detect/> since it was taking a lot of memory when loading system
> fonts (causing https://jira.xwiki.org/browse/XWIKI-13307) which where
> totally useless for most of them. Ideally we should replace freefont
> with a more complete font pack if anything like this exist in a
> compatible license.
>
> In any case don't hesitate to create an issue on http://jira.xwiki.org
> about that.
>
> On Wed, Mar 8, 2017 at 3:48 PM, Billtec <[hidden email]> wrote:
>> Hi all,I'm using the xwiki v9.1.2 and I found that when exporting a page to
>> PDF, the non-English characters will be replaced by "#".I spent a lot of
>> time searching in jira.xwiki.org and found that there's similar issues like
>> XWIKI-4724 and XWIKI-1609. However, non of the solutions work on me.Finally
>> I found this commit log in github -
>> https://github.com/xwiki/xwiki-platform/commit/437f28bee5545670b2478c93498f2f261a021aa5
>> and I manually create a fop-config.xml file in xwiki\WEB-INF\classes folder,
>> with the "" not commented out. Then the problem solved.The question is that
>> why in 2015 the dev team want to comment out the auto-detect propert and it
>> seems that it results on the issue I mentioned above. Will it be fixed in
>> next release?Regards,Bill
>>
>>
>>
>> --
>> View this message in context: http://xwiki.475771.n2.nabble.com/Chinese-characters-will-be-replaced-by-when-exporting-a-page-to-PDF-tp7603012.html
>> Sent from the XWiki- Users mailing list archive at Nabble.com.
>
>
>
> --
> Thomas Mortagne



--
Thomas Mortagne
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Chinese characters will be replaced by # when exporting a page to PDF

Billtec
Is there anything I could do to make Chinese characters available in current release? Like manually replace the FreeFont with some other fonts?
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Chinese characters will be replaced by # when exporting a page to PDF

Thomas Mortagne
Administrator
As I said, the first thing to do is report the problem on
https://jira.xwiki.org.

Now for the how to fix it, looks like embedding a font supporting
Chinese characters might not be as easy as I first tough. I looked at
Noto font but the pack which support Chinese (not even talking about
Japanese and Korean) is 115MB...

We are not going to add 115MB (3 times actually that if we add
Japanese and Korean Nato fonts) to the default XWiki package so here
are some ideas:
* find a lighter font with full support for CJK (but it's possible
there is none)
* download the Chinese Noto pack either automatically (when zh is
enabled in the preferences for example) or on demand in the admin
* enable back <auto-detect/> but then we might get back memory issues
in systems with lots of fonts installed like we used to
* just explain how to install a font by hand (it should be nothing
more that extracting it directly in <xwikihome>/WEB-INF/fonts/ folder)

On Thu, Mar 9, 2017 at 2:38 PM, Billtec <[hidden email]> wrote:
> Is there anything I could do to make Chinese characters available in current
> release? Like manually replace the FreeFont with some other fonts?
>
>
>
> --
> View this message in context: http://xwiki.475771.n2.nabble.com/Chinese-characters-will-be-replaced-by-when-exporting-a-page-to-PDF-tp7603013p7603036.html
> Sent from the XWiki- Users mailing list archive at Nabble.com.



--
Thomas Mortagne
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Chinese characters will be replaced by # when exporting a page to PDF

Thomas Mortagne
Administrator
Actually I looked a bit more and we seems to also have fonts with some
Chinese support (uming font) so we might have another issue (or uming
font is really limited).

Anyway best is to create a jira issue about this and someone need to
debug FOP to understand what is going on exactly and what is supported
and not supported by the various fonts we have.


On Thu, Mar 9, 2017 at 3:06 PM, Thomas Mortagne
<[hidden email]> wrote:

> As I said, the first thing to do is report the problem on
> https://jira.xwiki.org.
>
> Now for the how to fix it, looks like embedding a font supporting
> Chinese characters might not be as easy as I first tough. I looked at
> Noto font but the pack which support Chinese (not even talking about
> Japanese and Korean) is 115MB...
>
> We are not going to add 115MB (3 times actually that if we add
> Japanese and Korean Nato fonts) to the default XWiki package so here
> are some ideas:
> * find a lighter font with full support for CJK (but it's possible
> there is none)
> * download the Chinese Noto pack either automatically (when zh is
> enabled in the preferences for example) or on demand in the admin
> * enable back <auto-detect/> but then we might get back memory issues
> in systems with lots of fonts installed like we used to
> * just explain how to install a font by hand (it should be nothing
> more that extracting it directly in <xwikihome>/WEB-INF/fonts/ folder)
>
> On Thu, Mar 9, 2017 at 2:38 PM, Billtec <[hidden email]> wrote:
>> Is there anything I could do to make Chinese characters available in current
>> release? Like manually replace the FreeFont with some other fonts?
>>
>>
>>
>> --
>> View this message in context: http://xwiki.475771.n2.nabble.com/Chinese-characters-will-be-replaced-by-when-exporting-a-page-to-PDF-tp7603013p7603036.html
>> Sent from the XWiki- Users mailing list archive at Nabble.com.
>
>
>
> --
> Thomas Mortagne



--
Thomas Mortagne
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Chinese characters will be replaced by # when exporting a page to PDF

Billtec
Thanks a lot for your reply. I've created an issue on jira.xwiki.org as XWIKI-14092.
Loading...