[Proposal] Weblate as XWiki's translation platform

classic Classic list List threaded Threaded
8 messages Options
Reply | Threaded
Open this post in threaded view
|

[Proposal] Weblate as XWiki's translation platform

Adel Atallah
Hello everyone,

As you may already know, we have thought about replacing the l10n platform
which is becoming too slow. Weblate seems to be a good replacement choice,
as it will able contributors to have their name in the commits and it has
every features needed to make translations.

One problem is that XWiki doesn't use a standard method to process
translation files. We can solve that by creating some scripts to convert
XWiki translation files into one that Weblate can understand.

A detailed solution can be found here :
http://design.xwiki.org/xwiki/bin/view/Proposal/WeblateasXWikistranslationplatform

Feel free to discuss it by responding here or directly in the design page.

Thanks,
<http://www.xwiki.com/> *Adel Atallah*
*Product developer intern*
[hidden email] <[hidden email]>
tel: +33 (0)6 12 96 35 06
Reply | Threaded
Open this post in threaded view
|

Re: [Proposal] Weblate as XWiki's translation platform

caubin
On 04/23/2018 12:18 PM, Adel Atallah wrote:
> Hello everyone,

Hi,

AFAICS Weblate looks nice, but haven't we looked at other alternatives ?

Here are some of them :

* http://jabylon.org/ (uses Java, so it might be easier to integrate)
* http://zanata.org/
* http://pootle.translatehouse.org/
* https://pontoon.mozilla.org/

Also, I think that it has already been discussed, either on
https://xwiki.markmail.org/search/?q=weblate or on IRC, but being able
to use xwiki.org as an identity provider on our translation platform
would be neat, so it could be something to look for in the possible
solutions.

> As you may already know, we have thought about replacing the l10n platform
> which is becoming too slow. Weblate seems to be a good replacement choice,
> as it will able contributors to have their name in the commits and it has
> every features needed to make translations.
>
> One problem is that XWiki doesn't use a standard method to process
> translation files. We can solve that by creating some scripts to convert
> XWiki translation files into one that Weblate can understand.
>
> A detailed solution can be found here :
> http://design.xwiki.org/xwiki/bin/view/Proposal/WeblateasXWikistranslationplatform

We'll indeed need to convert our translation files to a more "standard"
format, but I fear that using Python might decrease the maintainability
of those scripts.

WDYT ?

> Feel free to discuss it by responding here or directly in the design page.
>
> Thanks,
> <http://www.xwiki.com/> *Adel Atallah*
> *Product developer intern*
> [hidden email] <[hidden email]>
> tel: +33 (0)6 12 96 35 06
>

Thanks,
Clément
Reply | Threaded
Open this post in threaded view
|

Re: [Proposal] Weblate as XWiki's translation platform

Adel Atallah
Hi Clément,

Thanks for your feedback.

I think Weblate offers all the necessary features like git and github
integration, making PRs instead of directly pushing changes.
We should also be able to use xwiki.org as an identity provider using an
OAuth2 plugin.
If you see a missing feature that other tools have, we can consider them
for sure.

As for Python, why would it decrease maintainability and other what?

Thanks again,

<http://www.xwiki.com/> *Adel Atallah*
*Product developer intern*
[hidden email]
tel: +33 (0)6 12 96 35 06

On Tue, Apr 24, 2018 at 3:44 PM, Clément Aubin <[hidden email]> wrote:

> On 04/23/2018 12:18 PM, Adel Atallah wrote:
> > Hello everyone,
>
> Hi,
>
> AFAICS Weblate looks nice, but haven't we looked at other alternatives ?
>
> Here are some of them :
>
> * http://jabylon.org/ (uses Java, so it might be easier to integrate)
> * http://zanata.org/
> * http://pootle.translatehouse.org/
> * https://pontoon.mozilla.org/
>
> Also, I think that it has already been discussed, either on
> https://xwiki.markmail.org/search/?q=weblate or on IRC, but being able
> to use xwiki.org as an identity provider on our translation platform
> would be neat, so it could be something to look for in the possible
> solutions.
>
> > As you may already know, we have thought about replacing the l10n
> platform
> > which is becoming too slow. Weblate seems to be a good replacement
> choice,
> > as it will able contributors to have their name in the commits and it has
> > every features needed to make translations.
> >
> > One problem is that XWiki doesn't use a standard method to process
> > translation files. We can solve that by creating some scripts to convert
> > XWiki translation files into one that Weblate can understand.
> >
> > A detailed solution can be found here :
> > http://design.xwiki.org/xwiki/bin/view/Proposal/
> WeblateasXWikistranslationplatform
>
> We'll indeed need to convert our translation files to a more "standard"
> format, but I fear that using Python might decrease the maintainability
> of those scripts.
>
> WDYT ?
>
> > Feel free to discuss it by responding here or directly in the design
> page.
> >
> > Thanks,
> > <http://www.xwiki.com/> *Adel Atallah*
> > *Product developer intern*
> > [hidden email] <[hidden email]>
> > tel: +33 (0)6 12 96 35 06
> >
>
> Thanks,
> Clément
>
Reply | Threaded
Open this post in threaded view
|

Re: [Proposal] Weblate as XWiki's translation platform

vmassol
Administrator
In reply to this post by caubin


> On 24 Apr 2018, at 15:44, Clément Aubin <[hidden email]> wrote:
>
> On 04/23/2018 12:18 PM, Adel Atallah wrote:
>> Hello everyone,
>
> Hi,
>
> AFAICS Weblate looks nice, but haven't we looked at other alternatives ?
>
> Here are some of them :
>
> * http://jabylon.org/ (uses Java, so it might be easier to integrate)
> * http://zanata.org/
> * http://pootle.translatehouse.org/
> * https://pontoon.mozilla.org/
>
> Also, I think that it has already been discussed, either on
> https://xwiki.markmail.org/search/?q=weblate or on IRC, but being able
> to use xwiki.org as an identity provider on our translation platform
> would be neat, so it could be something to look for in the possible
> solutions.
>
>> As you may already know, we have thought about replacing the l10n platform
>> which is becoming too slow. Weblate seems to be a good replacement choice,
>> as it will able contributors to have their name in the commits and it has
>> every features needed to make translations.
>>
>> One problem is that XWiki doesn't use a standard method to process
>> translation files. We can solve that by creating some scripts to convert
>> XWiki translation files into one that Weblate can understand.
>>
>> A detailed solution can be found here :
>> http://design.xwiki.org/xwiki/bin/view/Proposal/WeblateasXWikistranslationplatform
>
> We'll indeed need to convert our translation files to a more "standard"
> format, but I fear that using Python might decrease the maintainability
> of those scripts.

- You said "Weblate is able to run scripts after pulling changes from the git repository and before committing changes.”. I don’t understand this sentence. You mean to sync files that are located in our Git repos to weblate (git —> weblate)? When you say “committing changes” you mean loading changes from git and inside weblate?

- You said "XML files with properties as content will be treated the same way as above”. I couldn’t find what you meant by “same way as above”.  Certainly you’ll need to generate some properties file out of the XML file, no?

- Globally I find what you describe very complex and a bit hackish. Especially the part about having some “.translation” directory which I assume you’ll want to commit in our git repos?

- Isn’t it possible to add some new file format inside weblate instead so that it understands properties inside XML files, etc.?

- Another idea: using weblate’s REST APIs to push/pull from weblate, using github web hooks to perform conversions:
— when a change is pushed to a translation file in github, run a script to import it inside weblate using weblate’s REST API
— when a change is done inside weblate and a PR is sent to our repos by weblate, run a script to convert the content to the proper format and modify the PR accordingly

Even that sounds complex IMO. Being able to teach weblate about our format would seem the simplest. If we could do this we could even send weblate a PR to support our format natively.

You also need to handle deprecated keys, ability to rename/move a key and all its translations, etc.

Thanks
-Vincent

>
> WDYT ?
>
>> Feel free to discuss it by responding here or directly in the design page.
>>
>> Thanks,
>> <http://www.xwiki.com/> *Adel Atallah*
>> *Product developer intern*
>> [hidden email] <[hidden email]>
>> tel: +33 (0)6 12 96 35 06
>>
>
> Thanks,
> Clément

Reply | Threaded
Open this post in threaded view
|

Re: [Proposal] Weblate as XWiki's translation platform

Adel Atallah
Hi Vincent,

- What I mean by "pulling changes" is to update the git repository (e.g.
xwiki-platform) used by Weblate to find translations, and what I mean by
"committing changes" is to create a commit with the new translations made
by a user.

- When I wrote “same way as above”, I meant that the script should extract
the content of the XML file and then treat it like a regular Java
properties file (as it should in the previous case).

- The ".translation" directory doesn't have to (and shouldn't) be committed
in the git repos.

- AFAICS, you can't simply add a new file format to weblate as it uses
another project for translations (
http://docs.translatehouse.org/projects/translate-toolkit/en/latest/formats/index.html)
and this tool also uses converters.

- About your other idea, this is exactly what the scripts are performing
but without using any API and using weblate's hooks.

- As for the other features, I don't think any of the suggested tools are
able to handle deprecated keys and renaming. We could just exclude the
deprecated section of the generated translations files so it doesn't appear
on Weblate.

I know that having to do what I described is a bit hackish but I think that
if we want to use a dedicated tool to deal with translations, we will have
to create a converter. You can find a list of converter used by the
translate toolkit here
: http://docs.translatehouse.org/projects/translate-toolkit/en/latest/commands/index.html
<http://docs.translatehouse.org/projects/translate-toolkit/en/latest/commands/index.html>
.

Thanks for your reply,

<http://www.xwiki.com/> *Adel Atallah*
*Product developer intern*
[hidden email]
tel: +33 (0)6 12 96 35 06

On Wed, Apr 25, 2018 at 10:47 AM, Vincent Massol <[hidden email]> wrote:

>
>
> > On 24 Apr 2018, at 15:44, Clément Aubin <[hidden email]> wrote:
> >
> > On 04/23/2018 12:18 PM, Adel Atallah wrote:
> >> Hello everyone,
> >
> > Hi,
> >
> > AFAICS Weblate looks nice, but haven't we looked at other alternatives ?
> >
> > Here are some of them :
> >
> > * http://jabylon.org/ (uses Java, so it might be easier to integrate)
> > * http://zanata.org/
> > * http://pootle.translatehouse.org/
> > * https://pontoon.mozilla.org/
> >
> > Also, I think that it has already been discussed, either on
> > https://xwiki.markmail.org/search/?q=weblate or on IRC, but being able
> > to use xwiki.org as an identity provider on our translation platform
> > would be neat, so it could be something to look for in the possible
> > solutions.
> >
> >> As you may already know, we have thought about replacing the l10n
> platform
> >> which is becoming too slow. Weblate seems to be a good replacement
> choice,
> >> as it will able contributors to have their name in the commits and it
> has
> >> every features needed to make translations.
> >>
> >> One problem is that XWiki doesn't use a standard method to process
> >> translation files. We can solve that by creating some scripts to convert
> >> XWiki translation files into one that Weblate can understand.
> >>
> >> A detailed solution can be found here :
> >> http://design.xwiki.org/xwiki/bin/view/Proposal/
> WeblateasXWikistranslationplatform
> >
> > We'll indeed need to convert our translation files to a more "standard"
> > format, but I fear that using Python might decrease the maintainability
> > of those scripts.
>
> - You said "Weblate is able to run scripts after pulling changes from the
> git repository and before committing changes.”. I don’t understand this
> sentence. You mean to sync files that are located in our Git repos to
> weblate (git —> weblate)? When you say “committing changes” you mean
> loading changes from git and inside weblate?
>
> - You said "XML files with properties as content will be treated the same
> way as above”. I couldn’t find what you meant by “same way as above”.
> Certainly you’ll need to generate some properties file out of the XML file,
> no?
>
> - Globally I find what you describe very complex and a bit hackish.
> Especially the part about having some “.translation” directory which I
> assume you’ll want to commit in our git repos?
>
> - Isn’t it possible to add some new file format inside weblate instead so
> that it understands properties inside XML files, etc.?
>
> - Another idea: using weblate’s REST APIs to push/pull from weblate, using
> github web hooks to perform conversions:
> — when a change is pushed to a translation file in github, run a script to
> import it inside weblate using weblate’s REST API
> — when a change is done inside weblate and a PR is sent to our repos by
> weblate, run a script to convert the content to the proper format and
> modify the PR accordingly
>
> Even that sounds complex IMO. Being able to teach weblate about our format
> would seem the simplest. If we could do this we could even send weblate a
> PR to support our format natively.
>
> You also need to handle deprecated keys, ability to rename/move a key and
> all its translations, etc.
>
> Thanks
> -Vincent
>
> >
> > WDYT ?
> >
> >> Feel free to discuss it by responding here or directly in the design
> page.
> >>
> >> Thanks,
> >> <http://www.xwiki.com/> *Adel Atallah*
> >> *Product developer intern*
> >> [hidden email] <[hidden email]>
> >> tel: +33 (0)6 12 96 35 06
> >>
> >
> > Thanks,
> > Clément
>
>
Reply | Threaded
Open this post in threaded view
|

Re: [Proposal] Weblate as XWiki's translation platform

Sergiu Dumitriu-3
In reply to this post by caubin
On 04/24/2018 09:44 AM, Clément Aubin wrote:

> On 04/23/2018 12:18 PM, Adel Atallah wrote:
>> Hello everyone,
>
> Hi,
>
> AFAICS Weblate looks nice, but haven't we looked at other alternatives ?
>
> Here are some of them :
>
> * http://jabylon.org/ (uses Java, so it might be easier to integrate)
> * http://zanata.org/
> * http://pootle.translatehouse.org/
> * https://pontoon.mozilla.org/

I've been quite happy with https://crowdin.com/ , which handles
ApplicationResources.properties very well, but isn't ideal for the .xml
translations. It's free for open source projects.

The problem with the .xml files is that apart from the translatable
content, which it handles well, it tries to commit the exact same XML
markup as in the "source" file, which doesn't really work for XWiki. As
a workaround, I just cat|sed the contents of the XML files and upload it
as a non-github-tracked resource. The alternative is to
rebase-interactive each pull request it creates to discard the commits
that try to re-format the XML files, but that gets tedious after a while.

> Also, I think that it has already been discussed, either on
> https://xwiki.markmail.org/search/?q=weblate or on IRC, but being able
> to use xwiki.org as an identity provider on our translation platform
> would be neat, so it could be something to look for in the possible
> solutions.
>
>> As you may already know, we have thought about replacing the l10n platform
>> which is becoming too slow. Weblate seems to be a good replacement choice,
>> as it will able contributors to have their name in the commits and it has
>> every features needed to make translations.
>>
>> One problem is that XWiki doesn't use a standard method to process
>> translation files. We can solve that by creating some scripts to convert
>> XWiki translation files into one that Weblate can understand.
>>
>> A detailed solution can be found here :
>> http://design.xwiki.org/xwiki/bin/view/Proposal/WeblateasXWikistranslationplatform
>
> We'll indeed need to convert our translation files to a more "standard"
> format, but I fear that using Python might decrease the maintainability
> of those scripts.
>
> WDYT ?
>
>> Feel free to discuss it by responding here or directly in the design page.
>>
>> Thanks,
>> <http://www.xwiki.com/> *Adel Atallah*
>> *Product developer intern*
>> [hidden email] <[hidden email]>
>> tel: +33 (0)6 12 96 35 06
>>
>
> Thanks,
> Clément
>


--
Sergiu Dumitriu
http://purl.org/net/sergiu
Reply | Threaded
Open this post in threaded view
|

Re: [Proposal] Weblate as XWiki's translation platform

Thomas Mortagne
Administrator
From what I remember https://crowdin.com/ quota for OpenSource plan
was too low for XWiki.

On Wed, Apr 25, 2018 at 10:15 PM, Sergiu Dumitriu <[hidden email]> wrote:

> On 04/24/2018 09:44 AM, Clément Aubin wrote:
>> On 04/23/2018 12:18 PM, Adel Atallah wrote:
>>> Hello everyone,
>>
>> Hi,
>>
>> AFAICS Weblate looks nice, but haven't we looked at other alternatives ?
>>
>> Here are some of them :
>>
>> * http://jabylon.org/ (uses Java, so it might be easier to integrate)
>> * http://zanata.org/
>> * http://pootle.translatehouse.org/
>> * https://pontoon.mozilla.org/
>
> I've been quite happy with https://crowdin.com/ , which handles
> ApplicationResources.properties very well, but isn't ideal for the .xml
> translations. It's free for open source projects.
>
> The problem with the .xml files is that apart from the translatable
> content, which it handles well, it tries to commit the exact same XML
> markup as in the "source" file, which doesn't really work for XWiki. As
> a workaround, I just cat|sed the contents of the XML files and upload it
> as a non-github-tracked resource. The alternative is to
> rebase-interactive each pull request it creates to discard the commits
> that try to re-format the XML files, but that gets tedious after a while.
>
>> Also, I think that it has already been discussed, either on
>> https://xwiki.markmail.org/search/?q=weblate or on IRC, but being able
>> to use xwiki.org as an identity provider on our translation platform
>> would be neat, so it could be something to look for in the possible
>> solutions.
>>
>>> As you may already know, we have thought about replacing the l10n platform
>>> which is becoming too slow. Weblate seems to be a good replacement choice,
>>> as it will able contributors to have their name in the commits and it has
>>> every features needed to make translations.
>>>
>>> One problem is that XWiki doesn't use a standard method to process
>>> translation files. We can solve that by creating some scripts to convert
>>> XWiki translation files into one that Weblate can understand.
>>>
>>> A detailed solution can be found here :
>>> http://design.xwiki.org/xwiki/bin/view/Proposal/WeblateasXWikistranslationplatform
>>
>> We'll indeed need to convert our translation files to a more "standard"
>> format, but I fear that using Python might decrease the maintainability
>> of those scripts.
>>
>> WDYT ?
>>
>>> Feel free to discuss it by responding here or directly in the design page.
>>>
>>> Thanks,
>>> <http://www.xwiki.com/> *Adel Atallah*
>>> *Product developer intern*
>>> [hidden email] <[hidden email]>
>>> tel: +33 (0)6 12 96 35 06
>>>
>>
>> Thanks,
>> Clément
>>
>
>
> --
> Sergiu Dumitriu
> http://purl.org/net/sergiu



--
Thomas Mortagne
Reply | Threaded
Open this post in threaded view
|

Re: [Proposal] Weblate as XWiki's translation platform

vmassol
Administrator


> On 26 Apr 2018, at 10:21, Thomas Mortagne <[hidden email]> wrote:
>
> From what I remember https://crowdin.com/ quota for OpenSource plan
> was too low for XWiki.

I also prefer weblate which is more open source than crowdin (which I don’t know if you can install locally and it’s a business and they could change their “opensource” plans at any time so we'd need a way to install it locally).

Thanks
-Vincent

>
> On Wed, Apr 25, 2018 at 10:15 PM, Sergiu Dumitriu <[hidden email]> wrote:
>> On 04/24/2018 09:44 AM, Clément Aubin wrote:
>>> On 04/23/2018 12:18 PM, Adel Atallah wrote:
>>>> Hello everyone,
>>>
>>> Hi,
>>>
>>> AFAICS Weblate looks nice, but haven't we looked at other alternatives ?
>>>
>>> Here are some of them :
>>>
>>> * http://jabylon.org/ (uses Java, so it might be easier to integrate)
>>> * http://zanata.org/
>>> * http://pootle.translatehouse.org/
>>> * https://pontoon.mozilla.org/
>>
>> I've been quite happy with https://crowdin.com/ , which handles
>> ApplicationResources.properties very well, but isn't ideal for the .xml
>> translations. It's free for open source projects.
>>
>> The problem with the .xml files is that apart from the translatable
>> content, which it handles well, it tries to commit the exact same XML
>> markup as in the "source" file, which doesn't really work for XWiki. As
>> a workaround, I just cat|sed the contents of the XML files and upload it
>> as a non-github-tracked resource. The alternative is to
>> rebase-interactive each pull request it creates to discard the commits
>> that try to re-format the XML files, but that gets tedious after a while.
>>
>>> Also, I think that it has already been discussed, either on
>>> https://xwiki.markmail.org/search/?q=weblate or on IRC, but being able
>>> to use xwiki.org as an identity provider on our translation platform
>>> would be neat, so it could be something to look for in the possible
>>> solutions.
>>>
>>>> As you may already know, we have thought about replacing the l10n platform
>>>> which is becoming too slow. Weblate seems to be a good replacement choice,
>>>> as it will able contributors to have their name in the commits and it has
>>>> every features needed to make translations.
>>>>
>>>> One problem is that XWiki doesn't use a standard method to process
>>>> translation files. We can solve that by creating some scripts to convert
>>>> XWiki translation files into one that Weblate can understand.
>>>>
>>>> A detailed solution can be found here :
>>>> http://design.xwiki.org/xwiki/bin/view/Proposal/WeblateasXWikistranslationplatform
>>>
>>> We'll indeed need to convert our translation files to a more "standard"
>>> format, but I fear that using Python might decrease the maintainability
>>> of those scripts.
>>>
>>> WDYT ?
>>>
>>>> Feel free to discuss it by responding here or directly in the design page.
>>>>
>>>> Thanks,
>>>> <http://www.xwiki.com/> *Adel Atallah*
>>>> *Product developer intern*
>>>> [hidden email] <[hidden email]>
>>>> tel: +33 (0)6 12 96 35 06
>>>>
>>>
>>> Thanks,
>>> Clément
>>>
>>
>>
>> --
>> Sergiu Dumitriu
>> http://purl.org/net/sergiu
>
>
>
> --
> Thomas Mortagne