[Brainstorming] Should we support multibranch translations?

classic Classic list List threaded Threaded
10 messages Options
Reply | Threaded
Open this post in threaded view
|

[Brainstorming] Should we support multibranch translations?

Adel Atallah
Hi,

Following my previous email on "How should we review translations?", I'd
like to know here if we should support automatic multibranch translations
in Weblate.

What I mean here is that with the old l10n platform, we would apply new
translations on multiple git branches (for some projects like XWiki
Platform). It was important to have new translations applied on LTS
releases and other branches.

The problem is that we can't tell Weblate to automatically push changes on
multiple branches. We have discuss the problem with the maintainer here:
https://github.com/WeblateOrg/weblate/issues/2016.
What we can do is to duplicate Weblate components (a component is just a
file to translate) for as many branches as we need. Making a change to a
translation key (e.g. tour.homepageTour.pageMenu.contentB) will propagate
the change to every other components with the same key. This way we can
have a PR made with the same change on every branch we want.

So here are the two options:

1) We keep the actual behavior
Pros:
  - We will only have one PR to review (on master branch)
Cons:
  - We will have to apply new changes to other branches ourselves when
needed

2) We duplicate components
Pros:
  - Changes will automatically be made for every specified branches
Cons:
  - Some work to do: we can't create all the new components by hand so we
will have to generate every components in some way
  - It will make Weblate much more complex because you can't hide
components (https://i.imgur.com/YJ8qtUz.png)

I prefer option 1 because it will make Weblate easier to use.

For option 2, we can also disable translation propagation and let people
make translations on the branch they want.

Thanks,
Adel
Reply | Threaded
Open this post in threaded view
|

Re: [Brainstorming] Should we support multibranch translations?

vmassol
Administrator
Hi Adel,

> On 18 May 2018, at 11:40, Adel Atallah <[hidden email]> wrote:
>
> Hi,
>
> Following my previous email on "How should we review translations?", I'd
> like to know here if we should support automatic multibranch translations
> in Weblate.
>
> What I mean here is that with the old l10n platform, we would apply new
> translations on multiple git branches (for some projects like XWiki
> Platform). It was important to have new translations applied on LTS
> releases and other branches.
>
> The problem is that we can't tell Weblate to automatically push changes on
> multiple branches. We have discuss the problem with the maintainer here:
> https://github.com/WeblateOrg/weblate/issues/2016.
> What we can do is to duplicate Weblate components (a component is just a
> file to translate) for as many branches as we need. Making a change to a
> translation key (e.g. tour.homepageTour.pageMenu.contentB) will propagate
> the change to every other components with the same key. This way we can
> have a PR made with the same change on every branch we want.
>
> So here are the two options:
>
> 1) We keep the actual behavior
> Pros:
>  - We will only have one PR to review (on master branch)
> Cons:
>  - We will have to apply new changes to other branches ourselves when
> needed

This is not fully the current behavior since right now the merge on a branch is done by the RM in one go for all translations.

With this proposal 1) someone (whom?) will need to merge the *various* commits done by the weblate PRs, on a need-be basis.

So this raises the following questions:
* Who is responsible for the branch merges and more specifically the LTS one. The RM?
* If so, what strategy do we decide, i.e. which translations do we want to merge or not? And what tool would be provide the RM or someone else to list all commits related to translations?

>
> 2) We duplicate components
> Pros:
>  - Changes will automatically be made for every specified branches
> Cons:
>  - Some work to do: we can't create all the new components by hand so we
> will have to generate every components in some way
>  - It will make Weblate much more complex because you can't hide
> components (https://i.imgur.com/YJ8qtUz.png)

This option 2 is complex because not only the hassle of creating and *Deleting* components (when the branch is closed) but also we need to decide which components to duplicate (there might components that only exist on master for ex). Ideally we would need a script to automatically add translation components for a branch.

If we can automate this then it’s not too bad but still complex. And indeed there’s the risk that users will translate branches by mistake instead of translating master.

> I prefer option 1 because it will make Weblate easier to use.
>
> For option 2, we can also disable translation propagation and let people
> make translations on the branch they want.

I can’t say which one I prefer yet because we need to answer the questions I raised for 1) first.

The general question is: what translations do we want to merge for the LTS branch? I think we can agree that we don’t really care about merging translations for the short-lived branches such as 10.4.x.

Thanks
-Vincent

>
> Thanks,
> Adel

Reply | Threaded
Open this post in threaded view
|

Re: [Brainstorming] Should we support multibranch translations?

Thomas Mortagne
Administrator
Option 2) would create too much of a mess on weblate side IMO (until
we can hide branches at least).

I would go for 1) for now and follow progress on Weblate product to
provide a clean solution for this use case.

That being said we need to find a solution for LTS (I don't think we
care about stable branch bugfixes releases and we could do it by hand
for RC branches since it's only 1 week usually). Here are some ideas:
a) it should not be hard to write a script which get all the weblate
commits from master since last weblate commit we can find in the
branch and cherry-pick them (probably also display a diff and ask for
confirmation for each of them). This would be executed before the
release by the release manager.
b) I guess it's possible to write or find a tool which automatically
create a pull request on the LTS branch when a weblate pull request is
applied
c) Anyone who apply a weblate pull request is responsible for applying
it on LTS branch. I don't trust us too much on that.



a does not seems complex to do (but of course someone need to spend time on it).
c does not require any tooling but I don't think it will work, I'm
sure we will keep forgetting to cherry-pick.


b would be nice if someone find a tool to do that. If not then I guess
the more realistic option is a.

On Fri, May 18, 2018 at 5:11 PM, Vincent Massol <[hidden email]> wrote:

> Hi Adel,
>
>> On 18 May 2018, at 11:40, Adel Atallah <[hidden email]> wrote:
>>
>> Hi,
>>
>> Following my previous email on "How should we review translations?", I'd
>> like to know here if we should support automatic multibranch translations
>> in Weblate.
>>
>> What I mean here is that with the old l10n platform, we would apply new
>> translations on multiple git branches (for some projects like XWiki
>> Platform). It was important to have new translations applied on LTS
>> releases and other branches.
>>
>> The problem is that we can't tell Weblate to automatically push changes on
>> multiple branches. We have discuss the problem with the maintainer here:
>> https://github.com/WeblateOrg/weblate/issues/2016.
>> What we can do is to duplicate Weblate components (a component is just a
>> file to translate) for as many branches as we need. Making a change to a
>> translation key (e.g. tour.homepageTour.pageMenu.contentB) will propagate
>> the change to every other components with the same key. This way we can
>> have a PR made with the same change on every branch we want.
>>
>> So here are the two options:
>>
>> 1) We keep the actual behavior
>> Pros:
>>  - We will only have one PR to review (on master branch)
>> Cons:
>>  - We will have to apply new changes to other branches ourselves when
>> needed
>
> This is not fully the current behavior since right now the merge on a branch is done by the RM in one go for all translations.
>
> With this proposal 1) someone (whom?) will need to merge the *various* commits done by the weblate PRs, on a need-be basis.
>
> So this raises the following questions:
> * Who is responsible for the branch merges and more specifically the LTS one. The RM?
> * If so, what strategy do we decide, i.e. which translations do we want to merge or not? And what tool would be provide the RM or someone else to list all commits related to translations?
>
>>
>> 2) We duplicate components
>> Pros:
>>  - Changes will automatically be made for every specified branches
>> Cons:
>>  - Some work to do: we can't create all the new components by hand so we
>> will have to generate every components in some way
>>  - It will make Weblate much more complex because you can't hide
>> components (https://i.imgur.com/YJ8qtUz.png)
>
> This option 2 is complex because not only the hassle of creating and *Deleting* components (when the branch is closed) but also we need to decide which components to duplicate (there might components that only exist on master for ex). Ideally we would need a script to automatically add translation components for a branch.
>
> If we can automate this then it’s not too bad but still complex. And indeed there’s the risk that users will translate branches by mistake instead of translating master.
>
>> I prefer option 1 because it will make Weblate easier to use.
>>
>> For option 2, we can also disable translation propagation and let people
>> make translations on the branch they want.
>
> I can’t say which one I prefer yet because we need to answer the questions I raised for 1) first.
>
> The general question is: what translations do we want to merge for the LTS branch? I think we can agree that we don’t really care about merging translations for the short-lived branches such as 10.4.x.
>
> Thanks
> -Vincent
>
>>
>> Thanks,
>> Adel
>



--
Thomas Mortagne
Reply | Threaded
Open this post in threaded view
|

Re: [Brainstorming] Should we support multibranch translations?

Ecaterina Moraru (Valica)
+1 for 1)
Make sure the commit has a marker like "[Translations]" or "[Weblate]" for
the the step in the release process, so that we can look for them in the
history in order to apply them, in case we really need them.
In practice we don't commit translations for LTS, because usually we make
changes in UI and we don't want to manually check and validate each
translation.

Thanks,
Caty


On Fri, May 18, 2018 at 5:47 PM, Thomas Mortagne <[hidden email]>
wrote:

> Option 2) would create too much of a mess on weblate side IMO (until
> we can hide branches at least).
>
> I would go for 1) for now and follow progress on Weblate product to
> provide a clean solution for this use case.
>
> That being said we need to find a solution for LTS (I don't think we
> care about stable branch bugfixes releases and we could do it by hand
> for RC branches since it's only 1 week usually). Here are some ideas:
> a) it should not be hard to write a script which get all the weblate
> commits from master since last weblate commit we can find in the
> branch and cherry-pick them (probably also display a diff and ask for
> confirmation for each of them). This would be executed before the
> release by the release manager.
> b) I guess it's possible to write or find a tool which automatically
> create a pull request on the LTS branch when a weblate pull request is
> applied
> c) Anyone who apply a weblate pull request is responsible for applying
> it on LTS branch. I don't trust us too much on that.
>
>
>
> a does not seems complex to do (but of course someone need to spend time
> on it).
> c does not require any tooling but I don't think it will work, I'm
> sure we will keep forgetting to cherry-pick.
>
>
> b would be nice if someone find a tool to do that. If not then I guess
> the more realistic option is a.
>
> On Fri, May 18, 2018 at 5:11 PM, Vincent Massol <[hidden email]>
> wrote:
> > Hi Adel,
> >
> >> On 18 May 2018, at 11:40, Adel Atallah <[hidden email]> wrote:
> >>
> >> Hi,
> >>
> >> Following my previous email on "How should we review translations?", I'd
> >> like to know here if we should support automatic multibranch
> translations
> >> in Weblate.
> >>
> >> What I mean here is that with the old l10n platform, we would apply new
> >> translations on multiple git branches (for some projects like XWiki
> >> Platform). It was important to have new translations applied on LTS
> >> releases and other branches.
> >>
> >> The problem is that we can't tell Weblate to automatically push changes
> on
> >> multiple branches. We have discuss the problem with the maintainer here:
> >> https://github.com/WeblateOrg/weblate/issues/2016.
> >> What we can do is to duplicate Weblate components (a component is just a
> >> file to translate) for as many branches as we need. Making a change to a
> >> translation key (e.g. tour.homepageTour.pageMenu.contentB) will
> propagate
> >> the change to every other components with the same key. This way we can
> >> have a PR made with the same change on every branch we want.
> >>
> >> So here are the two options:
> >>
> >> 1) We keep the actual behavior
> >> Pros:
> >>  - We will only have one PR to review (on master branch)
> >> Cons:
> >>  - We will have to apply new changes to other branches ourselves when
> >> needed
> >
> > This is not fully the current behavior since right now the merge on a
> branch is done by the RM in one go for all translations.
> >
> > With this proposal 1) someone (whom?) will need to merge the *various*
> commits done by the weblate PRs, on a need-be basis.
> >
> > So this raises the following questions:
> > * Who is responsible for the branch merges and more specifically the LTS
> one. The RM?
> > * If so, what strategy do we decide, i.e. which translations do we want
> to merge or not? And what tool would be provide the RM or someone else to
> list all commits related to translations?
> >
> >>
> >> 2) We duplicate components
> >> Pros:
> >>  - Changes will automatically be made for every specified branches
> >> Cons:
> >>  - Some work to do: we can't create all the new components by hand so we
> >> will have to generate every components in some way
> >>  - It will make Weblate much more complex because you can't hide
> >> components (https://i.imgur.com/YJ8qtUz.png)
> >
> > This option 2 is complex because not only the hassle of creating and
> *Deleting* components (when the branch is closed) but also we need to
> decide which components to duplicate (there might components that only
> exist on master for ex). Ideally we would need a script to automatically
> add translation components for a branch.
> >
> > If we can automate this then it’s not too bad but still complex. And
> indeed there’s the risk that users will translate branches by mistake
> instead of translating master.
> >
> >> I prefer option 1 because it will make Weblate easier to use.
> >>
> >> For option 2, we can also disable translation propagation and let people
> >> make translations on the branch they want.
> >
> > I can’t say which one I prefer yet because we need to answer the
> questions I raised for 1) first.
> >
> > The general question is: what translations do we want to merge for the
> LTS branch? I think we can agree that we don’t really care about merging
> translations for the short-lived branches such as 10.4.x.
> >
> > Thanks
> > -Vincent
> >
> >>
> >> Thanks,
> >> Adel
> >
>
>
>
> --
> Thomas Mortagne
>
Reply | Threaded
Open this post in threaded view
|

Re: [Brainstorming] Should we support multibranch translations?

Adel Atallah
So for 1), we still need to decide how we merge new translations into
LTS releases (or other branches).
An idea would be to write a script to let the RM apply new changes to
a specific branch.
One way to do it would be to write a script to find every translation
commits since a date then review and apply them.
An other way would be to use the list of translation files that we
already have and write a script to replace (checkout) those files from
master to the specified branch.

WDYT?

Thanks,
Adel
Adel Atallah
Product developer intern
[hidden email]
tel: +33 (0)6 12 96 35 06


On Fri, May 18, 2018 at 6:47 PM, Ecaterina Moraru (Valica)
<[hidden email]> wrote:

> +1 for 1)
> Make sure the commit has a marker like "[Translations]" or "[Weblate]" for
> the the step in the release process, so that we can look for them in the
> history in order to apply them, in case we really need them.
> In practice we don't commit translations for LTS, because usually we make
> changes in UI and we don't want to manually check and validate each
> translation.
>
> Thanks,
> Caty
>
>
> On Fri, May 18, 2018 at 5:47 PM, Thomas Mortagne <[hidden email]>
> wrote:
>
>> Option 2) would create too much of a mess on weblate side IMO (until
>> we can hide branches at least).
>>
>> I would go for 1) for now and follow progress on Weblate product to
>> provide a clean solution for this use case.
>>
>> That being said we need to find a solution for LTS (I don't think we
>> care about stable branch bugfixes releases and we could do it by hand
>> for RC branches since it's only 1 week usually). Here are some ideas:
>> a) it should not be hard to write a script which get all the weblate
>> commits from master since last weblate commit we can find in the
>> branch and cherry-pick them (probably also display a diff and ask for
>> confirmation for each of them). This would be executed before the
>> release by the release manager.
>> b) I guess it's possible to write or find a tool which automatically
>> create a pull request on the LTS branch when a weblate pull request is
>> applied
>> c) Anyone who apply a weblate pull request is responsible for applying
>> it on LTS branch. I don't trust us too much on that.
>>
>>
>>
>> a does not seems complex to do (but of course someone need to spend time
>> on it).
>> c does not require any tooling but I don't think it will work, I'm
>> sure we will keep forgetting to cherry-pick.
>>
>>
>> b would be nice if someone find a tool to do that. If not then I guess
>> the more realistic option is a.
>>
>> On Fri, May 18, 2018 at 5:11 PM, Vincent Massol <[hidden email]>
>> wrote:
>> > Hi Adel,
>> >
>> >> On 18 May 2018, at 11:40, Adel Atallah <[hidden email]> wrote:
>> >>
>> >> Hi,
>> >>
>> >> Following my previous email on "How should we review translations?", I'd
>> >> like to know here if we should support automatic multibranch
>> translations
>> >> in Weblate.
>> >>
>> >> What I mean here is that with the old l10n platform, we would apply new
>> >> translations on multiple git branches (for some projects like XWiki
>> >> Platform). It was important to have new translations applied on LTS
>> >> releases and other branches.
>> >>
>> >> The problem is that we can't tell Weblate to automatically push changes
>> on
>> >> multiple branches. We have discuss the problem with the maintainer here:
>> >> https://github.com/WeblateOrg/weblate/issues/2016.
>> >> What we can do is to duplicate Weblate components (a component is just a
>> >> file to translate) for as many branches as we need. Making a change to a
>> >> translation key (e.g. tour.homepageTour.pageMenu.contentB) will
>> propagate
>> >> the change to every other components with the same key. This way we can
>> >> have a PR made with the same change on every branch we want.
>> >>
>> >> So here are the two options:
>> >>
>> >> 1) We keep the actual behavior
>> >> Pros:
>> >>  - We will only have one PR to review (on master branch)
>> >> Cons:
>> >>  - We will have to apply new changes to other branches ourselves when
>> >> needed
>> >
>> > This is not fully the current behavior since right now the merge on a
>> branch is done by the RM in one go for all translations.
>> >
>> > With this proposal 1) someone (whom?) will need to merge the *various*
>> commits done by the weblate PRs, on a need-be basis.
>> >
>> > So this raises the following questions:
>> > * Who is responsible for the branch merges and more specifically the LTS
>> one. The RM?
>> > * If so, what strategy do we decide, i.e. which translations do we want
>> to merge or not? And what tool would be provide the RM or someone else to
>> list all commits related to translations?
>> >
>> >>
>> >> 2) We duplicate components
>> >> Pros:
>> >>  - Changes will automatically be made for every specified branches
>> >> Cons:
>> >>  - Some work to do: we can't create all the new components by hand so we
>> >> will have to generate every components in some way
>> >>  - It will make Weblate much more complex because you can't hide
>> >> components (https://i.imgur.com/YJ8qtUz.png)
>> >
>> > This option 2 is complex because not only the hassle of creating and
>> *Deleting* components (when the branch is closed) but also we need to
>> decide which components to duplicate (there might components that only
>> exist on master for ex). Ideally we would need a script to automatically
>> add translation components for a branch.
>> >
>> > If we can automate this then it’s not too bad but still complex. And
>> indeed there’s the risk that users will translate branches by mistake
>> instead of translating master.
>> >
>> >> I prefer option 1 because it will make Weblate easier to use.
>> >>
>> >> For option 2, we can also disable translation propagation and let people
>> >> make translations on the branch they want.
>> >
>> > I can’t say which one I prefer yet because we need to answer the
>> questions I raised for 1) first.
>> >
>> > The general question is: what translations do we want to merge for the
>> LTS branch? I think we can agree that we don’t really care about merging
>> translations for the short-lived branches such as 10.4.x.
>> >
>> > Thanks
>> > -Vincent
>> >
>> >>
>> >> Thanks,
>> >> Adel
>> >
>>
>>
>>
>> --
>> Thomas Mortagne
>>
Reply | Threaded
Open this post in threaded view
|

Re: [Brainstorming] Should we support multibranch translations?

vmassol
Administrator
FTR this is what I discussed with Adel and I asked him to post it here so that we can agree.

> On 22 May 2018, at 13:41, Adel Atallah <[hidden email]> wrote:
>
> So for 1), we still need to decide how we merge new translations into
> LTS releases (or other branches).

Idea 1:

> An idea would be to write a script to let the RM apply new changes to
> a specific branch.
> One way to do it would be to write a script to find every translation
> commits since a date then review and apply them.

More specifically this is about finding all commits done by a given user (the weblate user). In practice the commits are done under the translation contributor’s name so we would need to find the merge commit (done with the weblate user name) and find all associated commits (children?).

Idea 2:

> An other way would be to use the list of translation files that we
> already have and write a script to replace (checkout) those files from
> master to the specified branch.

The problem with this approach is that we can have commits related to the translation files that are not related to translations and thus generate false positives.

>
> WDYT?

Idea 3: For each translation files in the known list, query weblate (using the REST API) to get the latest translation and apply them locally.

In practice this will mean find keys in the translation file and replace with the value retrieved from weblate.

One improvement is that we could parse the keys from the local files and file updated translations for them from weblate.

The advantage of idea3 is that I don’t think there are false positives nor merge conflicts which can happen with ideas 1 and 2. The downside is that it may take a lot of time and a lot of REST calls to get them all.

For all these ideas, it’ll the responsibility of the user (RM?) to check the diff and decide to merge/push on the branch the full translations or only a subpart of it.

Aso note that we’re only talking about LTS here since we should not bother for the temporary branches (such as stable-10.4.x).

WDYT?

Thanks
-Vincent


>
> Thanks,
> Adel
> Adel Atallah
> Product developer intern
> [hidden email]
> tel: +33 (0)6 12 96 35 06
>
>
> On Fri, May 18, 2018 at 6:47 PM, Ecaterina Moraru (Valica)
> <[hidden email]> wrote:
>> +1 for 1)
>> Make sure the commit has a marker like "[Translations]" or "[Weblate]" for
>> the the step in the release process, so that we can look for them in the
>> history in order to apply them, in case we really need them.
>> In practice we don't commit translations for LTS, because usually we make
>> changes in UI and we don't want to manually check and validate each
>> translation.
>>
>> Thanks,
>> Caty
>>
>>
>> On Fri, May 18, 2018 at 5:47 PM, Thomas Mortagne <[hidden email]>
>> wrote:
>>
>>> Option 2) would create too much of a mess on weblate side IMO (until
>>> we can hide branches at least).
>>>
>>> I would go for 1) for now and follow progress on Weblate product to
>>> provide a clean solution for this use case.
>>>
>>> That being said we need to find a solution for LTS (I don't think we
>>> care about stable branch bugfixes releases and we could do it by hand
>>> for RC branches since it's only 1 week usually). Here are some ideas:
>>> a) it should not be hard to write a script which get all the weblate
>>> commits from master since last weblate commit we can find in the
>>> branch and cherry-pick them (probably also display a diff and ask for
>>> confirmation for each of them). This would be executed before the
>>> release by the release manager.
>>> b) I guess it's possible to write or find a tool which automatically
>>> create a pull request on the LTS branch when a weblate pull request is
>>> applied
>>> c) Anyone who apply a weblate pull request is responsible for applying
>>> it on LTS branch. I don't trust us too much on that.
>>>
>>>
>>>
>>> a does not seems complex to do (but of course someone need to spend time
>>> on it).
>>> c does not require any tooling but I don't think it will work, I'm
>>> sure we will keep forgetting to cherry-pick.
>>>
>>>
>>> b would be nice if someone find a tool to do that. If not then I guess
>>> the more realistic option is a.
>>>
>>> On Fri, May 18, 2018 at 5:11 PM, Vincent Massol <[hidden email]>
>>> wrote:
>>>> Hi Adel,
>>>>
>>>>> On 18 May 2018, at 11:40, Adel Atallah <[hidden email]> wrote:
>>>>>
>>>>> Hi,
>>>>>
>>>>> Following my previous email on "How should we review translations?", I'd
>>>>> like to know here if we should support automatic multibranch
>>> translations
>>>>> in Weblate.
>>>>>
>>>>> What I mean here is that with the old l10n platform, we would apply new
>>>>> translations on multiple git branches (for some projects like XWiki
>>>>> Platform). It was important to have new translations applied on LTS
>>>>> releases and other branches.
>>>>>
>>>>> The problem is that we can't tell Weblate to automatically push changes
>>> on
>>>>> multiple branches. We have discuss the problem with the maintainer here:
>>>>> https://github.com/WeblateOrg/weblate/issues/2016.
>>>>> What we can do is to duplicate Weblate components (a component is just a
>>>>> file to translate) for as many branches as we need. Making a change to a
>>>>> translation key (e.g. tour.homepageTour.pageMenu.contentB) will
>>> propagate
>>>>> the change to every other components with the same key. This way we can
>>>>> have a PR made with the same change on every branch we want.
>>>>>
>>>>> So here are the two options:
>>>>>
>>>>> 1) We keep the actual behavior
>>>>> Pros:
>>>>> - We will only have one PR to review (on master branch)
>>>>> Cons:
>>>>> - We will have to apply new changes to other branches ourselves when
>>>>> needed
>>>>
>>>> This is not fully the current behavior since right now the merge on a
>>> branch is done by the RM in one go for all translations.
>>>>
>>>> With this proposal 1) someone (whom?) will need to merge the *various*
>>> commits done by the weblate PRs, on a need-be basis.
>>>>
>>>> So this raises the following questions:
>>>> * Who is responsible for the branch merges and more specifically the LTS
>>> one. The RM?
>>>> * If so, what strategy do we decide, i.e. which translations do we want
>>> to merge or not? And what tool would be provide the RM or someone else to
>>> list all commits related to translations?
>>>>
>>>>>
>>>>> 2) We duplicate components
>>>>> Pros:
>>>>> - Changes will automatically be made for every specified branches
>>>>> Cons:
>>>>> - Some work to do: we can't create all the new components by hand so we
>>>>> will have to generate every components in some way
>>>>> - It will make Weblate much more complex because you can't hide
>>>>> components (https://i.imgur.com/YJ8qtUz.png)
>>>>
>>>> This option 2 is complex because not only the hassle of creating and
>>> *Deleting* components (when the branch is closed) but also we need to
>>> decide which components to duplicate (there might components that only
>>> exist on master for ex). Ideally we would need a script to automatically
>>> add translation components for a branch.
>>>>
>>>> If we can automate this then it’s not too bad but still complex. And
>>> indeed there’s the risk that users will translate branches by mistake
>>> instead of translating master.
>>>>
>>>>> I prefer option 1 because it will make Weblate easier to use.
>>>>>
>>>>> For option 2, we can also disable translation propagation and let people
>>>>> make translations on the branch they want.
>>>>
>>>> I can’t say which one I prefer yet because we need to answer the
>>> questions I raised for 1) first.
>>>>
>>>> The general question is: what translations do we want to merge for the
>>> LTS branch? I think we can agree that we don’t really care about merging
>>> translations for the short-lived branches such as 10.4.x.
>>>>
>>>> Thanks
>>>> -Vincent
>>>>
>>>>>
>>>>> Thanks,
>>>>> Adel
>>>>
>>>
>>>
>>>
>>> --
>>> Thomas Mortagne
>>>

Reply | Threaded
Open this post in threaded view
|

Re: [Brainstorming] Should we support multibranch translations?

Adel Atallah
On Tue, May 22, 2018 at 1:51 PM, Vincent Massol <[hidden email]> wrote:

> FTR this is what I discussed with Adel and I asked him to post it here so that we can agree.
>
>> On 22 May 2018, at 13:41, Adel Atallah <[hidden email]> wrote:
>>
>> So for 1), we still need to decide how we merge new translations into
>> LTS releases (or other branches).
>
> Idea 1:
>
>> An idea would be to write a script to let the RM apply new changes to
>> a specific branch.
>> One way to do it would be to write a script to find every translation
>> commits since a date then review and apply them.
>
> More specifically this is about finding all commits done by a given user (the weblate user). In practice the commits are done under the translation contributor’s name so we would need to find the merge commit (done with the weblate user name) and find all associated commits (children?).
>
> Idea 2:
>
>> An other way would be to use the list of translation files that we
>> already have and write a script to replace (checkout) those files from
>> master to the specified branch.
>
> The problem with this approach is that we can have commits related to the translation files that are not related to translations and thus generate false positives.
>
>>
>> WDYT?
>
> Idea 3: For each translation files in the known list, query weblate (using the REST API) to get the latest translation and apply them locally.
>
> In practice this will mean find keys in the translation file and replace with the value retrieved from weblate.

I don't think we need API calls for that, we can write a script that
will merge only the translations from two translation files (master
and LTS). It's a bit of work to do though.

>
> One improvement is that we could parse the keys from the local files and file updated translations for them from weblate.
>
> The advantage of idea3 is that I don’t think there are false positives nor merge conflicts which can happen with ideas 1 and 2. The downside is that it may take a lot of time and a lot of REST calls to get them all.
>
> For all these ideas, it’ll the responsibility of the user (RM?) to check the diff and decide to merge/push on the branch the full translations or only a subpart of it.
>
> Aso note that we’re only talking about LTS here since we should not bother for the temporary branches (such as stable-10.4.x).
>
> WDYT?
>
> Thanks
> -Vincent
>
>
>>
>> Thanks,
>> Adel
>> Adel Atallah
>> Product developer intern
>> [hidden email]
>> tel: +33 (0)6 12 96 35 06
>>
>>
>> On Fri, May 18, 2018 at 6:47 PM, Ecaterina Moraru (Valica)
>> <[hidden email]> wrote:
>>> +1 for 1)
>>> Make sure the commit has a marker like "[Translations]" or "[Weblate]" for
>>> the the step in the release process, so that we can look for them in the
>>> history in order to apply them, in case we really need them.
>>> In practice we don't commit translations for LTS, because usually we make
>>> changes in UI and we don't want to manually check and validate each
>>> translation.
>>>
>>> Thanks,
>>> Caty
>>>
>>>
>>> On Fri, May 18, 2018 at 5:47 PM, Thomas Mortagne <[hidden email]>
>>> wrote:
>>>
>>>> Option 2) would create too much of a mess on weblate side IMO (until
>>>> we can hide branches at least).
>>>>
>>>> I would go for 1) for now and follow progress on Weblate product to
>>>> provide a clean solution for this use case.
>>>>
>>>> That being said we need to find a solution for LTS (I don't think we
>>>> care about stable branch bugfixes releases and we could do it by hand
>>>> for RC branches since it's only 1 week usually). Here are some ideas:
>>>> a) it should not be hard to write a script which get all the weblate
>>>> commits from master since last weblate commit we can find in the
>>>> branch and cherry-pick them (probably also display a diff and ask for
>>>> confirmation for each of them). This would be executed before the
>>>> release by the release manager.
>>>> b) I guess it's possible to write or find a tool which automatically
>>>> create a pull request on the LTS branch when a weblate pull request is
>>>> applied
>>>> c) Anyone who apply a weblate pull request is responsible for applying
>>>> it on LTS branch. I don't trust us too much on that.
>>>>
>>>>
>>>>
>>>> a does not seems complex to do (but of course someone need to spend time
>>>> on it).
>>>> c does not require any tooling but I don't think it will work, I'm
>>>> sure we will keep forgetting to cherry-pick.
>>>>
>>>>
>>>> b would be nice if someone find a tool to do that. If not then I guess
>>>> the more realistic option is a.
>>>>
>>>> On Fri, May 18, 2018 at 5:11 PM, Vincent Massol <[hidden email]>
>>>> wrote:
>>>>> Hi Adel,
>>>>>
>>>>>> On 18 May 2018, at 11:40, Adel Atallah <[hidden email]> wrote:
>>>>>>
>>>>>> Hi,
>>>>>>
>>>>>> Following my previous email on "How should we review translations?", I'd
>>>>>> like to know here if we should support automatic multibranch
>>>> translations
>>>>>> in Weblate.
>>>>>>
>>>>>> What I mean here is that with the old l10n platform, we would apply new
>>>>>> translations on multiple git branches (for some projects like XWiki
>>>>>> Platform). It was important to have new translations applied on LTS
>>>>>> releases and other branches.
>>>>>>
>>>>>> The problem is that we can't tell Weblate to automatically push changes
>>>> on
>>>>>> multiple branches. We have discuss the problem with the maintainer here:
>>>>>> https://github.com/WeblateOrg/weblate/issues/2016.
>>>>>> What we can do is to duplicate Weblate components (a component is just a
>>>>>> file to translate) for as many branches as we need. Making a change to a
>>>>>> translation key (e.g. tour.homepageTour.pageMenu.contentB) will
>>>> propagate
>>>>>> the change to every other components with the same key. This way we can
>>>>>> have a PR made with the same change on every branch we want.
>>>>>>
>>>>>> So here are the two options:
>>>>>>
>>>>>> 1) We keep the actual behavior
>>>>>> Pros:
>>>>>> - We will only have one PR to review (on master branch)
>>>>>> Cons:
>>>>>> - We will have to apply new changes to other branches ourselves when
>>>>>> needed
>>>>>
>>>>> This is not fully the current behavior since right now the merge on a
>>>> branch is done by the RM in one go for all translations.
>>>>>
>>>>> With this proposal 1) someone (whom?) will need to merge the *various*
>>>> commits done by the weblate PRs, on a need-be basis.
>>>>>
>>>>> So this raises the following questions:
>>>>> * Who is responsible for the branch merges and more specifically the LTS
>>>> one. The RM?
>>>>> * If so, what strategy do we decide, i.e. which translations do we want
>>>> to merge or not? And what tool would be provide the RM or someone else to
>>>> list all commits related to translations?
>>>>>
>>>>>>
>>>>>> 2) We duplicate components
>>>>>> Pros:
>>>>>> - Changes will automatically be made for every specified branches
>>>>>> Cons:
>>>>>> - Some work to do: we can't create all the new components by hand so we
>>>>>> will have to generate every components in some way
>>>>>> - It will make Weblate much more complex because you can't hide
>>>>>> components (https://i.imgur.com/YJ8qtUz.png)
>>>>>
>>>>> This option 2 is complex because not only the hassle of creating and
>>>> *Deleting* components (when the branch is closed) but also we need to
>>>> decide which components to duplicate (there might components that only
>>>> exist on master for ex). Ideally we would need a script to automatically
>>>> add translation components for a branch.
>>>>>
>>>>> If we can automate this then it’s not too bad but still complex. And
>>>> indeed there’s the risk that users will translate branches by mistake
>>>> instead of translating master.
>>>>>
>>>>>> I prefer option 1 because it will make Weblate easier to use.
>>>>>>
>>>>>> For option 2, we can also disable translation propagation and let people
>>>>>> make translations on the branch they want.
>>>>>
>>>>> I can’t say which one I prefer yet because we need to answer the
>>>> questions I raised for 1) first.
>>>>>
>>>>> The general question is: what translations do we want to merge for the
>>>> LTS branch? I think we can agree that we don’t really care about merging
>>>> translations for the short-lived branches such as 10.4.x.
>>>>>
>>>>> Thanks
>>>>> -Vincent
>>>>>
>>>>>>
>>>>>> Thanks,
>>>>>> Adel
>>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> Thomas Mortagne
>>>>
>
Reply | Threaded
Open this post in threaded view
|

Re: [Brainstorming] Should we support multibranch translations?

Eduard Moraru
But what about the current (now old) translation process for LTS? AFAIR,
the RM was supposed to go through the translations and exclude some that
are OK for master, but not OK for LTS.

Did that ever happen in practice? Also, considering our translations
deprecation practice, what can go wrong at this point? If we deprecate a
translation on master and it ends up deprecated on LTS, it should still
work. Worst case, we end up with some new pages (new translations added on
master that are not relevant for LTS).

What I want to say is that it's not clear to me if we were in any way
better before than we are now (without any of the above proposed solutions).

Thanks,
Eduard

On Tue, May 22, 2018 at 2:59 PM, Adel Atallah <[hidden email]>
wrote:

> On Tue, May 22, 2018 at 1:51 PM, Vincent Massol <[hidden email]>
> wrote:
> > FTR this is what I discussed with Adel and I asked him to post it here
> so that we can agree.
> >
> >> On 22 May 2018, at 13:41, Adel Atallah <[hidden email]> wrote:
> >>
> >> So for 1), we still need to decide how we merge new translations into
> >> LTS releases (or other branches).
> >
> > Idea 1:
> >
> >> An idea would be to write a script to let the RM apply new changes to
> >> a specific branch.
> >> One way to do it would be to write a script to find every translation
> >> commits since a date then review and apply them.
> >
> > More specifically this is about finding all commits done by a given user
> (the weblate user). In practice the commits are done under the translation
> contributor’s name so we would need to find the merge commit (done with the
> weblate user name) and find all associated commits (children?).
> >
> > Idea 2:
> >
> >> An other way would be to use the list of translation files that we
> >> already have and write a script to replace (checkout) those files from
> >> master to the specified branch.
> >
> > The problem with this approach is that we can have commits related to
> the translation files that are not related to translations and thus
> generate false positives.
> >
> >>
> >> WDYT?
> >
> > Idea 3: For each translation files in the known list, query weblate
> (using the REST API) to get the latest translation and apply them locally.
> >
> > In practice this will mean find keys in the translation file and replace
> with the value retrieved from weblate.
>
> I don't think we need API calls for that, we can write a script that
> will merge only the translations from two translation files (master
> and LTS). It's a bit of work to do though.
>
> >
> > One improvement is that we could parse the keys from the local files and
> file updated translations for them from weblate.
> >
> > The advantage of idea3 is that I don’t think there are false positives
> nor merge conflicts which can happen with ideas 1 and 2. The downside is
> that it may take a lot of time and a lot of REST calls to get them all.
> >
> > For all these ideas, it’ll the responsibility of the user (RM?) to check
> the diff and decide to merge/push on the branch the full translations or
> only a subpart of it.
> >
> > Aso note that we’re only talking about LTS here since we should not
> bother for the temporary branches (such as stable-10.4.x).
> >
> > WDYT?
> >
> > Thanks
> > -Vincent
> >
> >
> >>
> >> Thanks,
> >> Adel
> >> Adel Atallah
> >> Product developer intern
> >> [hidden email]
> >> tel: +33 (0)6 12 96 35 06
> >>
> >>
> >> On Fri, May 18, 2018 at 6:47 PM, Ecaterina Moraru (Valica)
> >> <[hidden email]> wrote:
> >>> +1 for 1)
> >>> Make sure the commit has a marker like "[Translations]" or "[Weblate]"
> for
> >>> the the step in the release process, so that we can look for them in
> the
> >>> history in order to apply them, in case we really need them.
> >>> In practice we don't commit translations for LTS, because usually we
> make
> >>> changes in UI and we don't want to manually check and validate each
> >>> translation.
> >>>
> >>> Thanks,
> >>> Caty
> >>>
> >>>
> >>> On Fri, May 18, 2018 at 5:47 PM, Thomas Mortagne <
> [hidden email]>
> >>> wrote:
> >>>
> >>>> Option 2) would create too much of a mess on weblate side IMO (until
> >>>> we can hide branches at least).
> >>>>
> >>>> I would go for 1) for now and follow progress on Weblate product to
> >>>> provide a clean solution for this use case.
> >>>>
> >>>> That being said we need to find a solution for LTS (I don't think we
> >>>> care about stable branch bugfixes releases and we could do it by hand
> >>>> for RC branches since it's only 1 week usually). Here are some ideas:
> >>>> a) it should not be hard to write a script which get all the weblate
> >>>> commits from master since last weblate commit we can find in the
> >>>> branch and cherry-pick them (probably also display a diff and ask for
> >>>> confirmation for each of them). This would be executed before the
> >>>> release by the release manager.
> >>>> b) I guess it's possible to write or find a tool which automatically
> >>>> create a pull request on the LTS branch when a weblate pull request is
> >>>> applied
> >>>> c) Anyone who apply a weblate pull request is responsible for applying
> >>>> it on LTS branch. I don't trust us too much on that.
> >>>>
> >>>>
> >>>>
> >>>> a does not seems complex to do (but of course someone need to spend
> time
> >>>> on it).
> >>>> c does not require any tooling but I don't think it will work, I'm
> >>>> sure we will keep forgetting to cherry-pick.
> >>>>
> >>>>
> >>>> b would be nice if someone find a tool to do that. If not then I guess
> >>>> the more realistic option is a.
> >>>>
> >>>> On Fri, May 18, 2018 at 5:11 PM, Vincent Massol <[hidden email]>
> >>>> wrote:
> >>>>> Hi Adel,
> >>>>>
> >>>>>> On 18 May 2018, at 11:40, Adel Atallah <[hidden email]>
> wrote:
> >>>>>>
> >>>>>> Hi,
> >>>>>>
> >>>>>> Following my previous email on "How should we review
> translations?", I'd
> >>>>>> like to know here if we should support automatic multibranch
> >>>> translations
> >>>>>> in Weblate.
> >>>>>>
> >>>>>> What I mean here is that with the old l10n platform, we would apply
> new
> >>>>>> translations on multiple git branches (for some projects like XWiki
> >>>>>> Platform). It was important to have new translations applied on LTS
> >>>>>> releases and other branches.
> >>>>>>
> >>>>>> The problem is that we can't tell Weblate to automatically push
> changes
> >>>> on
> >>>>>> multiple branches. We have discuss the problem with the maintainer
> here:
> >>>>>> https://github.com/WeblateOrg/weblate/issues/2016.
> >>>>>> What we can do is to duplicate Weblate components (a component is
> just a
> >>>>>> file to translate) for as many branches as we need. Making a change
> to a
> >>>>>> translation key (e.g. tour.homepageTour.pageMenu.contentB) will
> >>>> propagate
> >>>>>> the change to every other components with the same key. This way we
> can
> >>>>>> have a PR made with the same change on every branch we want.
> >>>>>>
> >>>>>> So here are the two options:
> >>>>>>
> >>>>>> 1) We keep the actual behavior
> >>>>>> Pros:
> >>>>>> - We will only have one PR to review (on master branch)
> >>>>>> Cons:
> >>>>>> - We will have to apply new changes to other branches ourselves when
> >>>>>> needed
> >>>>>
> >>>>> This is not fully the current behavior since right now the merge on a
> >>>> branch is done by the RM in one go for all translations.
> >>>>>
> >>>>> With this proposal 1) someone (whom?) will need to merge the
> *various*
> >>>> commits done by the weblate PRs, on a need-be basis.
> >>>>>
> >>>>> So this raises the following questions:
> >>>>> * Who is responsible for the branch merges and more specifically the
> LTS
> >>>> one. The RM?
> >>>>> * If so, what strategy do we decide, i.e. which translations do we
> want
> >>>> to merge or not? And what tool would be provide the RM or someone
> else to
> >>>> list all commits related to translations?
> >>>>>
> >>>>>>
> >>>>>> 2) We duplicate components
> >>>>>> Pros:
> >>>>>> - Changes will automatically be made for every specified branches
> >>>>>> Cons:
> >>>>>> - Some work to do: we can't create all the new components by hand
> so we
> >>>>>> will have to generate every components in some way
> >>>>>> - It will make Weblate much more complex because you can't hide
> >>>>>> components (https://i.imgur.com/YJ8qtUz.png)
> >>>>>
> >>>>> This option 2 is complex because not only the hassle of creating and
> >>>> *Deleting* components (when the branch is closed) but also we need to
> >>>> decide which components to duplicate (there might components that only
> >>>> exist on master for ex). Ideally we would need a script to
> automatically
> >>>> add translation components for a branch.
> >>>>>
> >>>>> If we can automate this then it’s not too bad but still complex. And
> >>>> indeed there’s the risk that users will translate branches by mistake
> >>>> instead of translating master.
> >>>>>
> >>>>>> I prefer option 1 because it will make Weblate easier to use.
> >>>>>>
> >>>>>> For option 2, we can also disable translation propagation and let
> people
> >>>>>> make translations on the branch they want.
> >>>>>
> >>>>> I can’t say which one I prefer yet because we need to answer the
> >>>> questions I raised for 1) first.
> >>>>>
> >>>>> The general question is: what translations do we want to merge for
> the
> >>>> LTS branch? I think we can agree that we don’t really care about
> merging
> >>>> translations for the short-lived branches such as 10.4.x.
> >>>>>
> >>>>> Thanks
> >>>>> -Vincent
> >>>>>
> >>>>>>
> >>>>>> Thanks,
> >>>>>> Adel
> >>>>>
> >>>>
> >>>>
> >>>>
> >>>> --
> >>>> Thomas Mortagne
> >>>>
> >
>
Reply | Threaded
Open this post in threaded view
|

Re: [Brainstorming] Should we support multibranch translations?

vmassol
Administrator


> On 22 May 2018, at 14:07, Eduard Moraru <[hidden email]> wrote:
>
> But what about the current (now old) translation process for LTS? AFAIR,
> the RM was supposed to go through the translations and exclude some that
> are OK for master, but not OK for LTS.

Actually the exclude was optional and not a big deal IMO. Since we’re supposed to not break backward compat for translations and deprecate keys + introduce new ones, the worst that can happen is that new keys committed on LTS don’t get used. Not a big deal IMO.

> Did that ever happen in practice?

Yes from time to time but we should do it more since it’s logical that LTS gets the best possible translation applied (it’s supposed to be the most stable release after all and translation errors in the UI are bugs after all).

> Also, considering our translations
> deprecation practice, what can go wrong at this point? If we deprecate a
> translation on master and it ends up deprecated on LTS, it should still
> work. Worst case, we end up with some new pages (new translations added on
> master that are not relevant for LTS).

Exactly.

> What I want to say is that it's not clear to me if we were in any way
> better before than we are now (without any of the above proposed solutions).

The only problem is that we don’t have scripts anymore to do it the old way. Hence the 3 ideas/proposals in this email ;)

The main issue is that now we get PRs every day so there’s not a single commit to merge anymore. Hence ideas 1-3.

Thanks
-Vincent

>
> Thanks,
> Eduard
>
> On Tue, May 22, 2018 at 2:59 PM, Adel Atallah <[hidden email]>
> wrote:
>
>> On Tue, May 22, 2018 at 1:51 PM, Vincent Massol <[hidden email]>
>> wrote:
>>> FTR this is what I discussed with Adel and I asked him to post it here
>> so that we can agree.
>>>
>>>> On 22 May 2018, at 13:41, Adel Atallah <[hidden email]> wrote:
>>>>
>>>> So for 1), we still need to decide how we merge new translations into
>>>> LTS releases (or other branches).
>>>
>>> Idea 1:
>>>
>>>> An idea would be to write a script to let the RM apply new changes to
>>>> a specific branch.
>>>> One way to do it would be to write a script to find every translation
>>>> commits since a date then review and apply them.
>>>
>>> More specifically this is about finding all commits done by a given user
>> (the weblate user). In practice the commits are done under the translation
>> contributor’s name so we would need to find the merge commit (done with the
>> weblate user name) and find all associated commits (children?).
>>>
>>> Idea 2:
>>>
>>>> An other way would be to use the list of translation files that we
>>>> already have and write a script to replace (checkout) those files from
>>>> master to the specified branch.
>>>
>>> The problem with this approach is that we can have commits related to
>> the translation files that are not related to translations and thus
>> generate false positives.
>>>
>>>>
>>>> WDYT?
>>>
>>> Idea 3: For each translation files in the known list, query weblate
>> (using the REST API) to get the latest translation and apply them locally.
>>>
>>> In practice this will mean find keys in the translation file and replace
>> with the value retrieved from weblate.
>>
>> I don't think we need API calls for that, we can write a script that
>> will merge only the translations from two translation files (master
>> and LTS). It's a bit of work to do though.
>>
>>>
>>> One improvement is that we could parse the keys from the local files and
>> file updated translations for them from weblate.
>>>
>>> The advantage of idea3 is that I don’t think there are false positives
>> nor merge conflicts which can happen with ideas 1 and 2. The downside is
>> that it may take a lot of time and a lot of REST calls to get them all.
>>>
>>> For all these ideas, it’ll the responsibility of the user (RM?) to check
>> the diff and decide to merge/push on the branch the full translations or
>> only a subpart of it.
>>>
>>> Aso note that we’re only talking about LTS here since we should not
>> bother for the temporary branches (such as stable-10.4.x).
>>>
>>> WDYT?
>>>
>>> Thanks
>>> -Vincent
>>>
>>>
>>>>
>>>> Thanks,
>>>> Adel
>>>> Adel Atallah
>>>> Product developer intern
>>>> [hidden email]
>>>> tel: +33 (0)6 12 96 35 06
>>>>
>>>>
>>>> On Fri, May 18, 2018 at 6:47 PM, Ecaterina Moraru (Valica)
>>>> <[hidden email]> wrote:
>>>>> +1 for 1)
>>>>> Make sure the commit has a marker like "[Translations]" or "[Weblate]"
>> for
>>>>> the the step in the release process, so that we can look for them in
>> the
>>>>> history in order to apply them, in case we really need them.
>>>>> In practice we don't commit translations for LTS, because usually we
>> make
>>>>> changes in UI and we don't want to manually check and validate each
>>>>> translation.
>>>>>
>>>>> Thanks,
>>>>> Caty
>>>>>
>>>>>
>>>>> On Fri, May 18, 2018 at 5:47 PM, Thomas Mortagne <
>> [hidden email]>
>>>>> wrote:
>>>>>
>>>>>> Option 2) would create too much of a mess on weblate side IMO (until
>>>>>> we can hide branches at least).
>>>>>>
>>>>>> I would go for 1) for now and follow progress on Weblate product to
>>>>>> provide a clean solution for this use case.
>>>>>>
>>>>>> That being said we need to find a solution for LTS (I don't think we
>>>>>> care about stable branch bugfixes releases and we could do it by hand
>>>>>> for RC branches since it's only 1 week usually). Here are some ideas:
>>>>>> a) it should not be hard to write a script which get all the weblate
>>>>>> commits from master since last weblate commit we can find in the
>>>>>> branch and cherry-pick them (probably also display a diff and ask for
>>>>>> confirmation for each of them). This would be executed before the
>>>>>> release by the release manager.
>>>>>> b) I guess it's possible to write or find a tool which automatically
>>>>>> create a pull request on the LTS branch when a weblate pull request is
>>>>>> applied
>>>>>> c) Anyone who apply a weblate pull request is responsible for applying
>>>>>> it on LTS branch. I don't trust us too much on that.
>>>>>>
>>>>>>
>>>>>>
>>>>>> a does not seems complex to do (but of course someone need to spend
>> time
>>>>>> on it).
>>>>>> c does not require any tooling but I don't think it will work, I'm
>>>>>> sure we will keep forgetting to cherry-pick.
>>>>>>
>>>>>>
>>>>>> b would be nice if someone find a tool to do that. If not then I guess
>>>>>> the more realistic option is a.
>>>>>>
>>>>>> On Fri, May 18, 2018 at 5:11 PM, Vincent Massol <[hidden email]>
>>>>>> wrote:
>>>>>>> Hi Adel,
>>>>>>>
>>>>>>>> On 18 May 2018, at 11:40, Adel Atallah <[hidden email]>
>> wrote:
>>>>>>>>
>>>>>>>> Hi,
>>>>>>>>
>>>>>>>> Following my previous email on "How should we review
>> translations?", I'd
>>>>>>>> like to know here if we should support automatic multibranch
>>>>>> translations
>>>>>>>> in Weblate.
>>>>>>>>
>>>>>>>> What I mean here is that with the old l10n platform, we would apply
>> new
>>>>>>>> translations on multiple git branches (for some projects like XWiki
>>>>>>>> Platform). It was important to have new translations applied on LTS
>>>>>>>> releases and other branches.
>>>>>>>>
>>>>>>>> The problem is that we can't tell Weblate to automatically push
>> changes
>>>>>> on
>>>>>>>> multiple branches. We have discuss the problem with the maintainer
>> here:
>>>>>>>> https://github.com/WeblateOrg/weblate/issues/2016.
>>>>>>>> What we can do is to duplicate Weblate components (a component is
>> just a
>>>>>>>> file to translate) for as many branches as we need. Making a change
>> to a
>>>>>>>> translation key (e.g. tour.homepageTour.pageMenu.contentB) will
>>>>>> propagate
>>>>>>>> the change to every other components with the same key. This way we
>> can
>>>>>>>> have a PR made with the same change on every branch we want.
>>>>>>>>
>>>>>>>> So here are the two options:
>>>>>>>>
>>>>>>>> 1) We keep the actual behavior
>>>>>>>> Pros:
>>>>>>>> - We will only have one PR to review (on master branch)
>>>>>>>> Cons:
>>>>>>>> - We will have to apply new changes to other branches ourselves when
>>>>>>>> needed
>>>>>>>
>>>>>>> This is not fully the current behavior since right now the merge on a
>>>>>> branch is done by the RM in one go for all translations.
>>>>>>>
>>>>>>> With this proposal 1) someone (whom?) will need to merge the
>> *various*
>>>>>> commits done by the weblate PRs, on a need-be basis.
>>>>>>>
>>>>>>> So this raises the following questions:
>>>>>>> * Who is responsible for the branch merges and more specifically the
>> LTS
>>>>>> one. The RM?
>>>>>>> * If so, what strategy do we decide, i.e. which translations do we
>> want
>>>>>> to merge or not? And what tool would be provide the RM or someone
>> else to
>>>>>> list all commits related to translations?
>>>>>>>
>>>>>>>>
>>>>>>>> 2) We duplicate components
>>>>>>>> Pros:
>>>>>>>> - Changes will automatically be made for every specified branches
>>>>>>>> Cons:
>>>>>>>> - Some work to do: we can't create all the new components by hand
>> so we
>>>>>>>> will have to generate every components in some way
>>>>>>>> - It will make Weblate much more complex because you can't hide
>>>>>>>> components (https://i.imgur.com/YJ8qtUz.png)
>>>>>>>
>>>>>>> This option 2 is complex because not only the hassle of creating and
>>>>>> *Deleting* components (when the branch is closed) but also we need to
>>>>>> decide which components to duplicate (there might components that only
>>>>>> exist on master for ex). Ideally we would need a script to
>> automatically
>>>>>> add translation components for a branch.
>>>>>>>
>>>>>>> If we can automate this then it’s not too bad but still complex. And
>>>>>> indeed there’s the risk that users will translate branches by mistake
>>>>>> instead of translating master.
>>>>>>>
>>>>>>>> I prefer option 1 because it will make Weblate easier to use.
>>>>>>>>
>>>>>>>> For option 2, we can also disable translation propagation and let
>> people
>>>>>>>> make translations on the branch they want.
>>>>>>>
>>>>>>> I can’t say which one I prefer yet because we need to answer the
>>>>>> questions I raised for 1) first.
>>>>>>>
>>>>>>> The general question is: what translations do we want to merge for
>> the
>>>>>> LTS branch? I think we can agree that we don’t really care about
>> merging
>>>>>> translations for the short-lived branches such as 10.4.x.
>>>>>>>
>>>>>>> Thanks
>>>>>>> -Vincent
>>>>>>>
>>>>>>>>
>>>>>>>> Thanks,
>>>>>>>> Adel
>>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> Thomas Mortagne
>>>>>>
>>>
>>

Reply | Threaded
Open this post in threaded view
|

Re: [Brainstorming] Should we support multibranch translations?

vmassol
Administrator


> On 22 May 2018, at 14:13, Vincent Massol <[hidden email]> wrote:
>
>
>
>> On 22 May 2018, at 14:07, Eduard Moraru <[hidden email]> wrote:
>>
>> But what about the current (now old) translation process for LTS? AFAIR,
>> the RM was supposed to go through the translations and exclude some that
>> are OK for master, but not OK for LTS.
>
> Actually the exclude was optional and not a big deal IMO. Since we’re supposed to not break backward compat for translations and deprecate keys + introduce new ones, the worst that can happen is that new keys committed on LTS don’t get used. Not a big deal IMO.
>
>> Did that ever happen in practice?
>
> Yes from time to time but we should do it more since it’s logical that LTS gets the best possible translation applied (it’s supposed to be the most stable release after all and translation errors in the UI are bugs after all).
>
>> Also, considering our translations
>> deprecation practice, what can go wrong at this point? If we deprecate a
>> translation on master and it ends up deprecated on LTS, it should still
>> work. Worst case, we end up with some new pages (new translations added on
>> master that are not relevant for LTS).
>
> Exactly.

Note that idea 3 is better since it wouldn’t include new keys/files that are not used by the LTS version.

Thanks
-Vincent

>
>> What I want to say is that it's not clear to me if we were in any way
>> better before than we are now (without any of the above proposed solutions).
>
> The only problem is that we don’t have scripts anymore to do it the old way. Hence the 3 ideas/proposals in this email ;)
>
> The main issue is that now we get PRs every day so there’s not a single commit to merge anymore. Hence ideas 1-3.
>
> Thanks
> -Vincent
>
>>
>> Thanks,
>> Eduard
>>
>> On Tue, May 22, 2018 at 2:59 PM, Adel Atallah <[hidden email]>
>> wrote:
>>
>>> On Tue, May 22, 2018 at 1:51 PM, Vincent Massol <[hidden email]>
>>> wrote:
>>>> FTR this is what I discussed with Adel and I asked him to post it here
>>> so that we can agree.
>>>>
>>>>> On 22 May 2018, at 13:41, Adel Atallah <[hidden email]> wrote:
>>>>>
>>>>> So for 1), we still need to decide how we merge new translations into
>>>>> LTS releases (or other branches).
>>>>
>>>> Idea 1:
>>>>
>>>>> An idea would be to write a script to let the RM apply new changes to
>>>>> a specific branch.
>>>>> One way to do it would be to write a script to find every translation
>>>>> commits since a date then review and apply them.
>>>>
>>>> More specifically this is about finding all commits done by a given user
>>> (the weblate user). In practice the commits are done under the translation
>>> contributor’s name so we would need to find the merge commit (done with the
>>> weblate user name) and find all associated commits (children?).
>>>>
>>>> Idea 2:
>>>>
>>>>> An other way would be to use the list of translation files that we
>>>>> already have and write a script to replace (checkout) those files from
>>>>> master to the specified branch.
>>>>
>>>> The problem with this approach is that we can have commits related to
>>> the translation files that are not related to translations and thus
>>> generate false positives.
>>>>
>>>>>
>>>>> WDYT?
>>>>
>>>> Idea 3: For each translation files in the known list, query weblate
>>> (using the REST API) to get the latest translation and apply them locally.
>>>>
>>>> In practice this will mean find keys in the translation file and replace
>>> with the value retrieved from weblate.
>>>
>>> I don't think we need API calls for that, we can write a script that
>>> will merge only the translations from two translation files (master
>>> and LTS). It's a bit of work to do though.
>>>
>>>>
>>>> One improvement is that we could parse the keys from the local files and
>>> file updated translations for them from weblate.
>>>>
>>>> The advantage of idea3 is that I don’t think there are false positives
>>> nor merge conflicts which can happen with ideas 1 and 2. The downside is
>>> that it may take a lot of time and a lot of REST calls to get them all.
>>>>
>>>> For all these ideas, it’ll the responsibility of the user (RM?) to check
>>> the diff and decide to merge/push on the branch the full translations or
>>> only a subpart of it.
>>>>
>>>> Aso note that we’re only talking about LTS here since we should not
>>> bother for the temporary branches (such as stable-10.4.x).
>>>>
>>>> WDYT?
>>>>
>>>> Thanks
>>>> -Vincent
>>>>
>>>>
>>>>>
>>>>> Thanks,
>>>>> Adel
>>>>> Adel Atallah
>>>>> Product developer intern
>>>>> [hidden email]
>>>>> tel: +33 (0)6 12 96 35 06
>>>>>
>>>>>
>>>>> On Fri, May 18, 2018 at 6:47 PM, Ecaterina Moraru (Valica)
>>>>> <[hidden email]> wrote:
>>>>>> +1 for 1)
>>>>>> Make sure the commit has a marker like "[Translations]" or "[Weblate]"
>>> for
>>>>>> the the step in the release process, so that we can look for them in
>>> the
>>>>>> history in order to apply them, in case we really need them.
>>>>>> In practice we don't commit translations for LTS, because usually we
>>> make
>>>>>> changes in UI and we don't want to manually check and validate each
>>>>>> translation.
>>>>>>
>>>>>> Thanks,
>>>>>> Caty
>>>>>>
>>>>>>
>>>>>> On Fri, May 18, 2018 at 5:47 PM, Thomas Mortagne <
>>> [hidden email]>
>>>>>> wrote:
>>>>>>
>>>>>>> Option 2) would create too much of a mess on weblate side IMO (until
>>>>>>> we can hide branches at least).
>>>>>>>
>>>>>>> I would go for 1) for now and follow progress on Weblate product to
>>>>>>> provide a clean solution for this use case.
>>>>>>>
>>>>>>> That being said we need to find a solution for LTS (I don't think we
>>>>>>> care about stable branch bugfixes releases and we could do it by hand
>>>>>>> for RC branches since it's only 1 week usually). Here are some ideas:
>>>>>>> a) it should not be hard to write a script which get all the weblate
>>>>>>> commits from master since last weblate commit we can find in the
>>>>>>> branch and cherry-pick them (probably also display a diff and ask for
>>>>>>> confirmation for each of them). This would be executed before the
>>>>>>> release by the release manager.
>>>>>>> b) I guess it's possible to write or find a tool which automatically
>>>>>>> create a pull request on the LTS branch when a weblate pull request is
>>>>>>> applied
>>>>>>> c) Anyone who apply a weblate pull request is responsible for applying
>>>>>>> it on LTS branch. I don't trust us too much on that.
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> a does not seems complex to do (but of course someone need to spend
>>> time
>>>>>>> on it).
>>>>>>> c does not require any tooling but I don't think it will work, I'm
>>>>>>> sure we will keep forgetting to cherry-pick.
>>>>>>>
>>>>>>>
>>>>>>> b would be nice if someone find a tool to do that. If not then I guess
>>>>>>> the more realistic option is a.
>>>>>>>
>>>>>>> On Fri, May 18, 2018 at 5:11 PM, Vincent Massol <[hidden email]>
>>>>>>> wrote:
>>>>>>>> Hi Adel,
>>>>>>>>
>>>>>>>>> On 18 May 2018, at 11:40, Adel Atallah <[hidden email]>
>>> wrote:
>>>>>>>>>
>>>>>>>>> Hi,
>>>>>>>>>
>>>>>>>>> Following my previous email on "How should we review
>>> translations?", I'd
>>>>>>>>> like to know here if we should support automatic multibranch
>>>>>>> translations
>>>>>>>>> in Weblate.
>>>>>>>>>
>>>>>>>>> What I mean here is that with the old l10n platform, we would apply
>>> new
>>>>>>>>> translations on multiple git branches (for some projects like XWiki
>>>>>>>>> Platform). It was important to have new translations applied on LTS
>>>>>>>>> releases and other branches.
>>>>>>>>>
>>>>>>>>> The problem is that we can't tell Weblate to automatically push
>>> changes
>>>>>>> on
>>>>>>>>> multiple branches. We have discuss the problem with the maintainer
>>> here:
>>>>>>>>> https://github.com/WeblateOrg/weblate/issues/2016.
>>>>>>>>> What we can do is to duplicate Weblate components (a component is
>>> just a
>>>>>>>>> file to translate) for as many branches as we need. Making a change
>>> to a
>>>>>>>>> translation key (e.g. tour.homepageTour.pageMenu.contentB) will
>>>>>>> propagate
>>>>>>>>> the change to every other components with the same key. This way we
>>> can
>>>>>>>>> have a PR made with the same change on every branch we want.
>>>>>>>>>
>>>>>>>>> So here are the two options:
>>>>>>>>>
>>>>>>>>> 1) We keep the actual behavior
>>>>>>>>> Pros:
>>>>>>>>> - We will only have one PR to review (on master branch)
>>>>>>>>> Cons:
>>>>>>>>> - We will have to apply new changes to other branches ourselves when
>>>>>>>>> needed
>>>>>>>>
>>>>>>>> This is not fully the current behavior since right now the merge on a
>>>>>>> branch is done by the RM in one go for all translations.
>>>>>>>>
>>>>>>>> With this proposal 1) someone (whom?) will need to merge the
>>> *various*
>>>>>>> commits done by the weblate PRs, on a need-be basis.
>>>>>>>>
>>>>>>>> So this raises the following questions:
>>>>>>>> * Who is responsible for the branch merges and more specifically the
>>> LTS
>>>>>>> one. The RM?
>>>>>>>> * If so, what strategy do we decide, i.e. which translations do we
>>> want
>>>>>>> to merge or not? And what tool would be provide the RM or someone
>>> else to
>>>>>>> list all commits related to translations?
>>>>>>>>
>>>>>>>>>
>>>>>>>>> 2) We duplicate components
>>>>>>>>> Pros:
>>>>>>>>> - Changes will automatically be made for every specified branches
>>>>>>>>> Cons:
>>>>>>>>> - Some work to do: we can't create all the new components by hand
>>> so we
>>>>>>>>> will have to generate every components in some way
>>>>>>>>> - It will make Weblate much more complex because you can't hide
>>>>>>>>> components (https://i.imgur.com/YJ8qtUz.png)
>>>>>>>>
>>>>>>>> This option 2 is complex because not only the hassle of creating and
>>>>>>> *Deleting* components (when the branch is closed) but also we need to
>>>>>>> decide which components to duplicate (there might components that only
>>>>>>> exist on master for ex). Ideally we would need a script to
>>> automatically
>>>>>>> add translation components for a branch.
>>>>>>>>
>>>>>>>> If we can automate this then it’s not too bad but still complex. And
>>>>>>> indeed there’s the risk that users will translate branches by mistake
>>>>>>> instead of translating master.
>>>>>>>>
>>>>>>>>> I prefer option 1 because it will make Weblate easier to use.
>>>>>>>>>
>>>>>>>>> For option 2, we can also disable translation propagation and let
>>> people
>>>>>>>>> make translations on the branch they want.
>>>>>>>>
>>>>>>>> I can’t say which one I prefer yet because we need to answer the
>>>>>>> questions I raised for 1) first.
>>>>>>>>
>>>>>>>> The general question is: what translations do we want to merge for
>>> the
>>>>>>> LTS branch? I think we can agree that we don’t really care about
>>> merging
>>>>>>> translations for the short-lived branches such as 10.4.x.
>>>>>>>>
>>>>>>>> Thanks
>>>>>>>> -Vincent
>>>>>>>>
>>>>>>>>>
>>>>>>>>> Thanks,
>>>>>>>>> Adel
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> Thomas Mortagne
>>>>>>>
>>>>
>>>
>