[Brainstorming] Should we support only one skin in our functional tests or not?

classic Classic list List threaded Threaded
17 messages Options
Reply | Threaded
Open this post in threaded view
|

[Brainstorming] Should we support only one skin in our functional tests or not?

vmassol
Administrator
Hi devs,

ATM our selenium2-based test fwk for functional tests only supports one skin (Colibri).  

Guillaume is currently fixing the functional tests by changing our test framework to work with Flamingo.

After his changes are made our framework won’t work anymore with Colibri.

The question is:
* Is this ok?
* Do we want to change the framework to support multiple skins? In this case we would need to introduce Interfaces everywhere, for each Page Object to have various implementations.

One example of difference is Hover vs Clicking for the top level menus. Another example is that we don’t have anymore a contentmenu; it’s 3 buttons. Etc.

Any opinion?

Thanks
-Vincent
_______________________________________________
devs mailing list
[hidden email]
http://lists.xwiki.org/mailman/listinfo/devs
Reply | Threaded
Open this post in threaded view
|

Re: [Brainstorming] Should we support only one skin in our functional tests or not?

Sergiu Dumitriu-3
On 08/19/2014 12:12 PM, [hidden email] wrote:

> Hi devs,
>
> ATM our selenium2-based test fwk for functional tests only supports one skin (Colibri).  
>
> Guillaume is currently fixing the functional tests by changing our test framework to work with Flamingo.
>
> After his changes are made our framework won’t work anymore with Colibri.
>
> The question is:
> * Is this ok?
> * Do we want to change the framework to support multiple skins? In this case we would need to introduce Interfaces everywhere, for each Page Object to have various implementations.
>
> One example of difference is Hover vs Clicking for the top level menus. Another example is that we don’t have anymore a contentmenu; it’s 3 buttons. Etc.
>
> Any opinion?

Tests should work on different skins. At some point we did have some
support for skins, in the old xe-test-selenium module [1], but this
feature didn't survive the pageobjects revolution.

[1]
https://github.com/xwiki/xwiki-enterprise/tree/master/xwiki-enterprise-test/xwiki-enterprise-test-selenium/src/test/it/org/xwiki/test/selenium/framework

--
Sergiu Dumitriu
http://purl.org/net/sergiu
_______________________________________________
devs mailing list
[hidden email]
http://lists.xwiki.org/mailman/listinfo/devs
Reply | Threaded
Open this post in threaded view
|

Re: [Brainstorming] Should we support only one skin in our functional tests or not?

vmassol
Administrator
 



On 19 Aug 2014 at 19:53:54, Sergiu Dumitriu ([hidden email](mailto:[hidden email])) wrote:

> On 08/19/2014 12:12 PM, [hidden email] wrote:
> > Hi devs,
> >
> > ATM our selenium2-based test fwk for functional tests only supports one skin (Colibri).
> >
> > Guillaume is currently fixing the functional tests by changing our test framework to work with Flamingo.
> >
> > After his changes are made our framework won’t work anymore with Colibri.
> >
> > The question is:
> > * Is this ok?
> > * Do we want to change the framework to support multiple skins? In this case we would need to introduce Interfaces everywhere, for each Page Object to have various implementations.
> >
> > One example of difference is Hover vs Clicking for the top level menus. Another example is that we don’t have anymore a contentmenu; it’s 3 buttons. Etc.
> >
> > Any opinion?
>  
> Tests should work on different skins. At some point we did have some
> support for skins, in the old xe-test-selenium module [1], but this
> feature didn't survive the pageobjects revolution.

Yes (I coded it). The main reason it didn’t survive is because several skins didn’t survive and in the end we’ve been supporting only 1 skin for years.

The question are:
1) whether we’re going to support several skins in the future or not? We’ve already said we’re going drop Colibri when Flamingo is ready. Supporting several skins in the test framework is a lot of effort.
2) if we want to support several skins in the test fwk when is the right time. ATM we’re late for Flamingo as the default skin for 6.2 and doing this in 6.2 would mean postponing 6.2’s release by at least 1 to 2 weeks.

Thanks
-Vincent

> [1]
> https://github.com/xwiki/xwiki-enterprise/tree/master/xwiki-enterprise-test/xwiki-enterprise-test-selenium/src/test/it/org/xwiki/test/selenium/framework
>  
> --
> Sergiu Dumitriu
_______________________________________________
devs mailing list
[hidden email]
http://lists.xwiki.org/mailman/listinfo/devs
Reply | Threaded
Open this post in threaded view
|

Re: [Brainstorming] Should we support only one skin in our functional tests or not?

Eduard Moraru
Hi,

What I don`t fully understand is why changing a skin should affect the UI
tests of an application?

Page objects should be specific to the application's UI elements that are
custom coded to reflect the state of and to perform operations on the UI
elements of the tested application only. These UI elements should generally
be located inside the document's rendered content and should be minimally
impacted by a skin change. For me, an application should be limited as much
as possible to the document's rendered content.

When we are not testing applications, but the UI of XWiki itself, then
indeed, perhaps we should have both skin-specific page-objects and tests.

So my question is: How much is the current skin change affecting our UI
tests and at which level? Are application UI tests affected as well?

Thanks,
Eduard


On Tue, Aug 19, 2014 at 9:02 PM, [hidden email] <[hidden email]>
wrote:

>
>
>
>
> On 19 Aug 2014 at 19:53:54, Sergiu Dumitriu ([hidden email](mailto:
> [hidden email])) wrote:
>
> > On 08/19/2014 12:12 PM, [hidden email] wrote:
> > > Hi devs,
> > >
> > > ATM our selenium2-based test fwk for functional tests only supports
> one skin (Colibri).
> > >
> > > Guillaume is currently fixing the functional tests by changing our
> test framework to work with Flamingo.
> > >
> > > After his changes are made our framework won’t work anymore with
> Colibri.
> > >
> > > The question is:
> > > * Is this ok?
> > > * Do we want to change the framework to support multiple skins? In
> this case we would need to introduce Interfaces everywhere, for each Page
> Object to have various implementations.
> > >
> > > One example of difference is Hover vs Clicking for the top level
> menus. Another example is that we don’t have anymore a contentmenu; it’s 3
> buttons. Etc.
> > >
> > > Any opinion?
> >
> > Tests should work on different skins. At some point we did have some
> > support for skins, in the old xe-test-selenium module [1], but this
> > feature didn't survive the pageobjects revolution.
>
> Yes (I coded it). The main reason it didn’t survive is because several
> skins didn’t survive and in the end we’ve been supporting only 1 skin for
> years.
>
> The question are:
> 1) whether we’re going to support several skins in the future or not?
> We’ve already said we’re going drop Colibri when Flamingo is ready.
> Supporting several skins in the test framework is a lot of effort.
> 2) if we want to support several skins in the test fwk when is the right
> time. ATM we’re late for Flamingo as the default skin for 6.2 and doing
> this in 6.2 would mean postponing 6.2’s release by at least 1 to 2 weeks.
>
> Thanks
> -Vincent
>
> > [1]
> >
> https://github.com/xwiki/xwiki-enterprise/tree/master/xwiki-enterprise-test/xwiki-enterprise-test-selenium/src/test/it/org/xwiki/test/selenium/framework
> >
> > --
> > Sergiu Dumitriu
> _______________________________________________
> devs mailing list
> [hidden email]
> http://lists.xwiki.org/mailman/listinfo/devs
>
_______________________________________________
devs mailing list
[hidden email]
http://lists.xwiki.org/mailman/listinfo/devs
Reply | Threaded
Open this post in threaded view
|

Re: [Brainstorming] Should we support only one skin in our functional tests or not?

vmassol
Administrator
 



On 20 Aug 2014 at 10:51:51, Eduard Moraru ([hidden email](mailto:[hidden email])) wrote:

> Hi,
>  
> What I don`t fully understand is why changing a skin should affect the UI
> tests of an application?
>  
> Page objects should be specific to the application's UI elements that are
> custom coded to reflect the state of and to perform operations on the UI
> elements of the tested application only. These UI elements should generally
> be located inside the document's rendered content and should be minimally
> impacted by a skin change. For me, an application should be limited as much
> as possible to the document's rendered content.
>  
> When we are not testing applications, but the UI of XWiki itself, then
> indeed, perhaps we should have both skin-specific page-objects and tests.
>  
> So my question is: How much is the current skin change affecting our UI
> tests and at which level? Are application UI tests affected as well?

You can check by yourself from the commits done by GD yesterday and from the mail I sent yesterday about the failing tests in platform.

I haven’t counted them.

Still there are lots of UI tests about the XWiki UI so the question remains.

Thanks
-Vincent

> Thanks,
> Eduard
>  
>  
> On Tue, Aug 19, 2014 at 9:02 PM, [hidden email]  
> wrote:
>  
> >
> >
> >
> >
> > On 19 Aug 2014 at 19:53:54, Sergiu Dumitriu ([hidden email](mailto:
> > [hidden email])) wrote:
> >
> > > On 08/19/2014 12:12 PM, [hidden email] wrote:
> > > > Hi devs,
> > > >
> > > > ATM our selenium2-based test fwk for functional tests only supports
> > one skin (Colibri).
> > > >
> > > > Guillaume is currently fixing the functional tests by changing our
> > test framework to work with Flamingo.
> > > >
> > > > After his changes are made our framework won’t work anymore with
> > Colibri.
> > > >
> > > > The question is:
> > > > * Is this ok?
> > > > * Do we want to change the framework to support multiple skins? In
> > this case we would need to introduce Interfaces everywhere, for each Page
> > Object to have various implementations.
> > > >
> > > > One example of difference is Hover vs Clicking for the top level
> > menus. Another example is that we don’t have anymore a contentmenu; it’s 3
> > buttons. Etc.
> > > >
> > > > Any opinion?
> > >
> > > Tests should work on different skins. At some point we did have some
> > > support for skins, in the old xe-test-selenium module [1], but this
> > > feature didn't survive the pageobjects revolution.
> >
> > Yes (I coded it). The main reason it didn’t survive is because several
> > skins didn’t survive and in the end we’ve been supporting only 1 skin for
> > years.
> >
> > The question are:
> > 1) whether we’re going to support several skins in the future or not?
> > We’ve already said we’re going drop Colibri when Flamingo is ready.
> > Supporting several skins in the test framework is a lot of effort.
> > 2) if we want to support several skins in the test fwk when is the right
> > time. ATM we’re late for Flamingo as the default skin for 6.2 and doing
> > this in 6.2 would mean postponing 6.2’s release by at least 1 to 2 weeks.
> >
> > Thanks
> > -Vincent
> >
> > > [1]
> > >
> > https://github.com/xwiki/xwiki-enterprise/tree/master/xwiki-enterprise-test/xwiki-enterprise-test-selenium/src/test/it/org/xwiki/test/selenium/framework
> > >
> > > --
> > > Sergiu Dumitriu
> > _______________________________________________
> > devs mailing list
> > [hidden email]
> > http://lists.xwiki.org/mailman/listinfo/devs
> >
> _______________________________________________
> devs mailing list
> [hidden email]
> http://lists.xwiki.org/mailman/listinfo/devs
_______________________________________________
devs mailing list
[hidden email]
http://lists.xwiki.org/mailman/listinfo/devs
Reply | Threaded
Open this post in threaded view
|

Re: [Brainstorming] Should we support only one skin in our functional tests or not?

vmassol
Administrator
Just to give one example, AllDocsTest from the Index app is failing. One reason is that it logs the user out to verify that the LT doesn’t display the actions reserved for admins.

So it does a: 

    // Logs out to be guest
    page.logout();

And this in turn will use a skin UI element (the logout button at the top right).

Now you’re right that for this precise example, we could modify the test so that instead of clicking the logout button we log out the user by using TestUtils, i.e. not make it part of what we test. It could be cleaner. Of course we would need to have some other functional tests to prove that logout works fine when clicked but we do have that in this case.

But as I mentioned we still have a lot of skin-related test code and this pushes my question to this code.

Thanks
-Vincent

On 20 Aug 2014 at 10:57:19, [hidden email] ([hidden email](mailto:[hidden email])) wrote:

>  
>  
>  
>  
> On 20 Aug 2014 at 10:51:51, Eduard Moraru ([hidden email](mailto:[hidden email])) wrote:
>  
> > Hi,
> >
> > What I don`t fully understand is why changing a skin should affect the UI
> > tests of an application?
> >
> > Page objects should be specific to the application's UI elements that are
> > custom coded to reflect the state of and to perform operations on the UI
> > elements of the tested application only. These UI elements should generally
> > be located inside the document's rendered content and should be minimally
> > impacted by a skin change. For me, an application should be limited as much
> > as possible to the document's rendered content.
> >
> > When we are not testing applications, but the UI of XWiki itself, then
> > indeed, perhaps we should have both skin-specific page-objects and tests.
> >
> > So my question is: How much is the current skin change affecting our UI
> > tests and at which level? Are application UI tests affected as well?  
>  
> You can check by yourself from the commits done by GD yesterday and from the mail I sent yesterday about the failing tests in platform.  
>  
> I haven’t counted them.  
>  
> Still there are lots of UI tests about the XWiki UI so the question remains.  
>  
> Thanks  
> -Vincent
>  
> > Thanks,
> > Eduard
> >
> >
> > On Tue, Aug 19, 2014 at 9:02 PM, [hidden email]
> > wrote:
> >
> > >
> > >
> > >
> > >
> > > On 19 Aug 2014 at 19:53:54, Sergiu Dumitriu ([hidden email](mailto:
> > > [hidden email])) wrote:
> > >
> > > > On 08/19/2014 12:12 PM, [hidden email] wrote:
> > > > > Hi devs,
> > > > >
> > > > > ATM our selenium2-based test fwk for functional tests only supports
> > > one skin (Colibri).
> > > > >
> > > > > Guillaume is currently fixing the functional tests by changing our
> > > test framework to work with Flamingo.
> > > > >
> > > > > After his changes are made our framework won’t work anymore with
> > > Colibri.
> > > > >
> > > > > The question is:
> > > > > * Is this ok?
> > > > > * Do we want to change the framework to support multiple skins? In
> > > this case we would need to introduce Interfaces everywhere, for each Page
> > > Object to have various implementations.
> > > > >
> > > > > One example of difference is Hover vs Clicking for the top level
> > > menus. Another example is that we don’t have anymore a contentmenu; it’s 3
> > > buttons. Etc.
> > > > >
> > > > > Any opinion?
> > > >
> > > > Tests should work on different skins. At some point we did have some
> > > > support for skins, in the old xe-test-selenium module [1], but this
> > > > feature didn't survive the pageobjects revolution.
> > >
> > > Yes (I coded it). The main reason it didn’t survive is because several
> > > skins didn’t survive and in the end we’ve been supporting only 1 skin for
> > > years.
> > >
> > > The question are:
> > > 1) whether we’re going to support several skins in the future or not?
> > > We’ve already said we’re going drop Colibri when Flamingo is ready.
> > > Supporting several skins in the test framework is a lot of effort.
> > > 2) if we want to support several skins in the test fwk when is the right
> > > time. ATM we’re late for Flamingo as the default skin for 6.2 and doing
> > > this in 6.2 would mean postponing 6.2’s release by at least 1 to 2 weeks.
> > >
> > > Thanks
> > > -Vincent
> > >
> > > > [1]
> > > >
> > > https://github.com/xwiki/xwiki-enterprise/tree/master/xwiki-enterprise-test/xwiki-enterprise-test-selenium/src/test/it/org/xwiki/test/selenium/framework
> > > >
> > > > --
> > > > Sergiu Dumitriu
> > > _______________________________________________
> > > devs mailing list
> > > [hidden email]
> > > http://lists.xwiki.org/mailman/listinfo/devs
> > >
> > _______________________________________________
> > devs mailing list
> > [hidden email]
> > http://lists.xwiki.org/mailman/listinfo/devs

_______________________________________________
devs mailing list
[hidden email]
http://lists.xwiki.org/mailman/listinfo/devs
Reply | Threaded
Open this post in threaded view
|

Re: [Brainstorming] Should we support only one skin in our functional tests or not?

Marius Dumitru Florea
In reply to this post by vmassol
I'm fine with running the functional tests only for the default skin.
Note that although we support many browsers we run the functional
tests only for Firefox. So I think it's ok if we continue supporting
the Colibri skin for a few versions even if we stop running the
functional tests on it (because it's not the default skin any more).

Thanks,
Marius

On Tue, Aug 19, 2014 at 7:12 PM, [hidden email] <[hidden email]> wrote:

> Hi devs,
>
> ATM our selenium2-based test fwk for functional tests only supports one skin (Colibri).
>
> Guillaume is currently fixing the functional tests by changing our test framework to work with Flamingo.
>
> After his changes are made our framework won’t work anymore with Colibri.
>
> The question is:
> * Is this ok?
> * Do we want to change the framework to support multiple skins? In this case we would need to introduce Interfaces everywhere, for each Page Object to have various implementations.
>
> One example of difference is Hover vs Clicking for the top level menus. Another example is that we don’t have anymore a contentmenu; it’s 3 buttons. Etc.
>
> Any opinion?
>
> Thanks
> -Vincent
> _______________________________________________
> devs mailing list
> [hidden email]
> http://lists.xwiki.org/mailman/listinfo/devs
_______________________________________________
devs mailing list
[hidden email]
http://lists.xwiki.org/mailman/listinfo/devs
Reply | Threaded
Open this post in threaded view
|

Re: [Brainstorming] Should we support only one skin in our functional tests or not?

vmassol
Administrator
 



On 20 Aug 2014 at 15:55:59, Marius Dumitru Florea ([hidden email](mailto:[hidden email])) wrote:

> I'm fine with running the functional tests only for the default skin.
> Note that although we support many browsers we run the functional
> tests only for Firefox. 

Which is bad and not something we want ;)

Thanks
-Vincent

> So I think it's ok if we continue supporting
> the Colibri skin for a few versions even if we stop running the
> functional tests on it (because it's not the default skin any more).
>  
> Thanks,
> Marius
>  
> On Tue, Aug 19, 2014 at 7:12 PM, [hidden email] wrote:
> > Hi devs,
> >
> > ATM our selenium2-based test fwk for functional tests only supports one skin (Colibri).
> >
> > Guillaume is currently fixing the functional tests by changing our test framework to work with Flamingo.
> >
> > After his changes are made our framework won’t work anymore with Colibri.
> >
> > The question is:
> > * Is this ok?
> > * Do we want to change the framework to support multiple skins? In this case we would need to introduce Interfaces everywhere, for each Page Object to have various implementations.
> >
> > One example of difference is Hover vs Clicking for the top level menus. Another example is that we don’t have anymore a contentmenu; it’s 3 buttons. Etc.
> >
> > Any opinion?
> >
> > Thanks
> > -Vincent
_______________________________________________
devs mailing list
[hidden email]
http://lists.xwiki.org/mailman/listinfo/devs
Reply | Threaded
Open this post in threaded view
|

Re: [Brainstorming] Should we support only one skin in our functional tests or not?

Silvia Rusu
In reply to this post by Marius Dumitru Florea
Hi,

On Wed, Aug 20, 2014 at 4:55 PM, Marius Dumitru Florea
<[hidden email]> wrote:
> I'm fine with running the functional tests only for the default skin.

+1

> Note that although we support many browsers we run the functional
> tests only for Firefox. So I think it's ok if we continue supporting
> the Colibri skin for a few versions even if we stop running the
> functional tests on it (because it's not the default skin any more).

+1 for this as well. Andreea has been doing manual tests on Colibri in
parallel with Flamingo and will probably continue to do so for a few
more versions.

Thanks,
Silvia

>
> Thanks,
> Marius
>
> On Tue, Aug 19, 2014 at 7:12 PM, [hidden email] <[hidden email]> wrote:
>> Hi devs,
>>
>> ATM our selenium2-based test fwk for functional tests only supports one skin (Colibri).
>>
>> Guillaume is currently fixing the functional tests by changing our test framework to work with Flamingo.
>>
>> After his changes are made our framework won’t work anymore with Colibri.
>>
>> The question is:
>> * Is this ok?
>> * Do we want to change the framework to support multiple skins? In this case we would need to introduce Interfaces everywhere, for each Page Object to have various implementations.
>>
>> One example of difference is Hover vs Clicking for the top level menus. Another example is that we don’t have anymore a contentmenu; it’s 3 buttons. Etc.
>>
>> Any opinion?
>>
>> Thanks
>> -Vincent
>> _______________________________________________
>> devs mailing list
>> [hidden email]
>> http://lists.xwiki.org/mailman/listinfo/devs
> _______________________________________________
> devs mailing list
> [hidden email]
> http://lists.xwiki.org/mailman/listinfo/devs
_______________________________________________
devs mailing list
[hidden email]
http://lists.xwiki.org/mailman/listinfo/devs
Silvia Rusu
Tester & Documentation Writer - XWiki
http://twitter.com/silviarusu
Reply | Threaded
Open this post in threaded view
|

Re: [Brainstorming] Should we support only one skin in our functional tests or not?

Andreea Popescu
>
> > I'm fine with running the functional tests only for the default skin.
>
> +1
>
> > Note that although we support many browsers we run the functional
> > tests only for Firefox. So I think it's ok if we continue supporting
> > the Colibri skin for a few versions even if we stop running the
> > functional tests on it (because it's not the default skin any more).
>
> +1 for this as well. Andreea has been doing manual tests on Colibri in
> parallel with Flamingo and will probably continue to do so for a few
> more versions.
>

I agree with Silvia on this. So, I'm +1.

Thanks,
Andreea


On Thu, Aug 21, 2014 at 11:33 AM, Silvia Rusu <[hidden email]> wrote:

> Hi,
>
> On Wed, Aug 20, 2014 at 4:55 PM, Marius Dumitru Florea
> <[hidden email]> wrote:
> > I'm fine with running the functional tests only for the default skin.
>
> +1
>
> > Note that although we support many browsers we run the functional
> > tests only for Firefox. So I think it's ok if we continue supporting
> > the Colibri skin for a few versions even if we stop running the
> > functional tests on it (because it's not the default skin any more).
>
> +1 for this as well. Andreea has been doing manual tests on Colibri in
> parallel with Flamingo and will probably continue to do so for a few
> more versions.
>
> Thanks,
> Silvia
>
> >
> > Thanks,
> > Marius
> >
> > On Tue, Aug 19, 2014 at 7:12 PM, [hidden email] <[hidden email]>
> wrote:
> >> Hi devs,
> >>
> >> ATM our selenium2-based test fwk for functional tests only supports one
> skin (Colibri).
> >>
> >> Guillaume is currently fixing the functional tests by changing our test
> framework to work with Flamingo.
> >>
> >> After his changes are made our framework won’t work anymore with
> Colibri.
> >>
> >> The question is:
> >> * Is this ok?
> >> * Do we want to change the framework to support multiple skins? In this
> case we would need to introduce Interfaces everywhere, for each Page Object
> to have various implementations.
> >>
> >> One example of difference is Hover vs Clicking for the top level menus.
> Another example is that we don’t have anymore a contentmenu; it’s 3
> buttons. Etc.
> >>
> >> Any opinion?
> >>
> >> Thanks
> >> -Vincent
> >> _______________________________________________
> >> devs mailing list
> >> [hidden email]
> >> http://lists.xwiki.org/mailman/listinfo/devs
> > _______________________________________________
> > devs mailing list
> > [hidden email]
> > http://lists.xwiki.org/mailman/listinfo/devs
> _______________________________________________
> devs mailing list
> [hidden email]
> http://lists.xwiki.org/mailman/listinfo/devs
>
_______________________________________________
devs mailing list
[hidden email]
http://lists.xwiki.org/mailman/listinfo/devs
Reply | Threaded
Open this post in threaded view
|

Re: [Brainstorming] Should we support only one skin in our functional tests or not?

Denis Gervalle-2
In reply to this post by vmassol
On Wed, Aug 20, 2014 at 4:01 PM, [hidden email] <[hidden email]>
wrote:

>
>
>
>
> On 20 Aug 2014 at 15:55:59, Marius Dumitru Florea (
> [hidden email](mailto:[hidden email]))
> wrote:
>
> > I'm fine with running the functional tests only for the default skin.
> > Note that although we support many browsers we run the functional
> > tests only for Firefox.
>
> Which is bad and not something we want ;)
>

We want to have test on all browser with as many skin as possible,
including test of all combination of migrated version, and ... well, what
we want is not really compatible with true reality :)

So, taking into account our team, I do not think we can afford the support
of more than one skin. In practice, I doubt those that see our new skin
will want to stick with colibri for long, and those that will, will also
stick with the old xwiki version they have. The support for both colibri
and flamingo in parallel is also short term, so unless I over estimate the
workload, I am +1 to only support one skin in functional tests, and to also
put some reserves on the usage of colibri in upcoming versions.

Thanks,


>
> Thanks
> -Vincent
>
> > So I think it's ok if we continue supporting
> > the Colibri skin for a few versions even if we stop running the
> > functional tests on it (because it's not the default skin any more).
> >
> > Thanks,
> > Marius
> >
> > On Tue, Aug 19, 2014 at 7:12 PM, [hidden email] wrote:
> > > Hi devs,
> > >
> > > ATM our selenium2-based test fwk for functional tests only supports
> one skin (Colibri).
> > >
> > > Guillaume is currently fixing the functional tests by changing our
> test framework to work with Flamingo.
> > >
> > > After his changes are made our framework won’t work anymore with
> Colibri.
> > >
> > > The question is:
> > > * Is this ok?
> > > * Do we want to change the framework to support multiple skins? In
> this case we would need to introduce Interfaces everywhere, for each Page
> Object to have various implementations.
> > >
> > > One example of difference is Hover vs Clicking for the top level
> menus. Another example is that we don’t have anymore a contentmenu; it’s 3
> buttons. Etc.
> > >
> > > Any opinion?
> > >
> > > Thanks
> > > -Vincent
> _______________________________________________
> devs mailing list
> [hidden email]
> http://lists.xwiki.org/mailman/listinfo/devs
>



--
Denis Gervalle
SOFTEC sa - CEO
_______________________________________________
devs mailing list
[hidden email]
http://lists.xwiki.org/mailman/listinfo/devs
Reply | Threaded
Open this post in threaded view
|

Re: [Brainstorming] Should we support only one skin in our functional tests or not?

Manuel Smeria-2
Hi,

+1 for only testing on Flamingo.

Thanks,
Manuel


On Thu, Aug 21, 2014 at 12:57 PM, Denis Gervalle <[hidden email]> wrote:

> On Wed, Aug 20, 2014 at 4:01 PM, [hidden email] <[hidden email]>
> wrote:
>
> >
> >
> >
> >
> > On 20 Aug 2014 at 15:55:59, Marius Dumitru Florea (
> > [hidden email](mailto:[hidden email]))
> > wrote:
> >
> > > I'm fine with running the functional tests only for the default skin.
> > > Note that although we support many browsers we run the functional
> > > tests only for Firefox.
> >
> > Which is bad and not something we want ;)
> >
>
> We want to have test on all browser with as many skin as possible,
> including test of all combination of migrated version, and ... well, what
> we want is not really compatible with true reality :)
>
> So, taking into account our team, I do not think we can afford the support
> of more than one skin. In practice, I doubt those that see our new skin
> will want to stick with colibri for long, and those that will, will also
> stick with the old xwiki version they have. The support for both colibri
> and flamingo in parallel is also short term, so unless I over estimate the
> workload, I am +1 to only support one skin in functional tests, and to also
> put some reserves on the usage of colibri in upcoming versions.
>
> Thanks,
>
>
> >
> > Thanks
> > -Vincent
> >
> > > So I think it's ok if we continue supporting
> > > the Colibri skin for a few versions even if we stop running the
> > > functional tests on it (because it's not the default skin any more).
> > >
> > > Thanks,
> > > Marius
> > >
> > > On Tue, Aug 19, 2014 at 7:12 PM, [hidden email] wrote:
> > > > Hi devs,
> > > >
> > > > ATM our selenium2-based test fwk for functional tests only supports
> > one skin (Colibri).
> > > >
> > > > Guillaume is currently fixing the functional tests by changing our
> > test framework to work with Flamingo.
> > > >
> > > > After his changes are made our framework won’t work anymore with
> > Colibri.
> > > >
> > > > The question is:
> > > > * Is this ok?
> > > > * Do we want to change the framework to support multiple skins? In
> > this case we would need to introduce Interfaces everywhere, for each Page
> > Object to have various implementations.
> > > >
> > > > One example of difference is Hover vs Clicking for the top level
> > menus. Another example is that we don’t have anymore a contentmenu; it’s
> 3
> > buttons. Etc.
> > > >
> > > > Any opinion?
> > > >
> > > > Thanks
> > > > -Vincent
> > _______________________________________________
> > devs mailing list
> > [hidden email]
> > http://lists.xwiki.org/mailman/listinfo/devs
> >
>
>
>
> --
> Denis Gervalle
> SOFTEC sa - CEO
> _______________________________________________
> devs mailing list
> [hidden email]
> http://lists.xwiki.org/mailman/listinfo/devs
>
_______________________________________________
devs mailing list
[hidden email]
http://lists.xwiki.org/mailman/listinfo/devs
Reply | Threaded
Open this post in threaded view
|

Re: [Brainstorming] Should we support only one skin in our functional tests or not?

vmassol
Administrator
Just for the record: anyone saying +1 for only testing on Flaming must realize that they are basically saying that we will NOT support more than 1 skin (Because manual testing doesn’t scale).

Thanks
-Vincent 

On 21 Aug 2014 at 14:22:38, Manuel Smeria ([hidden email](mailto:[hidden email])) wrote:

> Hi,
>  
> +1 for only testing on Flamingo.
>  
> Thanks,
> Manuel
>  
>  
> On Thu, Aug 21, 2014 at 12:57 PM, Denis Gervalle wrote:
>  
> > On Wed, Aug 20, 2014 at 4:01 PM, [hidden email]  
> > wrote:
> >
> > >
> > >
> > >
> > >
> > > On 20 Aug 2014 at 15:55:59, Marius Dumitru Florea (
> > > [hidden email](mailto:[hidden email]))
> > > wrote:
> > >
> > > > I'm fine with running the functional tests only for the default skin.
> > > > Note that although we support many browsers we run the functional
> > > > tests only for Firefox.
> > >
> > > Which is bad and not something we want ;)
> > >
> >
> > We want to have test on all browser with as many skin as possible,
> > including test of all combination of migrated version, and ... well, what
> > we want is not really compatible with true reality :)
> >
> > So, taking into account our team, I do not think we can afford the support
> > of more than one skin. In practice, I doubt those that see our new skin
> > will want to stick with colibri for long, and those that will, will also
> > stick with the old xwiki version they have. The support for both colibri
> > and flamingo in parallel is also short term, so unless I over estimate the
> > workload, I am +1 to only support one skin in functional tests, and to also
> > put some reserves on the usage of colibri in upcoming versions.
> >
> > Thanks,
> >
> >
> > >
> > > Thanks
> > > -Vincent
> > >
> > > > So I think it's ok if we continue supporting
> > > > the Colibri skin for a few versions even if we stop running the
> > > > functional tests on it (because it's not the default skin any more).
> > > >
> > > > Thanks,
> > > > Marius
> > > >
> > > > On Tue, Aug 19, 2014 at 7:12 PM, [hidden email] wrote:
> > > > > Hi devs,
> > > > >
> > > > > ATM our selenium2-based test fwk for functional tests only supports
> > > one skin (Colibri).
> > > > >
> > > > > Guillaume is currently fixing the functional tests by changing our
> > > test framework to work with Flamingo.
> > > > >
> > > > > After his changes are made our framework won’t work anymore with
> > > Colibri.
> > > > >
> > > > > The question is:
> > > > > * Is this ok?
> > > > > * Do we want to change the framework to support multiple skins? In
> > > this case we would need to introduce Interfaces everywhere, for each Page
> > > Object to have various implementations.
> > > > >
> > > > > One example of difference is Hover vs Clicking for the top level
> > > menus. Another example is that we don’t have anymore a contentmenu; it’s
> > 3
> > > buttons. Etc.
> > > > >
> > > > > Any opinion?
> > > > >
> > > > > Thanks
> > > > > -Vincent
> > > _______________________________________________
> > > devs mailing list
> > > [hidden email]
> > > http://lists.xwiki.org/mailman/listinfo/devs
> > >
> >
> >
> >
> > --
> > Denis Gervalle
> > SOFTEC sa - CEO
> > _______________________________________________
> > devs mailing list
> > [hidden email]
> > http://lists.xwiki.org/mailman/listinfo/devs
> >
> _______________________________________________
> devs mailing list
> [hidden email]
> http://lists.xwiki.org/mailman/listinfo/devs
_______________________________________________
devs mailing list
[hidden email]
http://lists.xwiki.org/mailman/listinfo/devs
Reply | Threaded
Open this post in threaded view
|

Re: [Brainstorming] Should we support only one skin in our functional tests or not?

vmassol
Administrator
In reply to this post by Denis Gervalle-2
 



On 21 Aug 2014 at 11:58:01, Denis Gervalle ([hidden email](mailto:[hidden email])) wrote:

> On Wed, Aug 20, 2014 at 4:01 PM, [hidden email]  
> wrote:
>  
> >
> >
> >
> >
> > On 20 Aug 2014 at 15:55:59, Marius Dumitru Florea (
> > [hidden email](mailto:[hidden email]))
> > wrote:
> >
> > > I'm fine with running the functional tests only for the default skin.
> > > Note that although we support many browsers we run the functional
> > > tests only for Firefox.
> >
> > Which is bad and not something we want ;)
> >
>  
> We want to have test on all browser with as many skin as possible,
> including test of all combination of migrated version, and ... well, what
> we want is not really compatible with true reality :)

You’re mixing a bit everything in the same bag here :) There are 2 aspects:

- It’s specifically because we don’t have that much manpower that we must have our CI automatically test on various browsers/databases/OSes. This works well because the tests don’t need to be modified (in theory at least but mostly true) to be executed on various browsers/databases/OSes.

- for the various skins it’s different because you need to write several test frameworks (one per skin) so it’s indeed more work.

> So, taking into account our team, I do not think we can afford the support
> of more than one skin. In practice, I doubt those that see our new skin
> will want to stick with colibri for long, and those that will, will also
> stick with the old xwiki version they have. The support for both colibri
> and flamingo in parallel is also short term, so unless I over estimate the
> workload, I am +1 to only support one skin in functional tests, and to also
> put some reserves on the usage of colibri in upcoming versions.

It depends if we’re able to fulfil a large majority of use cases with only 1 skin or not.

The fact that we’ve been able to support only one skin shows how complex writing an XWiki skin is. If you check on extensions.xwiki.org you’ll see we have very few skins, especially compared to other tools like wordpress, drupal, etc. We’ve been saying this for years but we haven’t made progress. Quite the opposite, over the years, we’ve made writing skins more complex (for example right now if you want to write a new skin you need to make sure to implement all Extension Points in your skin if you want your skin to behave properly with extensions!).

Now one solution could be to have only 1 (or a few) base skins and improve Skin Themes so that users can do more and more in a Theme, thus allowing just to modify Themes and not a full skin. We started with Color Themes, added the ability to change the logo, and we are now moving to Themes with the ability to also change fonts. This is a step in the right direction I believe. In some future, we could maybe also support more Theme features. At some point we might need to fold default Panels layout (1 column, 2 columns, 3 columns, etc) and Panels selection into Themes too. Etc.

If we push this to a level where having 1 skin and using various Themes allow XWiki users to implement the UI they need for their project then we won’t need to support more than 1 skin.

But I fear this is just pushing the testing problem to Themes (but maybe to a lesser degree, depending on the freedom we give to Themes) ;) And if we wish to support several Themes we’ll need to test them…

Anyway, for 6.2 I agree we don’t have the time to make our UI test framework multiskin-enabled.

However, I believe we’ll need to refactor our test framework to allow supporting testing several UI. That will be useful when we need to do that but also for our users writing new skins and needing tests for them. I don’t think it’s that complicated to do.

Thanks
-Vincent

> Thanks,
>  
>  
> >
> > Thanks
> > -Vincent
> >
> > > So I think it's ok if we continue supporting
> > > the Colibri skin for a few versions even if we stop running the
> > > functional tests on it (because it's not the default skin any more).
> > >
> > > Thanks,
> > > Marius
> > >
> > > On Tue, Aug 19, 2014 at 7:12 PM, [hidden email] wrote:
> > > > Hi devs,
> > > >
> > > > ATM our selenium2-based test fwk for functional tests only supports
> > one skin (Colibri).
> > > >
> > > > Guillaume is currently fixing the functional tests by changing our
> > test framework to work with Flamingo.
> > > >
> > > > After his changes are made our framework won’t work anymore with
> > Colibri.
> > > >
> > > > The question is:
> > > > * Is this ok?
> > > > * Do we want to change the framework to support multiple skins? In
> > this case we would need to introduce Interfaces everywhere, for each Page
> > Object to have various implementations.
> > > >
> > > > One example of difference is Hover vs Clicking for the top level
> > menus. Another example is that we don’t have anymore a contentmenu; it’s 3
> > buttons. Etc.
> > > >
> > > > Any opinion?
> > > >
> > > > Thanks
> > > > -Vincent
> > _______________________________________________
> > devs mailing list
> > [hidden email]
> > http://lists.xwiki.org/mailman/listinfo/devs
> >
>  
>  
>  
> --
> Denis Gervalle
> SOFTEC sa - CEO
> _______________________________________________
> devs mailing list
> [hidden email]
> http://lists.xwiki.org/mailman/listinfo/devs
_______________________________________________
devs mailing list
[hidden email]
http://lists.xwiki.org/mailman/listinfo/devs
Reply | Threaded
Open this post in threaded view
|

Re: [Brainstorming] Should we support only one skin in our functional tests or not?

Denis Gervalle-2
On Thu, Aug 21, 2014 at 4:52 PM, [hidden email] <[hidden email]>
wrote:

>
>
>
>
> On 21 Aug 2014 at 11:58:01, Denis Gervalle ([hidden email](mailto:
> [hidden email])) wrote:
>
> > On Wed, Aug 20, 2014 at 4:01 PM, [hidden email]
> > wrote:
> >
> > >
> > >
> > >
> > >
> > > On 20 Aug 2014 at 15:55:59, Marius Dumitru Florea (
> > > [hidden email](mailto:[hidden email]))
> > > wrote:
> > >
> > > > I'm fine with running the functional tests only for the default skin.
> > > > Note that although we support many browsers we run the functional
> > > > tests only for Firefox.
> > >
> > > Which is bad and not something we want ;)
> > >
> >
> > We want to have test on all browser with as many skin as possible,
> > including test of all combination of migrated version, and ... well, what
> > we want is not really compatible with true reality :)
>
> You’re mixing a bit everything in the same bag here :) There are 2 aspects:
>
> - It’s specifically because we don’t have that much manpower that we must
> have our CI automatically test on various browsers/databases/OSes. This
> works well because the tests don’t need to be modified (in theory at least
> but mostly true) to be executed on various browsers/databases/OSes.
>

We could not say that test does not require some maintenance effort, and
supporting multiple skin will increase it.


> - for the various skins it’s different because you need to write several
> test frameworks (one per skin) so it’s indeed more work.
>

I have not the feeling we have the time for that, there are many other
aspect that also require our attention.


>
> > So, taking into account our team, I do not think we can afford the
> support
> > of more than one skin. In practice, I doubt those that see our new skin
> > will want to stick with colibri for long, and those that will, will also
> > stick with the old xwiki version they have. The support for both colibri
> > and flamingo in parallel is also short term, so unless I over estimate
> the
> > workload, I am +1 to only support one skin in functional tests, and to
> also
> > put some reserves on the usage of colibri in upcoming versions.
>
> It depends if we’re able to fulfil a large majority of use cases with only
> 1 skin or not.
>

I agree that if flavor came with different skins, we will need to support
testing those. This does not seems to be the plan at the moment. We have
taken the direction of supporting mobile using the same skin than desktop,
so the exact opposite of having multiple skins for different purposes.


> The fact that we’ve been able to support only one skin shows how complex
> writing an XWiki skin is. If you check on extensions.xwiki.org you’ll see
> we have very few skins, especially compared to other tools like wordpress,
> drupal, etc. We’ve been saying this for years but we haven’t made progress.
> Quite the opposite, over the years, we’ve made writing skins more complex
> (for example right now if you want to write a new skin you need to make
> sure to implement all Extension Points in your skin if you want your skin
> to behave properly with extensions!).
>

Comparing XWiki skins ans wordpress, drupal, etc... themes, is really
unfair. In fact, those products are simply not fully "skinable", they have
a single static skin with many options, and they have themes, which is not
exactly the same. The Skin is a very powerful feature of XWiki, since it
allows a complete flexibility of the XWiki UI when need arise.


> Now one solution could be to have only 1 (or a few) base skins and improve
> Skin Themes so that users can do more and more in a Theme, thus allowing
> just to modify Themes and not a full skin. We started with Color Themes,
> added the ability to change the logo, and we are now moving to Themes with
> the ability to also change fonts. This is a step in the right direction I
> believe. In some future, we could maybe also support more Theme features.
> At some point we might need to fold default Panels layout (1 column, 2
> columns, 3 columns, etc) and Panels selection into Themes too. Etc.
>

I completely agree that we should improve that aspect, and Flamingo will
really help. It should be able to take bootstrap themes and apply them to
XWiki. Changing and supporting multiple theme is only a matter of CSS, and
should not affect the functionalities, so testing different themes is not
really required IMO.


> If we push this to a level where having 1 skin and using various Themes
> allow XWiki users to implement the UI they need for their project then we
> won’t need to support more than 1 skin.
>

Users can always customize the skin, this does not change anything for us.


> But I fear this is just pushing the testing problem to Themes (but maybe
> to a lesser degree, depending on the freedom we give to Themes) ;) And if
> we wish to support several Themes we’ll need to test them…
>

We should not put in Themes what is a matter of Skin. The skin could have
many options, like other product does. Supporting multiple skin is actually
out of scope for the small theme we are.


> Anyway, for 6.2 I agree we don’t have the time to make our UI test
> framework multiskin-enabled.
>
> However, I believe we’ll need to refactor our test framework to allow
> supporting testing several UI. That will be useful when we need to do that
> but also for our users writing new skins and needing tests for them. I
> don’t think it’s that complicated to do.
>

I have nothing against, I just do not see it as a priority.


>
> Thanks
> -Vincent
>
> > Thanks,
> >
> >
> > >
> > > Thanks
> > > -Vincent
> > >
> > > > So I think it's ok if we continue supporting
> > > > the Colibri skin for a few versions even if we stop running the
> > > > functional tests on it (because it's not the default skin any more).
> > > >
> > > > Thanks,
> > > > Marius
> > > >
> > > > On Tue, Aug 19, 2014 at 7:12 PM, [hidden email] wrote:
> > > > > Hi devs,
> > > > >
> > > > > ATM our selenium2-based test fwk for functional tests only supports
> > > one skin (Colibri).
> > > > >
> > > > > Guillaume is currently fixing the functional tests by changing our
> > > test framework to work with Flamingo.
> > > > >
> > > > > After his changes are made our framework won’t work anymore with
> > > Colibri.
> > > > >
> > > > > The question is:
> > > > > * Is this ok?
> > > > > * Do we want to change the framework to support multiple skins? In
> > > this case we would need to introduce Interfaces everywhere, for each
> Page
> > > Object to have various implementations.
> > > > >
> > > > > One example of difference is Hover vs Clicking for the top level
> > > menus. Another example is that we don’t have anymore a contentmenu;
> it’s 3
> > > buttons. Etc.
> > > > >
> > > > > Any opinion?
> > > > >
> > > > > Thanks
> > > > > -Vincent
> > > _______________________________________________
> > > devs mailing list
> > > [hidden email]
> > > http://lists.xwiki.org/mailman/listinfo/devs
> > >
> >
> >
> >
> > --
> > Denis Gervalle
> > SOFTEC sa - CEO
> > _______________________________________________
> > devs mailing list
> > [hidden email]
> > http://lists.xwiki.org/mailman/listinfo/devs
> _______________________________________________
> devs mailing list
> [hidden email]
> http://lists.xwiki.org/mailman/listinfo/devs
>



--
Denis Gervalle
SOFTEC sa - CEO
_______________________________________________
devs mailing list
[hidden email]
http://lists.xwiki.org/mailman/listinfo/devs
Reply | Threaded
Open this post in threaded view
|

Re: [Brainstorming] Should we support only one skin in our functional tests or not?

Thomas Mortagne
Administrator
In reply to this post by vmassol
+1 for supporting only one skin in tests

On Tue, Aug 19, 2014 at 6:12 PM, [hidden email] <[hidden email]> wrote:

> Hi devs,
>
> ATM our selenium2-based test fwk for functional tests only supports one skin (Colibri).
>
> Guillaume is currently fixing the functional tests by changing our test framework to work with Flamingo.
>
> After his changes are made our framework won’t work anymore with Colibri.
>
> The question is:
> * Is this ok?
> * Do we want to change the framework to support multiple skins? In this case we would need to introduce Interfaces everywhere, for each Page Object to have various implementations.
>
> One example of difference is Hover vs Clicking for the top level menus. Another example is that we don’t have anymore a contentmenu; it’s 3 buttons. Etc.
>
> Any opinion?
>
> Thanks
> -Vincent
> _______________________________________________
> devs mailing list
> [hidden email]
> http://lists.xwiki.org/mailman/listinfo/devs



--
Thomas Mortagne
_______________________________________________
devs mailing list
[hidden email]
http://lists.xwiki.org/mailman/listinfo/devs
Reply | Threaded
Open this post in threaded view
|

Re: [Brainstorming] Should we support only one skin in our functional tests or not?

Eduard Moraru
+1 as well, since we already went in that direction with the tests and of
obvious resource limitation.

Thanks,
Eduard


On Wed, Aug 27, 2014 at 11:13 AM, Thomas Mortagne <[hidden email]
> wrote:

> +1 for supporting only one skin in tests
>
> On Tue, Aug 19, 2014 at 6:12 PM, [hidden email] <[hidden email]>
> wrote:
> > Hi devs,
> >
> > ATM our selenium2-based test fwk for functional tests only supports one
> skin (Colibri).
> >
> > Guillaume is currently fixing the functional tests by changing our test
> framework to work with Flamingo.
> >
> > After his changes are made our framework won’t work anymore with Colibri.
> >
> > The question is:
> > * Is this ok?
> > * Do we want to change the framework to support multiple skins? In this
> case we would need to introduce Interfaces everywhere, for each Page Object
> to have various implementations.
> >
> > One example of difference is Hover vs Clicking for the top level menus.
> Another example is that we don’t have anymore a contentmenu; it’s 3
> buttons. Etc.
> >
> > Any opinion?
> >
> > Thanks
> > -Vincent
> > _______________________________________________
> > devs mailing list
> > [hidden email]
> > http://lists.xwiki.org/mailman/listinfo/devs
>
>
>
> --
> Thomas Mortagne
> _______________________________________________
> devs mailing list
> [hidden email]
> http://lists.xwiki.org/mailman/listinfo/devs
>
_______________________________________________
devs mailing list
[hidden email]
http://lists.xwiki.org/mailman/listinfo/devs