Improving tests with Descartes

classic Classic list List threaded Threaded
9 messages Options
Reply | Threaded
Open this post in threaded view
|

Improving tests with Descartes

vmassol
Administrator
Hi devs,

It would be great if you could help improve our unit tests using Descartes. This is needed for the STAMP research project (https://www.stamp-project.eu/view/main/) and will benefit XWiki by having 2 effects:
* increasing the test coverage
* improving the tests themselves (increasing their mutation score)

Since 10.7 is 50% testing and 50% BFD, it would be great if you could spend all or a substantial part of your testing time working on this.

I propose the following strategy:
* You find a module you want to work on.
* In that module you run: mvn clean install -Pquality -Dxwiki.pitest.skip=false
* Then you check target/pit-reports/<date>/issues/index.html and verify if there are "pseudo tested" methods listed (when we have finished fixing all of those we can move to “partially tested methods”).
* If there are some, then please record the current jacoco threshold and the current mutation score.
* You can get the jacoco threshold by running "mvn clean install -Pquality -Dxwiki.pitest.skip=false -Dxwiki.pitest.mutationThreshold=100” (or by checking target/pit-reports/<date>/index.html, I haven’t checked yet if they are the same).
* You can get the current mutation score by checking target/pit-reports/<date>/index.html
* Then fix the test so that Descartes doesn’t report any pseudo tested or partially tested methods
* Update the jacoco threshold and the mutation scores in the pom.xml
* Send a PR on https://github.com/STAMP-project/descartes-usecases-output/tree/master/xwiki using the format already defined there.

WDYT? Doable?

Thanks
-Vincent

Reply | Threaded
Open this post in threaded view
|

Re: Improving tests with Descartes

vmassol
Administrator


> On 30 Jul 2018, at 12:39, Vincent Massol <[hidden email]> wrote:
>
> Hi devs,
>
> It would be great if you could help improve our unit tests using Descartes. This is needed for the STAMP research project (https://www.stamp-project.eu/view/main/) and will benefit XWiki by having 2 effects:
> * increasing the test coverage
> * improving the tests themselves (increasing their mutation score)
>
> Since 10.7 is 50% testing and 50% BFD, it would be great if you could spend all or a substantial part of your testing time working on this.
>
> I propose the following strategy:
> * You find a module you want to work on.
> * In that module you run: mvn clean install -Pquality -Dxwiki.pitest.skip=false

You can also run that at the top of a multimodule project and then find pseudo/partially tested methods with:

find . -name "methods.json" -exec egrep -oH "pseudo-tested|partially-tested" {} \;

Thanks
-Vincent

> * Then you check target/pit-reports/<date>/issues/index.html and verify if there are "pseudo tested" methods listed (when we have finished fixing all of those we can move to “partially tested methods”).
> * If there are some, then please record the current jacoco threshold and the current mutation score.
> * You can get the jacoco threshold by running "mvn clean install -Pquality -Dxwiki.pitest.skip=false -Dxwiki.pitest.mutationThreshold=100” (or by checking target/pit-reports/<date>/index.html, I haven’t checked yet if they are the same).
> * You can get the current mutation score by checking target/pit-reports/<date>/index.html
> * Then fix the test so that Descartes doesn’t report any pseudo tested or partially tested methods
> * Update the jacoco threshold and the mutation scores in the pom.xml
> * Send a PR on https://github.com/STAMP-project/descartes-usecases-output/tree/master/xwiki using the format already defined there.
>
> WDYT? Doable?
>
> Thanks
> -Vincent
>

Reply | Threaded
Open this post in threaded view
|

Re: Improving tests with Descartes

vmassol
Administrator
Note for Thomas (or whoever else is interested in improving the commons extension tests) there are plenty in

./xwiki-commons-core/xwiki-commons-extension/xwiki-commons-extension-api/target/pit-reports/201808071749/methods.json:pseudo-tested
pseudo-tested
pseudo-tested
partially-tested
partially-tested
pseudo-tested
pseudo-tested
pseudo-tested
pseudo-tested
pseudo-tested
partially-tested
pseudo-tested
pseudo-tested
partially-tested
pseudo-tested
partially-tested
pseudo-tested
pseudo-tested
pseudo-tested
pseudo-tested
pseudo-tested
pseudo-tested
pseudo-tested
partially-tested
partially-tested
pseudo-tested
pseudo-tested
pseudo-tested
pseudo-tested
partially-tested
pseudo-tested
pseudo-tested
pseudo-tested
pseudo-tested
pseudo-tested
pseudo-tested
pseudo-tested
pseudo-tested
partially-tested
pseudo-tested
pseudo-tested
pseudo-tested
pseudo-tested
pseudo-tested
pseudo-tested
pseudo-tested
pseudo-tested
partially-tested
pseudo-tested
pseudo-tested
pseudo-tested
pseudo-tested
pseudo-tested
pseudo-tested
pseudo-tested
partially-tested
partially-tested
pseudo-tested
partially-tested
partially-tested
partially-tested
partially-tested
partially-tested
pseudo-tested
pseudo-tested

Fixing those should yield a good increase in both mutation score and test coverage.

Thanks
-Vincent

> On 7 Aug 2018, at 18:05, Vincent Massol <[hidden email]> wrote:
>
>
>
>> On 30 Jul 2018, at 12:39, Vincent Massol <[hidden email]> wrote:
>>
>> Hi devs,
>>
>> It would be great if you could help improve our unit tests using Descartes. This is needed for the STAMP research project (https://www.stamp-project.eu/view/main/) and will benefit XWiki by having 2 effects:
>> * increasing the test coverage
>> * improving the tests themselves (increasing their mutation score)
>>
>> Since 10.7 is 50% testing and 50% BFD, it would be great if you could spend all or a substantial part of your testing time working on this.
>>
>> I propose the following strategy:
>> * You find a module you want to work on.
>> * In that module you run: mvn clean install -Pquality -Dxwiki.pitest.skip=false
>
> You can also run that at the top of a multimodule project and then find pseudo/partially tested methods with:
>
> find . -name "methods.json" -exec egrep -oH "pseudo-tested|partially-tested" {} \;
>
> Thanks
> -Vincent
>
>> * Then you check target/pit-reports/<date>/issues/index.html and verify if there are "pseudo tested" methods listed (when we have finished fixing all of those we can move to “partially tested methods”).
>> * If there are some, then please record the current jacoco threshold and the current mutation score.
>> * You can get the jacoco threshold by running "mvn clean install -Pquality -Dxwiki.pitest.skip=false -Dxwiki.pitest.mutationThreshold=100” (or by checking target/pit-reports/<date>/index.html, I haven’t checked yet if they are the same).
>> * You can get the current mutation score by checking target/pit-reports/<date>/index.html
>> * Then fix the test so that Descartes doesn’t report any pseudo tested or partially tested methods
>> * Update the jacoco threshold and the mutation scores in the pom.xml
>> * Send a PR on https://github.com/STAMP-project/descartes-usecases-output/tree/master/xwiki using the format already defined there.
>>
>> WDYT? Doable?
>>
>> Thanks
>> -Vincent

Reply | Threaded
Open this post in threaded view
|

Re: Improving tests with Descartes

vmassol
Administrator
In reply to this post by vmassol


> On 30 Jul 2018, at 12:39, Vincent Massol <[hidden email]> wrote:
>
> Hi devs,
>
> It would be great if you could help improve our unit tests using Descartes. This is needed for the STAMP research project (https://www.stamp-project.eu/view/main/) and will benefit XWiki by having 2 effects:
> * increasing the test coverage
> * improving the tests themselves (increasing their mutation score)
>
> Since 10.7 is 50% testing and 50% BFD, it would be great if you could spend all or a substantial part of your testing time working on this.
>
> I propose the following strategy:
> * You find a module you want to work on.
> * In that module you run: mvn clean install -Pquality -Dxwiki.pitest.skip=false
> * Then you check target/pit-reports/<date>/issues/index.html and verify if there are "pseudo tested" methods listed (when we have finished fixing all of those we can move to “partially tested methods”).
> * If there are some, then please record the current jacoco threshold and the current mutation score.
> * You can get the jacoco threshold by running "mvn clean install -Pquality -Dxwiki.pitest.skip=false -Dxwiki.pitest.mutationThreshold=100” (or by checking target/pit-reports/<date>/index.html, I haven’t checked yet if they are the same).

^^^^

Correction:

jacoco threshold:
mvn clean install -Pquality -Dxwiki.jacoco.instructionRatio=1.00

mutation threshold:
mvn clean install -Pquality -Dxwiki.pitest.skip=false -Dxwiki.pitest.mutationThreshold=100

Thanks
-Vincent

> * You can get the current mutation score by checking target/pit-reports/<date>/index.html
> * Then fix the test so that Descartes doesn’t report any pseudo tested or partially tested methods
> * Update the jacoco threshold and the mutation scores in the pom.xml
> * Send a PR on https://github.com/STAMP-project/descartes-usecases-output/tree/master/xwiki using the format already defined there.
>
> WDYT? Doable?
>
> Thanks
> -Vincent
>

Reply | Threaded
Open this post in threaded view
|

Re: Improving tests with Descartes

vmassol
Administrator
In reply to this post by vmassol


> On 30 Jul 2018, at 12:39, Vincent Massol <[hidden email]> wrote:
>
> Hi devs,
>
> It would be great if you could help improve our unit tests using Descartes. This is needed for the STAMP research project (https://www.stamp-project.eu/view/main/) and will benefit XWiki by having 2 effects:
> * increasing the test coverage
> * improving the tests themselves (increasing their mutation score)
>
> Since 10.7 is 50% testing and 50% BFD, it would be great if you could spend all or a substantial part of your testing time working on this.
>
> I propose the following strategy:
> * You find a module you want to work on.
> * In that module you run: mvn clean install -Pquality -Dxwiki.pitest.skip=false
> * Then you check target/pit-reports/<date>/issues/index.html and verify if there are "pseudo tested" methods listed (when we have finished fixing all of those we can move to “partially tested methods”).
> * If there are some, then please record the current jacoco threshold and the current mutation score.
> * You can get the jacoco threshold by running "mvn clean install -Pquality -Dxwiki.pitest.skip=false -Dxwiki.pitest.mutationThreshold=100” (or by checking target/pit-reports/<date>/index.html, I haven’t checked yet if they are the same).
> * You can get the current mutation score by checking target/pit-reports/<date>/index.html
> * Then fix the test so that Descartes doesn’t report any pseudo tested or partially tested methods
> * Update the jacoco threshold and the mutation scores in the pom.xml
> * Send a PR on https://github.com/STAMP-project/descartes-usecases-output/tree/master/xwiki using the format already defined there.
>
> WDYT? Doable?

Just fixed all errors on the commons-component-api module and reported the score upgrade:
https://github.com/STAMP-project/descartes-usecases-output/commit/bdc1ecb42b906b91fe5bdc09a2cae7aebdeb5146

Who’s next? :)

Thanks
-Vincent

>
> Thanks
> -Vincent
>

Reply | Threaded
Open this post in threaded view
|

Re: Improving tests with Descartes

Thomas Mortagne
Administrator
In reply to this post by vmassol
This is probably because xwiki-commons-extension-api contains a lot of
what is closer to integration than unit tests.

On Tue, Aug 7, 2018 at 6:07 PM, Vincent Massol <[hidden email]> wrote:

> Note for Thomas (or whoever else is interested in improving the commons extension tests) there are plenty in
>
> ./xwiki-commons-core/xwiki-commons-extension/xwiki-commons-extension-api/target/pit-reports/201808071749/methods.json:pseudo-tested
> pseudo-tested
> pseudo-tested
> partially-tested
> partially-tested
> pseudo-tested
> pseudo-tested
> pseudo-tested
> pseudo-tested
> pseudo-tested
> partially-tested
> pseudo-tested
> pseudo-tested
> partially-tested
> pseudo-tested
> partially-tested
> pseudo-tested
> pseudo-tested
> pseudo-tested
> pseudo-tested
> pseudo-tested
> pseudo-tested
> pseudo-tested
> partially-tested
> partially-tested
> pseudo-tested
> pseudo-tested
> pseudo-tested
> pseudo-tested
> partially-tested
> pseudo-tested
> pseudo-tested
> pseudo-tested
> pseudo-tested
> pseudo-tested
> pseudo-tested
> pseudo-tested
> pseudo-tested
> partially-tested
> pseudo-tested
> pseudo-tested
> pseudo-tested
> pseudo-tested
> pseudo-tested
> pseudo-tested
> pseudo-tested
> pseudo-tested
> partially-tested
> pseudo-tested
> pseudo-tested
> pseudo-tested
> pseudo-tested
> pseudo-tested
> pseudo-tested
> pseudo-tested
> partially-tested
> partially-tested
> pseudo-tested
> partially-tested
> partially-tested
> partially-tested
> partially-tested
> partially-tested
> pseudo-tested
> pseudo-tested
>
> Fixing those should yield a good increase in both mutation score and test coverage.
>
> Thanks
> -Vincent
>
>> On 7 Aug 2018, at 18:05, Vincent Massol <[hidden email]> wrote:
>>
>>
>>
>>> On 30 Jul 2018, at 12:39, Vincent Massol <[hidden email]> wrote:
>>>
>>> Hi devs,
>>>
>>> It would be great if you could help improve our unit tests using Descartes. This is needed for the STAMP research project (https://www.stamp-project.eu/view/main/) and will benefit XWiki by having 2 effects:
>>> * increasing the test coverage
>>> * improving the tests themselves (increasing their mutation score)
>>>
>>> Since 10.7 is 50% testing and 50% BFD, it would be great if you could spend all or a substantial part of your testing time working on this.
>>>
>>> I propose the following strategy:
>>> * You find a module you want to work on.
>>> * In that module you run: mvn clean install -Pquality -Dxwiki.pitest.skip=false
>>
>> You can also run that at the top of a multimodule project and then find pseudo/partially tested methods with:
>>
>> find . -name "methods.json" -exec egrep -oH "pseudo-tested|partially-tested" {} \;
>>
>> Thanks
>> -Vincent
>>
>>> * Then you check target/pit-reports/<date>/issues/index.html and verify if there are "pseudo tested" methods listed (when we have finished fixing all of those we can move to “partially tested methods”).
>>> * If there are some, then please record the current jacoco threshold and the current mutation score.
>>> * You can get the jacoco threshold by running "mvn clean install -Pquality -Dxwiki.pitest.skip=false -Dxwiki.pitest.mutationThreshold=100” (or by checking target/pit-reports/<date>/index.html, I haven’t checked yet if they are the same).
>>> * You can get the current mutation score by checking target/pit-reports/<date>/index.html
>>> * Then fix the test so that Descartes doesn’t report any pseudo tested or partially tested methods
>>> * Update the jacoco threshold and the mutation scores in the pom.xml
>>> * Send a PR on https://github.com/STAMP-project/descartes-usecases-output/tree/master/xwiki using the format already defined there.
>>>
>>> WDYT? Doable?
>>>
>>> Thanks
>>> -Vincent
>



--
Thomas Mortagne
Reply | Threaded
Open this post in threaded view
|

Re: Improving tests with Descartes

Thomas Mortagne
Administrator
Started looking at xwiki-commons-extension-api by the way.

On Thu, Aug 9, 2018 at 11:50 AM, Thomas Mortagne
<[hidden email]> wrote:

> This is probably because xwiki-commons-extension-api contains a lot of
> what is closer to integration than unit tests.
>
> On Tue, Aug 7, 2018 at 6:07 PM, Vincent Massol <[hidden email]> wrote:
>> Note for Thomas (or whoever else is interested in improving the commons extension tests) there are plenty in
>>
>> ./xwiki-commons-core/xwiki-commons-extension/xwiki-commons-extension-api/target/pit-reports/201808071749/methods.json:pseudo-tested
>> pseudo-tested
>> pseudo-tested
>> partially-tested
>> partially-tested
>> pseudo-tested
>> pseudo-tested
>> pseudo-tested
>> pseudo-tested
>> pseudo-tested
>> partially-tested
>> pseudo-tested
>> pseudo-tested
>> partially-tested
>> pseudo-tested
>> partially-tested
>> pseudo-tested
>> pseudo-tested
>> pseudo-tested
>> pseudo-tested
>> pseudo-tested
>> pseudo-tested
>> pseudo-tested
>> partially-tested
>> partially-tested
>> pseudo-tested
>> pseudo-tested
>> pseudo-tested
>> pseudo-tested
>> partially-tested
>> pseudo-tested
>> pseudo-tested
>> pseudo-tested
>> pseudo-tested
>> pseudo-tested
>> pseudo-tested
>> pseudo-tested
>> pseudo-tested
>> partially-tested
>> pseudo-tested
>> pseudo-tested
>> pseudo-tested
>> pseudo-tested
>> pseudo-tested
>> pseudo-tested
>> pseudo-tested
>> pseudo-tested
>> partially-tested
>> pseudo-tested
>> pseudo-tested
>> pseudo-tested
>> pseudo-tested
>> pseudo-tested
>> pseudo-tested
>> pseudo-tested
>> partially-tested
>> partially-tested
>> pseudo-tested
>> partially-tested
>> partially-tested
>> partially-tested
>> partially-tested
>> partially-tested
>> pseudo-tested
>> pseudo-tested
>>
>> Fixing those should yield a good increase in both mutation score and test coverage.
>>
>> Thanks
>> -Vincent
>>
>>> On 7 Aug 2018, at 18:05, Vincent Massol <[hidden email]> wrote:
>>>
>>>
>>>
>>>> On 30 Jul 2018, at 12:39, Vincent Massol <[hidden email]> wrote:
>>>>
>>>> Hi devs,
>>>>
>>>> It would be great if you could help improve our unit tests using Descartes. This is needed for the STAMP research project (https://www.stamp-project.eu/view/main/) and will benefit XWiki by having 2 effects:
>>>> * increasing the test coverage
>>>> * improving the tests themselves (increasing their mutation score)
>>>>
>>>> Since 10.7 is 50% testing and 50% BFD, it would be great if you could spend all or a substantial part of your testing time working on this.
>>>>
>>>> I propose the following strategy:
>>>> * You find a module you want to work on.
>>>> * In that module you run: mvn clean install -Pquality -Dxwiki.pitest.skip=false
>>>
>>> You can also run that at the top of a multimodule project and then find pseudo/partially tested methods with:
>>>
>>> find . -name "methods.json" -exec egrep -oH "pseudo-tested|partially-tested" {} \;
>>>
>>> Thanks
>>> -Vincent
>>>
>>>> * Then you check target/pit-reports/<date>/issues/index.html and verify if there are "pseudo tested" methods listed (when we have finished fixing all of those we can move to “partially tested methods”).
>>>> * If there are some, then please record the current jacoco threshold and the current mutation score.
>>>> * You can get the jacoco threshold by running "mvn clean install -Pquality -Dxwiki.pitest.skip=false -Dxwiki.pitest.mutationThreshold=100” (or by checking target/pit-reports/<date>/index.html, I haven’t checked yet if they are the same).
>>>> * You can get the current mutation score by checking target/pit-reports/<date>/index.html
>>>> * Then fix the test so that Descartes doesn’t report any pseudo tested or partially tested methods
>>>> * Update the jacoco threshold and the mutation scores in the pom.xml
>>>> * Send a PR on https://github.com/STAMP-project/descartes-usecases-output/tree/master/xwiki using the format already defined there.
>>>>
>>>> WDYT? Doable?
>>>>
>>>> Thanks
>>>> -Vincent
>>
>
>
>
> --
> Thomas Mortagne



--
Thomas Mortagne
Reply | Threaded
Open this post in threaded view
|

Re: Improving tests with Descartes

Thomas Mortagne
Administrator
Done: https://github.com/STAMP-project/descartes-usecases-output/pull/1

On Thu, Aug 9, 2018 at 3:07 PM, Thomas Mortagne
<[hidden email]> wrote:

> Started looking at xwiki-commons-extension-api by the way.
>
> On Thu, Aug 9, 2018 at 11:50 AM, Thomas Mortagne
> <[hidden email]> wrote:
>> This is probably because xwiki-commons-extension-api contains a lot of
>> what is closer to integration than unit tests.
>>
>> On Tue, Aug 7, 2018 at 6:07 PM, Vincent Massol <[hidden email]> wrote:
>>> Note for Thomas (or whoever else is interested in improving the commons extension tests) there are plenty in
>>>
>>> ./xwiki-commons-core/xwiki-commons-extension/xwiki-commons-extension-api/target/pit-reports/201808071749/methods.json:pseudo-tested
>>> pseudo-tested
>>> pseudo-tested
>>> partially-tested
>>> partially-tested
>>> pseudo-tested
>>> pseudo-tested
>>> pseudo-tested
>>> pseudo-tested
>>> pseudo-tested
>>> partially-tested
>>> pseudo-tested
>>> pseudo-tested
>>> partially-tested
>>> pseudo-tested
>>> partially-tested
>>> pseudo-tested
>>> pseudo-tested
>>> pseudo-tested
>>> pseudo-tested
>>> pseudo-tested
>>> pseudo-tested
>>> pseudo-tested
>>> partially-tested
>>> partially-tested
>>> pseudo-tested
>>> pseudo-tested
>>> pseudo-tested
>>> pseudo-tested
>>> partially-tested
>>> pseudo-tested
>>> pseudo-tested
>>> pseudo-tested
>>> pseudo-tested
>>> pseudo-tested
>>> pseudo-tested
>>> pseudo-tested
>>> pseudo-tested
>>> partially-tested
>>> pseudo-tested
>>> pseudo-tested
>>> pseudo-tested
>>> pseudo-tested
>>> pseudo-tested
>>> pseudo-tested
>>> pseudo-tested
>>> pseudo-tested
>>> partially-tested
>>> pseudo-tested
>>> pseudo-tested
>>> pseudo-tested
>>> pseudo-tested
>>> pseudo-tested
>>> pseudo-tested
>>> pseudo-tested
>>> partially-tested
>>> partially-tested
>>> pseudo-tested
>>> partially-tested
>>> partially-tested
>>> partially-tested
>>> partially-tested
>>> partially-tested
>>> pseudo-tested
>>> pseudo-tested
>>>
>>> Fixing those should yield a good increase in both mutation score and test coverage.
>>>
>>> Thanks
>>> -Vincent
>>>
>>>> On 7 Aug 2018, at 18:05, Vincent Massol <[hidden email]> wrote:
>>>>
>>>>
>>>>
>>>>> On 30 Jul 2018, at 12:39, Vincent Massol <[hidden email]> wrote:
>>>>>
>>>>> Hi devs,
>>>>>
>>>>> It would be great if you could help improve our unit tests using Descartes. This is needed for the STAMP research project (https://www.stamp-project.eu/view/main/) and will benefit XWiki by having 2 effects:
>>>>> * increasing the test coverage
>>>>> * improving the tests themselves (increasing their mutation score)
>>>>>
>>>>> Since 10.7 is 50% testing and 50% BFD, it would be great if you could spend all or a substantial part of your testing time working on this.
>>>>>
>>>>> I propose the following strategy:
>>>>> * You find a module you want to work on.
>>>>> * In that module you run: mvn clean install -Pquality -Dxwiki.pitest.skip=false
>>>>
>>>> You can also run that at the top of a multimodule project and then find pseudo/partially tested methods with:
>>>>
>>>> find . -name "methods.json" -exec egrep -oH "pseudo-tested|partially-tested" {} \;
>>>>
>>>> Thanks
>>>> -Vincent
>>>>
>>>>> * Then you check target/pit-reports/<date>/issues/index.html and verify if there are "pseudo tested" methods listed (when we have finished fixing all of those we can move to “partially tested methods”).
>>>>> * If there are some, then please record the current jacoco threshold and the current mutation score.
>>>>> * You can get the jacoco threshold by running "mvn clean install -Pquality -Dxwiki.pitest.skip=false -Dxwiki.pitest.mutationThreshold=100” (or by checking target/pit-reports/<date>/index.html, I haven’t checked yet if they are the same).
>>>>> * You can get the current mutation score by checking target/pit-reports/<date>/index.html
>>>>> * Then fix the test so that Descartes doesn’t report any pseudo tested or partially tested methods
>>>>> * Update the jacoco threshold and the mutation scores in the pom.xml
>>>>> * Send a PR on https://github.com/STAMP-project/descartes-usecases-output/tree/master/xwiki using the format already defined there.
>>>>>
>>>>> WDYT? Doable?
>>>>>
>>>>> Thanks
>>>>> -Vincent
>>>
>>
>>
>>
>> --
>> Thomas Mortagne
>
>
>
> --
> Thomas Mortagne



--
Thomas Mortagne
Reply | Threaded
Open this post in threaded view
|

Re: Improving tests with Descartes

Thomas Mortagne
Administrator
Note: Descartes (or Pitest) seems be a bit lost with tests located on
abstract classes. For example it pretty much react as if those 3
classes where not existing:
https://github.com/xwiki/xwiki-commons/tree/master/xwiki-commons-core/xwiki-commons-filter/xwiki-commons-filter-api/src/test/java/org/xwiki/filter/internal
(which makes the score way lower than it should I think).

On Thu, Aug 9, 2018 at 6:10 PM, Thomas Mortagne
<[hidden email]> wrote:

> Done: https://github.com/STAMP-project/descartes-usecases-output/pull/1
>
> On Thu, Aug 9, 2018 at 3:07 PM, Thomas Mortagne
> <[hidden email]> wrote:
>> Started looking at xwiki-commons-extension-api by the way.
>>
>> On Thu, Aug 9, 2018 at 11:50 AM, Thomas Mortagne
>> <[hidden email]> wrote:
>>> This is probably because xwiki-commons-extension-api contains a lot of
>>> what is closer to integration than unit tests.
>>>
>>> On Tue, Aug 7, 2018 at 6:07 PM, Vincent Massol <[hidden email]> wrote:
>>>> Note for Thomas (or whoever else is interested in improving the commons extension tests) there are plenty in
>>>>
>>>> ./xwiki-commons-core/xwiki-commons-extension/xwiki-commons-extension-api/target/pit-reports/201808071749/methods.json:pseudo-tested
>>>> pseudo-tested
>>>> pseudo-tested
>>>> partially-tested
>>>> partially-tested
>>>> pseudo-tested
>>>> pseudo-tested
>>>> pseudo-tested
>>>> pseudo-tested
>>>> pseudo-tested
>>>> partially-tested
>>>> pseudo-tested
>>>> pseudo-tested
>>>> partially-tested
>>>> pseudo-tested
>>>> partially-tested
>>>> pseudo-tested
>>>> pseudo-tested
>>>> pseudo-tested
>>>> pseudo-tested
>>>> pseudo-tested
>>>> pseudo-tested
>>>> pseudo-tested
>>>> partially-tested
>>>> partially-tested
>>>> pseudo-tested
>>>> pseudo-tested
>>>> pseudo-tested
>>>> pseudo-tested
>>>> partially-tested
>>>> pseudo-tested
>>>> pseudo-tested
>>>> pseudo-tested
>>>> pseudo-tested
>>>> pseudo-tested
>>>> pseudo-tested
>>>> pseudo-tested
>>>> pseudo-tested
>>>> partially-tested
>>>> pseudo-tested
>>>> pseudo-tested
>>>> pseudo-tested
>>>> pseudo-tested
>>>> pseudo-tested
>>>> pseudo-tested
>>>> pseudo-tested
>>>> pseudo-tested
>>>> partially-tested
>>>> pseudo-tested
>>>> pseudo-tested
>>>> pseudo-tested
>>>> pseudo-tested
>>>> pseudo-tested
>>>> pseudo-tested
>>>> pseudo-tested
>>>> partially-tested
>>>> partially-tested
>>>> pseudo-tested
>>>> partially-tested
>>>> partially-tested
>>>> partially-tested
>>>> partially-tested
>>>> partially-tested
>>>> pseudo-tested
>>>> pseudo-tested
>>>>
>>>> Fixing those should yield a good increase in both mutation score and test coverage.
>>>>
>>>> Thanks
>>>> -Vincent
>>>>
>>>>> On 7 Aug 2018, at 18:05, Vincent Massol <[hidden email]> wrote:
>>>>>
>>>>>
>>>>>
>>>>>> On 30 Jul 2018, at 12:39, Vincent Massol <[hidden email]> wrote:
>>>>>>
>>>>>> Hi devs,
>>>>>>
>>>>>> It would be great if you could help improve our unit tests using Descartes. This is needed for the STAMP research project (https://www.stamp-project.eu/view/main/) and will benefit XWiki by having 2 effects:
>>>>>> * increasing the test coverage
>>>>>> * improving the tests themselves (increasing their mutation score)
>>>>>>
>>>>>> Since 10.7 is 50% testing and 50% BFD, it would be great if you could spend all or a substantial part of your testing time working on this.
>>>>>>
>>>>>> I propose the following strategy:
>>>>>> * You find a module you want to work on.
>>>>>> * In that module you run: mvn clean install -Pquality -Dxwiki.pitest.skip=false
>>>>>
>>>>> You can also run that at the top of a multimodule project and then find pseudo/partially tested methods with:
>>>>>
>>>>> find . -name "methods.json" -exec egrep -oH "pseudo-tested|partially-tested" {} \;
>>>>>
>>>>> Thanks
>>>>> -Vincent
>>>>>
>>>>>> * Then you check target/pit-reports/<date>/issues/index.html and verify if there are "pseudo tested" methods listed (when we have finished fixing all of those we can move to “partially tested methods”).
>>>>>> * If there are some, then please record the current jacoco threshold and the current mutation score.
>>>>>> * You can get the jacoco threshold by running "mvn clean install -Pquality -Dxwiki.pitest.skip=false -Dxwiki.pitest.mutationThreshold=100” (or by checking target/pit-reports/<date>/index.html, I haven’t checked yet if they are the same).
>>>>>> * You can get the current mutation score by checking target/pit-reports/<date>/index.html
>>>>>> * Then fix the test so that Descartes doesn’t report any pseudo tested or partially tested methods
>>>>>> * Update the jacoco threshold and the mutation scores in the pom.xml
>>>>>> * Send a PR on https://github.com/STAMP-project/descartes-usecases-output/tree/master/xwiki using the format already defined there.
>>>>>>
>>>>>> WDYT? Doable?
>>>>>>
>>>>>> Thanks
>>>>>> -Vincent
>>>>
>>>
>>>
>>>
>>> --
>>> Thomas Mortagne
>>
>>
>>
>> --
>> Thomas Mortagne
>
>
>
> --
> Thomas Mortagne



--
Thomas Mortagne