Analysis of TPC loss on xwiki-commons-job since 2019-01-01

classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

Analysis of TPC loss on xwiki-commons-job since 2019-01-01

vmassol
Administrator
Hi Thomas and all,

I’ve just done the analysis of the xwiki-commons-job TPC loss displayed on http://maven.xwiki.org/site/clover/20190108/XWikiReport-20190101-2330-20190108-0150.html (and it’s still the case today on http://maven.xwiki.org/site/clover/20190202/XWikiReport-20190101-2330-20190202-0222.html).

xwiki-commons-job 84.3812 84.2416 -0.1395 -0.0002

There were only 2 classes that have had changes in TPC:
* DefaultJobProgress - lowered TPC
* AbstractRequest - increased TPC

But there’s more lowering than increasing globally which is why it’s in red.

Specifically the lowering happened in 2 places:
* http://maven.xwiki.org/site/clover/20190108/clover-commons+rendering+platform-20190108-0150/org/xwiki/job/internal/DefaultJobProgress.html?line=133#src-133 vs http://maven.xwiki.org/site/clover/20190101/clover-commons+rendering+platform-20190101-2330/org/xwiki/job/internal/DefaultJobProgress.html?line=133#src-133
* http://maven.xwiki.org/site/clover/20190108/clover-commons+rendering+platform-20190108-0150/org/xwiki/job/internal/DefaultJobProgress.html?line=219#src-219 vs http://maven.xwiki.org/site/clover/20190101/clover-commons+rendering+platform-20190101-2330/org/xwiki/job/internal/DefaultJobProgress.html?line=219#src-219

Note that no source code changed in the job module.

Conclusions:
* It was hard to track and I need to improve the report to show a package level difference too and not just modules, at least for modules going down. Actually, even better would be a class level difference too for modules going down.
* In this case, I believe we had some bugs in XWiki somewhere that led to hitting the "Could not find any matching step for source [{}]. Ignoring EndStepProgress.” error. Could someone confirm that, it rings a bell to me?
* It also shows there’s no module level tests (ie unit tests) that go on in this IF and it would be good to add one to prove that we get a log when we have an end without a start.

It’s interesting because this is a use case where our global TPC went down because we fixed a bug (and thus indirectly we enter into less code). It mostly highlights that we don’t tests this case and we should.

Thanks
-Vincent

Reply | Threaded
Open this post in threaded view
|

Re: Analysis of TPC loss on xwiki-commons-job since 2019-01-01

vmassol
Administrator
Forgot to mention that we have to be careful when reading Clover report data.

If you click on org.xwiki.job package you”ll see it gives:
* 88.2% for http://maven.xwiki.org/site/clover/20190101/clover-commons+rendering+platform-20190101-2330/org/xwiki/job/pkg-summary.html
* 88.7% for http://maven.xwiki.org/site/clover/20190108/clover-commons+rendering+platform-20190108-0150/org/xwiki/job/pkg-summary.html

So you may be tempted to think that we increased the coverage. However it’s not the case.

It says 15 files for org.xwiki.job but there are 28 in reality.

I don’t really know how Clover does its computation but it’s not correct.

FTR I spent 2 hours recomputing everything for xwiki-commons-job, by debugging my report code, to prove that it’s actually correct.

Thanks
-Vincent

> On 10 Feb 2019, at 13:55, Vincent Massol <[hidden email]> wrote:
>
> Hi Thomas and all,
>
> I’ve just done the analysis of the xwiki-commons-job TPC loss displayed on http://maven.xwiki.org/site/clover/20190108/XWikiReport-20190101-2330-20190108-0150.html (and it’s still the case today on http://maven.xwiki.org/site/clover/20190202/XWikiReport-20190101-2330-20190202-0222.html).
>
> xwiki-commons-job 84.3812 84.2416 -0.1395 -0.0002
>
> There were only 2 classes that have had changes in TPC:
> * DefaultJobProgress - lowered TPC
> * AbstractRequest - increased TPC
>
> But there’s more lowering than increasing globally which is why it’s in red.
>
> Specifically the lowering happened in 2 places:
> * http://maven.xwiki.org/site/clover/20190108/clover-commons+rendering+platform-20190108-0150/org/xwiki/job/internal/DefaultJobProgress.html?line=133#src-133 vs http://maven.xwiki.org/site/clover/20190101/clover-commons+rendering+platform-20190101-2330/org/xwiki/job/internal/DefaultJobProgress.html?line=133#src-133
> * http://maven.xwiki.org/site/clover/20190108/clover-commons+rendering+platform-20190108-0150/org/xwiki/job/internal/DefaultJobProgress.html?line=219#src-219 vs http://maven.xwiki.org/site/clover/20190101/clover-commons+rendering+platform-20190101-2330/org/xwiki/job/internal/DefaultJobProgress.html?line=219#src-219
>
> Note that no source code changed in the job module.
>
> Conclusions:
> * It was hard to track and I need to improve the report to show a package level difference too and not just modules, at least for modules going down. Actually, even better would be a class level difference too for modules going down.
> * In this case, I believe we had some bugs in XWiki somewhere that led to hitting the "Could not find any matching step for source [{}]. Ignoring EndStepProgress.” error. Could someone confirm that, it rings a bell to me?
> * It also shows there’s no module level tests (ie unit tests) that go on in this IF and it would be good to add one to prove that we get a log when we have an end without a start.
>
> It’s interesting because this is a use case where our global TPC went down because we fixed a bug (and thus indirectly we enter into less code). It mostly highlights that we don’t tests this case and we should.
>
> Thanks
> -Vincent
>