[GSOC Update 3] RedPen Integration

classic Classic list List threaded Threaded
6 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

[GSOC Update 3] RedPen Integration

deshengchuan
Hi all!
This is the third update regarding the progress of the Redpen integration project.
Some of the info can also be accessed in the design page over here:


I have created a UI on XWiki. The two main parts are such:
First, the application homepage will allow users to key in possible spelling and expression errors, and their recommended corrections. I have not started on building the backend in Java to process this database yet, but I'll start on it within the next couple of weeks.
Second, there is a setting page within the global administration wiki that allows the user to key in basic settings parameters e.g. setting maximum sentence length.

The functionality is still restricted to validating a wiki page on document save, but I will be implementing the function to run redpen validation as a Job next, right after i finish fixing and stabilising some issues within the code.
Any comments?
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: [GSOC Update 3] RedPen Integration

vmassol
Administrator
Hi DeSheng,

> On 27 Jun 2017, at 07:10, deshengchuan <[hidden email]> wrote:
>
> Hi all!
> This is the third update regarding the progress of the Redpen integration
> project.
> Some of the info can also be accessed in the design page over here:  
> <http://design.xwiki.org/xwiki/bin/view/Proposal/RedPenIntegration>  
>
> I have created a UI on XWiki. The two main parts are such:
> First, the application homepage will allow users to key in possible spelling
> and expression errors, and their recommended corrections. I have not started
> on building the backend in Java to process this database yet, but I'll start
> on it within the next couple of weeks.
> Second, there is a setting page within the global administration wiki that
> allows the user to key in basic settings parameters e.g. setting maximum
> sentence length.
>
> The functionality is still restricted to validating a wiki page on document
> save, but I will be implementing the function to run redpen validation as a
> Job next, right after i finish fixing and stabilising some issues within the
> code.
> Any comments?

Thanks. I think you’re on the right track and that you understand well the objectives and what needs to be done :)

Some questions/remarks:
* I still need to review your code in the coming few days
* Does it work? Am I able to get a RedPen validation when I save a page? (I need to test your extension now)
* In the ConfigurableClass, you should use a static list of Strings that lets the user enter the exclusions so that they can enter regex. Something similar to http://extensions.xwiki.org/xwiki/bin/view/Extension/Multiselect%20custom%20display I think this would be more powerful. IMO you should also have includes and not just excludes to cover all use cases.
* I see that you have implemented a UI to allow entering some words to be corrected. Does it mean that the RedPen execution not only validates content but is also able to modify it?

Thanks for the update!
-Vincent

> --
> View this message in context: http://xwiki.475771.n2.nabble.com/GSOC-Update-3-RedPen-Integration-tp7604255.html
> Sent from the XWiki- Dev mailing list archive at Nabble.com.

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: [GSOC Update 3] RedPen Integration

deshengchuan
vmassol wrote
Some questions/remarks:
* I still need to review your code in the coming few days
* Does it work? Am I able to get a RedPen validation when I save a page? (I need to test your extension now)
* In the ConfigurableClass, you should use a static list of Strings that lets the user enter the exclusions so that they can enter regex. Something similar to http://extensions.xwiki.org/xwiki/bin/view/Extension/Multiselect%20custom%20display I think this would be more powerful. IMO you should also have includes and not just excludes to cover all use cases.
* I see that you have implemented a UI to allow entering some words to be corrected. Does it mean that the RedPen execution not only validates content but is also able to modify it?

Thanks for the update!
-Vincent
Hi! Thanks in advance for the review :)

I am still working through issues in my code one at a time, and I don't think it would be working today. I will keep you updated on it, and if need be I can commit an earlier version, without much of the UI capabilities, which validates plain text on a page on save and appends the results at the bottom of the page.

After getting my code to work I'll edit the configurable class. I can see that includes and excludes would cover all use cases. I will work on editing the class such that the user can choose to include or exclude.

RedPen can take in a dictionary of words or expressions(in JSON format), but it doesn't alter the entries directly. Rather, it would output a message along with the other validation messages, with this one suggesting the word or expression replacement.
However, I think it is possible to implement a replacement function. Since the validation message provides the location of the error, we can extract the erroreous word or expression out and replace it with the corrected one. What do you think?
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: [GSOC Update 3] RedPen Integration

vmassol
Administrator

> On 27 Jun 2017, at 10:29, deshengchuan <[hidden email]> wrote:
>
> vmassol wrote
>> Some questions/remarks:
>> * I still need to review your code in the coming few days
>> * Does it work? Am I able to get a RedPen validation when I save a page?
>> (I need to test your extension now)
>> * In the ConfigurableClass, you should use a static list of Strings that
>> lets the user enter the exclusions so that they can enter regex. Something
>> similar to
>> http://extensions.xwiki.org/xwiki/bin/view/Extension/Multiselect%20custom%20display
>> I think this would be more powerful. IMO you should also have includes and
>> not just excludes to cover all use cases.
>> * I see that you have implemented a UI to allow entering some words to be
>> corrected. Does it mean that the RedPen execution not only validates
>> content but is also able to modify it?
>>
>> Thanks for the update!
>> -Vincent
>
> Hi! Thanks in advance for the review :)
>
> I am still working through issues in my code one at a time, and I don't
> think it would be working today. I will keep you updated on it, and if need
> be I can commit an earlier version, without much of the UI capabilities,
> which validates plain text on a page on save and appends the results at the
> bottom of the page.
>
> After getting my code to work I'll edit the configurable class. I can see
> that includes and excludes would cover all use cases. I will work on editing
> the class such that the user can choose to include or exclude.
>
> RedPen can take in a dictionary of words or expressions(in JSON format), but
> it doesn't alter the entries directly. Rather, it would output a message
> along with the other validation messages, with this one suggesting the word
> or expression replacement.
> However, I think it is possible to implement a replacement function. Since
> the validation message provides the location of the error, we can extract
> the erroreous word or expression out and replace it with the corrected one.
> What do you think?

I suggest that we focus on validation for the time being and get that to work nicely. When this is done, we can think about auto-fixing stuff. This would be doable in the listener, by changing the doc’s content before the save.

The part that worries me the most ATM is how to display nicely to the user the result of the validation. I don’t see how we can do that for now. Do you have an idea? I think this could go hand in hand with a feature we’ve been wanting for a long time (can’t fine the jira right now): the ability to capture logs when rendering a page and display them as notifications in the UI.

So the idea could be the following:
* The RedPen listener would issue warnings for example in the logs
* In the future when we implement the feature I mentioned, the user will be able to see them in the UI

Do you have a better idea?

I also think you need to include the concept of Validation Severity (i.e. warning, error). And when it’s a warning it should never stop saving the page. Only errors would do that.

WDYT?

Thanks
-Vincent

> --
> View this message in context: http://xwiki.475771.n2.nabble.com/GSOC-Update-3-RedPen-Integration-tp7604255p7604257.html
> Sent from the XWiki- Dev mailing list archive at Nabble.com.

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: [GSOC Update 3] RedPen Integration

vmassol
Administrator
I’ve started reviewing the code and there are some issues I believe. For example we need to agree on how to handle the configuration.

Generally speaking and in order to make sure you go in the right direction, I think you should regularly send implementation proposals to this list about how you’re planning to implement something so that we can help you and tell how how to do it in XWiki.

Regarding configuration, I’ve tried to put some info in comments.

Thanks
-Vincent

> On 28 Jun 2017, at 16:35, Vincent Massol <[hidden email]> wrote:
>
>>
>> On 27 Jun 2017, at 10:29, deshengchuan <[hidden email]> wrote:
>>
>> vmassol wrote
>>> Some questions/remarks:
>>> * I still need to review your code in the coming few days
>>> * Does it work? Am I able to get a RedPen validation when I save a page?
>>> (I need to test your extension now)
>>> * In the ConfigurableClass, you should use a static list of Strings that
>>> lets the user enter the exclusions so that they can enter regex. Something
>>> similar to
>>> http://extensions.xwiki.org/xwiki/bin/view/Extension/Multiselect%20custom%20display
>>> I think this would be more powerful. IMO you should also have includes and
>>> not just excludes to cover all use cases.
>>> * I see that you have implemented a UI to allow entering some words to be
>>> corrected. Does it mean that the RedPen execution not only validates
>>> content but is also able to modify it?
>>>
>>> Thanks for the update!
>>> -Vincent
>>
>> Hi! Thanks in advance for the review :)
>>
>> I am still working through issues in my code one at a time, and I don't
>> think it would be working today. I will keep you updated on it, and if need
>> be I can commit an earlier version, without much of the UI capabilities,
>> which validates plain text on a page on save and appends the results at the
>> bottom of the page.
>>
>> After getting my code to work I'll edit the configurable class. I can see
>> that includes and excludes would cover all use cases. I will work on editing
>> the class such that the user can choose to include or exclude.
>>
>> RedPen can take in a dictionary of words or expressions(in JSON format), but
>> it doesn't alter the entries directly. Rather, it would output a message
>> along with the other validation messages, with this one suggesting the word
>> or expression replacement.
>> However, I think it is possible to implement a replacement function. Since
>> the validation message provides the location of the error, we can extract
>> the erroreous word or expression out and replace it with the corrected one.
>> What do you think?
>
> I suggest that we focus on validation for the time being and get that to work nicely. When this is done, we can think about auto-fixing stuff. This would be doable in the listener, by changing the doc’s content before the save.
>
> The part that worries me the most ATM is how to display nicely to the user the result of the validation. I don’t see how we can do that for now. Do you have an idea? I think this could go hand in hand with a feature we’ve been wanting for a long time (can’t fine the jira right now): the ability to capture logs when rendering a page and display them as notifications in the UI.
>
> So the idea could be the following:
> * The RedPen listener would issue warnings for example in the logs
> * In the future when we implement the feature I mentioned, the user will be able to see them in the UI
>
> Do you have a better idea?
>
> I also think you need to include the concept of Validation Severity (i.e. warning, error). And when it’s a warning it should never stop saving the page. Only errors would do that.
>
> WDYT?
>
> Thanks
> -Vincent
>
>> --
>> View this message in context: http://xwiki.475771.n2.nabble.com/GSOC-Update-3-RedPen-Integration-tp7604255p7604257.html
>> Sent from the XWiki- Dev mailing list archive at Nabble.com.

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: [GSOC Update 3] RedPen Integration

deshengchuan
Hi, sorry for the late reply, was spending the time doing up the configuration handler over the rest of last week.


vmassol wrote
> The part that worries me the most ATM is how to display nicely to the user the result of the validation. I don’t see how we can do that for now. Do you have an idea? I think this could go hand in hand with a feature we’ve been wanting for a long time (can’t fine the jira right now): the ability to capture logs when rendering a page and display them as notifications in the UI.
>
> So the idea could be the following:
> * The RedPen listener would issue warnings for example in the logs
> * In the future when we implement the feature I mentioned, the user will be able to see them in the UI
I was looking around, and do you think for now, we could use a modal popup to indicate the errors and validation messages, aside from issuing warnings in the logs?


vmassol wrote
> I also think you need to include the concept of Validation Severity (i.e. warning, error). And when it’s a warning it should never stop saving the page. Only errors would do that.
As far as I know, whether the validation is a warning or an error depends on the ValidatorConfiguration object itself, for example, a SentenceLength validator gives out a warning message while a SuccessiveWord validator gives out an error.
I will work on it right away to split the error-giving validators and the warning-giving validators. For now, I'll just put all the validation messages on the logs. Is that okay?
Loading...