[Proposal] CAPTCHA module refactoring

classic Classic list List threaded Threaded
9 messages Options
Reply | Threaded
Open this post in threaded view
|

[Proposal] CAPTCHA module refactoring

Eduard Moraru
Hi, devs,

XWiki's current CAPTCHA module [1] is very old and outdated for a while now
and this is not news for anyone.

I see 2 major problems:
1) The obvious one is that we just need a technologically better CAPTCHA
implementation that the current JCaptcha-based one and JCaptcha is
discontinued.
2) The current architecture is outdated as well (i.e. wrapped around Struts
actions and initializing the VelocityContext with a custom binding) and
hard to use (i.e. you have to write your CAPTCHA UI every time you use it,
there is no rendering helper).

For 1), as both determined a few years ago [2] and also observed from
practice, the industry standard is now Google's reCaptcha service (either
its v2 or invisible version, or both), so we definitely need an
implementation that makes it easy to use in XWiki [3].

For 2), we need:
* to allow CAPTCHA implementations to provide their own rendering
** Can be done using a template .vm file (rendered with the
TemplateManager), a wiki page or directly from Java code.
** The implementation-specific parameters could be passed to
control/customize a particular rendering.
** The syntax ID may not be needed, since the only relevant output would be
html.
* to move to a ScriptService-based system and to leave the resource
(image/audio) accessing concern to the individual implementation (that may
choose to continue with Struts actions, or may choose to use something
lighter like a REST resource or even or even something more inventive, like
temporary attachments on some fixed page).
* an administration UI that configures the default CAPTCHA service and its
configuration
* to allow callers to use a different CAPTCHA than the default configured
one, as long as it is available (i.e. installed)

Examples:

= Display
$services.captcha.display() -- default implementation, current configuration
$services.captcha.recaptcha.display() -- explicitly requested
implementation, current/default configuration
$services.captcha.recaptcha.display({'theme' : 'dark', 'size' : 'compact',
'sitekey' : 'xyz'}) -- explicitly requested implementation and custom
configuration overwrites
$services.captcha.jcaptcha.display() -- other implementation,
current/default configuration

= Verification
$services.captcha.isValid() -- default implementation, current configuration
$services.captcha.recaptcha.isValid() -- explicitly requested
implementation, current/default configuration
$services.captcha.recaptcha.isValid({'secret' : 'xyz'}) -- explicitly
requested implementation and custom configuration overwrites
$services.captcha.jcaptcha.isValid() -- explicitly requested implementation
and custom configuration overwrites

Basically, the ScriptService API would be highly simplified to just
displaying and validating, while the components would consist of 2 parts:
* a generic Manager for the various implementations (listing,
getting/setting the default)
* various implementations, each responsible with rendering a form-friendly
CAPTCHA input and using the current request for extracting the information
they needs to read the user's answer and validate it.

WDYT?

Thanks,
Eduard

----------
[1] http://extensions.xwiki.org/xwiki/bin/view/Extension/Captcha%20Module
[2] http://design.xwiki.org/xwiki/bin/view/Proposal/CAPTCHAinvestigation70
[3] https://jira.xwiki.org/browse/XWIKI-12964
Reply | Threaded
Open this post in threaded view
|

Re: [Proposal] CAPTCHA module refactoring

Eduard Moraru
Question:

Do we need the captcha module to function without the DB as well?

This would make it more difficult to create the configuration UI (in
Administration) and would also mean that some captchas implementations
might not respect this constraint (i.e. choosing to implement their UI in a
wiki page instead).

IMO, it's not really worth the trouble and we should proceed to extending
the current captcha module into 2 parts: -api and -ui (the current module
would be converted from "jar" to "pom" and the current code will be
deprecated and moved to the new -api module).

Thanks,
Eduard

On Tue, May 22, 2018 at 8:35 PM, Eduard Moraru <[hidden email]> wrote:

> Hi, devs,
>
> XWiki's current CAPTCHA module [1] is very old and outdated for a while
> now and this is not news for anyone.
>
> I see 2 major problems:
> 1) The obvious one is that we just need a technologically better CAPTCHA
> implementation that the current JCaptcha-based one and JCaptcha is
> discontinued.
> 2) The current architecture is outdated as well (i.e. wrapped around
> Struts actions and initializing the VelocityContext with a custom binding)
> and hard to use (i.e. you have to write your CAPTCHA UI every time you use
> it, there is no rendering helper).
>
> For 1), as both determined a few years ago [2] and also observed from
> practice, the industry standard is now Google's reCaptcha service (either
> its v2 or invisible version, or both), so we definitely need an
> implementation that makes it easy to use in XWiki [3].
>
> For 2), we need:
> * to allow CAPTCHA implementations to provide their own rendering
> ** Can be done using a template .vm file (rendered with the
> TemplateManager), a wiki page or directly from Java code.
> ** The implementation-specific parameters could be passed to
> control/customize a particular rendering.
> ** The syntax ID may not be needed, since the only relevant output would
> be html.
> * to move to a ScriptService-based system and to leave the resource
> (image/audio) accessing concern to the individual implementation (that may
> choose to continue with Struts actions, or may choose to use something
> lighter like a REST resource or even or even something more inventive, like
> temporary attachments on some fixed page).
> * an administration UI that configures the default CAPTCHA service and its
> configuration
> * to allow callers to use a different CAPTCHA than the default configured
> one, as long as it is available (i.e. installed)
>
> Examples:
>
> = Display
> $services.captcha.display() -- default implementation, current
> configuration
> $services.captcha.recaptcha.display() -- explicitly requested
> implementation, current/default configuration
> $services.captcha.recaptcha.display({'theme' : 'dark', 'size' :
> 'compact', 'sitekey' : 'xyz'}) -- explicitly requested implementation and
> custom configuration overwrites
> $services.captcha.jcaptcha.display() -- other implementation,
> current/default configuration
>
> = Verification
> $services.captcha.isValid() -- default implementation, current
> configuration
> $services.captcha.recaptcha.isValid() -- explicitly requested
> implementation, current/default configuration
> $services.captcha.recaptcha.isValid({'secret' : 'xyz'}) -- explicitly
> requested implementation and custom configuration overwrites
> $services.captcha.jcaptcha.isValid() -- explicitly requested
> implementation and custom configuration overwrites
>
> Basically, the ScriptService API would be highly simplified to just
> displaying and validating, while the components would consist of 2 parts:
> * a generic Manager for the various implementations (listing,
> getting/setting the default)
> * various implementations, each responsible with rendering a form-friendly
> CAPTCHA input and using the current request for extracting the information
> they needs to read the user's answer and validate it.
>
> WDYT?
>
> Thanks,
> Eduard
>
> ----------
> [1] http://extensions.xwiki.org/xwiki/bin/view/Extension/Captcha%20Module
> [2] http://design.xwiki.org/xwiki/bin/view/Proposal/CAPTCHAinvestigation70
> [3] https://jira.xwiki.org/browse/XWIKI-12964
>
Reply | Threaded
Open this post in threaded view
|

Re: [Proposal] CAPTCHA module refactoring

caubin
Hi,

On 05/24/2018 05:15 PM, Eduard Moraru wrote:

> Question:
>
> Do we need the captcha module to function without the DB as well?
>
> This would make it more difficult to create the configuration UI (in
> Administration) and would also mean that some captchas implementations
> might not respect this constraint (i.e. choosing to implement their UI in a
> wiki page instead).
>
> IMO, it's not really worth the trouble and we should proceed to extending
> the current captcha module into 2 parts: -api and -ui (the current module
> would be converted from "jar" to "pom" and the current code will be
> deprecated and moved to the new -api module).

The only reason we could not to depend on the DB would be to transfer
parts of the captcha module to xwiki-commons IMO. For now, I don't see
any particular use case that could support this migration.

> Thanks,
> Eduard
>
> On Tue, May 22, 2018 at 8:35 PM, Eduard Moraru <[hidden email]> wrote:
>
>> Hi, devs,
>>
>> XWiki's current CAPTCHA module [1] is very old and outdated for a while
>> now and this is not news for anyone.
>>
>> I see 2 major problems:
>> 1) The obvious one is that we just need a technologically better CAPTCHA
>> implementation that the current JCaptcha-based one and JCaptcha is
>> discontinued.
>> 2) The current architecture is outdated as well (i.e. wrapped around
>> Struts actions and initializing the VelocityContext with a custom binding)
>> and hard to use (i.e. you have to write your CAPTCHA UI every time you use
>> it, there is no rendering helper).
>>
>> For 1), as both determined a few years ago [2] and also observed from
>> practice, the industry standard is now Google's reCaptcha service (either
>> its v2 or invisible version, or both), so we definitely need an
>> implementation that makes it easy to use in XWiki [3].
>>
>> For 2), we need:
>> * to allow CAPTCHA implementations to provide their own rendering
>> ** Can be done using a template .vm file (rendered with the
>> TemplateManager), a wiki page or directly from Java code.
>> ** The implementation-specific parameters could be passed to
>> control/customize a particular rendering.
>> ** The syntax ID may not be needed, since the only relevant output would
>> be html.
>> * to move to a ScriptService-based system and to leave the resource
>> (image/audio) accessing concern to the individual implementation (that may
>> choose to continue with Struts actions, or may choose to use something
>> lighter like a REST resource or even or even something more inventive, like
>> temporary attachments on some fixed page).
>> * an administration UI that configures the default CAPTCHA service and its
>> configuration
>> * to allow callers to use a different CAPTCHA than the default configured
>> one, as long as it is available (i.e. installed)
>>
>> Examples:
>>
>> = Display
>> $services.captcha.display() -- default implementation, current
>> configuration
>> $services.captcha.recaptcha.display() -- explicitly requested
>> implementation, current/default configuration
>> $services.captcha.recaptcha.display({'theme' : 'dark', 'size' :
>> 'compact', 'sitekey' : 'xyz'}) -- explicitly requested implementation and
>> custom configuration overwrites
>> $services.captcha.jcaptcha.display() -- other implementation,
>> current/default configuration
>>
>> = Verification
>> $services.captcha.isValid() -- default implementation, current
>> configuration
>> $services.captcha.recaptcha.isValid() -- explicitly requested
>> implementation, current/default configuration
>> $services.captcha.recaptcha.isValid({'secret' : 'xyz'}) -- explicitly
>> requested implementation and custom configuration overwrites
>> $services.captcha.jcaptcha.isValid() -- explicitly requested
>> implementation and custom configuration overwrites
>>
>> Basically, the ScriptService API would be highly simplified to just
>> displaying and validating, while the components would consist of 2 parts:
>> * a generic Manager for the various implementations (listing,
>> getting/setting the default)
>> * various implementations, each responsible with rendering a form-friendly
>> CAPTCHA input and using the current request for extracting the information
>> they needs to read the user's answer and validate it.
>>
>> WDYT?

I do agree that JCaptcha is getting quite old now, however, we might
have a hard time finding new FOSS and self-hosted captcha solutions (at
least, there's not a lot of interesting artifacts on maven central [1]).

IMO, we shouldn't rely on a captcha using an external verification
service (such as Google's captchas) as we might end up blocking users
that access their wiki with a filtered / controlled internet access. One
option could be to provide a challenge which asks the user to perform a
simple math operation, or respond to a control question instead, but I'm
not convinced of the efficiency of those solutions against real
captchas. As a captcha is usually quite simple to implement (apply
operations on a text written in an image in order to make it
non-readable by OCR software), we could also support our own version,
maybe based on the generation mechanism given by JCaptcha.

Thanks,
Clément

[1] https://search.maven.org/#search%7Cga%7C1%7Ccaptcha

>> Thanks,
>> Eduard
>>
>> ----------
>> [1] http://extensions.xwiki.org/xwiki/bin/view/Extension/Captcha%20Module
>> [2] http://design.xwiki.org/xwiki/bin/view/Proposal/CAPTCHAinvestigation70
>> [3] https://jira.xwiki.org/browse/XWIKI-12964
>>
Reply | Threaded
Open this post in threaded view
|

Re: [Proposal] CAPTCHA module refactoring

Thomas Mortagne
Administrator
I agree that the default captcha we provide should not depend on an
external service but it should just be a default. IMO the most
important is to make sure it's easy to choose a different captcha
implementation and we should most probably provide the Google one as
an option (through a contrib extension to better follow Google captcha
API evolution).

On Fri, May 25, 2018 at 12:37 AM, Clément Aubin <[hidden email]> wrote:

> Hi,
>
> On 05/24/2018 05:15 PM, Eduard Moraru wrote:
>> Question:
>>e
>> Do we need the captcha module to function without the DB as well?
>>
>> This would make it more difficult to create the configuration UI (in
>> Administration) and would also mean that some captchas implementations
>> might not respect this constraint (i.e. choosing to implement their UI in a
>> wiki page instead).
>>
>> IMO, it's not really worth the trouble and we should proceed to extending
>> the current captcha module into 2 parts: -api and -ui (the current module
>> would be converted from "jar" to "pom" and the current code will be
>> deprecated and moved to the new -api module).
>
> The only reason we could not to depend on the DB would be to transfer
> parts of the captcha module to xwiki-commons IMO. For now, I don't see
> any particular use case that could support this migration.
>
>> Thanks,
>> Eduard
>>
>> On Tue, May 22, 2018 at 8:35 PM, Eduard Moraru <[hidden email]> wrote:
>>
>>> Hi, devs,
>>>
>>> XWiki's current CAPTCHA module [1] is very old and outdated for a while
>>> now and this is not news for anyone.
>>>
>>> I see 2 major problems:
>>> 1) The obvious one is that we just need a technologically better CAPTCHA
>>> implementation that the current JCaptcha-based one and JCaptcha is
>>> discontinued.
>>> 2) The current architecture is outdated as well (i.e. wrapped around
>>> Struts actions and initializing the VelocityContext with a custom binding)
>>> and hard to use (i.e. you have to write your CAPTCHA UI every time you use
>>> it, there is no rendering helper).
>>>
>>> For 1), as both determined a few years ago [2] and also observed from
>>> practice, the industry standard is now Google's reCaptcha service (either
>>> its v2 or invisible version, or both), so we definitely need an
>>> implementation that makes it easy to use in XWiki [3].
>>>
>>> For 2), we need:
>>> * to allow CAPTCHA implementations to provide their own rendering
>>> ** Can be done using a template .vm file (rendered with the
>>> TemplateManager), a wiki page or directly from Java code.
>>> ** The implementation-specific parameters could be passed to
>>> control/customize a particular rendering.
>>> ** The syntax ID may not be needed, since the only relevant output would
>>> be html.
>>> * to move to a ScriptService-based system and to leave the resource
>>> (image/audio) accessing concern to the individual implementation (that may
>>> choose to continue with Struts actions, or may choose to use something
>>> lighter like a REST resource or even or even something more inventive, like
>>> temporary attachments on some fixed page).
>>> * an administration UI that configures the default CAPTCHA service and its
>>> configuration
>>> * to allow callers to use a different CAPTCHA than the default configured
>>> one, as long as it is available (i.e. installed)
>>>
>>> Examples:
>>>
>>> = Display
>>> $services.captcha.display() -- default implementation, current
>>> configuration
>>> $services.captcha.recaptcha.display() -- explicitly requested
>>> implementation, current/default configuration
>>> $services.captcha.recaptcha.display({'theme' : 'dark', 'size' :
>>> 'compact', 'sitekey' : 'xyz'}) -- explicitly requested implementation and
>>> custom configuration overwrites
>>> $services.captcha.jcaptcha.display() -- other implementation,
>>> current/default configuration
>>>
>>> = Verification
>>> $services.captcha.isValid() -- default implementation, current
>>> configuration
>>> $services.captcha.recaptcha.isValid() -- explicitly requested
>>> implementation, current/default configuration
>>> $services.captcha.recaptcha.isValid({'secret' : 'xyz'}) -- explicitly
>>> requested implementation and custom configuration overwrites
>>> $services.captcha.jcaptcha.isValid() -- explicitly requested
>>> implementation and custom configuration overwrites
>>>
>>> Basically, the ScriptService API would be highly simplified to just
>>> displaying and validating, while the components would consist of 2 parts:
>>> * a generic Manager for the various implementations (listing,
>>> getting/setting the default)
>>> * various implementations, each responsible with rendering a form-friendly
>>> CAPTCHA input and using the current request for extracting the information
>>> they needs to read the user's answer and validate it.
>>>
>>> WDYT?
>
> I do agree that JCaptcha is getting quite old now, however, we might
> have a hard time finding new FOSS and self-hosted captcha solutions (at
> least, there's not a lot of interesting artifacts on maven central [1]).
>
> IMO, we shouldn't rely on a captcha using an external verification
> service (such as Google's captchas) as we might end up blocking users
> that access their wiki with a filtered / controlled internet access. One
> option could be to provide a challenge which asks the user to perform a
> simple math operation, or respond to a control question instead, but I'm
> not convinced of the efficiency of those solutions against real
> captchas. As a captcha is usually quite simple to implement (apply
> operations on a text written in an image in order to make it
> non-readable by OCR software), we could also support our own version,
> maybe based on the generation mechanism given by JCaptcha.
>
> Thanks,
> Clément
>
> [1] https://search.maven.org/#search%7Cga%7C1%7Ccaptcha
>
>>> Thanks,
>>> Eduard
>>>
>>> ----------
>>> [1] http://extensions.xwiki.org/xwiki/bin/view/Extension/Captcha%20Module
>>> [2] http://design.xwiki.org/xwiki/bin/view/Proposal/CAPTCHAinvestigation70
>>> [3] https://jira.xwiki.org/browse/XWIKI-12964
>>>



--
Thomas Mortagne
Reply | Threaded
Open this post in threaded view
|

Re: [Proposal] CAPTCHA module refactoring

vmassol
Administrator
I also agree about this point that that we provide by default should not access anything external.

And I agree that other implementations (like google recaptcha) should be done as contrib extensions that can be installed in your wiki.

Thanks
-Vincent

> On 25 May 2018, at 08:51, Thomas Mortagne <[hidden email]> wrote:
>
> I agree that the default captcha we provide should not depend on an
> external service but it should just be a default. IMO the most
> important is to make sure it's easy to choose a different captcha
> implementation and we should most probably provide the Google one as
> an option (through a contrib extension to better follow Google captcha
> API evolution).
>
> On Fri, May 25, 2018 at 12:37 AM, Clément Aubin <[hidden email]> wrote:
>> Hi,
>>
>> On 05/24/2018 05:15 PM, Eduard Moraru wrote:
>>> Question:
>>> e
>>> Do we need the captcha module to function without the DB as well?
>>>
>>> This would make it more difficult to create the configuration UI (in
>>> Administration) and would also mean that some captchas implementations
>>> might not respect this constraint (i.e. choosing to implement their UI in a
>>> wiki page instead).
>>>
>>> IMO, it's not really worth the trouble and we should proceed to extending
>>> the current captcha module into 2 parts: -api and -ui (the current module
>>> would be converted from "jar" to "pom" and the current code will be
>>> deprecated and moved to the new -api module).
>>
>> The only reason we could not to depend on the DB would be to transfer
>> parts of the captcha module to xwiki-commons IMO. For now, I don't see
>> any particular use case that could support this migration.
>>
>>> Thanks,
>>> Eduard
>>>
>>> On Tue, May 22, 2018 at 8:35 PM, Eduard Moraru <[hidden email]> wrote:
>>>
>>>> Hi, devs,
>>>>
>>>> XWiki's current CAPTCHA module [1] is very old and outdated for a while
>>>> now and this is not news for anyone.
>>>>
>>>> I see 2 major problems:
>>>> 1) The obvious one is that we just need a technologically better CAPTCHA
>>>> implementation that the current JCaptcha-based one and JCaptcha is
>>>> discontinued.
>>>> 2) The current architecture is outdated as well (i.e. wrapped around
>>>> Struts actions and initializing the VelocityContext with a custom binding)
>>>> and hard to use (i.e. you have to write your CAPTCHA UI every time you use
>>>> it, there is no rendering helper).
>>>>
>>>> For 1), as both determined a few years ago [2] and also observed from
>>>> practice, the industry standard is now Google's reCaptcha service (either
>>>> its v2 or invisible version, or both), so we definitely need an
>>>> implementation that makes it easy to use in XWiki [3].
>>>>
>>>> For 2), we need:
>>>> * to allow CAPTCHA implementations to provide their own rendering
>>>> ** Can be done using a template .vm file (rendered with the
>>>> TemplateManager), a wiki page or directly from Java code.
>>>> ** The implementation-specific parameters could be passed to
>>>> control/customize a particular rendering.
>>>> ** The syntax ID may not be needed, since the only relevant output would
>>>> be html.
>>>> * to move to a ScriptService-based system and to leave the resource
>>>> (image/audio) accessing concern to the individual implementation (that may
>>>> choose to continue with Struts actions, or may choose to use something
>>>> lighter like a REST resource or even or even something more inventive, like
>>>> temporary attachments on some fixed page).
>>>> * an administration UI that configures the default CAPTCHA service and its
>>>> configuration
>>>> * to allow callers to use a different CAPTCHA than the default configured
>>>> one, as long as it is available (i.e. installed)
>>>>
>>>> Examples:
>>>>
>>>> = Display
>>>> $services.captcha.display() -- default implementation, current
>>>> configuration
>>>> $services.captcha.recaptcha.display() -- explicitly requested
>>>> implementation, current/default configuration
>>>> $services.captcha.recaptcha.display({'theme' : 'dark', 'size' :
>>>> 'compact', 'sitekey' : 'xyz'}) -- explicitly requested implementation and
>>>> custom configuration overwrites
>>>> $services.captcha.jcaptcha.display() -- other implementation,
>>>> current/default configuration
>>>>
>>>> = Verification
>>>> $services.captcha.isValid() -- default implementation, current
>>>> configuration
>>>> $services.captcha.recaptcha.isValid() -- explicitly requested
>>>> implementation, current/default configuration
>>>> $services.captcha.recaptcha.isValid({'secret' : 'xyz'}) -- explicitly
>>>> requested implementation and custom configuration overwrites
>>>> $services.captcha.jcaptcha.isValid() -- explicitly requested
>>>> implementation and custom configuration overwrites
>>>>
>>>> Basically, the ScriptService API would be highly simplified to just
>>>> displaying and validating, while the components would consist of 2 parts:
>>>> * a generic Manager for the various implementations (listing,
>>>> getting/setting the default)
>>>> * various implementations, each responsible with rendering a form-friendly
>>>> CAPTCHA input and using the current request for extracting the information
>>>> they needs to read the user's answer and validate it.
>>>>
>>>> WDYT?
>>
>> I do agree that JCaptcha is getting quite old now, however, we might
>> have a hard time finding new FOSS and self-hosted captcha solutions (at
>> least, there's not a lot of interesting artifacts on maven central [1]).
>>
>> IMO, we shouldn't rely on a captcha using an external verification
>> service (such as Google's captchas) as we might end up blocking users
>> that access their wiki with a filtered / controlled internet access. One
>> option could be to provide a challenge which asks the user to perform a
>> simple math operation, or respond to a control question instead, but I'm
>> not convinced of the efficiency of those solutions against real
>> captchas. As a captcha is usually quite simple to implement (apply
>> operations on a text written in an image in order to make it
>> non-readable by OCR software), we could also support our own version,
>> maybe based on the generation mechanism given by JCaptcha.
>>
>> Thanks,
>> Clément
>>
>> [1] https://search.maven.org/#search%7Cga%7C1%7Ccaptcha
>>
>>>> Thanks,
>>>> Eduard
>>>>
>>>> ----------
>>>> [1] http://extensions.xwiki.org/xwiki/bin/view/Extension/Captcha%20Module
>>>> [2] http://design.xwiki.org/xwiki/bin/view/Proposal/CAPTCHAinvestigation70
>>>> [3] https://jira.xwiki.org/browse/XWIKI-12964
>>>>
>
>
>
> --
> Thomas Mortagne

Reply | Threaded
Open this post in threaded view
|

Re: [Proposal] CAPTCHA module refactoring

Eduard Moraru
Hi,

On Fri, May 25, 2018 at 10:32 AM, Vincent Massol <[hidden email]> wrote:

> I also agree about this point that that we provide by default should not
> access anything external.
>
> And I agree that other implementations (like google recaptcha) should be
> done as contrib extensions that can be installed in your wiki.
>

I wouldn't directly say that I disagree with this statement, however I am
going to reiterate the conclusions we (at least I) had back when I did the
initial XWiki 7.0 CAPTCHA investigation regarding self hosted vs service
solution:

The conclusion was that the need for a CAPTCHA system arises in a public
environment/website. There is no plausible scenario where a
company/organization would want to setup a CAPTCHA on a private intranet or
internal system because it would be poitless to try to protect your system
from, e.g., bots registering fake accounts from inside your company's VPN
(or something like that). So, if the usecase is public endpoints/systems
that need to be protected by CAPTCHAs, then using a public service (such as
Google's reCAPTCHA) is perfectly plausible and not creating any technical
problems, since the client has access to download the necessary javascript
and the server (most likely) has external access to validate the response.

The above conclusion is also most likely the reason why self-hosted CAPTCHA
solutions have pretty much died out along the years, mostly due to Google's
reCAPTCHA that is enjoying a huge success, both in adoption and in
efficiency. I even found an asnwer from actual self-hosted CAPTCHA authors
recommending to use Google's service solution instead, even mentioning the
fact that it was helping (at some point) improve OCR on digializing text
books (now Google seems to have moved more to image recognition instead of
OCR, but still...).

https://stackoverflow.com/questions/810493/recommendations-for-java-captcha-libraries
(check https://stackoverflow.com/a/4528230/5198689 for a reply from
SimpleCaptcha's author)

SimpleCaptcha does seem to pop up in many discussions as the best Java
alternative so far. We might want to try and offer a contrib extension with
it as well.

Clement's suggestion of custom CAPTCHA solutions is not bad either. They
should be much easier to implement and switch to, with the new system.
Maybe it would be more appropriate at a per-depoyment level, instead of at
a product level, though.

If you ask me, it's Google's reCAPTCHA that we should use as a default
(since it's now the industry standard), while still bundling (at least for
a while) the old JCaptcha implementation (for backwards compatibility,
until users switch to the new system) and the new JCaptcha implementation
(with the new system). IMO, it would be nicer to offer the best solution as
a default and have the few cases that need the self-hosted solution switch
to JCaptcha or install some contrib extension based on SimpleCaptcha, or
whatever else is available.

Now, as Thomas mentioned, it would make more sense to implement it as a
Contrib extension (so that, in the near future, changes to reCAPTCHA could
be still used by older vesions; older as in stil using the new system, i.e.
10.5+, but not the latest XWiki version), so the only solution to have it
as default as well would mean for the Standard Flavor to bundle the
reCAPTCHA implementaion, as a Contrib extension. (Vincent, insert "-1" here
:) )

---

Side Note: Taking an approach, in terms of Admin UI, that is similar to
what we did for Search (transition and choice from Solr to Lucene), with
configuration options specific to each implementation.

---

Now, CAPTCHA is not enabled by default in XWiki, so admins still need to do
this. Once they enable it, if reCAPTCHA is the default, it won't work until
it's configured (with a Google API key, even for localhost). An interesting
thing we could do to avoid this problem and have the CAPTCHA not fail
completely, is to use Google's test keys (originally intended to not break
automated testing) by default which will make the CAPTCHA always validate,
while still being explicit that the configuration needs to be done. This
would make sure there is no friction in the process and we'll probably do
this by default for this implementation, regardless if we choose to use it
as XWiki's default or if it needs to be manually installed (from contrib).

https://developers.google.com/recaptcha/docs/faq#id-like-to-run-automated-tests-with-recaptcha-v2-what-should-i-do

Thanks,
Eduard


>
> Thanks
> -Vincent
>
> > On 25 May 2018, at 08:51, Thomas Mortagne <[hidden email]>
> wrote:
> >
> > I agree that the default captcha we provide should not depend on an
> > external service but it should just be a default. IMO the most
> > important is to make sure it's easy to choose a different captcha
> > implementation and we should most probably provide the Google one as
> > an option (through a contrib extension to better follow Google captcha
> > API evolution).
> >
> > On Fri, May 25, 2018 at 12:37 AM, Clément Aubin <[hidden email]>
> wrote:
> >> Hi,
> >>
> >> On 05/24/2018 05:15 PM, Eduard Moraru wrote:
> >>> Question:
> >>> e
> >>> Do we need the captcha module to function without the DB as well?
> >>>
> >>> This would make it more difficult to create the configuration UI (in
> >>> Administration) and would also mean that some captchas implementations
> >>> might not respect this constraint (i.e. choosing to implement their UI
> in a
> >>> wiki page instead).
> >>>
> >>> IMO, it's not really worth the trouble and we should proceed to
> extending
> >>> the current captcha module into 2 parts: -api and -ui (the current
> module
> >>> would be converted from "jar" to "pom" and the current code will be
> >>> deprecated and moved to the new -api module).
> >>
> >> The only reason we could not to depend on the DB would be to transfer
> >> parts of the captcha module to xwiki-commons IMO. For now, I don't see
> >> any particular use case that could support this migration.
> >>
> >>> Thanks,
> >>> Eduard
> >>>
> >>> On Tue, May 22, 2018 at 8:35 PM, Eduard Moraru <[hidden email]>
> wrote:
> >>>
> >>>> Hi, devs,
> >>>>
> >>>> XWiki's current CAPTCHA module [1] is very old and outdated for a
> while
> >>>> now and this is not news for anyone.
> >>>>
> >>>> I see 2 major problems:
> >>>> 1) The obvious one is that we just need a technologically better
> CAPTCHA
> >>>> implementation that the current JCaptcha-based one and JCaptcha is
> >>>> discontinued.
> >>>> 2) The current architecture is outdated as well (i.e. wrapped around
> >>>> Struts actions and initializing the VelocityContext with a custom
> binding)
> >>>> and hard to use (i.e. you have to write your CAPTCHA UI every time
> you use
> >>>> it, there is no rendering helper).
> >>>>
> >>>> For 1), as both determined a few years ago [2] and also observed from
> >>>> practice, the industry standard is now Google's reCaptcha service
> (either
> >>>> its v2 or invisible version, or both), so we definitely need an
> >>>> implementation that makes it easy to use in XWiki [3].
> >>>>
> >>>> For 2), we need:
> >>>> * to allow CAPTCHA implementations to provide their own rendering
> >>>> ** Can be done using a template .vm file (rendered with the
> >>>> TemplateManager), a wiki page or directly from Java code.
> >>>> ** The implementation-specific parameters could be passed to
> >>>> control/customize a particular rendering.
> >>>> ** The syntax ID may not be needed, since the only relevant output
> would
> >>>> be html.
> >>>> * to move to a ScriptService-based system and to leave the resource
> >>>> (image/audio) accessing concern to the individual implementation
> (that may
> >>>> choose to continue with Struts actions, or may choose to use something
> >>>> lighter like a REST resource or even or even something more
> inventive, like
> >>>> temporary attachments on some fixed page).
> >>>> * an administration UI that configures the default CAPTCHA service
> and its
> >>>> configuration
> >>>> * to allow callers to use a different CAPTCHA than the default
> configured
> >>>> one, as long as it is available (i.e. installed)
> >>>>
> >>>> Examples:
> >>>>
> >>>> = Display
> >>>> $services.captcha.display() -- default implementation, current
> >>>> configuration
> >>>> $services.captcha.recaptcha.display() -- explicitly requested
> >>>> implementation, current/default configuration
> >>>> $services.captcha.recaptcha.display({'theme' : 'dark', 'size' :
> >>>> 'compact', 'sitekey' : 'xyz'}) -- explicitly requested implementation
> and
> >>>> custom configuration overwrites
> >>>> $services.captcha.jcaptcha.display() -- other implementation,
> >>>> current/default configuration
> >>>>
> >>>> = Verification
> >>>> $services.captcha.isValid() -- default implementation, current
> >>>> configuration
> >>>> $services.captcha.recaptcha.isValid() -- explicitly requested
> >>>> implementation, current/default configuration
> >>>> $services.captcha.recaptcha.isValid({'secret' : 'xyz'}) -- explicitly
> >>>> requested implementation and custom configuration overwrites
> >>>> $services.captcha.jcaptcha.isValid() -- explicitly requested
> >>>> implementation and custom configuration overwrites
> >>>>
> >>>> Basically, the ScriptService API would be highly simplified to just
> >>>> displaying and validating, while the components would consist of 2
> parts:
> >>>> * a generic Manager for the various implementations (listing,
> >>>> getting/setting the default)
> >>>> * various implementations, each responsible with rendering a
> form-friendly
> >>>> CAPTCHA input and using the current request for extracting the
> information
> >>>> they needs to read the user's answer and validate it.
> >>>>
> >>>> WDYT?
> >>
> >> I do agree that JCaptcha is getting quite old now, however, we might
> >> have a hard time finding new FOSS and self-hosted captcha solutions (at
> >> least, there's not a lot of interesting artifacts on maven central [1]).
> >>
> >> IMO, we shouldn't rely on a captcha using an external verification
> >> service (such as Google's captchas) as we might end up blocking users
> >> that access their wiki with a filtered / controlled internet access. One
> >> option could be to provide a challenge which asks the user to perform a
> >> simple math operation, or respond to a control question instead, but I'm
> >> not convinced of the efficiency of those solutions against real
> >> captchas. As a captcha is usually quite simple to implement (apply
> >> operations on a text written in an image in order to make it
> >> non-readable by OCR software), we could also support our own version,
> >> maybe based on the generation mechanism given by JCaptcha.
> >>
> >> Thanks,
> >> Clément
> >>
> >> [1] https://search.maven.org/#search%7Cga%7C1%7Ccaptcha
> >>
> >>>> Thanks,
> >>>> Eduard
> >>>>
> >>>> ----------
> >>>> [1] http://extensions.xwiki.org/xwiki/bin/view/Extension/
> Captcha%20Module
> >>>> [2] http://design.xwiki.org/xwiki/bin/view/Proposal/
> CAPTCHAinvestigation70
> >>>> [3] https://jira.xwiki.org/browse/XWIKI-12964
> >>>>
> >
> >
> >
> > --
> > Thomas Mortagne
>
>
Reply | Threaded
Open this post in threaded view
|

Re: [Proposal] CAPTCHA module refactoring

vmassol
Administrator
Hi Edy,

> On 25 May 2018, at 14:02, Eduard Moraru <[hidden email]> wrote:
>
> Hi,
>
> On Fri, May 25, 2018 at 10:32 AM, Vincent Massol <[hidden email]> wrote:
>
>> I also agree about this point that that we provide by default should not
>> access anything external.
>>
>> And I agree that other implementations (like google recaptcha) should be
>> done as contrib extensions that can be installed in your wiki.
>>
>
> I wouldn't directly say that I disagree with this statement, however I am
> going to reiterate the conclusions we (at least I) had back when I did the
> initial XWiki 7.0 CAPTCHA investigation regarding self hosted vs service
> solution:
>
> The conclusion was that the need for a CAPTCHA system arises in a public
> environment/website. There is no plausible scenario where a
> company/organization would want to setup a CAPTCHA on a private intranet or
> internal system because it would be poitless to try to protect your system
> from, e.g., bots registering fake accounts from inside your company's VPN
> (or something like that). So, if the usecase is public endpoints/systems
> that need to be protected by CAPTCHAs, then using a public service (such as
> Google's reCAPTCHA) is perfectly plausible and not creating any technical
> problems, since the client has access to download the necessary javascript
> and the server (most likely) has external access to validate the response.

That’s a good point.

> The above conclusion is also most likely the reason why self-hosted CAPTCHA
> solutions have pretty much died out along the years, mostly due to Google's
> reCAPTCHA that is enjoying a huge success, both in adoption and in
> efficiency.

Yes, unless you don’t want to use google services for various reasons.

> I even found an asnwer from actual self-hosted CAPTCHA authors
> recommending to use Google's service solution instead, even mentioning the
> fact that it was helping (at some point) improve OCR on digializing text
> books (now Google seems to have moved more to image recognition instead of
> OCR, but still...).
>
> https://stackoverflow.com/questions/810493/recommendations-for-java-captcha-libraries
> (check https://stackoverflow.com/a/4528230/5198689 for a reply from
> SimpleCaptcha's author)
>
> SimpleCaptcha does seem to pop up in many discussions as the best Java
> alternative so far. We might want to try and offer a contrib extension with
> it as well.
>
> Clement's suggestion of custom CAPTCHA solutions is not bad either. They
> should be much easier to implement and switch to, with the new system.
> Maybe it would be more appropriate at a per-depoyment level, instead of at
> a product level, though.
>
> If you ask me, it's Google's reCAPTCHA that we should use as a default
> (since it's now the industry standard), while still bundling (at least for
> a while) the old JCaptcha implementation (for backwards compatibility,
> until users switch to the new system) and the new JCaptcha implementation
> (with the new system). IMO, it would be nicer to offer the best solution as
> a default and have the few cases that need the self-hosted solution switch
> to JCaptcha or install some contrib extension based on SimpleCaptcha, or
> whatever else is available.
>
> Now, as Thomas mentioned, it would make more sense to implement it as a
> Contrib extension (so that, in the near future, changes to reCAPTCHA could
> be still used by older vesions; older as in stil using the new system, i.e.
> 10.5+, but not the latest XWiki version), so the only solution to have it
> as default as well would mean for the Standard Flavor to bundle the
> reCAPTCHA implementaion, as a Contrib extension. (Vincent, insert "-1" here
> :) )

So if we agree that captcha are only required in a public environment which has internet access then I think it would make sense to not have any Captcha implementation by default in XS and make it fully a contrib extension for those use cases (e.g. public web site use case).

And yes I wouldn’t like to bundle another contrib extension in XS before we conclude on this topic (we started a discussion but didn’t finish it AFAIK).

> ---
>
> Side Note: Taking an approach, in terms of Admin UI, that is similar to
> what we did for Search (transition and choice from Solr to Lucene), with
> configuration options specific to each implementation.

One option is to have the captcha API could be in XS and no implementation provided by default and users would need to install an extension to get an implementation.

The other option is to introduce UIXP instead and fully move the code to the extension(s).

Thanks
-Vincent

> ---
>
> Now, CAPTCHA is not enabled by default in XWiki, so admins still need to do
> this. Once they enable it, if reCAPTCHA is the default, it won't work until
> it's configured (with a Google API key, even for localhost). An interesting
> thing we could do to avoid this problem and have the CAPTCHA not fail
> completely, is to use Google's test keys (originally intended to not break
> automated testing) by default which will make the CAPTCHA always validate,
> while still being explicit that the configuration needs to be done. This
> would make sure there is no friction in the process and we'll probably do
> this by default for this implementation, regardless if we choose to use it
> as XWiki's default or if it needs to be manually installed (from contrib).
>
> https://developers.google.com/recaptcha/docs/faq#id-like-to-run-automated-tests-with-recaptcha-v2-what-should-i-do
>
> Thanks,
> Eduard
>
>
>>
>> Thanks
>> -Vincent
>>
>>> On 25 May 2018, at 08:51, Thomas Mortagne <[hidden email]>
>> wrote:
>>>
>>> I agree that the default captcha we provide should not depend on an
>>> external service but it should just be a default. IMO the most
>>> important is to make sure it's easy to choose a different captcha
>>> implementation and we should most probably provide the Google one as
>>> an option (through a contrib extension to better follow Google captcha
>>> API evolution).
>>>
>>> On Fri, May 25, 2018 at 12:37 AM, Clément Aubin <[hidden email]>
>> wrote:
>>>> Hi,
>>>>
>>>> On 05/24/2018 05:15 PM, Eduard Moraru wrote:
>>>>> Question:
>>>>> e
>>>>> Do we need the captcha module to function without the DB as well?
>>>>>
>>>>> This would make it more difficult to create the configuration UI (in
>>>>> Administration) and would also mean that some captchas implementations
>>>>> might not respect this constraint (i.e. choosing to implement their UI
>> in a
>>>>> wiki page instead).
>>>>>
>>>>> IMO, it's not really worth the trouble and we should proceed to
>> extending
>>>>> the current captcha module into 2 parts: -api and -ui (the current
>> module
>>>>> would be converted from "jar" to "pom" and the current code will be
>>>>> deprecated and moved to the new -api module).
>>>>
>>>> The only reason we could not to depend on the DB would be to transfer
>>>> parts of the captcha module to xwiki-commons IMO. For now, I don't see
>>>> any particular use case that could support this migration.
>>>>
>>>>> Thanks,
>>>>> Eduard
>>>>>
>>>>> On Tue, May 22, 2018 at 8:35 PM, Eduard Moraru <[hidden email]>
>> wrote:
>>>>>
>>>>>> Hi, devs,
>>>>>>
>>>>>> XWiki's current CAPTCHA module [1] is very old and outdated for a
>> while
>>>>>> now and this is not news for anyone.
>>>>>>
>>>>>> I see 2 major problems:
>>>>>> 1) The obvious one is that we just need a technologically better
>> CAPTCHA
>>>>>> implementation that the current JCaptcha-based one and JCaptcha is
>>>>>> discontinued.
>>>>>> 2) The current architecture is outdated as well (i.e. wrapped around
>>>>>> Struts actions and initializing the VelocityContext with a custom
>> binding)
>>>>>> and hard to use (i.e. you have to write your CAPTCHA UI every time
>> you use
>>>>>> it, there is no rendering helper).
>>>>>>
>>>>>> For 1), as both determined a few years ago [2] and also observed from
>>>>>> practice, the industry standard is now Google's reCaptcha service
>> (either
>>>>>> its v2 or invisible version, or both), so we definitely need an
>>>>>> implementation that makes it easy to use in XWiki [3].
>>>>>>
>>>>>> For 2), we need:
>>>>>> * to allow CAPTCHA implementations to provide their own rendering
>>>>>> ** Can be done using a template .vm file (rendered with the
>>>>>> TemplateManager), a wiki page or directly from Java code.
>>>>>> ** The implementation-specific parameters could be passed to
>>>>>> control/customize a particular rendering.
>>>>>> ** The syntax ID may not be needed, since the only relevant output
>> would
>>>>>> be html.
>>>>>> * to move to a ScriptService-based system and to leave the resource
>>>>>> (image/audio) accessing concern to the individual implementation
>> (that may
>>>>>> choose to continue with Struts actions, or may choose to use something
>>>>>> lighter like a REST resource or even or even something more
>> inventive, like
>>>>>> temporary attachments on some fixed page).
>>>>>> * an administration UI that configures the default CAPTCHA service
>> and its
>>>>>> configuration
>>>>>> * to allow callers to use a different CAPTCHA than the default
>> configured
>>>>>> one, as long as it is available (i.e. installed)
>>>>>>
>>>>>> Examples:
>>>>>>
>>>>>> = Display
>>>>>> $services.captcha.display() -- default implementation, current
>>>>>> configuration
>>>>>> $services.captcha.recaptcha.display() -- explicitly requested
>>>>>> implementation, current/default configuration
>>>>>> $services.captcha.recaptcha.display({'theme' : 'dark', 'size' :
>>>>>> 'compact', 'sitekey' : 'xyz'}) -- explicitly requested implementation
>> and
>>>>>> custom configuration overwrites
>>>>>> $services.captcha.jcaptcha.display() -- other implementation,
>>>>>> current/default configuration
>>>>>>
>>>>>> = Verification
>>>>>> $services.captcha.isValid() -- default implementation, current
>>>>>> configuration
>>>>>> $services.captcha.recaptcha.isValid() -- explicitly requested
>>>>>> implementation, current/default configuration
>>>>>> $services.captcha.recaptcha.isValid({'secret' : 'xyz'}) -- explicitly
>>>>>> requested implementation and custom configuration overwrites
>>>>>> $services.captcha.jcaptcha.isValid() -- explicitly requested
>>>>>> implementation and custom configuration overwrites
>>>>>>
>>>>>> Basically, the ScriptService API would be highly simplified to just
>>>>>> displaying and validating, while the components would consist of 2
>> parts:
>>>>>> * a generic Manager for the various implementations (listing,
>>>>>> getting/setting the default)
>>>>>> * various implementations, each responsible with rendering a
>> form-friendly
>>>>>> CAPTCHA input and using the current request for extracting the
>> information
>>>>>> they needs to read the user's answer and validate it.
>>>>>>
>>>>>> WDYT?
>>>>
>>>> I do agree that JCaptcha is getting quite old now, however, we might
>>>> have a hard time finding new FOSS and self-hosted captcha solutions (at
>>>> least, there's not a lot of interesting artifacts on maven central [1]).
>>>>
>>>> IMO, we shouldn't rely on a captcha using an external verification
>>>> service (such as Google's captchas) as we might end up blocking users
>>>> that access their wiki with a filtered / controlled internet access. One
>>>> option could be to provide a challenge which asks the user to perform a
>>>> simple math operation, or respond to a control question instead, but I'm
>>>> not convinced of the efficiency of those solutions against real
>>>> captchas. As a captcha is usually quite simple to implement (apply
>>>> operations on a text written in an image in order to make it
>>>> non-readable by OCR software), we could also support our own version,
>>>> maybe based on the generation mechanism given by JCaptcha.
>>>>
>>>> Thanks,
>>>> Clément
>>>>
>>>> [1] https://search.maven.org/#search%7Cga%7C1%7Ccaptcha
>>>>
>>>>>> Thanks,
>>>>>> Eduard
>>>>>>
>>>>>> ----------
>>>>>> [1] http://extensions.xwiki.org/xwiki/bin/view/Extension/
>> Captcha%20Module
>>>>>> [2] http://design.xwiki.org/xwiki/bin/view/Proposal/
>> CAPTCHAinvestigation70
>>>>>> [3] https://jira.xwiki.org/browse/XWIKI-12964
>>>>>>
>>>
>>>
>>>
>>> --
>>> Thomas Mortagne
>>
>>

Reply | Threaded
Open this post in threaded view
|

Re: [Proposal] CAPTCHA module refactoring

Eduard Moraru
On Fri, May 25, 2018 at 4:55 PM, Vincent Massol <[hidden email]> wrote:

> Hi Edy,
>
> > On 25 May 2018, at 14:02, Eduard Moraru <[hidden email]> wrote:
> >
> > Hi,
> >
> > On Fri, May 25, 2018 at 10:32 AM, Vincent Massol <[hidden email]>
> wrote:
> >
> >> I also agree about this point that that we provide by default should not
> >> access anything external.
> >>
> >> And I agree that other implementations (like google recaptcha) should be
> >> done as contrib extensions that can be installed in your wiki.
> >>
> >
> > I wouldn't directly say that I disagree with this statement, however I am
> > going to reiterate the conclusions we (at least I) had back when I did
> the
> > initial XWiki 7.0 CAPTCHA investigation regarding self hosted vs service
> > solution:
> >
> > The conclusion was that the need for a CAPTCHA system arises in a public
> > environment/website. There is no plausible scenario where a
> > company/organization would want to setup a CAPTCHA on a private intranet
> or
> > internal system because it would be poitless to try to protect your
> system
> > from, e.g., bots registering fake accounts from inside your company's VPN
> > (or something like that). So, if the usecase is public endpoints/systems
> > that need to be protected by CAPTCHAs, then using a public service (such
> as
> > Google's reCAPTCHA) is perfectly plausible and not creating any technical
> > problems, since the client has access to download the necessary
> javascript
> > and the server (most likely) has external access to validate the
> response.
>
> That’s a good point.
>
> > The above conclusion is also most likely the reason why self-hosted
> CAPTCHA
> > solutions have pretty much died out along the years, mostly due to
> Google's
> > reCAPTCHA that is enjoying a huge success, both in adoption and in
> > efficiency.
>
> Yes, unless you don’t want to use google services for various reasons.
>
> > I even found an asnwer from actual self-hosted CAPTCHA authors
> > recommending to use Google's service solution instead, even mentioning
> the
> > fact that it was helping (at some point) improve OCR on digializing text
> > books (now Google seems to have moved more to image recognition instead
> of
> > OCR, but still...).
> >
> > https://stackoverflow.com/questions/810493/recommendations-for-java-
> captcha-libraries
> > (check https://stackoverflow.com/a/4528230/5198689 for a reply from
> > SimpleCaptcha's author)
> >
> > SimpleCaptcha does seem to pop up in many discussions as the best Java
> > alternative so far. We might want to try and offer a contrib extension
> with
> > it as well.
> >
> > Clement's suggestion of custom CAPTCHA solutions is not bad either. They
> > should be much easier to implement and switch to, with the new system.
> > Maybe it would be more appropriate at a per-depoyment level, instead of
> at
> > a product level, though.
> >
> > If you ask me, it's Google's reCAPTCHA that we should use as a default
> > (since it's now the industry standard), while still bundling (at least
> for
> > a while) the old JCaptcha implementation (for backwards compatibility,
> > until users switch to the new system) and the new JCaptcha implementation
> > (with the new system). IMO, it would be nicer to offer the best solution
> as
> > a default and have the few cases that need the self-hosted solution
> switch
> > to JCaptcha or install some contrib extension based on SimpleCaptcha, or
> > whatever else is available.
> >
> > Now, as Thomas mentioned, it would make more sense to implement it as a
> > Contrib extension (so that, in the near future, changes to reCAPTCHA
> could
> > be still used by older vesions; older as in stil using the new system,
> i.e.
> > 10.5+, but not the latest XWiki version), so the only solution to have it
> > as default as well would mean for the Standard Flavor to bundle the
> > reCAPTCHA implementaion, as a Contrib extension. (Vincent, insert "-1"
> here
> > :) )
>
> So if we agree that captcha are only required in a public environment
> which has internet access then I think it would make sense to not have any
> Captcha implementation by default in XS and make it fully a contrib
> extension for those use cases (e.g. public web site use case).
>

Remember that Registration and Comments both allow enabling Captchas
individually, so it's a feature that XS still needs, at least at the
API/configuration level.


>
> And yes I wouldn’t like to bundle another contrib extension in XS before
> we conclude on this topic (we started a discussion but didn’t finish it
> AFAIK).
>
> > ---
> >
> > Side Note: Taking an approach, in terms of Admin UI, that is similar to
> > what we did for Search (transition and choice from Solr to Lucene), with
> > configuration options specific to each implementation.
>
> One option is to have the captcha API could be in XS and no implementation
> provided by default and users would need to install an extension to get an
> implementation.
>

Sure, however the problem with the current Captcha Module (
http://extensions.xwiki.org/xwiki/bin/view/Extension/Captcha%20Module#HGettingthecaptchaimage
) is that it is implemented with a Struts Action (bound in
struts-config.xml for "/imagecaptcha/") that anyone asking for a CAPTCHA
should use to generate the CAPTCHA image and use it in an <img
src=".../imagecaptcha/"/> tag. That makes it impossible to extract it as an
extension.

So we have a backwards compatibility problem in the fact that we need to
keep supporting:
* Users of the "/imagecaptcha/" action
* Users of custom CAPTCHA systems, build on top of the old CaptchaVerifier
system (
http://extensions.xwiki.org/xwiki/bin/view/Extension/Captcha%20Module#HHowaboutimplementingyourowncaptchatype3F
)
...and give them time to transition.

Because of this, I plan to move everything found in the current
xwiki-platform-captcha module to xwiki-platform-captcha-jcaptcha-api,
together with the integration with the new system, and continue bundling it
in XS. This way, both implementations should still work.

So we end up with the following layout:

xwiki-platform-captcha
+ xwiki-platform-captcha-api
+ xwiki-platform-captcha-ui
+ xwiki-platform-captcha-jcaptcha
++ xwiki-platform-captcha-jcaptcha-api
++ xwiki-platform-captcha-jcaptcha-ui

captcha-recaptcha (contrib, depends on xwiki-platform-captcha)
+ captcha-recaptcha-api
+ captcha-recaptcha-ui

Thanks,
Eduard


>
> The other option is to introduce UIXP instead and fully move the code to
> the extension(s).
>
> Thanks
> -Vincent
>
> > ---
> >
> > Now, CAPTCHA is not enabled by default in XWiki, so admins still need to
> do
> > this. Once they enable it, if reCAPTCHA is the default, it won't work
> until
> > it's configured (with a Google API key, even for localhost). An
> interesting
> > thing we could do to avoid this problem and have the CAPTCHA not fail
> > completely, is to use Google's test keys (originally intended to not
> break
> > automated testing) by default which will make the CAPTCHA always
> validate,
> > while still being explicit that the configuration needs to be done. This
> > would make sure there is no friction in the process and we'll probably do
> > this by default for this implementation, regardless if we choose to use
> it
> > as XWiki's default or if it needs to be manually installed (from
> contrib).
> >
> > https://developers.google.com/recaptcha/docs/faq#id-like-to-
> run-automated-tests-with-recaptcha-v2-what-should-i-do
> >
> > Thanks,
> > Eduard
> >
> >
> >>
> >> Thanks
> >> -Vincent
> >>
> >>> On 25 May 2018, at 08:51, Thomas Mortagne <[hidden email]>
> >> wrote:
> >>>
> >>> I agree that the default captcha we provide should not depend on an
> >>> external service but it should just be a default. IMO the most
> >>> important is to make sure it's easy to choose a different captcha
> >>> implementation and we should most probably provide the Google one as
> >>> an option (through a contrib extension to better follow Google captcha
> >>> API evolution).
> >>>
> >>> On Fri, May 25, 2018 at 12:37 AM, Clément Aubin <[hidden email]>
> >> wrote:
> >>>> Hi,
> >>>>
> >>>> On 05/24/2018 05:15 PM, Eduard Moraru wrote:
> >>>>> Question:
> >>>>> e
> >>>>> Do we need the captcha module to function without the DB as well?
> >>>>>
> >>>>> This would make it more difficult to create the configuration UI (in
> >>>>> Administration) and would also mean that some captchas
> implementations
> >>>>> might not respect this constraint (i.e. choosing to implement their
> UI
> >> in a
> >>>>> wiki page instead).
> >>>>>
> >>>>> IMO, it's not really worth the trouble and we should proceed to
> >> extending
> >>>>> the current captcha module into 2 parts: -api and -ui (the current
> >> module
> >>>>> would be converted from "jar" to "pom" and the current code will be
> >>>>> deprecated and moved to the new -api module).
> >>>>
> >>>> The only reason we could not to depend on the DB would be to transfer
> >>>> parts of the captcha module to xwiki-commons IMO. For now, I don't see
> >>>> any particular use case that could support this migration.
> >>>>
> >>>>> Thanks,
> >>>>> Eduard
> >>>>>
> >>>>> On Tue, May 22, 2018 at 8:35 PM, Eduard Moraru <[hidden email]
> >
> >> wrote:
> >>>>>
> >>>>>> Hi, devs,
> >>>>>>
> >>>>>> XWiki's current CAPTCHA module [1] is very old and outdated for a
> >> while
> >>>>>> now and this is not news for anyone.
> >>>>>>
> >>>>>> I see 2 major problems:
> >>>>>> 1) The obvious one is that we just need a technologically better
> >> CAPTCHA
> >>>>>> implementation that the current JCaptcha-based one and JCaptcha is
> >>>>>> discontinued.
> >>>>>> 2) The current architecture is outdated as well (i.e. wrapped around
> >>>>>> Struts actions and initializing the VelocityContext with a custom
> >> binding)
> >>>>>> and hard to use (i.e. you have to write your CAPTCHA UI every time
> >> you use
> >>>>>> it, there is no rendering helper).
> >>>>>>
> >>>>>> For 1), as both determined a few years ago [2] and also observed
> from
> >>>>>> practice, the industry standard is now Google's reCaptcha service
> >> (either
> >>>>>> its v2 or invisible version, or both), so we definitely need an
> >>>>>> implementation that makes it easy to use in XWiki [3].
> >>>>>>
> >>>>>> For 2), we need:
> >>>>>> * to allow CAPTCHA implementations to provide their own rendering
> >>>>>> ** Can be done using a template .vm file (rendered with the
> >>>>>> TemplateManager), a wiki page or directly from Java code.
> >>>>>> ** The implementation-specific parameters could be passed to
> >>>>>> control/customize a particular rendering.
> >>>>>> ** The syntax ID may not be needed, since the only relevant output
> >> would
> >>>>>> be html.
> >>>>>> * to move to a ScriptService-based system and to leave the resource
> >>>>>> (image/audio) accessing concern to the individual implementation
> >> (that may
> >>>>>> choose to continue with Struts actions, or may choose to use
> something
> >>>>>> lighter like a REST resource or even or even something more
> >> inventive, like
> >>>>>> temporary attachments on some fixed page).
> >>>>>> * an administration UI that configures the default CAPTCHA service
> >> and its
> >>>>>> configuration
> >>>>>> * to allow callers to use a different CAPTCHA than the default
> >> configured
> >>>>>> one, as long as it is available (i.e. installed)
> >>>>>>
> >>>>>> Examples:
> >>>>>>
> >>>>>> = Display
> >>>>>> $services.captcha.display() -- default implementation, current
> >>>>>> configuration
> >>>>>> $services.captcha.recaptcha.display() -- explicitly requested
> >>>>>> implementation, current/default configuration
> >>>>>> $services.captcha.recaptcha.display({'theme' : 'dark', 'size' :
> >>>>>> 'compact', 'sitekey' : 'xyz'}) -- explicitly requested
> implementation
> >> and
> >>>>>> custom configuration overwrites
> >>>>>> $services.captcha.jcaptcha.display() -- other implementation,
> >>>>>> current/default configuration
> >>>>>>
> >>>>>> = Verification
> >>>>>> $services.captcha.isValid() -- default implementation, current
> >>>>>> configuration
> >>>>>> $services.captcha.recaptcha.isValid() -- explicitly requested
> >>>>>> implementation, current/default configuration
> >>>>>> $services.captcha.recaptcha.isValid({'secret' : 'xyz'}) --
> explicitly
> >>>>>> requested implementation and custom configuration overwrites
> >>>>>> $services.captcha.jcaptcha.isValid() -- explicitly requested
> >>>>>> implementation and custom configuration overwrites
> >>>>>>
> >>>>>> Basically, the ScriptService API would be highly simplified to just
> >>>>>> displaying and validating, while the components would consist of 2
> >> parts:
> >>>>>> * a generic Manager for the various implementations (listing,
> >>>>>> getting/setting the default)
> >>>>>> * various implementations, each responsible with rendering a
> >> form-friendly
> >>>>>> CAPTCHA input and using the current request for extracting the
> >> information
> >>>>>> they needs to read the user's answer and validate it.
> >>>>>>
> >>>>>> WDYT?
> >>>>
> >>>> I do agree that JCaptcha is getting quite old now, however, we might
> >>>> have a hard time finding new FOSS and self-hosted captcha solutions
> (at
> >>>> least, there's not a lot of interesting artifacts on maven central
> [1]).
> >>>>
> >>>> IMO, we shouldn't rely on a captcha using an external verification
> >>>> service (such as Google's captchas) as we might end up blocking users
> >>>> that access their wiki with a filtered / controlled internet access.
> One
> >>>> option could be to provide a challenge which asks the user to perform
> a
> >>>> simple math operation, or respond to a control question instead, but
> I'm
> >>>> not convinced of the efficiency of those solutions against real
> >>>> captchas. As a captcha is usually quite simple to implement (apply
> >>>> operations on a text written in an image in order to make it
> >>>> non-readable by OCR software), we could also support our own version,
> >>>> maybe based on the generation mechanism given by JCaptcha.
> >>>>
> >>>> Thanks,
> >>>> Clément
> >>>>
> >>>> [1] https://search.maven.org/#search%7Cga%7C1%7Ccaptcha
> >>>>
> >>>>>> Thanks,
> >>>>>> Eduard
> >>>>>>
> >>>>>> ----------
> >>>>>> [1] http://extensions.xwiki.org/xwiki/bin/view/Extension/
> >> Captcha%20Module
> >>>>>> [2] http://design.xwiki.org/xwiki/bin/view/Proposal/
> >> CAPTCHAinvestigation70
> >>>>>> [3] https://jira.xwiki.org/browse/XWIKI-12964
> >>>>>>
> >>>
> >>>
> >>>
> >>> --
> >>> Thomas Mortagne
> >>
> >>
>
>
Reply | Threaded
Open this post in threaded view
|

Re: [Proposal] CAPTCHA module refactoring

vmassol
Administrator
Hi Edy,

> On 25 May 2018, at 18:24, Eduard Moraru <[hidden email]> wrote:

[snip]

>>> Now, as Thomas mentioned, it would make more sense to implement it as a
>>> Contrib extension (so that, in the near future, changes to reCAPTCHA
>> could
>>> be still used by older vesions; older as in stil using the new system,
>> i.e.
>>> 10.5+, but not the latest XWiki version), so the only solution to have it
>>> as default as well would mean for the Standard Flavor to bundle the
>>> reCAPTCHA implementaion, as a Contrib extension. (Vincent, insert "-1"
>> here
>>> :) )
>>
>> So if we agree that captcha are only required in a public environment
>> which has internet access then I think it would make sense to not have any
>> Captcha implementation by default in XS and make it fully a contrib
>> extension for those use cases (e.g. public web site use case).
>>
>
> Remember that Registration and Comments both allow enabling Captchas
> individually, so it's a feature that XS still needs, at least at the
> API/configuration level.

Yes, that’s what I mentioned below in my previous answer.

Either at API level (but without an impl.) or as UIXPs and extensions.

>
>
>>
>> And yes I wouldn’t like to bundle another contrib extension in XS before
>> we conclude on this topic (we started a discussion but didn’t finish it
>> AFAIK).
>>
>>> ---
>>>
>>> Side Note: Taking an approach, in terms of Admin UI, that is similar to
>>> what we did for Search (transition and choice from Solr to Lucene), with
>>> configuration options specific to each implementation.
>>
>> One option is to have the captcha API could be in XS and no implementation
>> provided by default and users would need to install an extension to get an
>> implementation.
>>
>
> Sure, however the problem with the current Captcha Module (
> http://extensions.xwiki.org/xwiki/bin/view/Extension/Captcha%20Module#HGettingthecaptchaimage
> ) is that it is implemented with a Struts Action (bound in
> struts-config.xml for "/imagecaptcha/“)

I don’t see this point as a problem. You can just extract it and bundle it in a contrib extension and have some manual installation instructions to enable it, by modifying struts-config.xml.

BTW what’s the new approach? Probably needs a new ResourceHandle to introduce a new URL type to validate captchas, right?

>  that anyone asking for a CAPTCHA
> should use to generate the CAPTCHA image and use it in an <img
> src=".../imagecaptcha/"/> tag. That makes it impossible to extract it as an
> extension.

This can be replaced by the call to the new URL type. The new resource handler would check which impl to use and it’s the old jcatpcha one, it would redirect to its /imagecaptcha/ URL for example.

> So we have a backwards compatibility problem in the fact that we need to
> keep supporting:
> * Users of the "/imagecaptcha/" action
> * Users of custom CAPTCHA systems, build on top of the old CaptchaVerifier
> system (
> http://extensions.xwiki.org/xwiki/bin/view/Extension/Captcha%20Module#HHowaboutimplementingyourowncaptchatype3F
> )
> ...and give them time to transition.
>
> Because of this, I plan to move everything found in the current
> xwiki-platform-captcha module to xwiki-platform-captcha-jcaptcha-api,
> together with the integration with the new system, and continue bundling it
> in XS. This way, both implementations should still work.
>
> So we end up with the following layout:
>
> xwiki-platform-captcha
> + xwiki-platform-captcha-api
> + xwiki-platform-captcha-ui
> + xwiki-platform-captcha-jcaptcha
> ++ xwiki-platform-captcha-jcaptcha-api
> ++ xwiki-platform-captcha-jcaptcha-ui
>
> captcha-recaptcha (contrib, depends on xwiki-platform-captcha)
> + captcha-recaptcha-api
> + captcha-recaptcha-ui

My POV:

* Don’t touch the old jcaptcha system and fully extract it to a contrib extension. And have instructions to install it in XWiki (with manual steps).
* Introduce the new more generic captcha APIs and UIs in xwiki-platform but don’t bundle it in XS by default
* Don’t provide any impl by default in XS either
* Add a new contrib extension for recaptcha and have it depends on the new captcha API module from platform

WDYT?

Thanks
-Vincent

>
> Thanks,
> Eduard
>
>
>>
>> The other option is to introduce UIXP instead and fully move the code to
>> the extension(s).
>>
>> Thanks
>> -Vincent
>>
>>> ---
>>>
>>> Now, CAPTCHA is not enabled by default in XWiki, so admins still need to
>> do
>>> this. Once they enable it, if reCAPTCHA is the default, it won't work
>> until
>>> it's configured (with a Google API key, even for localhost). An
>> interesting
>>> thing we could do to avoid this problem and have the CAPTCHA not fail
>>> completely, is to use Google's test keys (originally intended to not
>> break
>>> automated testing) by default which will make the CAPTCHA always
>> validate,
>>> while still being explicit that the configuration needs to be done. This
>>> would make sure there is no friction in the process and we'll probably do
>>> this by default for this implementation, regardless if we choose to use
>> it
>>> as XWiki's default or if it needs to be manually installed (from
>> contrib).
>>>
>>> https://developers.google.com/recaptcha/docs/faq#id-like-to-
>> run-automated-tests-with-recaptcha-v2-what-should-i-do
>>>
>>> Thanks,
>>> Eduard
>>>
>>>
>>>>
>>>> Thanks
>>>> -Vincent
>>>>
>>>>> On 25 May 2018, at 08:51, Thomas Mortagne <[hidden email]>
>>>> wrote:
>>>>>
>>>>> I agree that the default captcha we provide should not depend on an
>>>>> external service but it should just be a default. IMO the most
>>>>> important is to make sure it's easy to choose a different captcha
>>>>> implementation and we should most probably provide the Google one as
>>>>> an option (through a contrib extension to better follow Google captcha
>>>>> API evolution).
>>>>>
>>>>> On Fri, May 25, 2018 at 12:37 AM, Clément Aubin <[hidden email]>
>>>> wrote:
>>>>>> Hi,
>>>>>>
>>>>>> On 05/24/2018 05:15 PM, Eduard Moraru wrote:
>>>>>>> Question:
>>>>>>> e
>>>>>>> Do we need the captcha module to function without the DB as well?
>>>>>>>
>>>>>>> This would make it more difficult to create the configuration UI (in
>>>>>>> Administration) and would also mean that some captchas
>> implementations
>>>>>>> might not respect this constraint (i.e. choosing to implement their
>> UI
>>>> in a
>>>>>>> wiki page instead).
>>>>>>>
>>>>>>> IMO, it's not really worth the trouble and we should proceed to
>>>> extending
>>>>>>> the current captcha module into 2 parts: -api and -ui (the current
>>>> module
>>>>>>> would be converted from "jar" to "pom" and the current code will be
>>>>>>> deprecated and moved to the new -api module).
>>>>>>
>>>>>> The only reason we could not to depend on the DB would be to transfer
>>>>>> parts of the captcha module to xwiki-commons IMO. For now, I don't see
>>>>>> any particular use case that could support this migration.
>>>>>>
>>>>>>> Thanks,
>>>>>>> Eduard
>>>>>>>
>>>>>>> On Tue, May 22, 2018 at 8:35 PM, Eduard Moraru <[hidden email]
>>>
>>>> wrote:
>>>>>>>
>>>>>>>> Hi, devs,
>>>>>>>>
>>>>>>>> XWiki's current CAPTCHA module [1] is very old and outdated for a
>>>> while
>>>>>>>> now and this is not news for anyone.
>>>>>>>>
>>>>>>>> I see 2 major problems:
>>>>>>>> 1) The obvious one is that we just need a technologically better
>>>> CAPTCHA
>>>>>>>> implementation that the current JCaptcha-based one and JCaptcha is
>>>>>>>> discontinued.
>>>>>>>> 2) The current architecture is outdated as well (i.e. wrapped around
>>>>>>>> Struts actions and initializing the VelocityContext with a custom
>>>> binding)
>>>>>>>> and hard to use (i.e. you have to write your CAPTCHA UI every time
>>>> you use
>>>>>>>> it, there is no rendering helper).
>>>>>>>>
>>>>>>>> For 1), as both determined a few years ago [2] and also observed
>> from
>>>>>>>> practice, the industry standard is now Google's reCaptcha service
>>>> (either
>>>>>>>> its v2 or invisible version, or both), so we definitely need an
>>>>>>>> implementation that makes it easy to use in XWiki [3].
>>>>>>>>
>>>>>>>> For 2), we need:
>>>>>>>> * to allow CAPTCHA implementations to provide their own rendering
>>>>>>>> ** Can be done using a template .vm file (rendered with the
>>>>>>>> TemplateManager), a wiki page or directly from Java code.
>>>>>>>> ** The implementation-specific parameters could be passed to
>>>>>>>> control/customize a particular rendering.
>>>>>>>> ** The syntax ID may not be needed, since the only relevant output
>>>> would
>>>>>>>> be html.
>>>>>>>> * to move to a ScriptService-based system and to leave the resource
>>>>>>>> (image/audio) accessing concern to the individual implementation
>>>> (that may
>>>>>>>> choose to continue with Struts actions, or may choose to use
>> something
>>>>>>>> lighter like a REST resource or even or even something more
>>>> inventive, like
>>>>>>>> temporary attachments on some fixed page).
>>>>>>>> * an administration UI that configures the default CAPTCHA service
>>>> and its
>>>>>>>> configuration
>>>>>>>> * to allow callers to use a different CAPTCHA than the default
>>>> configured
>>>>>>>> one, as long as it is available (i.e. installed)
>>>>>>>>
>>>>>>>> Examples:
>>>>>>>>
>>>>>>>> = Display
>>>>>>>> $services.captcha.display() -- default implementation, current
>>>>>>>> configuration
>>>>>>>> $services.captcha.recaptcha.display() -- explicitly requested
>>>>>>>> implementation, current/default configuration
>>>>>>>> $services.captcha.recaptcha.display({'theme' : 'dark', 'size' :
>>>>>>>> 'compact', 'sitekey' : 'xyz'}) -- explicitly requested
>> implementation
>>>> and
>>>>>>>> custom configuration overwrites
>>>>>>>> $services.captcha.jcaptcha.display() -- other implementation,
>>>>>>>> current/default configuration
>>>>>>>>
>>>>>>>> = Verification
>>>>>>>> $services.captcha.isValid() -- default implementation, current
>>>>>>>> configuration
>>>>>>>> $services.captcha.recaptcha.isValid() -- explicitly requested
>>>>>>>> implementation, current/default configuration
>>>>>>>> $services.captcha.recaptcha.isValid({'secret' : 'xyz'}) --
>> explicitly
>>>>>>>> requested implementation and custom configuration overwrites
>>>>>>>> $services.captcha.jcaptcha.isValid() -- explicitly requested
>>>>>>>> implementation and custom configuration overwrites
>>>>>>>>
>>>>>>>> Basically, the ScriptService API would be highly simplified to just
>>>>>>>> displaying and validating, while the components would consist of 2
>>>> parts:
>>>>>>>> * a generic Manager for the various implementations (listing,
>>>>>>>> getting/setting the default)
>>>>>>>> * various implementations, each responsible with rendering a
>>>> form-friendly
>>>>>>>> CAPTCHA input and using the current request for extracting the
>>>> information
>>>>>>>> they needs to read the user's answer and validate it.
>>>>>>>>
>>>>>>>> WDYT?
>>>>>>
>>>>>> I do agree that JCaptcha is getting quite old now, however, we might
>>>>>> have a hard time finding new FOSS and self-hosted captcha solutions
>> (at
>>>>>> least, there's not a lot of interesting artifacts on maven central
>> [1]).
>>>>>>
>>>>>> IMO, we shouldn't rely on a captcha using an external verification
>>>>>> service (such as Google's captchas) as we might end up blocking users
>>>>>> that access their wiki with a filtered / controlled internet access.
>> One
>>>>>> option could be to provide a challenge which asks the user to perform
>> a
>>>>>> simple math operation, or respond to a control question instead, but
>> I'm
>>>>>> not convinced of the efficiency of those solutions against real
>>>>>> captchas. As a captcha is usually quite simple to implement (apply
>>>>>> operations on a text written in an image in order to make it
>>>>>> non-readable by OCR software), we could also support our own version,
>>>>>> maybe based on the generation mechanism given by JCaptcha.
>>>>>>
>>>>>> Thanks,
>>>>>> Clément
>>>>>>
>>>>>> [1] https://search.maven.org/#search%7Cga%7C1%7Ccaptcha
>>>>>>
>>>>>>>> Thanks,
>>>>>>>> Eduard
>>>>>>>>
>>>>>>>> ----------
>>>>>>>> [1] http://extensions.xwiki.org/xwiki/bin/view/Extension/
>>>> Captcha%20Module
>>>>>>>> [2] http://design.xwiki.org/xwiki/bin/view/Proposal/
>>>> CAPTCHAinvestigation70
>>>>>>>> [3] https://jira.xwiki.org/browse/XWIKI-12964
>>>>>>>>
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Thomas Mortagne