[Idea] Make Markdown syntax available as an optional subset of the XWiki 2.1 syntax

classic Classic list List threaded Threaded
9 messages Options
Reply | Threaded
Open this post in threaded view
|

[Idea] Make Markdown syntax available as an optional subset of the XWiki 2.1 syntax

Guillaume Lerouge
Hi Devs,

as you probably noticed it, Markdown has recently become somewhat of a
reference syntax for many developer tools, most notably GitHub. I have
recently discussed with teams using XWiki who are also using GitHub and
Slack and who are interested in being able to use Markdown syntax inside
XWiki.

Although it is already possible to use Markdown syntax in XWiki in a
limited way <http://rendering.xwiki.org/xwiki/bin/view/Main/Markdown11>, I
don't think that a wiki could really work with only Markdown, due to the
following limitations:

   1. We have invested a lot in the XWiki rendering and the XWiki 2.1
   syntax in order to make them address a lot of use cases and work seamlessly
   with the WYSIWYG editor
   2. Conversely, Markdown syntax is very limited by design
   <https://daringfireball.net/projects/markdown/syntax> and does not
   support many of the important features of XWiki syntax, forcing users to
   rely on HTML for a lot of use cases

What's interesting however is that Markdown syntax is very close to XWiki
syntax in a number of regards, notably line breaks, bold and lists. Some
notable differences include the syntax for links, images and code blocks.

My line of thinking is the following: what if we made it possible as an
option for users of XWiki 2.1 syntax to have XWiki interpret the main
elements of Markdown syntax? In practice, this would mean adding a set of
5-10 additional rules to the rendering engine.

The obvious benefit would be to improve adoption of XWiki by dev teams who
are already familiar with Markdown. I don't see any obvious drawbacks
(besides the need to code and maintain the feature of course), but I
clearly don't master all the subtleties of the XWiki rendering engine.

Thanks,

Guillaume
_______________________________________________
devs mailing list
[hidden email]
http://lists.xwiki.org/mailman/listinfo/devs
Reply | Threaded
Open this post in threaded view
|

Re: [Idea] Make Markdown syntax available as an optional subset of the XWiki 2.1 syntax

vmassol
Administrator
Hi Guillaume,

On 25 Nov 2015 at 09:05:31, Guillaume Lerouge ([hidden email](mailto:[hidden email])) wrote:

> Hi Devs,
>  
> as you probably noticed it, Markdown has recently become somewhat of a
> reference syntax for many developer tools, most notably GitHub. I have
> recently discussed with teams using XWiki who are also using GitHub and
> Slack and who are interested in being able to use Markdown syntax inside
> XWiki.
>  
> Although it is already possible to use Markdown syntax in XWiki in a
> limited way , I
> don't think that a wiki could really work with only Markdown, due to the
> following limitations:
>  
> 1. We have invested a lot in the XWiki rendering and the XWiki 2.1
> syntax in order to make them address a lot of use cases and work seamlessly
> with the WYSIWYG editor
> 2. Conversely, Markdown syntax is very limited by design
> and does not
> support many of the important features of XWiki syntax, forcing users to
> rely on HTML for a lot of use cases
>  
> What's interesting however is that Markdown syntax is very close to XWiki
> syntax in a number of regards, notably line breaks, bold and lists. Some
> notable differences include the syntax for links, images and code blocks.
>  
> My line of thinking is the following: what if we made it possible as an
> option for users of XWiki 2.1 syntax to have XWiki interpret the main
> elements of Markdown syntax? In practice, this would mean adding a set of
> 5-10 additional rules to the rendering engine.
>  
> The obvious benefit would be to improve adoption of XWiki by dev teams who
> are already familiar with Markdown. I don't see any obvious drawbacks
> (besides the need to code and maintain the feature of course), but I
> clearly don't master all the subtleties of the XWiki rendering engine.

There are lots of differences between Markdown and XWiki Syntax, they’re completely different actually (if I had to give a ratio I’d say 90% different). Also note that white spaces have meanings in Markdown.

You should know that each syntax has a parser. We’re using the pegdown parser to parse Markdown (we don’t do any work on our side on the parsing, we just adapt their events to the XWiki Rendering Events). And for the XWiki Syntax parser they are javacc-based parsers.

So theoretically it would be possible to implement a new XWiki Syntax altogether, although very difficult (XWiki Markdown Syntax 1.0 for example), but there are many drawbacks. Just to give 2:

1) It would be very very costly to develop and maintain (including having to develop a renderer for it! Something we haven’t even done for markdown because of lack of time/resource, so imagine having to code a new parser also).
2) It would confuse everyone: users knowing markdown and users knowing the xwiki syntax since nobody would recognize their syntax.

Last I want to mention that markdown is no longer the hip syntax. It’s been asciidoc for a while now. I’ve tried to integrate it several times but it’s missing a good java parser right now (they’re working on it, albeit slowly), see: http://jira.xwiki.org/browse/XRENDERING-323

Thanks for the idea though!

If we had some time to spend on this topic, I would spend it by implementing a Renderer for the Markdown syntax so that users could use Markdown in the WYSIWYG editor. Just note that this is quite complex to do and we would need to verify that we can express all HTML constructions in markdown.

Thanks
-Vincent

> Thanks,
>  
> Guillaume
> _______________________________________________
> devs mailing list
> [hidden email]
> http://lists.xwiki.org/mailman/listinfo/devs

_______________________________________________
devs mailing list
[hidden email]
http://lists.xwiki.org/mailman/listinfo/devs
Reply | Threaded
Open this post in threaded view
|

Re: [Idea] Make Markdown syntax available as an optional subset of the XWiki 2.1 syntax

Caleb James DeLisle-3
In reply to this post by Guillaume Lerouge
I would say generally speaking the longer we wait to align our syntax
with markdown the more non-standard and irrelvant our syntax will become.

If we were building XWiki from scratch today, we would be royal fools not
to adhere to what is essentially a universal standard for syntax.

But even though we're not building it today, many people are trying it
today and for someone who doesn't know or care about the bad old days
before markup was standardized, we're just another incompatible format
with no significant market share.

Same story is true for Bootstrap. 5 years ago there was no coherent
standard, now there is.

tl;dr it sucks but we have to.

Thanks,
Caleb


On 25/11/15 09:05, Guillaume Lerouge wrote:

> Hi Devs,
>
> as you probably noticed it, Markdown has recently become somewhat of a
> reference syntax for many developer tools, most notably GitHub. I have
> recently discussed with teams using XWiki who are also using GitHub and
> Slack and who are interested in being able to use Markdown syntax inside
> XWiki.
>
> Although it is already possible to use Markdown syntax in XWiki in a
> limited way <http://rendering.xwiki.org/xwiki/bin/view/Main/Markdown11>, I
> don't think that a wiki could really work with only Markdown, due to the
> following limitations:
>
>     1. We have invested a lot in the XWiki rendering and the XWiki 2.1
>     syntax in order to make them address a lot of use cases and work seamlessly
>     with the WYSIWYG editor
>     2. Conversely, Markdown syntax is very limited by design
>     <https://daringfireball.net/projects/markdown/syntax> and does not
>     support many of the important features of XWiki syntax, forcing users to
>     rely on HTML for a lot of use cases
>
> What's interesting however is that Markdown syntax is very close to XWiki
> syntax in a number of regards, notably line breaks, bold and lists. Some
> notable differences include the syntax for links, images and code blocks.
>
> My line of thinking is the following: what if we made it possible as an
> option for users of XWiki 2.1 syntax to have XWiki interpret the main
> elements of Markdown syntax? In practice, this would mean adding a set of
> 5-10 additional rules to the rendering engine.
>
> The obvious benefit would be to improve adoption of XWiki by dev teams who
> are already familiar with Markdown. I don't see any obvious drawbacks
> (besides the need to code and maintain the feature of course), but I
> clearly don't master all the subtleties of the XWiki rendering engine.
>
> Thanks,
>
> Guillaume
> _______________________________________________
> devs mailing list
> [hidden email]
> http://lists.xwiki.org/mailman/listinfo/devs
>
_______________________________________________
devs mailing list
[hidden email]
http://lists.xwiki.org/mailman/listinfo/devs
Reply | Threaded
Open this post in threaded view
|

Re: [Idea] Make Markdown syntax available as an optional subset of the XWiki 2.1 syntax

Paul Libbrecht-2
Hello all,

Caleb thus wrote:
> I would say generally speaking the longer we wait to align our syntax
> with markdown the more non-standard and irrelvant our syntax will become.
This is pretty harsh. Do you know if others are doing such a slow
convergence?
I think Markdown is great for elementary tasks such as a readme but I've
seen it fail for bigger (and long term) things.
> If we were building XWiki from scratch today, we would be royal fools not
> to adhere to what is essentially a universal standard for syntax.
Erm... provided it would do all we wish it to do.

E.g. I've never seen "macros" in markdown be put to use in widespread
infrastructures whereas many wiki syntaxes have some (GitHub has none as
far as I can tell).

Addressing that point might answer Guillaume's original intent, which
seems to answer a fairly precise desire. In particular, I'd be quite
fine if we enriched our MarkDown processor with some forms of macros but
nothing more of the xwiki syntax.

I've always seen markdown as "when it's simple I use it" otherwise, just
don't bother and use html.
Guillaume, should that be broken? I'd understand yes, but please confirm.
Also, how would you have recognition? E.g. how should the table syntax
be recognized?

paul

> But even though we're not building it today, many people are trying it
> today and for someone who doesn't know or care about the bad old days
> before markup was standardized, we're just another incompatible format
> with no significant market share.
>
> Same story is true for Bootstrap. 5 years ago there was no coherent
> standard, now there is.
>
> tl;dr it sucks but we have to.
>
> Thanks,
> Caleb
>
>
>
> _______________________________________________
> devs mailing list
> [hidden email]
> http://lists.xwiki.org/mailman/listinfo/devs
> Guillaume Lerouge <mailto:[hidden email]>
> 25 novembre 2015 09:05
> Hi Devs,
>
> as you probably noticed it, Markdown has recently become somewhat of a
> reference syntax for many developer tools, most notably GitHub. I have
> recently discussed with teams using XWiki who are also using GitHub and
> Slack and who are interested in being able to use Markdown syntax inside
> XWiki.
>
> Although it is already possible to use Markdown syntax in XWiki in a
> limited way <http://rendering.xwiki.org/xwiki/bin/view/Main/Markdown11>, I
> don't think that a wiki could really work with only Markdown, due to the
> following limitations:
>
> 1. We have invested a lot in the XWiki rendering and the XWiki 2.1
> syntax in order to make them address a lot of use cases and work
> seamlessly
> with the WYSIWYG editor
> 2. Conversely, Markdown syntax is very limited by design
> <https://daringfireball.net/projects/markdown/syntax> and does not
> support many of the important features of XWiki syntax, forcing users to
> rely on HTML for a lot of use cases
>
> What's interesting however is that Markdown syntax is very close to XWiki
> syntax in a number of regards, notably line breaks, bold and lists. Some
> notable differences include the syntax for links, images and code blocks.
>
> My line of thinking is the following: what if we made it possible as an
> option for users of XWiki 2.1 syntax to have XWiki interpret the main
> elements of Markdown syntax? In practice, this would mean adding a set of
> 5-10 additional rules to the rendering engine.
>
> The obvious benefit would be to improve adoption of XWiki by dev teams who
> are already familiar with Markdown. I don't see any obvious drawbacks
> (besides the need to code and maintain the feature of course), but I
> clearly don't master all the subtleties of the XWiki rendering engine.
>
> Thanks,
>
> Guillaume
> _______________________________________________
> devs mailing list
> [hidden email]
> http://lists.xwiki.org/mailman/listinfo/devs

_______________________________________________
devs mailing list
[hidden email]
http://lists.xwiki.org/mailman/listinfo/devs
Reply | Threaded
Open this post in threaded view
|

Re: [Idea] Make Markdown syntax available as an optional subset of the XWiki 2.1 syntax

vmassol
Administrator
Hi Paul,


On 27 Nov 2015 at 13:06:20, Paul Libbrecht ([hidden email](mailto:[hidden email])) wrote:

> Hello all,
>  
> Caleb thus wrote:
> > I would say generally speaking the longer we wait to align our syntax
> > with markdown the more non-standard and irrelvant our syntax will become.
> This is pretty harsh. Do you know if others are doing such a slow
> convergence?
> I think Markdown is great for elementary tasks such as a readme but I've
> seen it fail for bigger (and long term) things.
> > If we were building XWiki from scratch today, we would be royal fools not
> > to adhere to what is essentially a universal standard for syntax.
> Erm... provided it would do all we wish it to do.
>  
> E.g. I've never seen "macros" in markdown be put to use in widespread
> infrastructures whereas many wiki syntaxes have some (GitHub has none as
> far as I can tell).
>  
> Addressing that point might answer Guillaume's original intent, which
> seems to answer a fairly precise desire. In particular, I'd be quite
> fine if we enriched our MarkDown processor with some forms of macros but
> nothing more of the xwiki syntax.

See http://rendering.xwiki.org/xwiki/bin/view/Main/Markdown11 :)

Thanks
-Vincent

> I've always seen markdown as "when it's simple I use it" otherwise, just
> don't bother and use html.
> Guillaume, should that be broken? I'd understand yes, but please confirm.
> Also, how would you have recognition? E.g. how should the table syntax
> be recognized?
>  
> paul
>  
> > But even though we're not building it today, many people are trying it
> > today and for someone who doesn't know or care about the bad old days
> > before markup was standardized, we're just another incompatible format
> > with no significant market share.
> >
> > Same story is true for Bootstrap. 5 years ago there was no coherent
> > standard, now there is.
> >
> > tl;dr it sucks but we have to.
> >
> > Thanks,
> > Caleb
> >
> >
> >
> > _______________________________________________
> > devs mailing list
> > [hidden email]
> > http://lists.xwiki.org/mailman/listinfo/devs
> > Guillaume Lerouge  
> > 25 novembre 2015 09:05
> > Hi Devs,
> >
> > as you probably noticed it, Markdown has recently become somewhat of a
> > reference syntax for many developer tools, most notably GitHub. I have
> > recently discussed with teams using XWiki who are also using GitHub and
> > Slack and who are interested in being able to use Markdown syntax inside
> > XWiki.
> >
> > Although it is already possible to use Markdown syntax in XWiki in a
> > limited way , I
> > don't think that a wiki could really work with only Markdown, due to the
> > following limitations:
> >
> > 1. We have invested a lot in the XWiki rendering and the XWiki 2.1
> > syntax in order to make them address a lot of use cases and work
> > seamlessly
> > with the WYSIWYG editor
> > 2. Conversely, Markdown syntax is very limited by design
> > and does not
> > support many of the important features of XWiki syntax, forcing users to
> > rely on HTML for a lot of use cases
> >
> > What's interesting however is that Markdown syntax is very close to XWiki
> > syntax in a number of regards, notably line breaks, bold and lists. Some
> > notable differences include the syntax for links, images and code blocks.
> >
> > My line of thinking is the following: what if we made it possible as an
> > option for users of XWiki 2.1 syntax to have XWiki interpret the main
> > elements of Markdown syntax? In practice, this would mean adding a set of
> > 5-10 additional rules to the rendering engine.
> >
> > The obvious benefit would be to improve adoption of XWiki by dev teams who
> > are already familiar with Markdown. I don't see any obvious drawbacks
> > (besides the need to code and maintain the feature of course), but I
> > clearly don't master all the subtleties of the XWiki rendering engine.
> >
> > Thanks,
> >
> > Guillaume
> > _______________________________________________
> > devs mailing list
> > [hidden email]
> > http://lists.xwiki.org/mailman/listinfo/devs
>  
> _______________________________________________
> devs mailing list
> [hidden email]
> http://lists.xwiki.org/mailman/listinfo/devs
_______________________________________________
devs mailing list
[hidden email]
http://lists.xwiki.org/mailman/listinfo/devs
Reply | Threaded
Open this post in threaded view
|

Re: [Idea] Make Markdown syntax available as an optional subset of the XWiki 2.1 syntax

Paul Libbrecht-2
In reply to this post by vmassol
Cool.

Would it be possible to join the fashion claim by adding compatibility
statements?
E.g., does that make it compatible with some markdown variant? Is
GitHub's MarkDown somewhere close?

paul

[hidden email] wrote:
> > Addressing that point might answer Guillaume's original intent, which
> > seems to answer a fairly precise desire. In particular, I'd be quite
> > fine if we enriched our MarkDown processor with some forms of macros but
> > nothing more of the xwiki syntax.
>
> See http://rendering.xwiki.org/xwiki/bin/view/Main/Markdown11 :)
>

_______________________________________________
devs mailing list
[hidden email]
http://lists.xwiki.org/mailman/listinfo/devs
Reply | Threaded
Open this post in threaded view
|

Re: [Idea] Make Markdown syntax available as an optional subset of the XWiki 2.1 syntax

vmassol
Administrator
Hi Paul,

Cool. 

Would it be possible to join the fashion claim by adding compatibility 
statements? 
E.g., does that make it compatible with some markdown variant? Is 
GitHub's MarkDown somewhere close? 


It’s already mentioned on that page: "The Markdown 1.0 syntax in XWiki supports only the features provided by the PegDown parser, “

You then need to go to the Pegdown page to see what markdown it supports:

https://github.com/sirthias/pegdown

Thanks

-Vincent



paul 

[hidden email] wrote: 
> > Addressing that point might answer Guillaume's original intent, which 
> > seems to answer a fairly precise desire. In particular, I'd be quite 
> > fine if we enriched our MarkDown processor with some forms of macros but 
> > nothing more of the xwiki syntax. 

> See http://rendering.xwiki.org/xwiki/bin/view/Main/Markdown11 :) 


_______________________________________________ 
devs mailing list 
[hidden email] 
http://lists.xwiki.org/mailman/listinfo/devs 
_______________________________________________
devs mailing list
[hidden email]
http://lists.xwiki.org/mailman/listinfo/devs
Reply | Threaded
Open this post in threaded view
|

Re: [Idea] Make Markdown syntax available as an optional subset of the XWiki 2.1 syntax

Guillaume Lerouge
Hi Paul,

I was hoping for a "best-of-both-worlds" approach: keep the usual XWiki
syntax working as is, and add rules to our rendering engine for the
markdown syntax elements that are different. This way one could use both
XWiki syntax and markdown syntax in the same document seamlessly (a bit
like right now you can mix HTML content with markdown content), without
breaking the WYSIWYG editor.

I think Vincent made it abundantly clear that this was utopian and I was
underestimating the underlying complexity :-)

As for the relative popularity of markdown syntax WRT to ASCIIdoc and XWiki
syntax, I don't have hard numbers, but right now Markdown is the de facto
standard for many developer-focused apps in a way that the other 2 are not.

Thanks,

Guillaume

On Fri, Nov 27, 2015 at 2:13 PM, [hidden email] <[hidden email]>
wrote:

> Hi Paul,
>
> Cool.
>
> Would it be possible to join the fashion claim by adding compatibility
> statements?
> E.g., does that make it compatible with some markdown variant? Is
> GitHub's MarkDown somewhere close?
>
>
> It’s already mentioned on that page: "The Markdown 1.0 syntax in XWiki
> supports only the features provided by the PegDown parser, “
>
> You then need to go to the Pegdown page to see what markdown it supports:
>
> https://github.com/sirthias/pegdown
>
> Thanks
>
> -Vincent
>
>
>
> paul
>
> [hidden email] wrote:
> > > Addressing that point might answer Guillaume's original intent, which
> > > seems to answer a fairly precise desire. In particular, I'd be quite
> > > fine if we enriched our MarkDown processor with some forms of macros
> but
> > > nothing more of the xwiki syntax.
> >
> > See http://rendering.xwiki.org/xwiki/bin/view/Main/Markdown11 :)
> >
>
> _______________________________________________
> devs mailing list
> [hidden email]
> http://lists.xwiki.org/mailman/listinfo/devs
> _______________________________________________
> devs mailing list
> [hidden email]
> http://lists.xwiki.org/mailman/listinfo/devs
>
_______________________________________________
devs mailing list
[hidden email]
http://lists.xwiki.org/mailman/listinfo/devs
Reply | Threaded
Open this post in threaded view
|

Re: [xwiki-users] [Idea] Make Markdown syntax available as an optional subset of the XWiki 2.1 syntax

Caleb James DeLisle-3
In reply to this post by vmassol
The point is not to use someone else's code or even to duplicate
the logic to a perfect binary-compatible tee, the point is to be
"standard enough" that it's familiar to people who come in from
Github, StackOverflow, Slack, Reddit etc.

The php-inspired macro idea is another attempt at making it more
familiar to people and thus more intuitive.

Thanks,
Caleb



On 27/11/15 16:54, [hidden email] wrote:

>
>
>
> On 27 Nov 2015 at 16:41:34, Paul Libbrecht ([hidden email](mailto:[hidden email])) wrote:
>
>>
>>
>> Caleb James DeLisle wrote:
>>> I had just imagined an extension to the markdown standard, not sure
>>> exactly
>>> how macros ought to be implemented... One possibility:
>>>
>>>>> $doc.getFullName()
>>> ?>
>>>
>>> or
>>>
>>> ?>
>> Hey, that is very "standard" in the sense of PHP-ish.
>> I like it but I am sure it can clash heavily somewhere.
>>
>> Another proposal is here:
>> https://github.com/codingcoop/markdown-macros
>> More compact... thus less portable…
>
> Can you explain why you’re trying to do something that’s already implemented? You don’t like our current implementation?
>
> Thanks
> -Vincent
>
>> Paul
>
> _______________________________________________
> users mailing list
> [hidden email]
> http://lists.xwiki.org/mailman/listinfo/users
>
_______________________________________________
devs mailing list
[hidden email]
http://lists.xwiki.org/mailman/listinfo/devs