Improving the XWiki Development environment for traditional Java/JS Developers

classic Classic list List threaded Threaded
24 messages Options
12
Reply | Threaded
Open this post in threaded view
|

Improving the XWiki Development environment for traditional Java/JS Developers

Ludovic Dubost
While XWiki has many great advantages as a framework which takes car of
persistence, forms, user management, content management and provides tons
of APIs, when traditional developers want to extend XWiki there are facing
a few difficulties:

- they are lost in the myriad of XWiki APIs, and there is no completion
- they don't get good visibility of the code available in their application
- it is complicated to use editors which have syntax highlighting
- they cannot use their favorite IDE

On the framework side there are also some improvements which could make
XWiki even more killer:

- easier integration of advanced JS framework
- advanced integration of a high level JS framework that has templating
(maybe angular js)
- better validation functions
- easier way to add REST APIs
- more XWiki and better documented javascript apis


Here are some proposals to help fix the tools issues. Three approaches can
be looked at:

1/ Live Sync between an XWiki Instance and and improved maven xar file
structure, allowing to use any local IDE on XWiki code

First it should be possible to use any IDE on the maven xar file structure,
allowing to open the content and textarea fields of all XML files.
For this XWiki XML files should externalize the content and textarea fields
in separate files with extensions based on their content type.
The maven xar format should be able to clean the maven structure to do this
splitting and should also be able to build the XAR from all the files.

Finally a program should allow to do a live sync between a local or remote
wiki instance and the maven project. Any change in either the wiki or the
file system should be replicated to the other party.
So if you run "git pull" then your local or remote wiki would be updated.
If a change is made in the wiki, the change would show up in the file
system and your IDE would refresh it.

The sync program would keep a state of the version in the wiki, in order to
be able to merge changes if changes occur in both places between two sync.
This tool could be easily launched with "mvn xar:sync"

Syntax highlighting for XWiki Syntax+Velocity+Groovy would be developped
for the most popular editors.
When syncing the tool could show syntax checking error messages.

2/ Integration with Eclipse

Based on XEclipse, we would build an Eclipse plugin to be able to connect
to an XWiki instance and load a specified list of spaces. Then each space
would be organized by the type of code in this space. Content and Textarea
fields would be made visible as editable content in Eclipse.
The plugin should detect the type of code in each of the content or
textarea fields (velocity/html, groovy, javascript, css, translations,
xwiki syntax) and use the appropriate editor.

Finally a completion module could be provided by loading from the server a
list of available APIs.

The same plugin could also be able to organize the content from a local
maven project (based on 1/) and provide completion to such a project.

Live syntax checking could be provided.

3/ Integration of a WebIDE in XWiki

Based on Javascript WebIDE and web code editor softwares (Orion, Cloud9,
exoIDE, codemirror, etc..), we would provide a view on the code inside the
XWiki instance.
Code would be organized in the same way as in the Eclipse plugin and
appropriate editors would be used depending on the code type.
Completion could be provided in the velocity and groovy editors and
eventually in Javascript

Two views should be available, one for each AWM application, and one with
all the code in the Wiki.

Live syntax checking could be provided.

Solution 1/ is very powerful because it let's the user decide which is his
development environment, even if the tree structure won't be perfect. Still
he can see all the code in the wiki and work on it, including searching.
Committing is made very easy. There are some risks involved in the sync
process, particularly of you have multiple developers syncing local code
with the same dev server.
User can switch from local sync (local wiki) to remove sync (remove shared
wiki) and can therefore work offline more easily. Editing can be fully done
offline.

Providing syntax highlighting for many editors might prove difficult.

Solution 2/ is providing a nice XWiki environment inside Eclipse, without
the need for a local copy of the code. Committing would happen using the
browser.

Solution 3/ is the long term bet as the future is to have everything in the
web. Content is only in one place which makes things a little easier.
Development environment needs no setup.
However this is more "new" for developers which need to adopt the platform.

Live syntax checking is hard to provide but would be quite useful.
Alternatively mvn xar:format could also provide syntax checking for XWiki
syntax, velocity, groovy, js and css.

WDYT ? Which approaches do you believe would be the most promising.

Ludovic

--
Ludovic Dubost
Founder and CEO
Blog: http://blog.ludovic.org/
XWiki: http://www.xwiki.com
Skype: ldubost GTalk: ldubost
_______________________________________________
devs mailing list
[hidden email]
http://lists.xwiki.org/mailman/listinfo/devs
Reply | Threaded
Open this post in threaded view
|

Re: Improving the XWiki Development environment for traditional Java/JS Developers

vmassol
Administrator
Hi ludovic,

All good points that I think all of us would like to see addressed as well.

Replies inline.

On Jun 29, 2013, at 11:41 AM, Ludovic Dubost <[hidden email]> wrote:

> While XWiki has many great advantages as a framework which takes car of
> persistence, forms, user management, content management and provides tons
> of APIs, when traditional developers want to extend XWiki there are facing
> a few difficulties:
>
> - they are lost in the myriad of XWiki APIs, and there is no completion

Note that Edy and myself created an autcompletion extension last year but we didn't get to publish it on e.x.o (it's here ATM:  https://github.com/xwiki-contrib/wiki-editor-devtools).

I'm planning to continue and finish it this year during the coming hackathon in July.

> - they don't get good visibility of the code available in their application
> - it is complicated to use editors which have syntax highlighting
> - they cannot use their favorite IDE
>
> On the framework side there are also some improvements which could make
> XWiki even more killer:
>
> - easier integration of advanced JS framework
> - advanced integration of a high level JS framework that has templating
> (maybe angular js)
> - better validation functions
> - easier way to add REST APIs
> - more XWiki and better documented javascript apis
>
>
> Here are some proposals to help fix the tools issues. Three approaches can
> be looked at:
>
> 1/ Live Sync between an XWiki Instance and and improved maven xar file
> structure, allowing to use any local IDE on XWiki code
>
> First it should be possible to use any IDE on the maven xar file structure,
> allowing to open the content and textarea fields of all XML files.
> For this XWiki XML files should externalize the content and textarea fields
> in separate files with extensions based on their content type.
> The maven xar format should be able to clean the maven structure to do this
> splitting and should also be able to build the XAR from all the files.
>
> Finally a program should allow to do a live sync between a local or remote
> wiki instance and the maven project. Any change in either the wiki or the
> file system should be replicated to the other party.
> So if you run "git pull" then your local or remote wiki would be updated.
> If a change is made in the wiki, the change would show up in the file
> system and your IDE would refresh it.
>
> The sync program would keep a state of the version in the wiki, in order to
> be able to merge changes if changes occur in both places between two sync.
> This tool could be easily launched with "mvn xar:sync"
>
> Syntax highlighting for XWiki Syntax+Velocity+Groovy would be developped
> for the most popular editors.
> When syncing the tool could show syntax checking error messages.
>
> 2/ Integration with Eclipse
>
> Based on XEclipse, we would build an Eclipse plugin to be able to connect
> to an XWiki instance and load a specified list of spaces. Then each space
> would be organized by the type of code in this space. Content and Textarea
> fields would be made visible as editable content in Eclipse.
> The plugin should detect the type of code in each of the content or
> textarea fields (velocity/html, groovy, javascript, css, translations,
> xwiki syntax) and use the appropriate editor.
>
> Finally a completion module could be provided by loading from the server a
> list of available APIs.
>
> The same plugin could also be able to organize the content from a local
> maven project (based on 1/) and provide completion to such a project.
>
> Live syntax checking could be provided.
>
> 3/ Integration of a WebIDE in XWiki
>
> Based on Javascript WebIDE and web code editor softwares (Orion, Cloud9,
> exoIDE, codemirror, etc..), we would provide a view on the code inside the
> XWiki instance.
> Code would be organized in the same way as in the Eclipse plugin and
> appropriate editors would be used depending on the code type.
> Completion could be provided in the velocity and groovy editors and
> eventually in Javascript
>
> Two views should be available, one for each AWM application, and one with
> all the code in the Wiki.
>
> Live syntax checking could be provided.
>
> Solution 1/ is very powerful because it let's the user decide which is his
> development environment, even if the tree structure won't be perfect. Still
> he can see all the code in the wiki and work on it, including searching.
> Committing is made very easy. There are some risks involved in the sync
> process, particularly of you have multiple developers syncing local code
> with the same dev server.
> User can switch from local sync (local wiki) to remove sync (remove shared
> wiki) and can therefore work offline more easily. Editing can be fully done
> offline.
>
> Providing syntax highlighting for many editors might prove difficult.
>
> Solution 2/ is providing a nice XWiki environment inside Eclipse, without
> the need for a local copy of the code. Committing would happen using the
> browser.
>
> Solution 3/ is the long term bet as the future is to have everything in the
> web. Content is only in one place which makes things a little easier.
> Development environment needs no setup.
> However this is more "new" for developers which need to adopt the platform.
>
> Live syntax checking is hard to provide but would be quite useful.
> Alternatively mvn xar:format could also provide syntax checking for XWiki
> syntax, velocity, groovy, js and css.
>
> WDYT ? Which approaches do you believe would be the most promising.

Various comments:

* I've always been a big fan of XEclipse but it seems I've been the only one. Whenever I've proposed to work on it and proposed ideas about it I was told that it's not the future and the future is developing in the IDE using javascript. While this is probably true, I don't see it happening with the same ease as your current standalone IDE before 5 years at least.

* Your point 1) about exposing the XWiki model on the file system so that any IDE can use it is what I've been working on for a long time. (albeit very slowly). This is one of the reason I'm working on the new model, so that I can develop a file-based implementation. Note that I've just started developing one using Git as the backend in a branch and I've defined a way to map the XWiki model to a filesystem. I'll create a page on the Design space of dev.xwiki.org with what I have so far so that we can discuss it.

* Existing WebIDE solutions are too early IMO. BTW exoIDE is a no go since it handles the storage too using JCR and you cannot use it as a library. It would be good to have someone research the state of open source WebIDE (with compatible licenses) hat can be used a libraries but I fear that we won't find much.

So I see 3 steps:
* 1) Finish the syntax highlighting and autocompletion extension and publish it. Then include it in the platform if other devs agree
* 2) Build on top of XEclipse to bring it to the next level
* 3) In // we continue working on the new model

Now you point 1) is interesting but is not easy to implement. Basically it means implementing a VFS (Virtual File System) for the IDE. All IDEs have a VFS so it's just about having an implementation that takes its data from a remote XWiki server using REST but it might be slow and will need a lot of caching and thus sync,  which we need anyway for offline editing features (and is currently implemented in XEclipse). Thus the XEclpse way is the easiest for short term progress on this IMO.

I also think XEclipse could be seen as an admin center (ie. perform admin tasks on several XWiki instances, monitor its health, etc), but that's another topic.

Thanks
-Vincent

> Ludovic

_______________________________________________
devs mailing list
[hidden email]
http://lists.xwiki.org/mailman/listinfo/devs
Reply | Threaded
Open this post in threaded view
|

Re: Improving the XWiki Development environment for traditional Java/JS Developers

Caleb James DeLisle
In reply to this post by Ludovic Dubost
Making things easy for developers is quite interesting to me because I'm
a developer and I like scratching my own itches.

more inline:

On 06/29/2013 05:41 AM, Ludovic Dubost wrote:

> While XWiki has many great advantages as a framework which takes car of
> persistence, forms, user management, content management and provides tons
> of APIs, when traditional developers want to extend XWiki there are facing
> a few difficulties:
>
> - they are lost in the myriad of XWiki APIs, and there is no completion
> - they don't get good visibility of the code available in their application
> - it is complicated to use editors which have syntax highlighting
> - they cannot use their favorite IDE
>
> On the framework side there are also some improvements which could make
> XWiki even more killer:
>
> - easier integration of advanced JS framework
> - advanced integration of a high level JS framework that has templating
> (maybe angular js)
> - better validation functions
> - easier way to add REST APIs
> - more XWiki and better documented javascript apis
>
>
> Here are some proposals to help fix the tools issues. Three approaches can
> be looked at:
>
> 1/ Live Sync between an XWiki Instance and and improved maven xar file
> structure, allowing to use any local IDE on XWiki code
>
> First it should be possible to use any IDE on the maven xar file structure,
> allowing to open the content and textarea fields of all XML files.
> For this XWiki XML files should externalize the content and textarea fields
> in separate files with extensions based on their content type.
> The maven xar format should be able to clean the maven structure to do this
> splitting and should also be able to build the XAR from all the files.
>
> Finally a program should allow to do a live sync between a local or remote
> wiki instance and the maven project. Any change in either the wiki or the
> file system should be replicated to the other party.
> So if you run "git pull" then your local or remote wiki would be updated.
> If a change is made in the wiki, the change would show up in the file
> system and your IDE would refresh it.
>
> The sync program would keep a state of the version in the wiki, in order to
> be able to merge changes if changes occur in both places between two sync.
> This tool could be easily launched with "mvn xar:sync"
>
> Syntax highlighting for XWiki Syntax+Velocity+Groovy would be developped
> for the most popular editors.
> When syncing the tool could show syntax checking error messages.


I actually implemented part of this for my own personal toolkit.
I presented it on the list to great fanfair here:
http://xwiki.475771.n2.nabble.com/A-new-XWiki-Model-sort-of-td7585722.html

I have the xar generation and push to XWiki in a 4 or 5 second post operation.
The format is more makefile-esque so it's not really good for live sync but
I have been using it and have not yet thrown it across the room which is a
good indication that it's probably technologically sound.


>
> 2/ Integration with Eclipse
>
> Based on XEclipse, we would build an Eclipse plugin to be able to connect
> to an XWiki instance and load a specified list of spaces. Then each space
> would be organized by the type of code in this space. Content and Textarea
> fields would be made visible as editable content in Eclipse.
> The plugin should detect the type of code in each of the content or
> textarea fields (velocity/html, groovy, javascript, css, translations,
> xwiki syntax) and use the appropriate editor.
>
> Finally a completion module could be provided by loading from the server a
> list of available APIs.
>
> The same plugin could also be able to organize the content from a local
> maven project (based on 1/) and provide completion to such a project.
>
> Live syntax checking could be provided.
>
> 3/ Integration of a WebIDE in XWiki
>
> Based on Javascript WebIDE and web code editor softwares (Orion, Cloud9,
> exoIDE, codemirror, etc..), we would provide a view on the code inside the
> XWiki instance.
> Code would be organized in the same way as in the Eclipse plugin and
> appropriate editors would be used depending on the code type.
> Completion could be provided in the velocity and groovy editors and
> eventually in Javascript
>
> Two views should be available, one for each AWM application, and one with
> all the code in the Wiki.
>
> Live syntax checking could be provided.


If we decide this is a priority, it's something which I could investigate
and would be able to reuse experience and tools from the Resilience project.


>
> Solution 1/ is very powerful because it let's the user decide which is his
> development environment, even if the tree structure won't be perfect. Still
> he can see all the code in the wiki and work on it, including searching.
> Committing is made very easy. There are some risks involved in the sync
> process, particularly of you have multiple developers syncing local code
> with the same dev server.
> User can switch from local sync (local wiki) to remove sync (remove shared
> wiki) and can therefore work offline more easily. Editing can be fully done
> offline.
>
> Providing syntax highlighting for many editors might prove difficult.
>
> Solution 2/ is providing a nice XWiki environment inside Eclipse, without
> the need for a local copy of the code. Committing would happen using the
> browser.
>
> Solution 3/ is the long term bet as the future is to have everything in the
> web. Content is only in one place which makes things a little easier.
> Development environment needs no setup.
> However this is more "new" for developers which need to adopt the platform.
>
> Live syntax checking is hard to provide but would be quite useful.
> Alternatively mvn xar:format could also provide syntax checking for XWiki
> syntax, velocity, groovy, js and css.
>
> WDYT ? Which approaches do you believe would be the most promising.


#1 can be many things. It could be as simple as my little .xar generator
with ability to post to a live wiki. It could be as complex as an XWiki
filesystem which you'd mount with nfs or something.
If we commit to a directory structure, it is going to require a lot of
thought to make that structure coherent and once we commit to it, we're
kind of stuck with it.

#3 is easy at first then it gets hard. We can pull in an editor and if
we find an editor which supports ctags then we get completion for a
reasonable price. The problem is ctags (and most editors) don't support
velocity so if we want to autocomplete velocity we end up hacking
something together and making a few demos then it's forgotten, or worse
we write everything in-house to really support production ready velocity
completion and then we're saddled with maintaining another 15k LoC that
no other project has any interest in.
http://ctags.sourceforge.net/languages.html



If what we want is extensions then we should go after extensions directly.
The fastest growing extension repository is npm, while it's smaller than
maven or rubygems, it's getting new packages at almost twice the rate of
maven, the next competitor. My recommendation is to move in the
#1 direction but specifically targeting XWiki extensions by copying the
npm workflow.


Thanks,
Caleb

>
> Ludovic
>

_______________________________________________
devs mailing list
[hidden email]
http://lists.xwiki.org/mailman/listinfo/devs
Reply | Threaded
Open this post in threaded view
|

Re: Improving the XWiki Development environment for traditional Java/JS Developers

Caleb James DeLisle
oops "The fastest growing extension repository is npm, while it's smaller than
maven or rubygems, it's getting new packages at almost twice the rate of
maven, the next competitor" [citation needed]

http://modulecounts.com/


Thanks,
Caleb


On 06/29/2013 04:41 PM, Caleb James DeLisle wrote:

> Making things easy for developers is quite interesting to me because I'm
> a developer and I like scratching my own itches.
>
> more inline:
>
> On 06/29/2013 05:41 AM, Ludovic Dubost wrote:
>> While XWiki has many great advantages as a framework which takes car of
>> persistence, forms, user management, content management and provides tons
>> of APIs, when traditional developers want to extend XWiki there are facing
>> a few difficulties:
>>
>> - they are lost in the myriad of XWiki APIs, and there is no completion
>> - they don't get good visibility of the code available in their application
>> - it is complicated to use editors which have syntax highlighting
>> - they cannot use their favorite IDE
>>
>> On the framework side there are also some improvements which could make
>> XWiki even more killer:
>>
>> - easier integration of advanced JS framework
>> - advanced integration of a high level JS framework that has templating
>> (maybe angular js)
>> - better validation functions
>> - easier way to add REST APIs
>> - more XWiki and better documented javascript apis
>>
>>
>> Here are some proposals to help fix the tools issues. Three approaches can
>> be looked at:
>>
>> 1/ Live Sync between an XWiki Instance and and improved maven xar file
>> structure, allowing to use any local IDE on XWiki code
>>
>> First it should be possible to use any IDE on the maven xar file structure,
>> allowing to open the content and textarea fields of all XML files.
>> For this XWiki XML files should externalize the content and textarea fields
>> in separate files with extensions based on their content type.
>> The maven xar format should be able to clean the maven structure to do this
>> splitting and should also be able to build the XAR from all the files.
>>
>> Finally a program should allow to do a live sync between a local or remote
>> wiki instance and the maven project. Any change in either the wiki or the
>> file system should be replicated to the other party.
>> So if you run "git pull" then your local or remote wiki would be updated.
>> If a change is made in the wiki, the change would show up in the file
>> system and your IDE would refresh it.
>>
>> The sync program would keep a state of the version in the wiki, in order to
>> be able to merge changes if changes occur in both places between two sync.
>> This tool could be easily launched with "mvn xar:sync"
>>
>> Syntax highlighting for XWiki Syntax+Velocity+Groovy would be developped
>> for the most popular editors.
>> When syncing the tool could show syntax checking error messages.
>
>
> I actually implemented part of this for my own personal toolkit.
> I presented it on the list to great fanfair here:
> http://xwiki.475771.n2.nabble.com/A-new-XWiki-Model-sort-of-td7585722.html
>
> I have the xar generation and push to XWiki in a 4 or 5 second post operation.
> The format is more makefile-esque so it's not really good for live sync but
> I have been using it and have not yet thrown it across the room which is a
> good indication that it's probably technologically sound.
>
>
>>
>> 2/ Integration with Eclipse
>>
>> Based on XEclipse, we would build an Eclipse plugin to be able to connect
>> to an XWiki instance and load a specified list of spaces. Then each space
>> would be organized by the type of code in this space. Content and Textarea
>> fields would be made visible as editable content in Eclipse.
>> The plugin should detect the type of code in each of the content or
>> textarea fields (velocity/html, groovy, javascript, css, translations,
>> xwiki syntax) and use the appropriate editor.
>>
>> Finally a completion module could be provided by loading from the server a
>> list of available APIs.
>>
>> The same plugin could also be able to organize the content from a local
>> maven project (based on 1/) and provide completion to such a project.
>>
>> Live syntax checking could be provided.
>>
>> 3/ Integration of a WebIDE in XWiki
>>
>> Based on Javascript WebIDE and web code editor softwares (Orion, Cloud9,
>> exoIDE, codemirror, etc..), we would provide a view on the code inside the
>> XWiki instance.
>> Code would be organized in the same way as in the Eclipse plugin and
>> appropriate editors would be used depending on the code type.
>> Completion could be provided in the velocity and groovy editors and
>> eventually in Javascript
>>
>> Two views should be available, one for each AWM application, and one with
>> all the code in the Wiki.
>>
>> Live syntax checking could be provided.
>
>
> If we decide this is a priority, it's something which I could investigate
> and would be able to reuse experience and tools from the Resilience project.
>
>
>>
>> Solution 1/ is very powerful because it let's the user decide which is his
>> development environment, even if the tree structure won't be perfect. Still
>> he can see all the code in the wiki and work on it, including searching.
>> Committing is made very easy. There are some risks involved in the sync
>> process, particularly of you have multiple developers syncing local code
>> with the same dev server.
>> User can switch from local sync (local wiki) to remove sync (remove shared
>> wiki) and can therefore work offline more easily. Editing can be fully done
>> offline.
>>
>> Providing syntax highlighting for many editors might prove difficult.
>>
>> Solution 2/ is providing a nice XWiki environment inside Eclipse, without
>> the need for a local copy of the code. Committing would happen using the
>> browser.
>>
>> Solution 3/ is the long term bet as the future is to have everything in the
>> web. Content is only in one place which makes things a little easier.
>> Development environment needs no setup.
>> However this is more "new" for developers which need to adopt the platform.
>>
>> Live syntax checking is hard to provide but would be quite useful.
>> Alternatively mvn xar:format could also provide syntax checking for XWiki
>> syntax, velocity, groovy, js and css.
>>
>> WDYT ? Which approaches do you believe would be the most promising.
>
>
> #1 can be many things. It could be as simple as my little .xar generator
> with ability to post to a live wiki. It could be as complex as an XWiki
> filesystem which you'd mount with nfs or something.
> If we commit to a directory structure, it is going to require a lot of
> thought to make that structure coherent and once we commit to it, we're
> kind of stuck with it.
>
> #3 is easy at first then it gets hard. We can pull in an editor and if
> we find an editor which supports ctags then we get completion for a
> reasonable price. The problem is ctags (and most editors) don't support
> velocity so if we want to autocomplete velocity we end up hacking
> something together and making a few demos then it's forgotten, or worse
> we write everything in-house to really support production ready velocity
> completion and then we're saddled with maintaining another 15k LoC that
> no other project has any interest in.
> http://ctags.sourceforge.net/languages.html
>
>
>
> If what we want is extensions then we should go after extensions directly.
> The fastest growing extension repository is npm, while it's smaller than
> maven or rubygems, it's getting new packages at almost twice the rate of
> maven, the next competitor. My recommendation is to move in the
> #1 direction but specifically targeting XWiki extensions by copying the
> npm workflow.
>
>
> Thanks,
> Caleb
>
>>
>> Ludovic
>>
>
> _______________________________________________
> devs mailing list
> [hidden email]
> http://lists.xwiki.org/mailman/listinfo/devs
>

_______________________________________________
devs mailing list
[hidden email]
http://lists.xwiki.org/mailman/listinfo/devs
Reply | Threaded
Open this post in threaded view
|

Re: Improving the XWiki Development environment for traditional Java/JS Developers

Ludovic Dubost
In reply to this post by Caleb James DeLisle
2013/6/29 Caleb James DeLisle <[hidden email]>

> Making things easy for developers is quite interesting to me because I'm
> a developer and I like scratching my own itches.
>
> more inline:
>
> On 06/29/2013 05:41 AM, Ludovic Dubost wrote:
> > While XWiki has many great advantages as a framework which takes car of
> > persistence, forms, user management, content management and provides tons
> > of APIs, when traditional developers want to extend XWiki there are
> facing
> > a few difficulties:
> >
> > - they are lost in the myriad of XWiki APIs, and there is no completion
> > - they don't get good visibility of the code available in their
> application
> > - it is complicated to use editors which have syntax highlighting
> > - they cannot use their favorite IDE
> >
> > On the framework side there are also some improvements which could make
> > XWiki even more killer:
> >
> > - easier integration of advanced JS framework
> > - advanced integration of a high level JS framework that has templating
> > (maybe angular js)
> > - better validation functions
> > - easier way to add REST APIs
> > - more XWiki and better documented javascript apis
> >
> >
> > Here are some proposals to help fix the tools issues. Three approaches
> can
> > be looked at:
> >
> > 1/ Live Sync between an XWiki Instance and and improved maven xar file
> > structure, allowing to use any local IDE on XWiki code
> >
> > First it should be possible to use any IDE on the maven xar file
> structure,
> > allowing to open the content and textarea fields of all XML files.
> > For this XWiki XML files should externalize the content and textarea
> fields
> > in separate files with extensions based on their content type.
> > The maven xar format should be able to clean the maven structure to do
> this
> > splitting and should also be able to build the XAR from all the files.
> >
> > Finally a program should allow to do a live sync between a local or
> remote
> > wiki instance and the maven project. Any change in either the wiki or the
> > file system should be replicated to the other party.
> > So if you run "git pull" then your local or remote wiki would be updated.
> > If a change is made in the wiki, the change would show up in the file
> > system and your IDE would refresh it.
> >
> > The sync program would keep a state of the version in the wiki, in order
> to
> > be able to merge changes if changes occur in both places between two
> sync.
> > This tool could be easily launched with "mvn xar:sync"
> >
> > Syntax highlighting for XWiki Syntax+Velocity+Groovy would be developped
> > for the most popular editors.
> > When syncing the tool could show syntax checking error messages.
>
>
> I actually implemented part of this for my own personal toolkit.
> I presented it on the list to great fanfair here:
> http://xwiki.475771.n2.nabble.com/A-new-XWiki-Model-sort-of-td7585722.html
>
> I have the xar generation and push to XWiki in a 4 or 5 second post
> operation.
> The format is more makefile-esque so it's not really good for live sync but
> I have been using it and have not yet thrown it across the room which is a
> good indication that it's probably technologically sound.
>

Yes this is similar. However your implementation is in Javascript and we
need an implementation based on the same code as xar:format as we would
also need to read from the wiki to the file system and make sure that the
xml files are properly formatted. We need xar:format compatible with this.

>
> >
> > 2/ Integration with Eclipse
> >
> > Based on XEclipse, we would build an Eclipse plugin to be able to connect
> > to an XWiki instance and load a specified list of spaces. Then each space
> > would be organized by the type of code in this space. Content and
> Textarea
> > fields would be made visible as editable content in Eclipse.
> > The plugin should detect the type of code in each of the content or
> > textarea fields (velocity/html, groovy, javascript, css, translations,
> > xwiki syntax) and use the appropriate editor.
> >
> > Finally a completion module could be provided by loading from the server
> a
> > list of available APIs.
> >
> > The same plugin could also be able to organize the content from a local
> > maven project (based on 1/) and provide completion to such a project.
> >
> > Live syntax checking could be provided.
> >
> > 3/ Integration of a WebIDE in XWiki
> >
> > Based on Javascript WebIDE and web code editor softwares (Orion, Cloud9,
> > exoIDE, codemirror, etc..), we would provide a view on the code inside
> the
> > XWiki instance.
> > Code would be organized in the same way as in the Eclipse plugin and
> > appropriate editors would be used depending on the code type.
> > Completion could be provided in the velocity and groovy editors and
> > eventually in Javascript
> >
> > Two views should be available, one for each AWM application, and one with
> > all the code in the Wiki.
> >
> > Live syntax checking could be provided.
>
>
> If we decide this is a priority, it's something which I could investigate
> and would be able to reuse experience and tools from the Resilience
> project.
>
>
> >
> > Solution 1/ is very powerful because it let's the user decide which is
> his
> > development environment, even if the tree structure won't be perfect.
> Still
> > he can see all the code in the wiki and work on it, including searching.
> > Committing is made very easy. There are some risks involved in the sync
> > process, particularly of you have multiple developers syncing local code
> > with the same dev server.
> > User can switch from local sync (local wiki) to remove sync (remove
> shared
> > wiki) and can therefore work offline more easily. Editing can be fully
> done
> > offline.
> >
> > Providing syntax highlighting for many editors might prove difficult.
> >
> > Solution 2/ is providing a nice XWiki environment inside Eclipse, without
> > the need for a local copy of the code. Committing would happen using the
> > browser.
> >
> > Solution 3/ is the long term bet as the future is to have everything in
> the
> > web. Content is only in one place which makes things a little easier.
> > Development environment needs no setup.
> > However this is more "new" for developers which need to adopt the
> platform.
> >
> > Live syntax checking is hard to provide but would be quite useful.
> > Alternatively mvn xar:format could also provide syntax checking for XWiki
> > syntax, velocity, groovy, js and css.
> >
> > WDYT ? Which approaches do you believe would be the most promising.
>
>
> #1 can be many things. It could be as simple as my little .xar generator
> with ability to post to a live wiki. It could be as complex as an XWiki
> filesystem which you'd mount with nfs or something.
> If we commit to a directory structure, it is going to require a lot of
> thought to make that structure coherent and once we commit to it, we're
> kind of stuck with it.
>

I believe we would need to stick with what the maven xar plugin will like
to be able to build, but at the same time looks nice enough in an IDE.

>
> #3 is easy at first then it gets hard. We can pull in an editor and if
> we find an editor which supports ctags then we get completion for a
> reasonable price. The problem is ctags (and most editors) don't support
> velocity so if we want to autocomplete velocity we end up hacking
> something together and making a few demos then it's forgotten, or worse
> we write everything in-house to really support production ready velocity
> completion and then we're saddled with maintaining another 15k LoC that
> no other project has any interest in.
> http://ctags.sourceforge.net/languages.html
>
>
>
Well we don't have much choice here. Velocity is not that well supported.
We also have XWiki Syntax which can have subparts using velocity, groovy or
anything else.
We will need to do some work here.

We could simplify if we also extracted the content of some macros into
external files, like the velocity and groovy macros. But still velocity is
velocity + html + potentially XWiki syntax, so we need some custom handling
here.


>
> If what we want is extensions then we should go after extensions directly.
> The fastest growing extension repository is npm, while it's smaller than
> maven or rubygems, it's getting new packages at almost twice the rate of
> maven, the next competitor. My recommendation is to move in the
> #1 direction but specifically targeting XWiki extensions by copying the
> npm workflow.
>

We have a package manager. Now we need to make it easier to create the
package from code you have in XWiki.
But this is not fully related to the development tool which is something
else. Right now you can create a XAR from xwiki in just a few clicks and
commit it to GitHub (including the pom creation), using the GitHubApp. But
this does not solve the IDE issue which is something else. However it's
true that if your get your code in the file system we need to give a quick
and easy way to create the pom.

My purpose right now is not about the extension creation workflow. It's
more about getting traditional developers to like the XWiki dev
environment. I had multiple discussions with developers and while
developers like a lot everything XWiki provides to speed up development
from a framework perspective, they get cooled off by the developping in the
browser concept. This is less true when the objective is an Intranet app on
an existing instance. This is more true of web-agency style development or
larger development projects.

I firmly believe the future is in a WebIDE, but it will take time to
takeover all developers. I like the #1 approach because it's not that
complicated. It's the GitHubApp the way around with your file system with
the "live" aspect. I've started some java code and found some nice library
to watch changes in a directory. I'm already close to have an initial
comparison of a maven xar project with a remote instance. The bigger work
is to define how we split the code files (attachments, content fields,
textarea fields and get maven xar plugin to support this). And once we have
code like that you can use any IDE which is quite cool.

Ludovic


>
> Thanks,
> Caleb
>
> >
> > Ludovic
> >
>
> _______________________________________________
> devs mailing list
> [hidden email]
> http://lists.xwiki.org/mailman/listinfo/devs
>



--
Ludovic Dubost
Founder and CEO
Blog: http://blog.ludovic.org/
XWiki: http://www.xwiki.com
Skype: ldubost GTalk: ldubost
_______________________________________________
devs mailing list
[hidden email]
http://lists.xwiki.org/mailman/listinfo/devs
Reply | Threaded
Open this post in threaded view
|

Re: Improving the XWiki Development environment for traditional Java/JS Developers

Ludovic Dubost
In reply to this post by Ludovic Dubost
I've started some prototype code for 1/ at

https://github.com/ldubost/xwiki-projectsync/blob/master/src/main/java/org/xwiki/contrib/projectsync/ProjectSync.java

Currently if you point it to a maven xar module with the command line

-format /path/to/root/dir/of/project/

it will create some files for each content, textarea or attachments inside
an XML file

- Some code is also done to compare a wiki instance with that repository
(no command line yet to launch it)
- Some code allows to monitor changes in a directory (but nothing yet
happens)

It allows to see what it could mean to have the xml files spread in
multiple code files.
One big step is to decide if this is something we want because it is a
prerequesit for such a tool.

Ludovic






2013/6/29 Ludovic Dubost <[hidden email]>

>
> While XWiki has many great advantages as a framework which takes car of
> persistence, forms, user management, content management and provides tons
> of APIs, when traditional developers want to extend XWiki there are facing
> a few difficulties:
>
> - they are lost in the myriad of XWiki APIs, and there is no completion
> - they don't get good visibility of the code available in their application
> - it is complicated to use editors which have syntax highlighting
> - they cannot use their favorite IDE
>
> On the framework side there are also some improvements which could make
> XWiki even more killer:
>
> - easier integration of advanced JS framework
> - advanced integration of a high level JS framework that has templating
> (maybe angular js)
> - better validation functions
> - easier way to add REST APIs
> - more XWiki and better documented javascript apis
>
>
> Here are some proposals to help fix the tools issues. Three approaches can
> be looked at:
>
> 1/ Live Sync between an XWiki Instance and and improved maven xar file
> structure, allowing to use any local IDE on XWiki code
>
> First it should be possible to use any IDE on the maven xar file
> structure, allowing to open the content and textarea fields of all XML
> files.
> For this XWiki XML files should externalize the content and textarea
> fields in separate files with extensions based on their content type.
> The maven xar format should be able to clean the maven structure to do
> this splitting and should also be able to build the XAR from all the files.
>
> Finally a program should allow to do a live sync between a local or remote
> wiki instance and the maven project. Any change in either the wiki or the
> file system should be replicated to the other party.
> So if you run "git pull" then your local or remote wiki would be updated.
> If a change is made in the wiki, the change would show up in the file
> system and your IDE would refresh it.
>
> The sync program would keep a state of the version in the wiki, in order
> to be able to merge changes if changes occur in both places between two
> sync.
> This tool could be easily launched with "mvn xar:sync"
>
> Syntax highlighting for XWiki Syntax+Velocity+Groovy would be developped
> for the most popular editors.
> When syncing the tool could show syntax checking error messages.
>
> 2/ Integration with Eclipse
>
> Based on XEclipse, we would build an Eclipse plugin to be able to connect
> to an XWiki instance and load a specified list of spaces. Then each space
> would be organized by the type of code in this space. Content and Textarea
> fields would be made visible as editable content in Eclipse.
> The plugin should detect the type of code in each of the content or
> textarea fields (velocity/html, groovy, javascript, css, translations,
> xwiki syntax) and use the appropriate editor.
>
> Finally a completion module could be provided by loading from the server a
> list of available APIs.
>
> The same plugin could also be able to organize the content from a local
> maven project (based on 1/) and provide completion to such a project.
>
> Live syntax checking could be provided.
>
> 3/ Integration of a WebIDE in XWiki
>
> Based on Javascript WebIDE and web code editor softwares (Orion, Cloud9,
> exoIDE, codemirror, etc..), we would provide a view on the code inside the
> XWiki instance.
> Code would be organized in the same way as in the Eclipse plugin and
> appropriate editors would be used depending on the code type.
> Completion could be provided in the velocity and groovy editors and
> eventually in Javascript
>
> Two views should be available, one for each AWM application, and one with
> all the code in the Wiki.
>
> Live syntax checking could be provided.
>
> Solution 1/ is very powerful because it let's the user decide which is his
> development environment, even if the tree structure won't be perfect. Still
> he can see all the code in the wiki and work on it, including searching.
> Committing is made very easy. There are some risks involved in the sync
> process, particularly of you have multiple developers syncing local code
> with the same dev server.
> User can switch from local sync (local wiki) to remove sync (remove shared
> wiki) and can therefore work offline more easily. Editing can be fully done
> offline.
>
> Providing syntax highlighting for many editors might prove difficult.
>
> Solution 2/ is providing a nice XWiki environment inside Eclipse, without
> the need for a local copy of the code. Committing would happen using the
> browser.
>
> Solution 3/ is the long term bet as the future is to have everything in
> the web. Content is only in one place which makes things a little easier.
> Development environment needs no setup.
> However this is more "new" for developers which need to adopt the
> platform.
>
> Live syntax checking is hard to provide but would be quite useful.
> Alternatively mvn xar:format could also provide syntax checking for XWiki
> syntax, velocity, groovy, js and css.
>
> WDYT ? Which approaches do you believe would be the most promising.
>
> Ludovic
>
> --
> Ludovic Dubost
> Founder and CEO
> Blog: http://blog.ludovic.org/
> XWiki: http://www.xwiki.com
> Skype: ldubost GTalk: ldubost
>



--
Ludovic Dubost
Founder and CEO
Blog: http://blog.ludovic.org/
XWiki: http://www.xwiki.com
Skype: ldubost GTalk: ldubost
_______________________________________________
devs mailing list
[hidden email]
http://lists.xwiki.org/mailman/listinfo/devs
Reply | Threaded
Open this post in threaded view
|

Re: Improving the XWiki Development environment for traditional Java/JS Developers

Ludovic Dubost
Here is the result of the filemanager application project opened in Eclipse
after running the tool to extract the files:

http://uploadpie.com/tFaIF

Ludovic


2013/7/1 Ludovic Dubost <[hidden email]>

> I've started some prototype code for 1/ at
>
>
> https://github.com/ldubost/xwiki-projectsync/blob/master/src/main/java/org/xwiki/contrib/projectsync/ProjectSync.java
>
> Currently if you point it to a maven xar module with the command line
>
> -format /path/to/root/dir/of/project/
>
> it will create some files for each content, textarea or attachments inside
> an XML file
>
> - Some code is also done to compare a wiki instance with that repository
> (no command line yet to launch it)
> - Some code allows to monitor changes in a directory (but nothing yet
> happens)
>
> It allows to see what it could mean to have the xml files spread in
> multiple code files.
> One big step is to decide if this is something we want because it is a
> prerequesit for such a tool.
>
> Ludovic
>
>
>
>
>
>
> 2013/6/29 Ludovic Dubost <[hidden email]>
>
>>
>> While XWiki has many great advantages as a framework which takes car of
>> persistence, forms, user management, content management and provides tons
>> of APIs, when traditional developers want to extend XWiki there are facing
>> a few difficulties:
>>
>> - they are lost in the myriad of XWiki APIs, and there is no completion
>> - they don't get good visibility of the code available in their
>> application
>> - it is complicated to use editors which have syntax highlighting
>> - they cannot use their favorite IDE
>>
>> On the framework side there are also some improvements which could make
>> XWiki even more killer:
>>
>> - easier integration of advanced JS framework
>> - advanced integration of a high level JS framework that has templating
>> (maybe angular js)
>> - better validation functions
>> - easier way to add REST APIs
>> - more XWiki and better documented javascript apis
>>
>>
>> Here are some proposals to help fix the tools issues. Three approaches
>> can be looked at:
>>
>> 1/ Live Sync between an XWiki Instance and and improved maven xar file
>> structure, allowing to use any local IDE on XWiki code
>>
>> First it should be possible to use any IDE on the maven xar file
>> structure, allowing to open the content and textarea fields of all XML
>> files.
>> For this XWiki XML files should externalize the content and textarea
>> fields in separate files with extensions based on their content type.
>> The maven xar format should be able to clean the maven structure to do
>> this splitting and should also be able to build the XAR from all the files.
>>
>> Finally a program should allow to do a live sync between a local or
>> remote wiki instance and the maven project. Any change in either the wiki
>> or the file system should be replicated to the other party.
>> So if you run "git pull" then your local or remote wiki would be updated.
>> If a change is made in the wiki, the change would show up in the file
>> system and your IDE would refresh it.
>>
>> The sync program would keep a state of the version in the wiki, in order
>> to be able to merge changes if changes occur in both places between two
>> sync.
>> This tool could be easily launched with "mvn xar:sync"
>>
>> Syntax highlighting for XWiki Syntax+Velocity+Groovy would be developped
>> for the most popular editors.
>> When syncing the tool could show syntax checking error messages.
>>
>> 2/ Integration with Eclipse
>>
>> Based on XEclipse, we would build an Eclipse plugin to be able to connect
>> to an XWiki instance and load a specified list of spaces. Then each space
>> would be organized by the type of code in this space. Content and Textarea
>> fields would be made visible as editable content in Eclipse.
>> The plugin should detect the type of code in each of the content or
>> textarea fields (velocity/html, groovy, javascript, css, translations,
>> xwiki syntax) and use the appropriate editor.
>>
>> Finally a completion module could be provided by loading from the server
>> a list of available APIs.
>>
>> The same plugin could also be able to organize the content from a local
>> maven project (based on 1/) and provide completion to such a project.
>>
>> Live syntax checking could be provided.
>>
>> 3/ Integration of a WebIDE in XWiki
>>
>> Based on Javascript WebIDE and web code editor softwares (Orion, Cloud9,
>> exoIDE, codemirror, etc..), we would provide a view on the code inside the
>> XWiki instance.
>> Code would be organized in the same way as in the Eclipse plugin and
>> appropriate editors would be used depending on the code type.
>> Completion could be provided in the velocity and groovy editors and
>> eventually in Javascript
>>
>> Two views should be available, one for each AWM application, and one with
>> all the code in the Wiki.
>>
>> Live syntax checking could be provided.
>>
>> Solution 1/ is very powerful because it let's the user decide which is
>> his development environment, even if the tree structure won't be perfect.
>> Still he can see all the code in the wiki and work on it, including
>> searching. Committing is made very easy. There are some risks involved in
>> the sync process, particularly of you have multiple developers syncing
>> local code with the same dev server.
>> User can switch from local sync (local wiki) to remove sync (remove
>> shared wiki) and can therefore work offline more easily. Editing can be
>> fully done offline.
>>
>> Providing syntax highlighting for many editors might prove difficult.
>>
>> Solution 2/ is providing a nice XWiki environment inside Eclipse, without
>> the need for a local copy of the code. Committing would happen using the
>> browser.
>>
>> Solution 3/ is the long term bet as the future is to have everything in
>> the web. Content is only in one place which makes things a little easier.
>> Development environment needs no setup.
>> However this is more "new" for developers which need to adopt the
>> platform.
>>
>> Live syntax checking is hard to provide but would be quite useful.
>> Alternatively mvn xar:format could also provide syntax checking for XWiki
>> syntax, velocity, groovy, js and css.
>>
>> WDYT ? Which approaches do you believe would be the most promising.
>>
>> Ludovic
>>
>> --
>> Ludovic Dubost
>> Founder and CEO
>> Blog: http://blog.ludovic.org/
>> XWiki: http://www.xwiki.com
>> Skype: ldubost GTalk: ldubost
>>
>
>
>
> --
> Ludovic Dubost
> Founder and CEO
> Blog: http://blog.ludovic.org/
> XWiki: http://www.xwiki.com
> Skype: ldubost GTalk: ldubost
>



--
Ludovic Dubost
Founder and CEO
Blog: http://blog.ludovic.org/
XWiki: http://www.xwiki.com
Skype: ldubost GTalk: ldubost
_______________________________________________
devs mailing list
[hidden email]
http://lists.xwiki.org/mailman/listinfo/devs
Reply | Threaded
Open this post in threaded view
|

Re: Improving the XWiki Development environment for traditional Java/JS Developers

Ludovic Dubost
In reply to this post by vmassol
2013/6/29 Vincent Massol <[hidden email]>

> Hi ludovic,
>
> All good points that I think all of us would like to see addressed as well.
>
> Replies inline.
>
> On Jun 29, 2013, at 11:41 AM, Ludovic Dubost <[hidden email]> wrote:
>
> > While XWiki has many great advantages as a framework which takes car of
> > persistence, forms, user management, content management and provides tons
> > of APIs, when traditional developers want to extend XWiki there are
> facing
> > a few difficulties:
> >
> > - they are lost in the myriad of XWiki APIs, and there is no completion
>
> Note that Edy and myself created an autcompletion extension last year but
> we didn't get to publish it on e.x.o (it's here ATM:
> https://github.com/xwiki-contrib/wiki-editor-devtools).
>

Cool. I did not know it had been published.


>
> I'm planning to continue and finish it this year during the coming
> hackathon in July.
>

Yes this one is very important as it would help a lot newcomers.
Note: the scripting reference documentation is usually very appreciated,
but is not known to developers. In the same way, little developer know that
they can install some extension to edit textarea fields in an outside
editor (we should document them somewhere)


> > - they don't get good visibility of the code available in their
> application
> > - it is complicated to use editors which have syntax highlighting
> > - they cannot use their favorite IDE
> >
> > On the framework side there are also some improvements which could make
> > XWiki even more killer:
> >
> > - easier integration of advanced JS framework
> > - advanced integration of a high level JS framework that has templating
> > (maybe angular js)
> > - better validation functions
> > - easier way to add REST APIs
> > - more XWiki and better documented javascript apis
> >
> >
> > Here are some proposals to help fix the tools issues. Three approaches
> can
> > be looked at:
> >
> > 1/ Live Sync between an XWiki Instance and and improved maven xar file
> > structure, allowing to use any local IDE on XWiki code
> >
> > First it should be possible to use any IDE on the maven xar file
> structure,
> > allowing to open the content and textarea fields of all XML files.
> > For this XWiki XML files should externalize the content and textarea
> fields
> > in separate files with extensions based on their content type.
> > The maven xar format should be able to clean the maven structure to do
> this
> > splitting and should also be able to build the XAR from all the files.
> >
> > Finally a program should allow to do a live sync between a local or
> remote
> > wiki instance and the maven project. Any change in either the wiki or the
> > file system should be replicated to the other party.
> > So if you run "git pull" then your local or remote wiki would be updated.
> > If a change is made in the wiki, the change would show up in the file
> > system and your IDE would refresh it.
> >
> > The sync program would keep a state of the version in the wiki, in order
> to
> > be able to merge changes if changes occur in both places between two
> sync.
> > This tool could be easily launched with "mvn xar:sync"
> >
> > Syntax highlighting for XWiki Syntax+Velocity+Groovy would be developped
> > for the most popular editors.
> > When syncing the tool could show syntax checking error messages.
> >
> > 2/ Integration with Eclipse
> >
> > Based on XEclipse, we would build an Eclipse plugin to be able to connect
> > to an XWiki instance and load a specified list of spaces. Then each space
> > would be organized by the type of code in this space. Content and
> Textarea
> > fields would be made visible as editable content in Eclipse.
> > The plugin should detect the type of code in each of the content or
> > textarea fields (velocity/html, groovy, javascript, css, translations,
> > xwiki syntax) and use the appropriate editor.
> >
> > Finally a completion module could be provided by loading from the server
> a
> > list of available APIs.
> >
> > The same plugin could also be able to organize the content from a local
> > maven project (based on 1/) and provide completion to such a project.
> >
> > Live syntax checking could be provided.
> >
> > 3/ Integration of a WebIDE in XWiki
> >
> > Based on Javascript WebIDE and web code editor softwares (Orion, Cloud9,
> > exoIDE, codemirror, etc..), we would provide a view on the code inside
> the
> > XWiki instance.
> > Code would be organized in the same way as in the Eclipse plugin and
> > appropriate editors would be used depending on the code type.
> > Completion could be provided in the velocity and groovy editors and
> > eventually in Javascript
> >
> > Two views should be available, one for each AWM application, and one with
> > all the code in the Wiki.
> >
> > Live syntax checking could be provided.
> >
> > Solution 1/ is very powerful because it let's the user decide which is
> his
> > development environment, even if the tree structure won't be perfect.
> Still
> > he can see all the code in the wiki and work on it, including searching.
> > Committing is made very easy. There are some risks involved in the sync
> > process, particularly of you have multiple developers syncing local code
> > with the same dev server.
> > User can switch from local sync (local wiki) to remove sync (remove
> shared
> > wiki) and can therefore work offline more easily. Editing can be fully
> done
> > offline.
> >
> > Providing syntax highlighting for many editors might prove difficult.
> >
> > Solution 2/ is providing a nice XWiki environment inside Eclipse, without
> > the need for a local copy of the code. Committing would happen using the
> > browser.
> >
> > Solution 3/ is the long term bet as the future is to have everything in
> the
> > web. Content is only in one place which makes things a little easier.
> > Development environment needs no setup.
> > However this is more "new" for developers which need to adopt the
> platform.
> >
> > Live syntax checking is hard to provide but would be quite useful.
> > Alternatively mvn xar:format could also provide syntax checking for XWiki
> > syntax, velocity, groovy, js and css.
> >
> > WDYT ? Which approaches do you believe would be the most promising.
>
> Various comments:
>
> * I've always been a big fan of XEclipse but it seems I've been the only
> one. Whenever I've proposed to work on it and proposed ideas about it I was
> told that it's not the future and the future is developing in the IDE using
> javascript. While this is probably true, I don't see it happening with the
> same ease as your current standalone IDE before 5 years at least.
>

I still believe WebIDE is the future. However, I'm more and more convince
that even if it's the future it will take more time for devs to switch.
Therefore if we want to have developers adopt XWiki more we need to play
nice and give them what they want.


>
> * Your point 1) about exposing the XWiki model on the file system so that
> any IDE can use it is what I've been working on for a long time. (albeit
> very slowly). This is one of the reason I'm working on the new model, so
> that I can develop a file-based implementation. Note that I've just started
> developing one using Git as the backend in a branch and I've defined a way
> to map the XWiki model to a filesystem. I'll create a page on the Design
> space of dev.xwiki.org with what I have so far so that we can discuss it.
>

I don't agree that 1/ is what you mean, althout there are some
similarities. I don't think that it's a good solution to have the developer
set up their wiki with a git backend. We need to allow developers to work
with any wiki install with as little modifications as possible.
Additionally you could want to have the code on your computer, and the dev
server on a remote server online. You could work on your code (though not
test it) offline or temporarly with a local install until you decide to
push your code to the online install.

Now the similarity could be in the storage of the XWiki content in the File
System. This is the big debate, and we should start it ASAP. What is the
structure that this file system storage would get. Then it can be used by
the maven build or by your FS storage or by the tool I'm thinking about
which could push local content to XWiki.

Right now I did

space/page.xml
space/page.properties (for translations files)
space/page.xwiki (for other)
space/page_classname_objectnumber_propertyname.js (for JS Extension)
space/page_classname_objectnumber_propertyname.css (for Stylesheet
Extension)
space/page_classname_objectnumber_propertyname.properties (for UIExtension
properties)
space/page_classname_objectnumber_propertyname.xwiki (for other)
space/page_classname_objectnumber_style.css (for skin fields)
space/page_classname_objectnumber_view.vm (for skin fields)
space/page_classname_attachmentname.ext (for attachments)

Some custom decisions based on objects was needed to have custom
extensions. Now we could use a special object for that, but this can be
complicated if you have multiple textarea in one document.
Currently pure velocity or groovy content is hard to detect as it often
uses the {{groovy}} or {{velocity}} macro which is in the content.
The drawback of this is that the xml files are quite in the way (although
less used) but they are necessary.

This naming scheme goes beyond just a file system view. It's also trying to
make the files look more developer friendly.


> * Existing WebIDE solutions are too early IMO. BTW exoIDE is a no go since
> it handles the storage too using JCR and you cannot use it as a library. It
> would be good to have someone research the state of open source WebIDE
> (with compatible licenses) hat can be used a libraries but I fear that we
> won't find much.
>

Orion from the Eclipse project could be interesting.


>
> So I see 3 steps:
> * 1) Finish the syntax highlighting and autocompletion extension and
> publish it. Then include it in the platform if other devs agree
>

Yes but that's not enough


> * 2) Build on top of XEclipse to bring it to the next level
>

This is not enough either. XEclipse is interesting but first it would have
to be integrated in the Eclipse workbench and not a standalone tool. And
even that it forces to use XEclipse.


> * 3) In // we continue working on the new model
>

This is different than what I propose for 1/. It does not address 1/ at
this point. I believe we need to address 1/


>
> Now you point 1) is interesting but is not easy to implement. Basically it
> means implementing a VFS (Virtual File System) for the IDE. All IDEs have a
> VFS so it's just about having an implementation that takes its data from a
> remote XWiki server using REST but it might be slow and will need a lot of
> caching and thus sync,  which we need anyway for offline editing features
> (and is currently implemented in XEclipse). Thus the XEclpse way is the
> easiest for short term progress on this IMO.
>

I don't think so. If we agree on a model to store the files in maven
projects then we need:

0/ Agree on the file system model
1/ Implement the xar plugin changes to support the new storage and have
xar:format allow to migrate to the new one
2/ In // work on a nice tool to manage the push/pull (which is what I
started and made some progress)
3/ Evolve this tool towards a sync

In Eclipse the task is different. It's about working on the tree view to
make it more developer friendly and integrate in the workbench instead of
being standalone.

>
> I also think XEclipse could be seen as an admin center (ie. perform admin
> tasks on several XWiki instances, monitor its health, etc), but that's
> another topic.
>
>
Right now XEclipse is more a content admin tool, and this was a bit a fail
because content admins are good with the browser. We missed the target.

Ludovic


> Thanks
> -Vincent
>
> > Ludovic
>
> _______________________________________________
> devs mailing list
> [hidden email]
> http://lists.xwiki.org/mailman/listinfo/devs
>



--
Ludovic Dubost
Founder and CEO
Blog: http://blog.ludovic.org/
XWiki: http://www.xwiki.com
Skype: ldubost GTalk: ldubost
_______________________________________________
devs mailing list
[hidden email]
http://lists.xwiki.org/mailman/listinfo/devs
Reply | Threaded
Open this post in threaded view
|

Re: Improving the XWiki Development environment for traditional Java/JS Developers

Paul Libbrecht-2
In reply to this post by Ludovic Dubost
Hello Ludovic,

First let me warmly thank you for such a thread we badly need such a thing that would unify the practice of source files that can be edited by desktop apps with some luxury (IDEs, design tools, ...) to the set of objects in the wiki.

I am not 100% sure that the full-expansion tool is really fundamental as it creates quite many objects and many text areas or attachment could be considered trivial.
(e.g. contents such as a line that does just an include could just be a line in the XML file)

I would envision something a  flexible xar plugin with two extensions:

- parsing that would include xinclude, supporting attachments includes as well, the xml remains the main page container but can be enriched, when the developer says, by external files. The xar pugin would only load xml files and files referenced in there. These could be a velocity source for a text-area, doing html, js, or css, or could be a graphics in attachment.

- a sync tool that would download and upload all necessary parts and write them, just as much as the XML says (that may be tricky to handle with new things, but I guess the default should just be xml and extraction would be manual, or through wiki)

Shouldn't the code of the xar plugin be extended for such?

Paul



On 1 juil. 2013, at 00:59, Ludovic Dubost wrote:

> I've started some prototype code for 1/ at
>
> https://github.com/ldubost/xwiki-projectsync/blob/master/src/main/java/org/xwiki/contrib/projectsync/ProjectSync.java
>
> Currently if you point it to a maven xar module with the command line
>
> -format /path/to/root/dir/of/project/
>
> it will create some files for each content, textarea or attachments inside
> an XML file
>
> - Some code is also done to compare a wiki instance with that repository
> (no command line yet to launch it)
> - Some code allows to monitor changes in a directory (but nothing yet
> happens)
>
> It allows to see what it could mean to have the xml files spread in
> multiple code files.
> One big step is to decide if this is something we want because it is a
> prerequesit for such a tool.
>
> Ludovic
>
>
>
>
>
>
> 2013/6/29 Ludovic Dubost <[hidden email]>
>
>>
>> While XWiki has many great advantages as a framework which takes car of
>> persistence, forms, user management, content management and provides tons
>> of APIs, when traditional developers want to extend XWiki there are facing
>> a few difficulties:
>>
>> - they are lost in the myriad of XWiki APIs, and there is no completion
>> - they don't get good visibility of the code available in their application
>> - it is complicated to use editors which have syntax highlighting
>> - they cannot use their favorite IDE
>>
>> On the framework side there are also some improvements which could make
>> XWiki even more killer:
>>
>> - easier integration of advanced JS framework
>> - advanced integration of a high level JS framework that has templating
>> (maybe angular js)
>> - better validation functions
>> - easier way to add REST APIs
>> - more XWiki and better documented javascript apis
>>
>>
>> Here are some proposals to help fix the tools issues. Three approaches can
>> be looked at:
>>
>> 1/ Live Sync between an XWiki Instance and and improved maven xar file
>> structure, allowing to use any local IDE on XWiki code
>>
>> First it should be possible to use any IDE on the maven xar file
>> structure, allowing to open the content and textarea fields of all XML
>> files.
>> For this XWiki XML files should externalize the content and textarea
>> fields in separate files with extensions based on their content type.
>> The maven xar format should be able to clean the maven structure to do
>> this splitting and should also be able to build the XAR from all the files.
>>
>> Finally a program should allow to do a live sync between a local or remote
>> wiki instance and the maven project. Any change in either the wiki or the
>> file system should be replicated to the other party.
>> So if you run "git pull" then your local or remote wiki would be updated.
>> If a change is made in the wiki, the change would show up in the file
>> system and your IDE would refresh it.
>>
>> The sync program would keep a state of the version in the wiki, in order
>> to be able to merge changes if changes occur in both places between two
>> sync.
>> This tool could be easily launched with "mvn xar:sync"
>>
>> Syntax highlighting for XWiki Syntax+Velocity+Groovy would be developped
>> for the most popular editors.
>> When syncing the tool could show syntax checking error messages.
>>
>> 2/ Integration with Eclipse
>>
>> Based on XEclipse, we would build an Eclipse plugin to be able to connect
>> to an XWiki instance and load a specified list of spaces. Then each space
>> would be organized by the type of code in this space. Content and Textarea
>> fields would be made visible as editable content in Eclipse.
>> The plugin should detect the type of code in each of the content or
>> textarea fields (velocity/html, groovy, javascript, css, translations,
>> xwiki syntax) and use the appropriate editor.
>>
>> Finally a completion module could be provided by loading from the server a
>> list of available APIs.
>>
>> The same plugin could also be able to organize the content from a local
>> maven project (based on 1/) and provide completion to such a project.
>>
>> Live syntax checking could be provided.
>>
>> 3/ Integration of a WebIDE in XWiki
>>
>> Based on Javascript WebIDE and web code editor softwares (Orion, Cloud9,
>> exoIDE, codemirror, etc..), we would provide a view on the code inside the
>> XWiki instance.
>> Code would be organized in the same way as in the Eclipse plugin and
>> appropriate editors would be used depending on the code type.
>> Completion could be provided in the velocity and groovy editors and
>> eventually in Javascript
>>
>> Two views should be available, one for each AWM application, and one with
>> all the code in the Wiki.
>>
>> Live syntax checking could be provided.
>>
>> Solution 1/ is very powerful because it let's the user decide which is his
>> development environment, even if the tree structure won't be perfect. Still
>> he can see all the code in the wiki and work on it, including searching.
>> Committing is made very easy. There are some risks involved in the sync
>> process, particularly of you have multiple developers syncing local code
>> with the same dev server.
>> User can switch from local sync (local wiki) to remove sync (remove shared
>> wiki) and can therefore work offline more easily. Editing can be fully done
>> offline.
>>
>> Providing syntax highlighting for many editors might prove difficult.
>>
>> Solution 2/ is providing a nice XWiki environment inside Eclipse, without
>> the need for a local copy of the code. Committing would happen using the
>> browser.
>>
>> Solution 3/ is the long term bet as the future is to have everything in
>> the web. Content is only in one place which makes things a little easier.
>> Development environment needs no setup.
>> However this is more "new" for developers which need to adopt the
>> platform.
>>
>> Live syntax checking is hard to provide but would be quite useful.
>> Alternatively mvn xar:format could also provide syntax checking for XWiki
>> syntax, velocity, groovy, js and css.
>>
>> WDYT ? Which approaches do you believe would be the most promising.
>>
>> Ludovic
>>
>> --
>> Ludovic Dubost
>> Founder and CEO
>> Blog: http://blog.ludovic.org/
>> XWiki: http://www.xwiki.com
>> Skype: ldubost GTalk: ldubost
>>
>
>
>
> --
> Ludovic Dubost
> Founder and CEO
> Blog: http://blog.ludovic.org/
> XWiki: http://www.xwiki.com
> Skype: ldubost GTalk: ldubost
> _______________________________________________
> devs mailing list
> [hidden email]
> http://lists.xwiki.org/mailman/listinfo/devs

_______________________________________________
devs mailing list
[hidden email]
http://lists.xwiki.org/mailman/listinfo/devs
Reply | Threaded
Open this post in threaded view
|

Re: Improving the XWiki Development environment for traditional Java/JS Developers

Ludovic Dubost
Le mercredi 3 juillet 2013, Paul Libbrecht a écrit :

> Hello Ludovic,
>
> First let me warmly thank you for such a thread we badly need such a thing
> that would unify the practice of source files that can be edited by desktop
> apps with some luxury (IDEs, design tools, ...) to the set of objects in
> the wiki.
>
> I am not 100% sure that the full-expansion tool is really fundamental as
> it creates quite many objects and many text areas or attachment could be
> considered trivial.
> (e.g. contents such as a line that does just an include could just be a
> line in the XML file)
>
>


> I would envision something a  flexible xar plugin with two extensions:
>
> - parsing that would include xinclude, supporting attachments includes as
> well, the xml remains the main page container but can be enriched, when the
> developer says, by external files. The xar pugin would only load xml files
> and files referenced in there. These could be a velocity source for a
> text-area, doing html, js, or css, or could be a graphics in attachment.

I understand your approach. Now was is complex here is that it's the
developer that decides to extract a fuels as a separate file. It's great
one one side as we get good extension choice and naming. What is
complicated is that you need a tool and UI to do this in the XWIKi commit
apps, in the IDE or on the command line otherwise devs won't do it and even
with the tools it might not happen. You get incoherence and we need
coherence here if we want developers especially newbies to understand XWiki

We need to quickly decide what the target file system view should be, on
top of which we can have a "virtual view" for xeclipse



>
> - a sync tool that would download and upload all necessary parts and write
> them, just as much as the XML says (that may be tricky to handle with new
> things, but I guess the default should just be xml and extraction would be
> manual, or through wiki)
>
> Shouldn't the code of the xar plugin be extended for such?
>
> Paul
>
>
>
> On 1 juil. 2013, at 00:59, Ludovic Dubost wrote:
>
> > I've started some prototype code for 1/ at
> >
> >
> https://github.com/ldubost/xwiki-projectsync/blob/master/src/main/java/org/xwiki/contrib/projectsync/ProjectSync.java
> >
> > Currently if you point it to a maven xar module with the command line
> >
> > -format /path/to/root/dir/of/project/
> >
> > it will create some files for each content, textarea or attachments
> inside
> > an XML file
> >
> > - Some code is also done to compare a wiki instance with that repository
> > (no command line yet to launch it)
> > - Some code allows to monitor changes in a directory (but nothing yet
> > happens)
> >
> > It allows to see what it could mean to have the xml files spread in
> > multiple code files.
> > One big step is to decide if this is something we want because it is a
> > prerequesit for such a tool.
> >
> > Ludovic
> >
> >
> >
> >
> >
> >
> > 2013/6/29 Ludovic Dubost <[hidden email]>
> >
> >>
> >> While XWiki has many great advantages as a framework which takes car of
> >> persistence, forms, user management, content management and provides
> tons
> >> of APIs, when traditional developers want to extend XWiki there are
> facing
> >> a few difficulties:
> >>
> >> - they are lost in the myriad of XWiki APIs, and there is no completion
> >> - they don't get good visibility of the code available in their
> application
> >> - it is complicated to use editors which have syntax highlighting
> >> - they cannot use their favorite IDE
> >>
> >> On the framework side there are also some improvements which could make
> >> XWiki even more killer:
> >>
> >> - easier integration of advanced JS framework
> >> - advanced integration of a high level JS framework that has templating
> >> (maybe angular js)
> >> - better validation functions
> >> - easier way to add REST APIs
> >> - more XWiki and better documented javascript apis
> >>
> >>
> >> Here are some proposals to help fix the tools issues. Three approaches
> can
> >> be looked at:
> >>
> >> 1/ Live Sync between an XWiki Instance and and improved maven xar file
> >> structure, allowing to use any local IDE on XWiki code
> >>
> >> First it should be possible to use any IDE on the maven xar file
> >> structure, allowing to open the content and textarea fields of all XML
> >> files.
> >> For this XWiki XML files should externalize the content and textarea
> >> fields in separate files with extensions based on their content type.
> >> The maven xar format should be able to clean the maven structure to do
> >> this splitting and should also be able to build the XAR from all the
> files.
> >>
> >> Finally a program should allow to do a live sync between a local or
> remote
> >> wiki instance and the maven project. Any change in either the wiki or
> the
> >> file system should be replicated to the other party.
> >> So if you run "git pull" then your local or remote wiki would be
> updated.
> >> If a change is made in the wiki, the change would show up in the file
> >> system and your IDE would refresh it.
> >>
> >> The sync program would keep a state of the version in the wiki, in order
> >> to be able to merge changes if changes occur in both places between two
> >> sync.
> >> This tool could be easily launched with "mvn xar:sync"
> >>
> >> Syntax highlighting for XWiki Syntax+Velocity+Groovy would be developped
> >> for the most popular editors.
> >> When syncing the tool could show syntax checking error messages.
> >>
> >> 2/ Integration with Eclipse
> >>
> >> Based on XEclipse, we would build an Eclipse plugin to be able to
> connect
> >> to an XWiki instance and load a specified list of spaces. Then each
> space
> >> would be organized by the type of code in this space. Content and
> Textarea
> >> fields would be made visible as editable content in Eclipse.
> >> The plugin should detect the type of code in each of the content or
> >> textarea fields (velocity/html, groovy, javascript, css, translations,
> >> xwiki syntax) and use the appropriate editor.<>
> _______________________________________________
> > devs mailing list
> > [hidden email] <javascript:;>
> > http://lists.xwiki.org/mailman/listinfo/devs
>
> _______________________________________________
> devs mailing list
> [hidden email] <javascript:;>
> http://lists.xwiki.org/mailman/listinfo/devs
>


--
Sent from Mobile
_______________________________________________
devs mailing list
[hidden email]
http://lists.xwiki.org/mailman/listinfo/devs
Reply | Threaded
Open this post in threaded view
|

Re: Improving the XWiki Development environment for traditional Java/JS Developers

Paul Libbrecht-2

On 4 juil. 2013, at 08:14, Ludovic Dubost wrote:

>> - parsing that would include xinclude, supporting attachments includes as
>> well, the xml remains the main page container but can be enriched, when the
>> developer says, by external files. The xar pugin would only load xml files
>> and files referenced in there. These could be a velocity source for a
>> text-area, doing html, js, or css, or could be a graphics in attachment.
>
> I understand your approach. Now was is complex here is that it's the
> developer that decides to extract a fuels as a separate file. It's great
> one one side as we get good extension choice and naming. What is
> complicated is that you need a tool and UI to do this in the XWIKi commit
> apps, in the IDE or on the command line otherwise devs won't do it and even
> with the tools it might not happen. You get incoherence and we need
> coherence here if we want developers especially newbies to understand XWiki

Normal developers keep doing nose estimates to perform design decisions and that one would be just one of them. XWiki should not be concerned with that, only the xar plugin and possibly the xar export, following instructions of the developers.

I note that it is very common that the file-name is not well defined.
In my curriki sources, where the wiki project has a directory src/main/pages/ where page sources' text are saved, I have rather inconsistent file names. At least .grv, .vm, and .properties.
And each has its utility.
And I'd need a lot more.
Automating the naming of such a thing means that you add types to textareas fields, I do not think we're ready for it, are we?

The developers would edit the xml to "include" an external file and of course create the related file.
The xar plugin would perform the necessary include and leave a processing instruction so that the export can do the converse.

I am happy to try to code this.
Is that the place to edit for such a modification?
https://github.com/xwiki/xwiki-commons/tree/master/xwiki-commons-tools/xwiki-commons-tool-xar/

> We need to quickly decide what the target file system view should be, on
> top of which we can have a "virtual view" for xeclipse

That seems more like a UI decision.
I believe the screenshot you provided showed well how suddenly big it can become if it is fully automated.
That's ok when you look at UIs but that is not the tight economy of "overseeable files" that developers like to have.

For example, I think an important criterion is readibility of sources and their searchability but ensuring such criteria is the developers' job, not a universal thing.

If, in a project, for example, a large amount of source files is spread in xwiki pages, these sources, and almost only them, would be extacted. If a large amount of jpegs, these would be extracted (hence their metadata be searchable by, say, spotlight).

paul
_______________________________________________
devs mailing list
[hidden email]
http://lists.xwiki.org/mailman/listinfo/devs
Reply | Threaded
Open this post in threaded view
|

Re: Improving the XWiki Development environment for traditional Java/JS Developers

vmassol
Administrator
In reply to this post by vmassol

On Jun 29, 2013, at 1:04 PM, Vincent Massol <[hidden email]> wrote:

> Hi ludovic,
>
> All good points that I think all of us would like to see addressed as well.
>
> Replies inline.
>
> On Jun 29, 2013, at 11:41 AM, Ludovic Dubost <[hidden email]> wrote:
>
>> While XWiki has many great advantages as a framework which takes car of
>> persistence, forms, user management, content management and provides tons
>> of APIs, when traditional developers want to extend XWiki there are facing
>> a few difficulties:
>>
>> - they are lost in the myriad of XWiki APIs, and there is no completion
>
> Note that Edy and myself created an autcompletion extension last year but we didn't get to publish it on e.x.o (it's here ATM:  https://github.com/xwiki-contrib/wiki-editor-devtools).
>
> I'm planning to continue and finish it this year during the coming hackathon in July.
>
>> - they don't get good visibility of the code available in their application
>> - it is complicated to use editors which have syntax highlighting
>> - they cannot use their favorite IDE
>>
>> On the framework side there are also some improvements which could make
>> XWiki even more killer:
>>
>> - easier integration of advanced JS framework
>> - advanced integration of a high level JS framework that has templating
>> (maybe angular js)
>> - better validation functions
>> - easier way to add REST APIs
>> - more XWiki and better documented javascript apis
>>
>>
>> Here are some proposals to help fix the tools issues. Three approaches can
>> be looked at:
>>
>> 1/ Live Sync between an XWiki Instance and and improved maven xar file
>> structure, allowing to use any local IDE on XWiki code
>>
>> First it should be possible to use any IDE on the maven xar file structure,
>> allowing to open the content and textarea fields of all XML files.
>> For this XWiki XML files should externalize the content and textarea fields
>> in separate files with extensions based on their content type.
>> The maven xar format should be able to clean the maven structure to do this
>> splitting and should also be able to build the XAR from all the files.
>>
>> Finally a program should allow to do a live sync between a local or remote
>> wiki instance and the maven project. Any change in either the wiki or the
>> file system should be replicated to the other party.
>> So if you run "git pull" then your local or remote wiki would be updated.
>> If a change is made in the wiki, the change would show up in the file
>> system and your IDE would refresh it.
>>
>> The sync program would keep a state of the version in the wiki, in order to
>> be able to merge changes if changes occur in both places between two sync.
>> This tool could be easily launched with "mvn xar:sync"
>>
>> Syntax highlighting for XWiki Syntax+Velocity+Groovy would be developped
>> for the most popular editors.
>> When syncing the tool could show syntax checking error messages.
>>
>> 2/ Integration with Eclipse
>>
>> Based on XEclipse, we would build an Eclipse plugin to be able to connect
>> to an XWiki instance and load a specified list of spaces. Then each space
>> would be organized by the type of code in this space. Content and Textarea
>> fields would be made visible as editable content in Eclipse.
>> The plugin should detect the type of code in each of the content or
>> textarea fields (velocity/html, groovy, javascript, css, translations,
>> xwiki syntax) and use the appropriate editor.
>>
>> Finally a completion module could be provided by loading from the server a
>> list of available APIs.
>>
>> The same plugin could also be able to organize the content from a local
>> maven project (based on 1/) and provide completion to such a project.
>>
>> Live syntax checking could be provided.
>>
>> 3/ Integration of a WebIDE in XWiki
>>
>> Based on Javascript WebIDE and web code editor softwares (Orion, Cloud9,
>> exoIDE, codemirror, etc..), we would provide a view on the code inside the
>> XWiki instance.
>> Code would be organized in the same way as in the Eclipse plugin and
>> appropriate editors would be used depending on the code type.
>> Completion could be provided in the velocity and groovy editors and
>> eventually in Javascript
>>
>> Two views should be available, one for each AWM application, and one with
>> all the code in the Wiki.
>>
>> Live syntax checking could be provided.
>>
>> Solution 1/ is very powerful because it let's the user decide which is his
>> development environment, even if the tree structure won't be perfect. Still
>> he can see all the code in the wiki and work on it, including searching.
>> Committing is made very easy. There are some risks involved in the sync
>> process, particularly of you have multiple developers syncing local code
>> with the same dev server.
>> User can switch from local sync (local wiki) to remove sync (remove shared
>> wiki) and can therefore work offline more easily. Editing can be fully done
>> offline.
>>
>> Providing syntax highlighting for many editors might prove difficult.
>>
>> Solution 2/ is providing a nice XWiki environment inside Eclipse, without
>> the need for a local copy of the code. Committing would happen using the
>> browser.
>>
>> Solution 3/ is the long term bet as the future is to have everything in the
>> web. Content is only in one place which makes things a little easier.
>> Development environment needs no setup.
>> However this is more "new" for developers which need to adopt the platform.
>>
>> Live syntax checking is hard to provide but would be quite useful.
>> Alternatively mvn xar:format could also provide syntax checking for XWiki
>> syntax, velocity, groovy, js and css.
>>
>> WDYT ? Which approaches do you believe would be the most promising.
>
> Various comments:
>
> * I've always been a big fan of XEclipse but it seems I've been the only one. Whenever I've proposed to work on it and proposed ideas about it I was told that it's not the future and the future is developing in the IDE using javascript. While this is probably true, I don't see it happening with the same ease as your current standalone IDE before 5 years at least.

On this topic of 5 years just seen this today:
http://www.theserverside.com/feature/Cloud-based-IDEs-to-replace-desktop-development-tools-in-five-years

Thanks
-Vincent

> * Your point 1) about exposing the XWiki model on the file system so that any IDE can use it is what I've been working on for a long time. (albeit very slowly). This is one of the reason I'm working on the new model, so that I can develop a file-based implementation. Note that I've just started developing one using Git as the backend in a branch and I've defined a way to map the XWiki model to a filesystem. I'll create a page on the Design space of dev.xwiki.org with what I have so far so that we can discuss it.
>
> * Existing WebIDE solutions are too early IMO. BTW exoIDE is a no go since it handles the storage too using JCR and you cannot use it as a library. It would be good to have someone research the state of open source WebIDE (with compatible licenses) hat can be used a libraries but I fear that we won't find much.
>
> So I see 3 steps:
> * 1) Finish the syntax highlighting and autocompletion extension and publish it. Then include it in the platform if other devs agree
> * 2) Build on top of XEclipse to bring it to the next level
> * 3) In // we continue working on the new model
>
> Now you point 1) is interesting but is not easy to implement. Basically it means implementing a VFS (Virtual File System) for the IDE. All IDEs have a VFS so it's just about having an implementation that takes its data from a remote XWiki server using REST but it might be slow and will need a lot of caching and thus sync,  which we need anyway for offline editing features (and is currently implemented in XEclipse). Thus the XEclpse way is the easiest for short term progress on this IMO.
>
> I also think XEclipse could be seen as an admin center (ie. perform admin tasks on several XWiki instances, monitor its health, etc), but that's another topic.
>
> Thanks
> -Vincent
>
>> Ludovic
>

_______________________________________________
devs mailing list
[hidden email]
http://lists.xwiki.org/mailman/listinfo/devs
Reply | Threaded
Open this post in threaded view
|

Re: Improving the XWiki Development environment for traditional Java/JS Developers

Paul Libbrecht-2
In reply to this post by Paul Libbrecht-2
Hello all,

this thread has been quiet.

Is the answer to the question below simply yes?

Am I going to stumble in Guillaume's projected work?

thanks in advance.

Paul


On 4 juil. 2013, at 09:07, Paul Libbrecht wrote:

> The developers would edit the xml to "include" an external file and of course create the related file.
> The xar plugin would perform the necessary include and leave a processing instruction so that the export can do the converse.
>
> I am happy to try to code this.
> Is that the place to edit for such a modification?
> https://github.com/xwiki/xwiki-commons/tree/master/xwiki-commons-tools/xwiki-commons-tool-xar/

_______________________________________________
devs mailing list
[hidden email]
http://lists.xwiki.org/mailman/listinfo/devs
Reply | Threaded
Open this post in threaded view
|

Re: Improving the XWiki Development environment for traditional Java/JS Developers

Paul Libbrecht-2
On 4 juil. 2013, at 09:07, Paul Libbrecht wrote:
> The developers would edit the xml to "include" an external file and of course create the related file.
> The xar plugin would perform the necessary include and leave a processing instruction so that the export can do the converse.
>
> I am happy to try to code this.
> Is that the place to edit for such a modification?
> https://github.com/xwiki/xwiki-commons/tree/master/xwiki-commons-tools/xwiki-commons-tool-xar/

So I've looked around a bit because just now would be a good time and I had an issue related to the Package class.

I understand XWiki is using dom4j as internal xml representation.
I understand every time the xml is read, it is using SAXReader.

My first step would be to support xinclude parse=text in xml documents.
That would allow me to package the source files (especially of the page content) in the xar zips together with the xml sources and thus have a source tree made of short handwritten xml sources, and content sources.

Later on, this could be generalized to attachments, the maven xar plugin would need to convert the binaries to base64 which would also be included using xinclude.

Now, I am left with finding the right place where the xar import of XWiki ingests the XML and add the XInclude feature to it (which Xerces supports) (other places do not need the xinclude).

I have found com.xpn.xwiki.plugin.packaging.Package in the method fromXml and readFromXML.
Are there the only places?

These method sound called from readFromDir and Import which I would reformulate to allow relative resolution (URLs need to be provided to the XML parser then). This will create a dom that has "forgotten" the XInclude (except maybe an invisible trace) and thus would not bother further xml (de)-serialization. Alternatively, I'd post process the dom4j tree to perform the xinclude (might be easier).

thank you for your guidance through the sea of xwiki code.

Paul

_______________________________________________
devs mailing list
[hidden email]
http://lists.xwiki.org/mailman/listinfo/devs
Reply | Threaded
Open this post in threaded view
|

Re: Improving the XWiki Development environment for traditional Java/JS Developers

Thomas Mortagne
Administrator
On Thu, Aug 8, 2013 at 10:50 PM, Paul Libbrecht <[hidden email]> wrote:

> On 4 juil. 2013, at 09:07, Paul Libbrecht wrote:
>> The developers would edit the xml to "include" an external file and of course create the related file.
>> The xar plugin would perform the necessary include and leave a processing instruction so that the export can do the converse.
>>
>> I am happy to try to code this.
>> Is that the place to edit for such a modification?
>> https://github.com/xwiki/xwiki-commons/tree/master/xwiki-commons-tools/xwiki-commons-tool-xar/
>
> So I've looked around a bit because just now would be a good time and I had an issue related to the Package class.
>
> I understand XWiki is using dom4j as internal xml representation.
> I understand every time the xml is read, it is using SAXReader.
>
> My first step would be to support xinclude parse=text in xml documents.
> That would allow me to package the source files (especially of the page content) in the xar zips together with the xml sources and thus have a source tree made of short handwritten xml sources, and content sources.
>
> Later on, this could be generalized to attachments, the maven xar plugin would need to convert the binaries to base64 which would also be included using xinclude.
>
> Now, I am left with finding the right place where the xar import of XWiki ingests the XML and add the XInclude feature to it (which Xerces supports) (other places do not need the xinclude).
>
> I have found com.xpn.xwiki.plugin.packaging.Package in the method fromXml and readFromXML.
> Are there the only places?
>
> These method sound called from readFromDir and Import which I would reformulate to allow relative resolution (URLs need to be provided to the XML parser then). This will create a dom that has "forgotten" the XInclude (except maybe an invisible trace) and thus would not bother further xml (de)-serialization. Alternatively, I'd post process the dom4j tree to perform the xinclude (might be easier).
>
> thank you for your guidance through the sea of xwiki code.

About that, I'm currently (or more precisely when I'm back from
holidays in about 2 weeks or I will have problems :)) in the process
of rewriting completely XAR import/export as wiki stream modules
before including it as standard module hopefully in the 5.2 timeframe.
The main difference from low level XML point of view is that it's now
all based on StAX and there is not a line of dom4j or any other
external XML library in XAR modules (except a bit of
https://java.net/projects/stax-utils).

The current state is a pretty much done core and most of the API and a
started XAR output module (document/attachment part is pretty much
done, still need to work on generic events for xclass/xobjects). See
https://github.com/xwiki-contrib/wiki-stream.

It would probably be better to forget about Package class and look
directly at wiki stream.

>
> Paul
>
> _______________________________________________
> devs mailing list
> [hidden email]
> http://lists.xwiki.org/mailman/listinfo/devs



--
Thomas Mortagne
_______________________________________________
devs mailing list
[hidden email]
http://lists.xwiki.org/mailman/listinfo/devs
Reply | Threaded
Open this post in threaded view
|

Re: Improving the XWiki Development environment for traditional Java/JS Developers

Ludovic Dubost
In reply to this post by Paul Libbrecht-2
At the XWiki SAS Seminar we had a meeting discussing possible new directory
structures for the maven build.

The notes are pretty rough as I did not have time to pass again on them and
put more details:

http://dev.xwiki.org/xwiki/bin/view/Design/DirectoryStructureforXWikiApplications

Ludovic


2013/8/8 Paul Libbrecht <[hidden email]>

> On 4 juil. 2013, at 09:07, Paul Libbrecht wrote:
> > The developers would edit the xml to "include" an external file and of
> course create the related file.
> > The xar plugin would perform the necessary include and leave a
> processing instruction so that the export can do the converse.
> >
> > I am happy to try to code this.
> > Is that the place to edit for such a modification?
> >
> https://github.com/xwiki/xwiki-commons/tree/master/xwiki-commons-tools/xwiki-commons-tool-xar/
>
> So I've looked around a bit because just now would be a good time and I
> had an issue related to the Package class.
>
> I understand XWiki is using dom4j as internal xml representation.
> I understand every time the xml is read, it is using SAXReader.
>
> My first step would be to support xinclude parse=text in xml documents.
> That would allow me to package the source files (especially of the page
> content) in the xar zips together with the xml sources and thus have a
> source tree made of short handwritten xml sources, and content sources.
>
> Later on, this could be generalized to attachments, the maven xar plugin
> would need to convert the binaries to base64 which would also be included
> using xinclude.
>
> Now, I am left with finding the right place where the xar import of XWiki
> ingests the XML and add the XInclude feature to it (which Xerces supports)
> (other places do not need the xinclude).
>
> I have found com.xpn.xwiki.plugin.packaging.Package in the method fromXml
> and readFromXML.
> Are there the only places?
>
> These method sound called from readFromDir and Import which I would
> reformulate to allow relative resolution (URLs need to be provided to the
> XML parser then). This will create a dom that has "forgotten" the XInclude
> (except maybe an invisible trace) and thus would not bother further xml
> (de)-serialization. Alternatively, I'd post process the dom4j tree to
> perform the xinclude (might be easier).
>
> thank you for your guidance through the sea of xwiki code.
>
> Paul
>
> _______________________________________________
> devs mailing list
> [hidden email]
> http://lists.xwiki.org/mailman/listinfo/devs
>



--
Ludovic Dubost
Founder and CEO
Blog: http://blog.ludovic.org/
XWiki: http://www.xwiki.com
Skype: ldubost GTalk: ldubost
_______________________________________________
devs mailing list
[hidden email]
http://lists.xwiki.org/mailman/listinfo/devs
Reply | Threaded
Open this post in threaded view
|

Re: Improving the XWiki Development environment for traditional Java/JS Developers

Paul Libbrecht-2
In reply to this post by Thomas Mortagne
> https://java.net/projects/stax-utils).

XInclude support may be a bit weak there but I found a processor that would map stream to stream (https://github.com/etourdot/xincproc).
In any case, for attachments, one would also wish base64 encoding (so that files are... erm... files!) which is not in XInclude so would need a (non-standardized) extension.

> The current state is a pretty much done core and most of the API and a
> started XAR output module (document/attachment part is pretty much
> done, still need to work on generic events for xclass/xobjects). See
> https://github.com/xwiki-contrib/wiki-stream.

Will this include an uploader for a single xml file (and its inclusions) into a remote wiki?
That would be the really useful thing that is often breaking for me (I have a command-line uploader but it seems to create fuss recently).

> It would probably be better to forget about Package class and look
> directly at wiki stream.

Can you give a little "handbook" for the case of:
- producing a xar from a set of xml sources
- uploading a set of xml files
?

Thanks in advance.

Paul
_______________________________________________
devs mailing list
[hidden email]
http://lists.xwiki.org/mailman/listinfo/devs
Reply | Threaded
Open this post in threaded view
|

Re: Improving the XWiki Development environment for traditional Java/JS Developers

Fabio Mancinelli-4
Hi Paul,

as Ludovic said, at the seminar we had some brainstorming about this.
To make things advance I wrote a first attempt at something that should be
usable: XWikiFS - https://github.com/fmancinelli/xwikifs

It basically parses a directory structure and builds a XAR from it.
I used "enhanced" YAML files for laying out stuff. This enhancement
consists in the use of "references" that allows you to externalize content.

For example in the YAML describing the page content
https://github.com/fmancinelli/xwikifs/blob/master/src/test/resources/Main.WebHome/document.xwd
you
have a "-> content.xwiki" saying that the actual content is in the
content.xwiki file.

This allows to keep and organize things better (e.g. for textarea object
fields)

There's this classinfo directory that contains the class file descriptions
for the classes used in the objects. It's there because this information is
needed in the XAR. If someone knows how to build a XAR with objects without
including (redundant) class information it would be great (and we could get
rid of this directory as well).

The idea is that we can "monitor" the XWikiFS for changes and automatically
upload them to the main wiki (handling the merging) and viceversa.

It's a first iteration, so if you have any comments don't hesitate.

Thanks,
Fabio


On Wed, Aug 21, 2013 at 9:52 PM, Paul Libbrecht <[hidden email]> wrote:

> > https://java.net/projects/stax-utils).
>
> XInclude support may be a bit weak there but I found a processor that
> would map stream to stream (https://github.com/etourdot/xincproc).
> In any case, for attachments, one would also wish base64 encoding (so that
> files are... erm... files!) which is not in XInclude so would need a
> (non-standardized) extension.
>
> > The current state is a pretty much done core and most of the API and a
> > started XAR output module (document/attachment part is pretty much
> > done, still need to work on generic events for xclass/xobjects). See
> > https://github.com/xwiki-contrib/wiki-stream.
>
> Will this include an uploader for a single xml file (and its inclusions)
> into a remote wiki?
> That would be the really useful thing that is often breaking for me (I
> have a command-line uploader but it seems to create fuss recently).
>
> > It would probably be better to forget about Package class and look
> > directly at wiki stream.
>
> Can you give a little "handbook" for the case of:
> - producing a xar from a set of xml sources
> - uploading a set of xml files
> ?
>
> Thanks in advance.
>
> Paul
> _______________________________________________
> devs mailing list
> [hidden email]
> http://lists.xwiki.org/mailman/listinfo/devs
>
_______________________________________________
devs mailing list
[hidden email]
http://lists.xwiki.org/mailman/listinfo/devs
Reply | Threaded
Open this post in threaded view
|

Re: Improving the XWiki Development environment for traditional Java/JS Developers

Caleb James DeLisle-2
This may or may not be important, NTFS forbids the : symbol in a filename.
https://github.com/fmancinelli/xwikifs/tree/master/src/test/resources/Main.WebHome/objects/XWiki.XWikiComments:0

Thanks,
Caleb

On 08/22/2013 06:09 PM, Fabio Mancinelli wrote:

> Hi Paul,
>
> as Ludovic said, at the seminar we had some brainstorming about this.
> To make things advance I wrote a first attempt at something that should be
> usable: XWikiFS - https://github.com/fmancinelli/xwikifs
>
> It basically parses a directory structure and builds a XAR from it.
> I used "enhanced" YAML files for laying out stuff. This enhancement
> consists in the use of "references" that allows you to externalize content.
>
> For example in the YAML describing the page content
> https://github.com/fmancinelli/xwikifs/blob/master/src/test/resources/Main.WebHome/document.xwd
> you
> have a "-> content.xwiki" saying that the actual content is in the
> content.xwiki file.
>
> This allows to keep and organize things better (e.g. for textarea object
> fields)
>
> There's this classinfo directory that contains the class file descriptions
> for the classes used in the objects. It's there because this information is
> needed in the XAR. If someone knows how to build a XAR with objects without
> including (redundant) class information it would be great (and we could get
> rid of this directory as well).
>
> The idea is that we can "monitor" the XWikiFS for changes and automatically
> upload them to the main wiki (handling the merging) and viceversa.
>
> It's a first iteration, so if you have any comments don't hesitate.
>
> Thanks,
> Fabio
>
>
> On Wed, Aug 21, 2013 at 9:52 PM, Paul Libbrecht <[hidden email]> wrote:
>
>>> https://java.net/projects/stax-utils).
>>
>> XInclude support may be a bit weak there but I found a processor that
>> would map stream to stream (https://github.com/etourdot/xincproc).
>> In any case, for attachments, one would also wish base64 encoding (so that
>> files are... erm... files!) which is not in XInclude so would need a
>> (non-standardized) extension.
>>
>>> The current state is a pretty much done core and most of the API and a
>>> started XAR output module (document/attachment part is pretty much
>>> done, still need to work on generic events for xclass/xobjects). See
>>> https://github.com/xwiki-contrib/wiki-stream.
>>
>> Will this include an uploader for a single xml file (and its inclusions)
>> into a remote wiki?
>> That would be the really useful thing that is often breaking for me (I
>> have a command-line uploader but it seems to create fuss recently).
>>
>>> It would probably be better to forget about Package class and look
>>> directly at wiki stream.
>>
>> Can you give a little "handbook" for the case of:
>> - producing a xar from a set of xml sources
>> - uploading a set of xml files
>> ?
>>
>> Thanks in advance.
>>
>> Paul
>> _______________________________________________
>> devs mailing list
>> [hidden email]
>> http://lists.xwiki.org/mailman/listinfo/devs
>>
> _______________________________________________
> devs mailing list
> [hidden email]
> http://lists.xwiki.org/mailman/listinfo/devs
>
_______________________________________________
devs mailing list
[hidden email]
http://lists.xwiki.org/mailman/listinfo/devs
Reply | Threaded
Open this post in threaded view
|

Re: Improving the XWiki Development environment for traditional Java/JS Developers

Paul Libbrecht-2
In reply to this post by Fabio Mancinelli-4
Hello Fabio,

Thanks for your attempt.
I see in there the following:
Space.Name
|
+- document.xwd
+- class.xwc (optional)
+- objects (optional)
|  |
|  +- classinfo
|  |  |
|  |  +- Space.Name1.xwc
|  |  +- ...
|  +- Space.Name-N.xwo
|  +- ...
+- attachments (optional)
   |
   +- file
   +- ....
and I have four issues:
- the attachments do not have a free positioning: attachments are commonly referenced in web-editing fashions when editing with some web-editor (e.g. an IDE or DreamWeaver) and their referencing mechanisms often follows the directory paths. This has made the success of PHP and JSP. I see no reasons to not put it here. Sometimes you need more flexibility and then you accept to loose the verification abilities (e.g. you could import a part of that from Word or Firefox, and this dictates a different directory layout).
- the space/page structure is not following a directory structure (one directory per space). While this could certainly be discussable in some cases, it still appears to me as the most useful positioning but is not there. Again, this allows links to be somewhat better checked (and auto-completed, and...).
- I see .xwd and .xwo files in there. Avoiding to fix the filename extensions appears crucial to me. I want to use IntelliJ's .vm support by using the .vm extension for many pages but sometimes, these should be translations!
- Finally, I lack a possibility to put in a separate file a textarea-field (e.g. a panel content) whereas this sounds to be common-practice in xwiki.

... and a taste issue: I am not very friend with serialization languages which claim to do almost the same as XML but are just not XML (YAML, YML, ...). The toolset for XML is sooooo widespread that loosing this means you just need a bunch of extensions for just about any editor and this makes things longer to adopt. But that is personal for sure.

<rant>
Let's agree before JSON is imposed on us everywhere!
</rant>
;-)

I agree that the full XInclude is probably too much, but I do not think we need to achieve it fully.

Paul

> as Ludovic said, at the seminar we had some brainstorming about this.
> To make things advance I wrote a first attempt at something that should be
> usable: XWikiFS - https://github.com/fmancinelli/xwikifs
>
> It basically parses a directory structure and builds a XAR from it.
> I used "enhanced" YAML files for laying out stuff. This enhancement
> consists in the use of "references" that allows you to externalize content.
>
> For example in the YAML describing the page content
> https://github.com/fmancinelli/xwikifs/blob/master/src/test/resources/Main.WebHome/document.xwd
> you
> have a "-> content.xwiki" saying that the actual content is in the
> content.xwiki file.
>
> This allows to keep and organize things better (e.g. for textarea object
> fields)
>
> There's this classinfo directory that contains the class file descriptions
> for the classes used in the objects. It's there because this information is
> needed in the XAR. If someone knows how to build a XAR with objects without
> including (redundant) class information it would be great (and we could get
> rid of this directory as well).
>
> The idea is that we can "monitor" the XWikiFS for changes and automatically
> upload them to the main wiki (handling the merging) and viceversa.
>
> It's a first iteration, so if you have any comments don't hesitate.

_______________________________________________
devs mailing list
[hidden email]
http://lists.xwiki.org/mailman/listinfo/devs
12