[BRAINSTORM] You're rewriting XWiki, you can use any technology in the world, describe it.

classic Classic list List threaded Threaded
10 messages Options
Reply | Threaded
Open this post in threaded view
|

[BRAINSTORM] You're rewriting XWiki, you can use any technology in the world, describe it.

Caleb James DeLisle
This is the premise, you are going to write a new XWiki.
You are not bound by any backward compatibility requirements.
Use any technology or combination of technologies.
PHP? C++? Golang? Node.js? Java? Your call!

Describe it in as much detail as possible.
What frameworks will you use? What ORM? what databases?

Why will it perform well?
How will you get new features into the hands of users quickly?
How will you avoid this:
http://steve-yegge.blogspot.ca/2007/12/codes-worst-enemy.html
and this: http://www.laputan.org/mud/

Think big and go wild!

Thanks,
Caleb

_______________________________________________
devs mailing list
[hidden email]
http://lists.xwiki.org/mailman/listinfo/devs
Reply | Threaded
Open this post in threaded view
|

Re: [BRAINSTORM] You're rewriting XWiki, you can use any technology in the world, describe it.

Jérôme Velociter
Hi Caleb,

Great thread, good exercice.

Like many of us I imagine I've been giving that question some thoughts
already. I've written an article exactly one year ago about this topic
[1] (not on rewritting XWiki per-se, but on writing a similar
application/platform). My thoughts have partly evolved since then,
although I still think most of what I've laid down there at the time.

Basically I would go like this :

- I would stick to the JVM, and to Java.
- I would make it a RESTful web service. And probably even several if I
were to be iso in terms of feature. An example of a feature that I would
see live in a separated web service (so different process and different
API host or subdomain) would be anything open office related. It does
not belong to the base service, and would actually benefit being hosted
on the same machine as an open office server. This is an example, I
believe there are other features that do not belong in the base service
(Scheduling ? Mailing ? etc.). It's SOA, though the base service (the
base "wiki engine" let say) has to be self-sufficient and offer
something that work standalone (even if it means no watch list, no
office import, etc.)
- Today I'm not so sure about JAX-RS as I was in my article. It's nice
and and brings a lot "for free" but now I also think it has some
limitations that can be penalties, such as limited dynamicity in the way
you can play with routes. Though maybe this is mitigated by JAX-RS 2.0
filters.
- Front-end templating would be done with dumb templating, for example
handlebars.js or dust.js ; with a flexible intermediary layer for
preparing the "context" needed for such templating mechanisms (written
as groovy or JS scripts).
- I would go for a RDBM (like Postgres) without an "industrial ORM". I
think hibernate and friends bring too much complexity for too little
value. They tend to impose themselves on the DB design instead of having
DB design and object design exists independently and meet at the O/R
layer. They make it hard for the developer to know what is executed
against the DB (the "gut feeling" can be wrong) and when. Also lazy
fetching does not solve any actual problem IMHO.
- I would make anything query related (like searching for objects when
building applications) happen against the search engine and not the DB.
- I would use elasticsearch as a search engine. It can be very easily
embedded (for development and simple deployments) or run as a separate
cluster, it's schemaless and RESTful by nature.
- I would embrace JavaScript fully on the "client" for changing state.
It would be hard for me not to throw AngularJS in the mix.
- In terms of UX, I would make the "administration" a different UI/UX
than the wiki itself, and make it an actual "back office".
- ... it's missing a lot of details, I'll come back if I have some time

Actually I've been trying to put most of those ideas into action for
some months, into a e-commerce/marketplace platform I'm building [2].

Now, I'm very eager to read what others think.

Jerome

[1] http://velociter.fr/journal/my-idea-of-a-modern-web-app-on-the-jvm
[2] https://github.com/mayocat/mayocat-shop/


On 03/27/2013 02:12 AM, Caleb James DeLisle wrote:

> This is the premise, you are going to write a new XWiki.
> You are not bound by any backward compatibility requirements.
> Use any technology or combination of technologies.
> PHP? C++? Golang? Node.js? Java? Your call!
>
> Describe it in as much detail as possible.
> What frameworks will you use? What ORM? what databases?
>
> Why will it perform well?
> How will you get new features into the hands of users quickly?
> How will you avoid this:
> http://steve-yegge.blogspot.ca/2007/12/codes-worst-enemy.html
> and this: http://www.laputan.org/mud/
>
> Think big and go wild!
>
> Thanks,
> Caleb
>
> _______________________________________________
> devs mailing list
> [hidden email]
> http://lists.xwiki.org/mailman/listinfo/devs

_______________________________________________
devs mailing list
[hidden email]
http://lists.xwiki.org/mailman/listinfo/devs
Reply | Threaded
Open this post in threaded view
|

Re: [BRAINSTORM] You're rewriting XWiki, you can use any technology in the world, describe it.

Fabio Mancinelli-4
In reply to this post by Caleb James DeLisle
On Wed, Mar 27, 2013 at 2:12 AM, Caleb James DeLisle
<[hidden email]> wrote:

> This is the premise, you are going to write a new XWiki.
> You are not bound by any backward compatibility requirements.
> Use any technology or combination of technologies.
> PHP? C++? Golang? Node.js? Java? Your call!
>
> Describe it in as much detail as possible.
> What frameworks will you use? What ORM? what databases?
>
> Why will it perform well?
> How will you get new features into the hands of users quickly?
> How will you avoid this:
> http://steve-yegge.blogspot.ca/2007/12/codes-worst-enemy.html
> and this: http://www.laputan.org/mud/
>
> Think big and go wild!
>
I would try to do mostly the following: rewrite the core using a
functional approach, trying to stay away as much as possible from
mutable state and side effects functions (correctly isolating and
limiting those parts that needs for some reason to introduce
side-effects) - This would almost completely rule out concurrency
issues, though introducing some little performance penalties.

In order to do so you would need to use a language that is functional
and supports persistent data structure (otherwise you're doomed to
endless memory copying). Since I would stick to the JVM this means
that, AFAIK, you basically have two choices:

1) Scala
2) Clojure

I would choose Clojure which is several orders of magnitude simpler
than Scala and also doesn't suffer of the runtime upgrade problems
that Scala has.

I think that Clojure community is very
vibrant, responsive, friendly and there is a strong professional
support because eminent people of the community founded a company
that's basing all its business on Clojure.

So basically I would (try to) rewrite the core using Clojure and see
how it goes, at least to put into practice what is being preached
recently by a lot of people. I think there's a lot of truth there
indeed, but I would also like to check :)

This, in some ways, also contadicts my idea that for developing
maintainable software we should use type
checked languages :) However Clojure has a library for performing some
type checking so maybe there's not too much contradiction here.
Unfortunately Scala, which has a great type-checker, seems to be too
messy to be used.

Ideally, it could be very interesting to put this in practice by
providing an implementation for the new model backed by Clojure :)

For the client interactions'll use very simple and composable REST frameworks.
Clojure has plenty of these, but even if we stick to java, a JAX-RS
would do just fine.

I have not a strong opinion on this but maybe it could be interesting
to explore the idea of using a language with a more sane semantics for
writing frontend Javascript.
Something like Dart or a language that is able to compile to
Javascript. Clojure has clojurescript, Haskell has Fay, etc.

-Fabio

> Thanks,
> Caleb
>
> _______________________________________________
> devs mailing list
> [hidden email]
> http://lists.xwiki.org/mailman/listinfo/devs
_______________________________________________
devs mailing list
[hidden email]
http://lists.xwiki.org/mailman/listinfo/devs
Reply | Threaded
Open this post in threaded view
|

Re: [BRAINSTORM] You're rewriting XWiki, you can use any technology in the world, describe it.

Ludovic Dubost
In reply to this post by Jérôme Velociter
Hi Caleb,

From my point of view, before starting to list technologies that would be
great for a next generation XWiki you first need to define what your exact
target is for XWiki. It is very important to do that because when I look at
Jerome's answer I don't necessarly have the feeling he is looking at
exactly the same objective as I would. While his technology choices might
make a lot of sense for an eCommerce platform, they might not in the
Intranet concept.

This is a first big significant task which depending on the answers will
serve as input on the technology stack to use. Indeed since 2003 when I
started XWiki a lot of new technologies and also use cases or delivery
methods have florished. For instance Cloud is now a very important delivery
method of Web Applications and wether your write for a Cloud service, for
Software deployment or for both this might lead to different technology
choices. Mobile is another key delivery method that was not there in 2003.
On the technology side, SOA, REST, Javascript are everywhere. NoSQL is also
a key technology to consider.

So let me first try to address the question of what a new XWiki could be
used for, what it is currently used for, what are the strength of XWiki and
it's weaknesses.

First the initial objective which is the reason for XWiki:

- building flexible collaborative intranets on top of a Wiki, allowing to
quickly build applications, integrate these applications with the content

Then a lot of things that users, clients have asked for, developers have
done on their own:

- providing Knowledge based solutions
- building applications on top of the "Application" development features of
XWiki (the extension of the previous use case)
- providing a full Intranet solution, including what XWiki does natively,
but also providing "Advanced Profiles", "Groups", "Social networks", a set
of Collaborative Applications (Forum, files, etc..)
- building public or internal content web sites with limited editors (CMS
use case)
- building a web application using XWiki as a framework, using many or
little of the XWiki existing functions
- just doing a very nice Wiki in an Enterprise or Public context
- doing any of these in the "Cloud" context (multi tenant) and allowing
users to subscribe to an "use-case" that would be built using the XWiki
technology

As you can see this is close to the "flavor" discussion we already have and
I've voluntarily listed the first three which are the flavors we are
currently discussing because this is at XWiki SAS what we see organizations
we discuss with are asking for most.
We have seen the other uses cases also but either we had difficulties
addressing them or it was less natural to use XWiki to address them.

The "framework" use case is a very important one here, because if we think
XWiki's objective is to be a Framework objective for ANY web application,
then this might be incompatible with the Intranet, Knowledge Base and even
the Wiki use case.
For instance Jerome's use case does not fit in the "Intranet" use case and
the problem he has are very different from those you have in an Intranet
environment.

The "Cloud" one is a very big one also and can lead to radical different
technology choices. As we can see many new cloud services are build around
NoSQL and there are some good reason for that. It's because the Cloud
provider wants a unique platform for all his customers and wants to run the
service in a fully multi-tenant way on top of this unique storage mecanism.
The objective here is to reduce the operating cost to the maximum, in order
to be able to provide a "freemium" service. However this choice might have
a lot of benefits in the "Cloud" context, but it is a radical choice which
has a lot of impact if you also want to provide your service as "Software"
like we do for XWiki. It complexifies the installation, reduces the
features of your storage (very little relational queries), is a radical new
technology to learn to build on which is still very young (choosen a NoSQL
platform today is a very risky bet and supporting multiple technologies is
a nightmare). Now not supporting NoSQL does not necessarly mean you cannot
go "Cloud" but this goes against the way of doing business in the cloud and
complexifies the cloud platform, has scalability issues, etc. Now it's
impossible to ignore "Cloud" but it's important to understand which type of
"Cloud". Is is a unique Cloud or is it SaaS where your objective is to be
able to "host" the service for users and customers. The first choice might
mean going NoSQL, the second might mean sticking with traditional
Databases. The first choice might mean that you will have a hard time
convince "software" users because you also need to convince them to go
NoSQL.

For me the biggest success we have is for "Intranets" and it has always
been the key objective, and what we see is that the "Intranet" use case has
lead us to have to address the "full Intranet" solution but also the "CMS"
one as some organizations want a unique platform for both publishing and
collaboration. There is a lot of pressure also to support all "Social
Network" features as there are obvious bridges between a software that
would be the main places to manage profile, groups and discussions and
another software that want to manage information, knowledge and
applications.

For me the biggest strength of XWiki is it's flexibility to build
Applications and to Customize it. The first objective of XWiki was to allow
organizations to setup a wiki start sharing knowledge and then extend their
wiki with applications that would organize the information in a better way.
This need is still here today (Podio is a good example of the Application
concept without the Wiki part to glue everything together). I believe many
CMS include more and more "application" features so that you can extend
your web site with more advance organizations and features (but CMS don't
have the same collaborative aspects). The power of XWiki is to allow to do
this fully from your web browser and to do it fast. It is to allow power
users (not developers) to participate because they don't need full
developer skill (this one is very important when it comes to the technology
choices of how XWiki gets extensible). The power of XWiki is to not limit
the customizations capabilities to the content but also allow to customize
the UI by overidding all templates and then morph the initial web site you
created into something else. This power is want allowed XWiki to kind of be
a full "framework" for web development, but it could be that we have pushed
it too far or not far enough. These strength are particularly talking for
some of the users case listed before and for others it can actually be a
weakness.

The weaknesses of XWiki depend on the use cases. For the "Intranet" use
case, our weaknesses have been to not work enough on the UI, to not adapt
enough to the actual intranet use cases (this is where flavors come in).
Another weakness is not enough separation (but not too much) between
content and applications. Applications live in XWiki page but we don't have
enough markers to separate them and not enough "SOA" at the XWiki
Application level. We can fix this. Other weaknesses are how to address
mobile, how to address AJAX development and still allowing power-users (not
developers) to participate in this development. We also took too much time
to build AppWithinMinutes. There are challenges to allowing power-users to
develop and still allow this development to be continued by real developers.

On the "framework" use-case it's another analysis. Do we need to allow
development in the Wiki if XWiki's job is to compete with web-development
frameworks ? Can we go without hosting the development in an IDE and if so
should it be Eclipse or a Web based IDE ?
Is it compatible to provide development features that are more easily
accessible to non power-user (like velocity is) or can we go with more
advanced technologies. Etc... there are many questions coming from that.

There are many other very interesting questions that will yield different
answers. For instance should we do a full AJAX UI. What are the
consequences of it. If we go full AJAX, what about URLs ? What about search
engine indexing. Can we do CMS if we are full AJAX. Full AJAX means more
complex to customize also depending on how it's developped.

To summarize we need to "choose" first what we want XWiki to be. Once we
all agree on this then we can answer how to build the next generation
XWiki. I don't disagree that we should think about this, because it's going
to be 10 years in November since I started building XWiki and of course
technologies evolve and we need to be able to adapt.

I'm looking forward to your opinions on that. Then I propose we discuss for
each of the use-cases we consider are the future of XWiki what are the
right technologies. Then we also look at the compatibility of technologies
with each other. This will give us a lot of answers. One being in which
direction do we need to put our technological effort on. Another being more
clear about what users and developers can expect from XWiki.

Ludovic





2013/3/27 Jerome Velociter <[hidden email]>

> Hi Caleb,
>
> Great thread, good exercice.
>
> Like many of us I imagine I've been giving that question some thoughts
> already. I've written an article exactly one year ago about this topic [1]
> (not on rewritting XWiki per-se, but on writing a similar
> application/platform). My thoughts have partly evolved since then, although
> I still think most of what I've laid down there at the time.
>
> Basically I would go like this :
>
> - I would stick to the JVM, and to Java.
> - I would make it a RESTful web service. And probably even several if I
> were to be iso in terms of feature. An example of a feature that I would
> see live in a separated web service (so different process and different API
> host or subdomain) would be anything open office related. It does not
> belong to the base service, and would actually benefit being hosted on the
> same machine as an open office server. This is an example, I believe there
> are other features that do not belong in the base service (Scheduling ?
> Mailing ? etc.). It's SOA, though the base service (the base "wiki engine"
> let say) has to be self-sufficient and offer something that work standalone
> (even if it means no watch list, no office import, etc.)
> - Today I'm not so sure about JAX-RS as I was in my article. It's nice and
> and brings a lot "for free" but now I also think it has some limitations
> that can be penalties, such as limited dynamicity in the way you can play
> with routes. Though maybe this is mitigated by JAX-RS 2.0 filters.
> - Front-end templating would be done with dumb templating, for example
> handlebars.js or dust.js ; with a flexible intermediary layer for preparing
> the "context" needed for such templating mechanisms (written as groovy or
> JS scripts).
> - I would go for a RDBM (like Postgres) without an "industrial ORM". I
> think hibernate and friends bring too much complexity for too little value.
> They tend to impose themselves on the DB design instead of having DB design
> and object design exists independently and meet at the O/R layer. They make
> it hard for the developer to know what is executed against the DB (the "gut
> feeling" can be wrong) and when. Also lazy fetching does not solve any
> actual problem IMHO.
> - I would make anything query related (like searching for objects when
> building applications) happen against the search engine and not the DB.
> - I would use elasticsearch as a search engine. It can be very easily
> embedded (for development and simple deployments) or run as a separate
> cluster, it's schemaless and RESTful by nature.
> - I would embrace JavaScript fully on the "client" for changing state. It
> would be hard for me not to throw AngularJS in the mix.
> - In terms of UX, I would make the "administration" a different UI/UX than
> the wiki itself, and make it an actual "back office".
> - ... it's missing a lot of details, I'll come back if I have some time
>
> Actually I've been trying to put most of those ideas into action for some
> months, into a e-commerce/marketplace platform I'm building [2].
>
> Now, I'm very eager to read what others think.
>
> Jerome
>
> [1] http://velociter.fr/journal/**my-idea-of-a-modern-web-app-**on-the-jvm<http://velociter.fr/journal/my-idea-of-a-modern-web-app-on-the-jvm>
> [2] https://github.com/mayocat/**mayocat-shop/<https://github.com/mayocat/mayocat-shop/>
>
>
>
> On 03/27/2013 02:12 AM, Caleb James DeLisle wrote:
>
>> This is the premise, you are going to write a new XWiki.
>> You are not bound by any backward compatibility requirements.
>> Use any technology or combination of technologies.
>> PHP? C++? Golang? Node.js? Java? Your call!
>>
>> Describe it in as much detail as possible.
>> What frameworks will you use? What ORM? what databases?
>>
>> Why will it perform well?
>> How will you get new features into the hands of users quickly?
>> How will you avoid this:
>> http://steve-yegge.blogspot.**ca/2007/12/codes-worst-enemy.**html<http://steve-yegge.blogspot.ca/2007/12/codes-worst-enemy.html>
>> and this: http://www.laputan.org/mud/
>>
>> Think big and go wild!
>>
>> Thanks,
>> Caleb
>>
>> ______________________________**_________________
>> devs mailing list
>> [hidden email]
>> http://lists.xwiki.org/**mailman/listinfo/devs<http://lists.xwiki.org/mailman/listinfo/devs>
>>
>
> ______________________________**_________________
> devs mailing list
> [hidden email]
> http://lists.xwiki.org/**mailman/listinfo/devs<http://lists.xwiki.org/mailman/listinfo/devs>
>



--
Ludovic Dubost
Founder and CEO
Blog: http://blog.ludovic.org/
XWiki: http://www.xwiki.com
Skype: ldubost GTalk: ldubost
_______________________________________________
devs mailing list
[hidden email]
http://lists.xwiki.org/mailman/listinfo/devs
Reply | Threaded
Open this post in threaded view
|

Re: [BRAINSTORM] You're rewriting XWiki, you can use any technology in the world, describe it.

Jérôme Velociter
Hi Ludovic, all

You're right, those are interesting questions to ask ourselves before
going into specific technologies.

Let me try to sum them up for reference :

* Should it be possible/easy for users to download and try XWiki on
their machine ?

* Should it be easy for users to run XWiki on their own server/VPS ?

* Should it be easy to run XWiki on popular PaaS providers platforms ?
More specically :
** SQL-backed ones : Heroky, Cloudbees, etc.
** NoSQL-backed ones : Google app engine, etc.

* Should it be possible to run XWiki in corporate environments ? More
specifically :
** Should it be possible to run XWiki on traditionnal (should I say
"legacy") systems (say websphere + oracle)
** Should it be possible to run XWiki on the more modern systems of
corporate environments (say tomcat+mysql)

Please add more if I missed some.

Jerome

On 03/27/2013 10:30 AM, Ludovic Dubost wrote:

> Hi Caleb,
>
>  From my point of view, before starting to list technologies that would be
> great for a next generation XWiki you first need to define what your exact
> target is for XWiki. It is very important to do that because when I look at
> Jerome's answer I don't necessarly have the feeling he is looking at
> exactly the same objective as I would. While his technology choices might
> make a lot of sense for an eCommerce platform, they might not in the
> Intranet concept.
>
> This is a first big significant task which depending on the answers will
> serve as input on the technology stack to use. Indeed since 2003 when I
> started XWiki a lot of new technologies and also use cases or delivery
> methods have florished. For instance Cloud is now a very important delivery
> method of Web Applications and wether your write for a Cloud service, for
> Software deployment or for both this might lead to different technology
> choices. Mobile is another key delivery method that was not there in 2003.
> On the technology side, SOA, REST, Javascript are everywhere. NoSQL is also
> a key technology to consider.
>
> So let me first try to address the question of what a new XWiki could be
> used for, what it is currently used for, what are the strength of XWiki and
> it's weaknesses.
>
> First the initial objective which is the reason for XWiki:
>
> - building flexible collaborative intranets on top of a Wiki, allowing to
> quickly build applications, integrate these applications with the content
>
> Then a lot of things that users, clients have asked for, developers have
> done on their own:
>
> - providing Knowledge based solutions
> - building applications on top of the "Application" development features of
> XWiki (the extension of the previous use case)
> - providing a full Intranet solution, including what XWiki does natively,
> but also providing "Advanced Profiles", "Groups", "Social networks", a set
> of Collaborative Applications (Forum, files, etc..)
> - building public or internal content web sites with limited editors (CMS
> use case)
> - building a web application using XWiki as a framework, using many or
> little of the XWiki existing functions
> - just doing a very nice Wiki in an Enterprise or Public context
> - doing any of these in the "Cloud" context (multi tenant) and allowing
> users to subscribe to an "use-case" that would be built using the XWiki
> technology
>
> As you can see this is close to the "flavor" discussion we already have and
> I've voluntarily listed the first three which are the flavors we are
> currently discussing because this is at XWiki SAS what we see organizations
> we discuss with are asking for most.
> We have seen the other uses cases also but either we had difficulties
> addressing them or it was less natural to use XWiki to address them.
>
> The "framework" use case is a very important one here, because if we think
> XWiki's objective is to be a Framework objective for ANY web application,
> then this might be incompatible with the Intranet, Knowledge Base and even
> the Wiki use case.
> For instance Jerome's use case does not fit in the "Intranet" use case and
> the problem he has are very different from those you have in an Intranet
> environment.
>
> The "Cloud" one is a very big one also and can lead to radical different
> technology choices. As we can see many new cloud services are build around
> NoSQL and there are some good reason for that. It's because the Cloud
> provider wants a unique platform for all his customers and wants to run the
> service in a fully multi-tenant way on top of this unique storage mecanism.
> The objective here is to reduce the operating cost to the maximum, in order
> to be able to provide a "freemium" service. However this choice might have
> a lot of benefits in the "Cloud" context, but it is a radical choice which
> has a lot of impact if you also want to provide your service as "Software"
> like we do for XWiki. It complexifies the installation, reduces the
> features of your storage (very little relational queries), is a radical new
> technology to learn to build on which is still very young (choosen a NoSQL
> platform today is a very risky bet and supporting multiple technologies is
> a nightmare). Now not supporting NoSQL does not necessarly mean you cannot
> go "Cloud" but this goes against the way of doing business in the cloud and
> complexifies the cloud platform, has scalability issues, etc. Now it's
> impossible to ignore "Cloud" but it's important to understand which type of
> "Cloud". Is is a unique Cloud or is it SaaS where your objective is to be
> able to "host" the service for users and customers. The first choice might
> mean going NoSQL, the second might mean sticking with traditional
> Databases. The first choice might mean that you will have a hard time
> convince "software" users because you also need to convince them to go
> NoSQL.
>
> For me the biggest success we have is for "Intranets" and it has always
> been the key objective, and what we see is that the "Intranet" use case has
> lead us to have to address the "full Intranet" solution but also the "CMS"
> one as some organizations want a unique platform for both publishing and
> collaboration. There is a lot of pressure also to support all "Social
> Network" features as there are obvious bridges between a software that
> would be the main places to manage profile, groups and discussions and
> another software that want to manage information, knowledge and
> applications.
>
> For me the biggest strength of XWiki is it's flexibility to build
> Applications and to Customize it. The first objective of XWiki was to allow
> organizations to setup a wiki start sharing knowledge and then extend their
> wiki with applications that would organize the information in a better way.
> This need is still here today (Podio is a good example of the Application
> concept without the Wiki part to glue everything together). I believe many
> CMS include more and more "application" features so that you can extend
> your web site with more advance organizations and features (but CMS don't
> have the same collaborative aspects). The power of XWiki is to allow to do
> this fully from your web browser and to do it fast. It is to allow power
> users (not developers) to participate because they don't need full
> developer skill (this one is very important when it comes to the technology
> choices of how XWiki gets extensible). The power of XWiki is to not limit
> the customizations capabilities to the content but also allow to customize
> the UI by overidding all templates and then morph the initial web site you
> created into something else. This power is want allowed XWiki to kind of be
> a full "framework" for web development, but it could be that we have pushed
> it too far or not far enough. These strength are particularly talking for
> some of the users case listed before and for others it can actually be a
> weakness.
>
> The weaknesses of XWiki depend on the use cases. For the "Intranet" use
> case, our weaknesses have been to not work enough on the UI, to not adapt
> enough to the actual intranet use cases (this is where flavors come in).
> Another weakness is not enough separation (but not too much) between
> content and applications. Applications live in XWiki page but we don't have
> enough markers to separate them and not enough "SOA" at the XWiki
> Application level. We can fix this. Other weaknesses are how to address
> mobile, how to address AJAX development and still allowing power-users (not
> developers) to participate in this development. We also took too much time
> to build AppWithinMinutes. There are challenges to allowing power-users to
> develop and still allow this development to be continued by real developers.
>
> On the "framework" use-case it's another analysis. Do we need to allow
> development in the Wiki if XWiki's job is to compete with web-development
> frameworks ? Can we go without hosting the development in an IDE and if so
> should it be Eclipse or a Web based IDE ?
> Is it compatible to provide development features that are more easily
> accessible to non power-user (like velocity is) or can we go with more
> advanced technologies. Etc... there are many questions coming from that.
>
> There are many other very interesting questions that will yield different
> answers. For instance should we do a full AJAX UI. What are the
> consequences of it. If we go full AJAX, what about URLs ? What about search
> engine indexing. Can we do CMS if we are full AJAX. Full AJAX means more
> complex to customize also depending on how it's developped.
>
> To summarize we need to "choose" first what we want XWiki to be. Once we
> all agree on this then we can answer how to build the next generation
> XWiki. I don't disagree that we should think about this, because it's going
> to be 10 years in November since I started building XWiki and of course
> technologies evolve and we need to be able to adapt.
>
> I'm looking forward to your opinions on that. Then I propose we discuss for
> each of the use-cases we consider are the future of XWiki what are the
> right technologies. Then we also look at the compatibility of technologies
> with each other. This will give us a lot of answers. One being in which
> direction do we need to put our technological effort on. Another being more
> clear about what users and developers can expect from XWiki.
>
> Ludovic
>
>
>
>
>
> 2013/3/27 Jerome Velociter <[hidden email]>
>
>> Hi Caleb,
>>
>> Great thread, good exercice.
>>
>> Like many of us I imagine I've been giving that question some thoughts
>> already. I've written an article exactly one year ago about this topic [1]
>> (not on rewritting XWiki per-se, but on writing a similar
>> application/platform). My thoughts have partly evolved since then, although
>> I still think most of what I've laid down there at the time.
>>
>> Basically I would go like this :
>>
>> - I would stick to the JVM, and to Java.
>> - I would make it a RESTful web service. And probably even several if I
>> were to be iso in terms of feature. An example of a feature that I would
>> see live in a separated web service (so different process and different API
>> host or subdomain) would be anything open office related. It does not
>> belong to the base service, and would actually benefit being hosted on the
>> same machine as an open office server. This is an example, I believe there
>> are other features that do not belong in the base service (Scheduling ?
>> Mailing ? etc.). It's SOA, though the base service (the base "wiki engine"
>> let say) has to be self-sufficient and offer something that work standalone
>> (even if it means no watch list, no office import, etc.)
>> - Today I'm not so sure about JAX-RS as I was in my article. It's nice and
>> and brings a lot "for free" but now I also think it has some limitations
>> that can be penalties, such as limited dynamicity in the way you can play
>> with routes. Though maybe this is mitigated by JAX-RS 2.0 filters.
>> - Front-end templating would be done with dumb templating, for example
>> handlebars.js or dust.js ; with a flexible intermediary layer for preparing
>> the "context" needed for such templating mechanisms (written as groovy or
>> JS scripts).
>> - I would go for a RDBM (like Postgres) without an "industrial ORM". I
>> think hibernate and friends bring too much complexity for too little value.
>> They tend to impose themselves on the DB design instead of having DB design
>> and object design exists independently and meet at the O/R layer. They make
>> it hard for the developer to know what is executed against the DB (the "gut
>> feeling" can be wrong) and when. Also lazy fetching does not solve any
>> actual problem IMHO.
>> - I would make anything query related (like searching for objects when
>> building applications) happen against the search engine and not the DB.
>> - I would use elasticsearch as a search engine. It can be very easily
>> embedded (for development and simple deployments) or run as a separate
>> cluster, it's schemaless and RESTful by nature.
>> - I would embrace JavaScript fully on the "client" for changing state. It
>> would be hard for me not to throw AngularJS in the mix.
>> - In terms of UX, I would make the "administration" a different UI/UX than
>> the wiki itself, and make it an actual "back office".
>> - ... it's missing a lot of details, I'll come back if I have some time
>>
>> Actually I've been trying to put most of those ideas into action for some
>> months, into a e-commerce/marketplace platform I'm building [2].
>>
>> Now, I'm very eager to read what others think.
>>
>> Jerome
>>
>> [1] http://velociter.fr/journal/**my-idea-of-a-modern-web-app-**on-the-jvm<http://velociter.fr/journal/my-idea-of-a-modern-web-app-on-the-jvm>
>> [2] https://github.com/mayocat/**mayocat-shop/<https://github.com/mayocat/mayocat-shop/>
>>
>>
>>
>> On 03/27/2013 02:12 AM, Caleb James DeLisle wrote:
>>
>>> This is the premise, you are going to write a new XWiki.
>>> You are not bound by any backward compatibility requirements.
>>> Use any technology or combination of technologies.
>>> PHP? C++? Golang? Node.js? Java? Your call!
>>>
>>> Describe it in as much detail as possible.
>>> What frameworks will you use? What ORM? what databases?
>>>
>>> Why will it perform well?
>>> How will you get new features into the hands of users quickly?
>>> How will you avoid this:
>>> http://steve-yegge.blogspot.**ca/2007/12/codes-worst-enemy.**html<http://steve-yegge.blogspot.ca/2007/12/codes-worst-enemy.html>
>>> and this: http://www.laputan.org/mud/
>>>
>>> Think big and go wild!
>>>
>>> Thanks,
>>> Caleb
>>>
>>> ______________________________**_________________
>>> devs mailing list
>>> [hidden email]
>>> http://lists.xwiki.org/**mailman/listinfo/devs<http://lists.xwiki.org/mailman/listinfo/devs>
>>>
>> ______________________________**_________________
>> devs mailing list
>> [hidden email]
>> http://lists.xwiki.org/**mailman/listinfo/devs<http://lists.xwiki.org/mailman/listinfo/devs>
>>
>
>

_______________________________________________
devs mailing list
[hidden email]
http://lists.xwiki.org/mailman/listinfo/devs
Reply | Threaded
Open this post in threaded view
|

Re: [BRAINSTORM] You're rewriting XWiki, you can use any technology in the world, describe it.

Caleb James DeLisle
In reply to this post by Ludovic Dubost
Wow, this has some great thoughts.

When I wrote up the mail I was thinking about XWiki as basically
what it is being used for right now. I want it to be easy for non-devs
to build basic applications and easy for the IT departments and project
managers (XWiki SAS) to make entire customized services from reusable
building blocks.

The most important part in my mind is search. A company can't just throw
their crown jewels up on the web and wait for google to index them.
We need to be able to take dirty data sets and respond to poorly formed
search queries with relevant information quickly. If we can do that
then XWiki will be first in the bookmarks list of the work computer,
just like google is on everybody's home computer.

Second is the user defined model and revision history which make XWiki
what it is. With a mutable model, the technical and non-technical users
at a customer site can work together applying script and domain knowledge
to clean up dirty datasets which might begin their life as a huge word
document or excel sheet and then work their way toward becoming a set of
XObjects with accompanying data collection APIs and report generation
scripts. Without revision history, you don't have the "contribute first
ask questions later" culture which makes wiki collaboration work.

Third is of course a simple way for project teams to build custom
web services. Users have similar needs but they also have edge cases and
may want the site to look completely different, this should be easy
for developers but need not be within reach for non-developers.




I used to think 1 hard drive was plenty for my desktop. When I began
to learn about ZFS I realized that actually I had much more data than
I had ever thought I had and I really cared about disk failure and silent
corruption and the performance of my harddrive was rather poor. Now
I know that I need 9 hard drives and 1 ssd cache device.

XWiki needs to be the ZFS of industrial intelligence.

We need to prove that all companies have much more data than they thought
they had, that all employees and customers produce data all the time
(the vast majority of which is wasted), and that all data can be converted
in to actionable intelligence through collaboration (otherwise it's noise).

Once that narrative becomes universal, the need for XWiki will be obvious.


More inline:


On 03/27/2013 05:30 AM, Ludovic Dubost wrote:

> Hi Caleb,
>
>>From my point of view, before starting to list technologies that would be
> great for a next generation XWiki you first need to define what your exact
> target is for XWiki. It is very important to do that because when I look at
> Jerome's answer I don't necessarly have the feeling he is looking at
> exactly the same objective as I would. While his technology choices might
> make a lot of sense for an eCommerce platform, they might not in the
> Intranet concept.
>
> This is a first big significant task which depending on the answers will
> serve as input on the technology stack to use. Indeed since 2003 when I
> started XWiki a lot of new technologies and also use cases or delivery
> methods have florished. For instance Cloud is now a very important delivery
> method of Web Applications and wether your write for a Cloud service, for
> Software deployment or for both this might lead to different technology
> choices. Mobile is another key delivery method that was not there in 2003.
> On the technology side, SOA, REST, Javascript are everywhere. NoSQL is also
> a key technology to consider.
>
> So let me first try to address the question of what a new XWiki could be
> used for, what it is currently used for, what are the strength of XWiki and
> it's weaknesses.
>
> First the initial objective which is the reason for XWiki:
>
> - building flexible collaborative intranets on top of a Wiki, allowing to
> quickly build applications, integrate these applications with the content
>
> Then a lot of things that users, clients have asked for, developers have
> done on their own:
>
> - providing Knowledge based solutions
> - building applications on top of the "Application" development features of
> XWiki (the extension of the previous use case)
> - providing a full Intranet solution, including what XWiki does natively,
> but also providing "Advanced Profiles", "Groups", "Social networks", a set
> of Collaborative Applications (Forum, files, etc..)
> - building public or internal content web sites with limited editors (CMS
> use case)
> - building a web application using XWiki as a framework, using many or
> little of the XWiki existing functions
> - just doing a very nice Wiki in an Enterprise or Public context
> - doing any of these in the "Cloud" context (multi tenant) and allowing
> users to subscribe to an "use-case" that would be built using the XWiki
> technology
>
> As you can see this is close to the "flavor" discussion we already have and
> I've voluntarily listed the first three which are the flavors we are
> currently discussing because this is at XWiki SAS what we see organizations
> we discuss with are asking for most.
> We have seen the other uses cases also but either we had difficulties
> addressing them or it was less natural to use XWiki to address them.
>
> The "framework" use case is a very important one here, because if we think
> XWiki's objective is to be a Framework objective for ANY web application,
> then this might be incompatible with the Intranet, Knowledge Base and even
> the Wiki use case.
> For instance Jerome's use case does not fit in the "Intranet" use case and
> the problem he has are very different from those you have in an Intranet
> environment.
>
> The "Cloud" one is a very big one also and can lead to radical different
> technology choices. As we can see many new cloud services are build around
> NoSQL and there are some good reason for that. It's because the Cloud
> provider wants a unique platform for all his customers and wants to run the
> service in a fully multi-tenant way on top of this unique storage mecanism.
> The objective here is to reduce the operating cost to the maximum, in order
> to be able to provide a "freemium" service. However this choice might have
> a lot of benefits in the "Cloud" context, but it is a radical choice which
> has a lot of impact if you also want to provide your service as "Software"
> like we do for XWiki. It complexifies the installation, reduces the
> features of your storage (very little relational queries), is a radical new
> technology to learn to build on which is still very young (choosen a NoSQL
> platform today is a very risky bet and supporting multiple technologies is
> a nightmare). Now not supporting NoSQL does not necessarly mean you cannot
> go "Cloud" but this goes against the way of doing business in the cloud and
> complexifies the cloud platform, has scalability issues, etc. Now it's
> impossible to ignore "Cloud" but it's important to understand which type of
> "Cloud". Is is a unique Cloud or is it SaaS where your objective is to be
> able to "host" the service for users and customers. The first choice might
> mean going NoSQL, the second might mean sticking with traditional
> Databases. The first choice might mean that you will have a hard time
> convince "software" users because you also need to convince them to go
> NoSQL.


I think that when we reach the stage where XWiki is "google for your
company". They will suddenly realize they have much more information than
mysql could hold without painful replication.

Of course for people who believe they will be the next twitter, a massively
scalable framework is quite interesting.


>
> For me the biggest success we have is for "Intranets" and it has always
> been the key objective, and what we see is that the "Intranet" use case has
> lead us to have to address the "full Intranet" solution but also the "CMS"
> one as some organizations want a unique platform for both publishing and
> collaboration. There is a lot of pressure also to support all "Social
> Network" features as there are obvious bridges between a software that
> would be the main places to manage profile, groups and discussions and
> another software that want to manage information, knowledge and
> applications.
>
> For me the biggest strength of XWiki is it's flexibility to build
> Applications and to Customize it. The first objective of XWiki was to allow
> organizations to setup a wiki start sharing knowledge and then extend their
> wiki with applications that would organize the information in a better way.
> This need is still here today (Podio is a good example of the Application
> concept without the Wiki part to glue everything together). I believe many
> CMS include more and more "application" features so that you can extend
> your web site with more advance organizations and features (but CMS don't
> have the same collaborative aspects). The power of XWiki is to allow to do
> this fully from your web browser and to do it fast. It is to allow power
> users (not developers) to participate because they don't need full
> developer skill (this one is very important when it comes to the technology
> choices of how XWiki gets extensible). The power of XWiki is to not limit
> the customizations capabilities to the content but also allow to customize
> the UI by overidding all templates and then morph the initial web site you
> created into something else. This power is want allowed XWiki to kind of be
> a full "framework" for web development, but it could be that we have pushed
> it too far or not far enough. These strength are particularly talking for
> some of the users case listed before and for others it can actually be a
> weakness.


I don't think we will get many technologically savvy customers who want
public websites (EG: ecommerce). The market is flooded with competing
frameworks and everybody has their own favorite language. Where we win is
"the framework with versioning and search built in".


>
> The weaknesses of XWiki depend on the use cases. For the "Intranet" use
> case, our weaknesses have been to not work enough on the UI, to not adapt
> enough to the actual intranet use cases (this is where flavors come in).
> Another weakness is not enough separation (but not too much) between
> content and applications. Applications live in XWiki page but we don't have
> enough markers to separate them and not enough "SOA" at the XWiki
> Application level. We can fix this. Other weaknesses are how to address
> mobile, how to address AJAX development and still allowing power-users (not
> developers) to participate in this development. We also took too much time
> to build AppWithinMinutes. There are challenges to allowing power-users to
> develop and still allow this development to be continued by real developers.
>
> On the "framework" use-case it's another analysis. Do we need to allow
> development in the Wiki if XWiki's job is to compete with web-development
> frameworks ? Can we go without hosting the development in an IDE and if so
> should it be Eclipse or a Web based IDE ?
> Is it compatible to provide development features that are more easily
> accessible to non power-user (like velocity is) or can we go with more
> advanced technologies. Etc... there are many questions coming from that.


I spent some time thinking about this and determined that if we are to
support either multi-tenent or multi-node systems, we really need to store
any code which might be changed in the database.

Even with myxwiki.org, editing a template must be done on both nodes.

One option would be for the scripting to be something the user can
git clone and push to a jgit server which is integrated in the wiki,
similar to how Heroku hosts apps.

While that is interesting, it doesn't rank on my list because it is a
significant challenge and is of little benefit to the intranet users.



Thanks,
Caleb



>
> There are many other very interesting questions that will yield different
> answers. For instance should we do a full AJAX UI. What are the
> consequences of it. If we go full AJAX, what about URLs ? What about search
> engine indexing. Can we do CMS if we are full AJAX. Full AJAX means more
> complex to customize also depending on how it's developped.
>
> To summarize we need to "choose" first what we want XWiki to be. Once we
> all agree on this then we can answer how to build the next generation
> XWiki. I don't disagree that we should think about this, because it's going
> to be 10 years in November since I started building XWiki and of course
> technologies evolve and we need to be able to adapt.
>
> I'm looking forward to your opinions on that. Then I propose we discuss for
> each of the use-cases we consider are the future of XWiki what are the
> right technologies. Then we also look at the compatibility of technologies
> with each other. This will give us a lot of answers. One being in which
> direction do we need to put our technological effort on. Another being more
> clear about what users and developers can expect from XWiki.
>
> Ludovic
>
>
>
>
>
> 2013/3/27 Jerome Velociter <[hidden email]>
>
>> Hi Caleb,
>>
>> Great thread, good exercice.
>>
>> Like many of us I imagine I've been giving that question some thoughts
>> already. I've written an article exactly one year ago about this topic [1]
>> (not on rewritting XWiki per-se, but on writing a similar
>> application/platform). My thoughts have partly evolved since then, although
>> I still think most of what I've laid down there at the time.
>>
>> Basically I would go like this :
>>
>> - I would stick to the JVM, and to Java.
>> - I would make it a RESTful web service. And probably even several if I
>> were to be iso in terms of feature. An example of a feature that I would
>> see live in a separated web service (so different process and different API
>> host or subdomain) would be anything open office related. It does not
>> belong to the base service, and would actually benefit being hosted on the
>> same machine as an open office server. This is an example, I believe there
>> are other features that do not belong in the base service (Scheduling ?
>> Mailing ? etc.). It's SOA, though the base service (the base "wiki engine"
>> let say) has to be self-sufficient and offer something that work standalone
>> (even if it means no watch list, no office import, etc.)
>> - Today I'm not so sure about JAX-RS as I was in my article. It's nice and
>> and brings a lot "for free" but now I also think it has some limitations
>> that can be penalties, such as limited dynamicity in the way you can play
>> with routes. Though maybe this is mitigated by JAX-RS 2.0 filters.
>> - Front-end templating would be done with dumb templating, for example
>> handlebars.js or dust.js ; with a flexible intermediary layer for preparing
>> the "context" needed for such templating mechanisms (written as groovy or
>> JS scripts).
>> - I would go for a RDBM (like Postgres) without an "industrial ORM". I
>> think hibernate and friends bring too much complexity for too little value.
>> They tend to impose themselves on the DB design instead of having DB design
>> and object design exists independently and meet at the O/R layer. They make
>> it hard for the developer to know what is executed against the DB (the "gut
>> feeling" can be wrong) and when. Also lazy fetching does not solve any
>> actual problem IMHO.
>> - I would make anything query related (like searching for objects when
>> building applications) happen against the search engine and not the DB.
>> - I would use elasticsearch as a search engine. It can be very easily
>> embedded (for development and simple deployments) or run as a separate
>> cluster, it's schemaless and RESTful by nature.
>> - I would embrace JavaScript fully on the "client" for changing state. It
>> would be hard for me not to throw AngularJS in the mix.
>> - In terms of UX, I would make the "administration" a different UI/UX than
>> the wiki itself, and make it an actual "back office".
>> - ... it's missing a lot of details, I'll come back if I have some time
>>
>> Actually I've been trying to put most of those ideas into action for some
>> months, into a e-commerce/marketplace platform I'm building [2].
>>
>> Now, I'm very eager to read what others think.
>>
>> Jerome
>>
>> [1] http://velociter.fr/journal/**my-idea-of-a-modern-web-app-**on-the-jvm<http://velociter.fr/journal/my-idea-of-a-modern-web-app-on-the-jvm>
>> [2] https://github.com/mayocat/**mayocat-shop/<https://github.com/mayocat/mayocat-shop/>
>>
>>
>>
>> On 03/27/2013 02:12 AM, Caleb James DeLisle wrote:
>>
>>> This is the premise, you are going to write a new XWiki.
>>> You are not bound by any backward compatibility requirements.
>>> Use any technology or combination of technologies.
>>> PHP? C++? Golang? Node.js? Java? Your call!
>>>
>>> Describe it in as much detail as possible.
>>> What frameworks will you use? What ORM? what databases?
>>>
>>> Why will it perform well?
>>> How will you get new features into the hands of users quickly?
>>> How will you avoid this:
>>> http://steve-yegge.blogspot.**ca/2007/12/codes-worst-enemy.**html<http://steve-yegge.blogspot.ca/2007/12/codes-worst-enemy.html>
>>> and this: http://www.laputan.org/mud/
>>>
>>> Think big and go wild!
>>>
>>> Thanks,
>>> Caleb
>>>
>>> ______________________________**_________________
>>> devs mailing list
>>> [hidden email]
>>> http://lists.xwiki.org/**mailman/listinfo/devs<http://lists.xwiki.org/mailman/listinfo/devs>
>>>
>>
>> ______________________________**_________________
>> devs mailing list
>> [hidden email]
>> http://lists.xwiki.org/**mailman/listinfo/devs<http://lists.xwiki.org/mailman/listinfo/devs>
>>
>
>
>

_______________________________________________
devs mailing list
[hidden email]
http://lists.xwiki.org/mailman/listinfo/devs
Reply | Threaded
Open this post in threaded view
|

Re: [BRAINSTORM] You're rewriting XWiki, you can use any technology in the world, describe it.

Caleb James DeLisle
In reply to this post by Caleb James DeLisle
My answer:

XWiki is by definition "the wiki with features", one of the most
important features is free-text search. The only serious text
search solutions are written in Java so the JVM must be a part
of my solution.

I would use two layers of code, the bottom layer being a set of
modules which do not have any means of communication among themselves
so that can not become intertwined. The top layer shall be written
in a popular scripting language and must be kept thin.

I have reviewed Jython, Jruby, Groovy, and Node.js and gave cursory
attention to Velocity, Scala, Rhino, and Clojure.

I am a fan of Node and considered running a Node process external
to the JVM but decided it is too difficult as it would require C++
to hook the garbage collection of "handles" in node and clear their
associated Java objects.

Velocity does not have the flexibility to represent models
and controllers and I don't think Groovy has grown a large enough
community.

Scala is very interesting and might come up in the future, after all,
Twitter is using it. There seems to be a solid community according
to one popularity ranking[1] and it's syntax is quite beautiful by
my standards. One can write functional code or imperative as they
choose so the learning curve is reasonable. It has been faulted for
long compile time. The build must be fast, otherwise developing is a
pain. That said I will continue to watch it.

We must always remember that even if our core devs like their IDEs,
contributors who want to write one small patch will not want to
spend hours importing XWiki so we must not make an IDE a requirement
for developing.

Clojure and Rhino are interesting but like Groovy, they don't have
sufficient communities to get me excited. Clojure is very complete
in it's vision but for people who are not familiar with functional
programming, it's parenthesis soup.

Python community is large and Django-on-Jython was a consideration
but Jython is not getting much attention and it is said to perform
poorly.

Ruby performance is bad but it's said to be a reasonably well
thought out language. It is one of the top ranking languages in terms
of popularity so more users will feel at home, the Jruby engine is
getting serious developer attention and the performance should be
okay since most of the heavy lifting should be implemented in Java
modules.

I choose Jruby for the popularity but continue to watch Scala.

For an MVC framework, I thought about Rails because it has the most
community backing but I have been playing with Rails and found it to
be overbuilt and reeking of jar hell. Sinatra to be far more simple
so I would select Sinatra or a similar lightweight framework and
possibly port to Rails later if need be.

"how to run your Ruby webapp on the XWiki2 Platform" would get
likely contributors excited who otherwise wouldn't care.


I recently had the pleasure of working with HAML template engine
which is a ruby-on-rails favorite. While I am quite impressed with
HAML, I think Jade is even more complete in it's vision. Jade has
been ported to Java so it should be faster than the Ruby
implementation. http://jade-lang.com/

I choose Jade for the views.


Storage
-------


A central part of Ruby on Rails (or Sinatra) is the simplicity of
ActiveRecord. Where Hibernate offers every option one could ever want,
ActiveRecord offers the options that everybody is likely to want in a
simple way.

Using Jruby for the front end, ActiveRecord is the default answer and
JDBC adapters for ActiveRecord on Jruby are available.

Getting the Ruby defined model to be accessible to the Java code is a
difficult problem but there has been some work on this front.
http://blog.liveramp.com/2011/03/28/bringing-rubys-activerecord-to-java/



User defined model is the original killer app of XWiki and since the
heavy normalization used in the current implementation poses some
performance issues, I would have to generate native objects at the
user's request and add and update the database tables on the fly to
handle them. This is a technical challenge but not one which I think
can be avoided.

Where to put the data? Mysql? integrated HSQL/Derby? Hadoop? Cassandra?
One thing I really don't want to deal with is customers who want to
store the data in a weird backward database and then report bugs in my
code because of problems they brought on themselves.

I have a huge soft spot for Cassandra, it allows the code to scale to
big data which we all know every enterprise has and just doesn't know
it yet ;) Since it can be integrated in the JVM without very much hacking,
it can keep the stack I test the same as the stack the user runs.
This means getting ActiveRecord to talk JDO or to talk directly to
Cassandra but the potential benefits are enormous:
"The only wiki which can seriously scale"
"Run your Ruby app in your own Cassandra cloud."
Since this exercise didn't come with a budget or a timeline, I'm
going to go long and say "integrated Cassandra node".

My budget/timeline answer would have been Mysql.


What will be Java
-----------------

So far this sounds a bit like a total port to Ruby. What will stay
in Java land? The answer is that heavy lifting and hot codepaths
will be in Java since it's faster but the Java code will be designed
to be generic in nature.

Wiki syntax rendering and Jade templating will be in Java.
Storage in Cassandra, search via Solr (or possibly ElasticSearch which
I just learned about) and doc import and pdf export would be all in
java (as extensions).


Why will this not become a ball of mud?
---------------------------------------

It should be easy to see why the java layer will not become a big ball
of mud. Each module has no means of communicating with any other module.
I have concluded that dependency injection is too dangerous to use on
a project. In a well designed project DI provides little in the way of
benefits but in a poorly designed project it acts as a Bank of Technical
Debt, sweeping design problems under the carpet and allowing them to
compound until the project becomes truly unmaintainable.

For example: A dependency injector which allows Search module to pull in
Permissions allows Search to:

A: alter the state of Permissions in a way that causes a bug which can
neither be seen in Search nor in Permissions alone but only in the two
together.

B: alter the state of Permissions in a way that causes a bug when Search
is not present, creating a surprise dependency.

Also my design would have no XWikiContext, no globally accessible thread
local ExecutionContext or similar. All of these designs provide back
channels through which modules can unintentionally communicate, leading
to scenarios A and B. A module must only work with the tools which it is
given by the code which called it.



Why will the Ruby code not become a ball of mud?
This is harder because it is the part which is designed to be easy to
alter, it is easy to alter because most customization of the look and
feel will be done here.

Step 1: Use a framework, follow best practices for the framework.
MVC, keep the logic in the controllers.

Step 2: When controllers get too big (1000 lines), start finding code
which can be moved down into the lower level. This is hard!
You have a big controller filled with logic that calls modules like
Search, Permissions, Database and Rendering and you have to subdivide
the logic. You have to write new Search APIs for the Search code which
can't touch Permissions, Rendering or Database. You can choose to write
a wrapper module which wraps Search and Database and provides a richer
API but your API must remain generic. If you are just rewriting the
ugly controller in Java, you're pushing the dirt around.

Suppose you don't keep up with it, what you get is a great big
controller which is an eye sore and nobody likes it.

You have obvious technical debt in one place.

This is infinitely better than shoving the code down into some module
which pulls in what it needs (wants) by dependency injection because
then what you get is subtle technical debt accumulating in places all
over the codebase.

In place of dependency injection, I would use a single initialization
script which lies on the boundary between code and configuration. This
script constructs each module, feeding them their dependencies as they
are constructed. This script is expected to look somewhat ugly, that's
because any dependency problems will concentrate in this one place so
it should be forgiven of some uglyness but watched as it is a metric
of overall project health.


Thanks,
Caleb








[1] http://techcrunch.com/2012/09/12/javascript-tops-latest-programming-language-popularity-ranking-from-redmonk/


On 03/26/2013 09:12 PM, Caleb James DeLisle wrote:

> This is the premise, you are going to write a new XWiki.
> You are not bound by any backward compatibility requirements.
> Use any technology or combination of technologies.
> PHP? C++? Golang? Node.js? Java? Your call!
>
> Describe it in as much detail as possible.
> What frameworks will you use? What ORM? what databases?
>
> Why will it perform well?
> How will you get new features into the hands of users quickly?
> How will you avoid this:
> http://steve-yegge.blogspot.ca/2007/12/codes-worst-enemy.html
> and this: http://www.laputan.org/mud/
>
> Think big and go wild!
>
> Thanks,
> Caleb
>
> _______________________________________________
> devs mailing list
> [hidden email]
> http://lists.xwiki.org/mailman/listinfo/devs
>

_______________________________________________
devs mailing list
[hidden email]
http://lists.xwiki.org/mailman/listinfo/devs
Reply | Threaded
Open this post in threaded view
|

Re: [BRAINSTORM] You're rewriting XWiki, you can use any technology in the world, describe it.

Jérôme Velociter
Le 27/03/13 16:06, Caleb James DeLisle a écrit :

> My answer:
>
> XWiki is by definition "the wiki with features", one of the most
> important features is free-text search. The only serious text
> search solutions are written in Java so the JVM must be a part
> of my solution.
>
> I would use two layers of code, the bottom layer being a set of
> modules which do not have any means of communication among themselves
> so that can not become intertwined. The top layer shall be written
> in a popular scripting language and must be kept thin.
>
> I have reviewed Jython, Jruby, Groovy, and Node.js and gave cursory
> attention to Velocity, Scala, Rhino, and Clojure.
>
> I am a fan of Node and considered running a Node process external
> to the JVM but decided it is too difficult as it would require C++
> to hook the garbage collection of "handles" in node and clear their
> associated Java objects.
>
> Velocity does not have the flexibility to represent models
> and controllers and I don't think Groovy has grown a large enough
> community.
>
> Scala is very interesting and might come up in the future, after all,
> Twitter is using it. There seems to be a solid community according
> to one popularity ranking[1] and it's syntax is quite beautiful by
> my standards. One can write functional code or imperative as they
> choose so the learning curve is reasonable. It has been faulted for
> long compile time. The build must be fast, otherwise developing is a
> pain. That said I will continue to watch it.
>
> We must always remember that even if our core devs like their IDEs,
> contributors who want to write one small patch will not want to
> spend hours importing XWiki so we must not make an IDE a requirement
> for developing.
>
> Clojure and Rhino are interesting but like Groovy, they don't have
> sufficient communities to get me excited. Clojure is very complete
> in it's vision but for people who are not familiar with functional
> programming, it's parenthesis soup.
>
> Python community is large and Django-on-Jython was a consideration
> but Jython is not getting much attention and it is said to perform
> poorly.
>
> Ruby performance is bad but it's said to be a reasonably well
> thought out language. It is one of the top ranking languages in terms
> of popularity so more users will feel at home, the Jruby engine is
> getting serious developer attention and the performance should be
> okay since most of the heavy lifting should be implemented in Java
> modules.
>
> I choose Jruby for the popularity but continue to watch Scala.
>
> For an MVC framework, I thought about Rails because it has the most
> community backing but I have been playing with Rails and found it to
> be overbuilt and reeking of jar hell. Sinatra to be far more simple
> so I would select Sinatra or a similar lightweight framework and
> possibly port to Rails later if need be.
>
> "how to run your Ruby webapp on the XWiki2 Platform" would get
> likely contributors excited who otherwise wouldn't care.
>
>
> I recently had the pleasure of working with HAML template engine
> which is a ruby-on-rails favorite. While I am quite impressed with
> HAML, I think Jade is even more complete in it's vision. Jade has
> been ported to Java so it should be faster than the Ruby
> implementation. http://jade-lang.com/
>
> I choose Jade for the views.
>
>
> Storage
> -------
>
>
> A central part of Ruby on Rails (or Sinatra) is the simplicity of
> ActiveRecord. Where Hibernate offers every option one could ever want,
> ActiveRecord offers the options that everybody is likely to want in a
> simple way.
>
> Using Jruby for the front end, ActiveRecord is the default answer and
> JDBC adapters for ActiveRecord on Jruby are available.
>
> Getting the Ruby defined model to be accessible to the Java code is a
> difficult problem but there has been some work on this front.
> http://blog.liveramp.com/2011/03/28/bringing-rubys-activerecord-to-java/
>
>
>
> User defined model is the original killer app of XWiki and since the
> heavy normalization used in the current implementation poses some
> performance issues, I would have to generate native objects at the
> user's request and add and update the database tables on the fly to
> handle them. This is a technical challenge but not one which I think
> can be avoided.
>
> Where to put the data? Mysql? integrated HSQL/Derby? Hadoop? Cassandra?
> One thing I really don't want to deal with is customers who want to
> store the data in a weird backward database and then report bugs in my
> code because of problems they brought on themselves.
>
> I have a huge soft spot for Cassandra, it allows the code to scale to
> big data which we all know every enterprise has and just doesn't know
> it yet ;) Since it can be integrated in the JVM without very much hacking,
> it can keep the stack I test the same as the stack the user runs.
> This means getting ActiveRecord to talk JDO or to talk directly to
> Cassandra but the potential benefits are enormous:
> "The only wiki which can seriously scale"
> "Run your Ruby app in your own Cassandra cloud."
> Since this exercise didn't come with a budget or a timeline, I'm
> going to go long and say "integrated Cassandra node".
>
> My budget/timeline answer would have been Mysql.
>
>
> What will be Java
> -----------------
>
> So far this sounds a bit like a total port to Ruby. What will stay
> in Java land? The answer is that heavy lifting and hot codepaths
> will be in Java since it's faster but the Java code will be designed
> to be generic in nature.
>
> Wiki syntax rendering and Jade templating will be in Java.
> Storage in Cassandra, search via Solr (or possibly ElasticSearch which
> I just learned about) and doc import and pdf export would be all in
> java (as extensions).
>
>
> Why will this not become a ball of mud?
> ---------------------------------------
>
> It should be easy to see why the java layer will not become a big ball
> of mud. Each module has no means of communicating with any other module.
> I have concluded that dependency injection is too dangerous to use on
> a project. In a well designed project DI provides little in the way of
> benefits but in a poorly designed project it acts as a Bank of Technical
> Debt, sweeping design problems under the carpet and allowing them to
> compound until the project becomes truly unmaintainable.
>
> For example: A dependency injector which allows Search module to pull in
> Permissions allows Search to:
>
> A: alter the state of Permissions in a way that causes a bug which can
> neither be seen in Search nor in Permissions alone but only in the two
> together.
>
> B: alter the state of Permissions in a way that causes a bug when Search
> is not present, creating a surprise dependency.
>
> Also my design would have no XWikiContext, no globally accessible thread
> local ExecutionContext or similar. All of these designs provide back
> channels through which modules can unintentionally communicate, leading
> to scenarios A and B. A module must only work with the tools which it is
> given by the code which called it.
>
>
>
> Why will the Ruby code not become a ball of mud?
> This is harder because it is the part which is designed to be easy to
> alter, it is easy to alter because most customization of the look and
> feel will be done here.
>
> Step 1: Use a framework, follow best practices for the framework.
> MVC, keep the logic in the controllers.
>
> Step 2: When controllers get too big (1000 lines), start finding code
> which can be moved down into the lower level. This is hard!
> You have a big controller filled with logic that calls modules like
> Search, Permissions, Database and Rendering and you have to subdivide
> the logic. You have to write new Search APIs for the Search code which
> can't touch Permissions, Rendering or Database. You can choose to write
> a wrapper module which wraps Search and Database and provides a richer
> API but your API must remain generic. If you are just rewriting the
> ugly controller in Java, you're pushing the dirt around.
>
> Suppose you don't keep up with it, what you get is a great big
> controller which is an eye sore and nobody likes it.
>
> You have obvious technical debt in one place.
>
> This is infinitely better than shoving the code down into some module
> which pulls in what it needs (wants) by dependency injection because
> then what you get is subtle technical debt accumulating in places all
> over the codebase.
>
> In place of dependency injection, I would use a single initialization
> script which lies on the boundary between code and configuration. This
> script constructs each module, feeding them their dependencies as they
> are constructed. This script is expected to look somewhat ugly, that's
> because any dependency problems will concentrate in this one place so
> it should be forgiven of some uglyness but watched as it is a metric
> of overall project health.


Isn't this what people do with Guice or Dagger when configuring instances ?

i.e. the :

     bind(TransactionLog.class).to(DatabaseTransactionLog.class);
bind(CreditCardProcessor.class).to(PaypalCreditCardProcessor.class);
     bind(BillingService.class).to(RealBillingService.class);

part of Guice documentation.

I interpret your reservations against DI as reservations against
auto-wiring of components. Is this correct ?

Jerome.

>
>
> Thanks,
> Caleb
>
>
>
>
>
>
>
>
> [1] http://techcrunch.com/2012/09/12/javascript-tops-latest-programming-language-popularity-ranking-from-redmonk/
>
>
> On 03/26/2013 09:12 PM, Caleb James DeLisle wrote:
>> This is the premise, you are going to write a new XWiki.
>> You are not bound by any backward compatibility requirements.
>> Use any technology or combination of technologies.
>> PHP? C++? Golang? Node.js? Java? Your call!
>>
>> Describe it in as much detail as possible.
>> What frameworks will you use? What ORM? what databases?
>>
>> Why will it perform well?
>> How will you get new features into the hands of users quickly?
>> How will you avoid this:
>> http://steve-yegge.blogspot.ca/2007/12/codes-worst-enemy.html
>> and this: http://www.laputan.org/mud/
>>
>> Think big and go wild!
>>
>> Thanks,
>> Caleb
>>
>> _______________________________________________
>> devs mailing list
>> [hidden email]
>> http://lists.xwiki.org/mailman/listinfo/devs
>>
> _______________________________________________
> devs mailing list
> [hidden email]
> http://lists.xwiki.org/mailman/listinfo/devs

_______________________________________________
devs mailing list
[hidden email]
http://lists.xwiki.org/mailman/listinfo/devs
Reply | Threaded
Open this post in threaded view
|

Re: [BRAINSTORM] You're rewriting XWiki, you can use any technology in the world, describe it.

Caleb James DeLisle


On 03/27/2013 11:22 AM, Jerome Velociter wrote:

> Le 27/03/13 16:06, Caleb James DeLisle a écrit :
>> My answer:
>>
>> XWiki is by definition "the wiki with features", one of the most
>> important features is free-text search. The only serious text
>> search solutions are written in Java so the JVM must be a part
>> of my solution.
>>
>> I would use two layers of code, the bottom layer being a set of
>> modules which do not have any means of communication among themselves
>> so that can not become intertwined. The top layer shall be written
>> in a popular scripting language and must be kept thin.
>>
>> I have reviewed Jython, Jruby, Groovy, and Node.js and gave cursory
>> attention to Velocity, Scala, Rhino, and Clojure.
>>
>> I am a fan of Node and considered running a Node process external
>> to the JVM but decided it is too difficult as it would require C++
>> to hook the garbage collection of "handles" in node and clear their
>> associated Java objects.
>>
>> Velocity does not have the flexibility to represent models
>> and controllers and I don't think Groovy has grown a large enough
>> community.
>>
>> Scala is very interesting and might come up in the future, after all,
>> Twitter is using it. There seems to be a solid community according
>> to one popularity ranking[1] and it's syntax is quite beautiful by
>> my standards. One can write functional code or imperative as they
>> choose so the learning curve is reasonable. It has been faulted for
>> long compile time. The build must be fast, otherwise developing is a
>> pain. That said I will continue to watch it.
>>
>> We must always remember that even if our core devs like their IDEs,
>> contributors who want to write one small patch will not want to
>> spend hours importing XWiki so we must not make an IDE a requirement
>> for developing.
>>
>> Clojure and Rhino are interesting but like Groovy, they don't have
>> sufficient communities to get me excited. Clojure is very complete
>> in it's vision but for people who are not familiar with functional
>> programming, it's parenthesis soup.
>>
>> Python community is large and Django-on-Jython was a consideration
>> but Jython is not getting much attention and it is said to perform
>> poorly.
>>
>> Ruby performance is bad but it's said to be a reasonably well
>> thought out language. It is one of the top ranking languages in terms
>> of popularity so more users will feel at home, the Jruby engine is
>> getting serious developer attention and the performance should be
>> okay since most of the heavy lifting should be implemented in Java
>> modules.
>>
>> I choose Jruby for the popularity but continue to watch Scala.
>>
>> For an MVC framework, I thought about Rails because it has the most
>> community backing but I have been playing with Rails and found it to
>> be overbuilt and reeking of jar hell. Sinatra to be far more simple
>> so I would select Sinatra or a similar lightweight framework and
>> possibly port to Rails later if need be.
>>
>> "how to run your Ruby webapp on the XWiki2 Platform" would get
>> likely contributors excited who otherwise wouldn't care.
>>
>>
>> I recently had the pleasure of working with HAML template engine
>> which is a ruby-on-rails favorite. While I am quite impressed with
>> HAML, I think Jade is even more complete in it's vision. Jade has
>> been ported to Java so it should be faster than the Ruby
>> implementation. http://jade-lang.com/
>>
>> I choose Jade for the views.
>>
>>
>> Storage
>> -------
>>
>>
>> A central part of Ruby on Rails (or Sinatra) is the simplicity of
>> ActiveRecord. Where Hibernate offers every option one could ever want,
>> ActiveRecord offers the options that everybody is likely to want in a
>> simple way.
>>
>> Using Jruby for the front end, ActiveRecord is the default answer and
>> JDBC adapters for ActiveRecord on Jruby are available.
>>
>> Getting the Ruby defined model to be accessible to the Java code is a
>> difficult problem but there has been some work on this front.
>> http://blog.liveramp.com/2011/03/28/bringing-rubys-activerecord-to-java/
>>
>>
>>
>> User defined model is the original killer app of XWiki and since the
>> heavy normalization used in the current implementation poses some
>> performance issues, I would have to generate native objects at the
>> user's request and add and update the database tables on the fly to
>> handle them. This is a technical challenge but not one which I think
>> can be avoided.
>>
>> Where to put the data? Mysql? integrated HSQL/Derby? Hadoop? Cassandra?
>> One thing I really don't want to deal with is customers who want to
>> store the data in a weird backward database and then report bugs in my
>> code because of problems they brought on themselves.
>>
>> I have a huge soft spot for Cassandra, it allows the code to scale to
>> big data which we all know every enterprise has and just doesn't know
>> it yet ;) Since it can be integrated in the JVM without very much hacking,
>> it can keep the stack I test the same as the stack the user runs.
>> This means getting ActiveRecord to talk JDO or to talk directly to
>> Cassandra but the potential benefits are enormous:
>> "The only wiki which can seriously scale"
>> "Run your Ruby app in your own Cassandra cloud."
>> Since this exercise didn't come with a budget or a timeline, I'm
>> going to go long and say "integrated Cassandra node".
>>
>> My budget/timeline answer would have been Mysql.
>>
>>
>> What will be Java
>> -----------------
>>
>> So far this sounds a bit like a total port to Ruby. What will stay
>> in Java land? The answer is that heavy lifting and hot codepaths
>> will be in Java since it's faster but the Java code will be designed
>> to be generic in nature.
>>
>> Wiki syntax rendering and Jade templating will be in Java.
>> Storage in Cassandra, search via Solr (or possibly ElasticSearch which
>> I just learned about) and doc import and pdf export would be all in
>> java (as extensions).
>>
>>
>> Why will this not become a ball of mud?
>> ---------------------------------------
>>
>> It should be easy to see why the java layer will not become a big ball
>> of mud. Each module has no means of communicating with any other module.
>> I have concluded that dependency injection is too dangerous to use on
>> a project. In a well designed project DI provides little in the way of
>> benefits but in a poorly designed project it acts as a Bank of Technical
>> Debt, sweeping design problems under the carpet and allowing them to
>> compound until the project becomes truly unmaintainable.
>>
>> For example: A dependency injector which allows Search module to pull in
>> Permissions allows Search to:
>>
>> A: alter the state of Permissions in a way that causes a bug which can
>> neither be seen in Search nor in Permissions alone but only in the two
>> together.
>>
>> B: alter the state of Permissions in a way that causes a bug when Search
>> is not present, creating a surprise dependency.
>>
>> Also my design would have no XWikiContext, no globally accessible thread
>> local ExecutionContext or similar. All of these designs provide back
>> channels through which modules can unintentionally communicate, leading
>> to scenarios A and B. A module must only work with the tools which it is
>> given by the code which called it.
>>
>>
>>
>> Why will the Ruby code not become a ball of mud?
>> This is harder because it is the part which is designed to be easy to
>> alter, it is easy to alter because most customization of the look and
>> feel will be done here.
>>
>> Step 1: Use a framework, follow best practices for the framework.
>> MVC, keep the logic in the controllers.
>>
>> Step 2: When controllers get too big (1000 lines), start finding code
>> which can be moved down into the lower level. This is hard!
>> You have a big controller filled with logic that calls modules like
>> Search, Permissions, Database and Rendering and you have to subdivide
>> the logic. You have to write new Search APIs for the Search code which
>> can't touch Permissions, Rendering or Database. You can choose to write
>> a wrapper module which wraps Search and Database and provides a richer
>> API but your API must remain generic. If you are just rewriting the
>> ugly controller in Java, you're pushing the dirt around.
>>
>> Suppose you don't keep up with it, what you get is a great big
>> controller which is an eye sore and nobody likes it.
>>
>> You have obvious technical debt in one place.
>>
>> This is infinitely better than shoving the code down into some module
>> which pulls in what it needs (wants) by dependency injection because
>> then what you get is subtle technical debt accumulating in places all
>> over the codebase.
>>
>> In place of dependency injection, I would use a single initialization
>> script which lies on the boundary between code and configuration. This
>> script constructs each module, feeding them their dependencies as they
>> are constructed. This script is expected to look somewhat ugly, that's
>> because any dependency problems will concentrate in this one place so
>> it should be forgiven of some uglyness but watched as it is a metric
>> of overall project health.
>
>
> Isn't this what people do with Guice or Dagger when configuring instances ?
>
> i.e. the :
>
>     bind(TransactionLog.class).to(DatabaseTransactionLog.class);
> bind(CreditCardProcessor.class).to(PaypalCreditCardProcessor.class);
>     bind(BillingService.class).to(RealBillingService.class);
>
> part of Guice documentation.
>
> I interpret your reservations against DI as reservations against auto-wiring of components. Is this correct ?

Hardwiring is not very cool but my main problem is that it allows for
what the big ball of mud people call "Byzantine back channels that circumvent
the system's original structure."

EG:

@Inject private ExecutionContext ec;
...
ec.put("hack", somePieceOfInformation);


...later, in a "completely separate module"....

@Inject private ExecutionContext ec;
...
x = ec.get("hack");


This is a textbook case and it's obvious something is wrong but this
pattern can be highly insidious. Seemingly innocent things like "putting
the User object in the Session" create surprise communication channels
which eventually make individual modules impossible to test without
elaborate mocks and impossible to debug without taking in to consideration
the state of the entire engine.

The problem is if you put in place the systems to slay this monster, DI
becomes far less useful so I would scrap DI entirely and accept the
additional difficulty with designing such things as runtime extension
loading.


Thanks,
Caleb

>
> Jerome.
>
>>
>>
>> Thanks,
>> Caleb
>>
>>
>>
>>
>>
>>
>>
>>
>> [1] http://techcrunch.com/2012/09/12/javascript-tops-latest-programming-language-popularity-ranking-from-redmonk/
>>
>>
>> On 03/26/2013 09:12 PM, Caleb James DeLisle wrote:
>>> This is the premise, you are going to write a new XWiki.
>>> You are not bound by any backward compatibility requirements.
>>> Use any technology or combination of technologies.
>>> PHP? C++? Golang? Node.js? Java? Your call!
>>>
>>> Describe it in as much detail as possible.
>>> What frameworks will you use? What ORM? what databases?
>>>
>>> Why will it perform well?
>>> How will you get new features into the hands of users quickly?
>>> How will you avoid this:
>>> http://steve-yegge.blogspot.ca/2007/12/codes-worst-enemy.html
>>> and this: http://www.laputan.org/mud/
>>>
>>> Think big and go wild!
>>>
>>> Thanks,
>>> Caleb
>>>
>>> _______________________________________________
>>> devs mailing list
>>> [hidden email]
>>> http://lists.xwiki.org/mailman/listinfo/devs
>>>
>> _______________________________________________
>> devs mailing list
>> [hidden email]
>> http://lists.xwiki.org/mailman/listinfo/devs
>
> _______________________________________________
> devs mailing list
> [hidden email]
> http://lists.xwiki.org/mailman/listinfo/devs
>

_______________________________________________
devs mailing list
[hidden email]
http://lists.xwiki.org/mailman/listinfo/devs
Reply | Threaded
Open this post in threaded view
|

Re: [BRAINSTORM] You're rewriting XWiki, you can use any technology in the world, describe it.

jerem
Hi,

If I had to rewrite xwiki, I wouldn't care too much of the "big ball of
mud" possibility, and just use technos, languages and patterns that I find
fun, and that would be easy to use, and attractive to most people, because
anyway it will never really evolve as you would expect. So let's have fun
with it, at least. What I like currently with components and DI is that
it's pretty simple and easy (KISS ?). Guice or spring/plexus wiring never
looked so sexy to me, but that's a very personal opinion. Also I think it's
really a (interesting) challenge, because currently xwiki succeeds pretty
well in most domains,

I don't know so much of the low-level core implementations, but IMO what is
mostly missing currently is a coherent set of graphical widgets. And the
possibility to very easily switch from one to another (possibly provided by
a third-party framework). Because this is what you see from xwiki when you
use it as a "customer", it's a wiki, so it's a web ui.
But even this could be improved with some well-thought new
extensions/integrations, and maybe better insulation of data/widget for
existing ones (livetables...). Livetables are great (because they fit with
xwiki model), but livetables miss many features (compared to an extjs grid
for example). I would like to have the best of both worlds when I develop
an app in xwiki. So I would really push on improving and adding great ui
components, bound to xwiki model when needed, and easily pluggable to any
other data source.

Thanks,
Jeremie




2013/3/28 Caleb James DeLisle <[hidden email]>

>
>
> On 03/27/2013 11:22 AM, Jerome Velociter wrote:
> > Le 27/03/13 16:06, Caleb James DeLisle a écrit :
> >> My answer:
> >>
> >> XWiki is by definition "the wiki with features", one of the most
> >> important features is free-text search. The only serious text
> >> search solutions are written in Java so the JVM must be a part
> >> of my solution.
> >>
> >> I would use two layers of code, the bottom layer being a set of
> >> modules which do not have any means of communication among themselves
> >> so that can not become intertwined. The top layer shall be written
> >> in a popular scripting language and must be kept thin.
> >>
> >> I have reviewed Jython, Jruby, Groovy, and Node.js and gave cursory
> >> attention to Velocity, Scala, Rhino, and Clojure.
> >>
> >> I am a fan of Node and considered running a Node process external
> >> to the JVM but decided it is too difficult as it would require C++
> >> to hook the garbage collection of "handles" in node and clear their
> >> associated Java objects.
> >>
> >> Velocity does not have the flexibility to represent models
> >> and controllers and I don't think Groovy has grown a large enough
> >> community.
> >>
> >> Scala is very interesting and might come up in the future, after all,
> >> Twitter is using it. There seems to be a solid community according
> >> to one popularity ranking[1] and it's syntax is quite beautiful by
> >> my standards. One can write functional code or imperative as they
> >> choose so the learning curve is reasonable. It has been faulted for
> >> long compile time. The build must be fast, otherwise developing is a
> >> pain. That said I will continue to watch it.
> >>
> >> We must always remember that even if our core devs like their IDEs,
> >> contributors who want to write one small patch will not want to
> >> spend hours importing XWiki so we must not make an IDE a requirement
> >> for developing.
> >>
> >> Clojure and Rhino are interesting but like Groovy, they don't have
> >> sufficient communities to get me excited. Clojure is very complete
> >> in it's vision but for people who are not familiar with functional
> >> programming, it's parenthesis soup.
> >>
> >> Python community is large and Django-on-Jython was a consideration
> >> but Jython is not getting much attention and it is said to perform
> >> poorly.
> >>
> >> Ruby performance is bad but it's said to be a reasonably well
> >> thought out language. It is one of the top ranking languages in terms
> >> of popularity so more users will feel at home, the Jruby engine is
> >> getting serious developer attention and the performance should be
> >> okay since most of the heavy lifting should be implemented in Java
> >> modules.
> >>
> >> I choose Jruby for the popularity but continue to watch Scala.
> >>
> >> For an MVC framework, I thought about Rails because it has the most
> >> community backing but I have been playing with Rails and found it to
> >> be overbuilt and reeking of jar hell. Sinatra to be far more simple
> >> so I would select Sinatra or a similar lightweight framework and
> >> possibly port to Rails later if need be.
> >>
> >> "how to run your Ruby webapp on the XWiki2 Platform" would get
> >> likely contributors excited who otherwise wouldn't care.
> >>
> >>
> >> I recently had the pleasure of working with HAML template engine
> >> which is a ruby-on-rails favorite. While I am quite impressed with
> >> HAML, I think Jade is even more complete in it's vision. Jade has
> >> been ported to Java so it should be faster than the Ruby
> >> implementation. http://jade-lang.com/
> >>
> >> I choose Jade for the views.
> >>
> >>
> >> Storage
> >> -------
> >>
> >>
> >> A central part of Ruby on Rails (or Sinatra) is the simplicity of
> >> ActiveRecord. Where Hibernate offers every option one could ever want,
> >> ActiveRecord offers the options that everybody is likely to want in a
> >> simple way.
> >>
> >> Using Jruby for the front end, ActiveRecord is the default answer and
> >> JDBC adapters for ActiveRecord on Jruby are available.
> >>
> >> Getting the Ruby defined model to be accessible to the Java code is a
> >> difficult problem but there has been some work on this front.
> >>
> http://blog.liveramp.com/2011/03/28/bringing-rubys-activerecord-to-java/
> >>
> >>
> >>
> >> User defined model is the original killer app of XWiki and since the
> >> heavy normalization used in the current implementation poses some
> >> performance issues, I would have to generate native objects at the
> >> user's request and add and update the database tables on the fly to
> >> handle them. This is a technical challenge but not one which I think
> >> can be avoided.
> >>
> >> Where to put the data? Mysql? integrated HSQL/Derby? Hadoop? Cassandra?
> >> One thing I really don't want to deal with is customers who want to
> >> store the data in a weird backward database and then report bugs in my
> >> code because of problems they brought on themselves.
> >>
> >> I have a huge soft spot for Cassandra, it allows the code to scale to
> >> big data which we all know every enterprise has and just doesn't know
> >> it yet ;) Since it can be integrated in the JVM without very much
> hacking,
> >> it can keep the stack I test the same as the stack the user runs.
> >> This means getting ActiveRecord to talk JDO or to talk directly to
> >> Cassandra but the potential benefits are enormous:
> >> "The only wiki which can seriously scale"
> >> "Run your Ruby app in your own Cassandra cloud."
> >> Since this exercise didn't come with a budget or a timeline, I'm
> >> going to go long and say "integrated Cassandra node".
> >>
> >> My budget/timeline answer would have been Mysql.
> >>
> >>
> >> What will be Java
> >> -----------------
> >>
> >> So far this sounds a bit like a total port to Ruby. What will stay
> >> in Java land? The answer is that heavy lifting and hot codepaths
> >> will be in Java since it's faster but the Java code will be designed
> >> to be generic in nature.
> >>
> >> Wiki syntax rendering and Jade templating will be in Java.
> >> Storage in Cassandra, search via Solr (or possibly ElasticSearch which
> >> I just learned about) and doc import and pdf export would be all in
> >> java (as extensions).
> >>
> >>
> >> Why will this not become a ball of mud?
> >> ---------------------------------------
> >>
> >> It should be easy to see why the java layer will not become a big ball
> >> of mud. Each module has no means of communicating with any other module.
> >> I have concluded that dependency injection is too dangerous to use on
> >> a project. In a well designed project DI provides little in the way of
> >> benefits but in a poorly designed project it acts as a Bank of Technical
> >> Debt, sweeping design problems under the carpet and allowing them to
> >> compound until the project becomes truly unmaintainable.
> >>
> >> For example: A dependency injector which allows Search module to pull in
> >> Permissions allows Search to:
> >>
> >> A: alter the state of Permissions in a way that causes a bug which can
> >> neither be seen in Search nor in Permissions alone but only in the two
> >> together.
> >>
> >> B: alter the state of Permissions in a way that causes a bug when Search
> >> is not present, creating a surprise dependency.
> >>
> >> Also my design would have no XWikiContext, no globally accessible thread
> >> local ExecutionContext or similar. All of these designs provide back
> >> channels through which modules can unintentionally communicate, leading
> >> to scenarios A and B. A module must only work with the tools which it is
> >> given by the code which called it.
> >>
> >>
> >>
> >> Why will the Ruby code not become a ball of mud?
> >> This is harder because it is the part which is designed to be easy to
> >> alter, it is easy to alter because most customization of the look and
> >> feel will be done here.
> >>
> >> Step 1: Use a framework, follow best practices for the framework.
> >> MVC, keep the logic in the controllers.
> >>
> >> Step 2: When controllers get too big (1000 lines), start finding code
> >> which can be moved down into the lower level. This is hard!
> >> You have a big controller filled with logic that calls modules like
> >> Search, Permissions, Database and Rendering and you have to subdivide
> >> the logic. You have to write new Search APIs for the Search code which
> >> can't touch Permissions, Rendering or Database. You can choose to write
> >> a wrapper module which wraps Search and Database and provides a richer
> >> API but your API must remain generic. If you are just rewriting the
> >> ugly controller in Java, you're pushing the dirt around.
> >>
> >> Suppose you don't keep up with it, what you get is a great big
> >> controller which is an eye sore and nobody likes it.
> >>
> >> You have obvious technical debt in one place.
> >>
> >> This is infinitely better than shoving the code down into some module
> >> which pulls in what it needs (wants) by dependency injection because
> >> then what you get is subtle technical debt accumulating in places all
> >> over the codebase.
> >>
> >> In place of dependency injection, I would use a single initialization
> >> script which lies on the boundary between code and configuration. This
> >> script constructs each module, feeding them their dependencies as they
> >> are constructed. This script is expected to look somewhat ugly, that's
> >> because any dependency problems will concentrate in this one place so
> >> it should be forgiven of some uglyness but watched as it is a metric
> >> of overall project health.
> >
> >
> > Isn't this what people do with Guice or Dagger when configuring
> instances ?
> >
> > i.e. the :
> >
> >     bind(TransactionLog.class).to(DatabaseTransactionLog.class);
> > bind(CreditCardProcessor.class).to(PaypalCreditCardProcessor.class);
> >     bind(BillingService.class).to(RealBillingService.class);
> >
> > part of Guice documentation.
> >
> > I interpret your reservations against DI as reservations against
> auto-wiring of components. Is this correct ?
>
> Hardwiring is not very cool but my main problem is that it allows for
> what the big ball of mud people call "Byzantine back channels that
> circumvent
> the system's original structure."
>
> EG:
>
> @Inject private ExecutionContext ec;
> ...
> ec.put("hack", somePieceOfInformation);
>
>
> ...later, in a "completely separate module"....
>
> @Inject private ExecutionContext ec;
> ...
> x = ec.get("hack");
>
>
> This is a textbook case and it's obvious something is wrong but this
> pattern can be highly insidious. Seemingly innocent things like "putting
> the User object in the Session" create surprise communication channels
> which eventually make individual modules impossible to test without
> elaborate mocks and impossible to debug without taking in to consideration
> the state of the entire engine.
>
> The problem is if you put in place the systems to slay this monster, DI
> becomes far less useful so I would scrap DI entirely and accept the
> additional difficulty with designing such things as runtime extension
> loading.
>
>
> Thanks,
> Caleb
>
> >
> > Jerome.
> >
> >>
> >>
> >> Thanks,
> >> Caleb
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >> [1]
> http://techcrunch.com/2012/09/12/javascript-tops-latest-programming-language-popularity-ranking-from-redmonk/
> >>
> >>
> >> On 03/26/2013 09:12 PM, Caleb James DeLisle wrote:
> >>> This is the premise, you are going to write a new XWiki.
> >>> You are not bound by any backward compatibility requirements.
> >>> Use any technology or combination of technologies.
> >>> PHP? C++? Golang? Node.js? Java? Your call!
> >>>
> >>> Describe it in as much detail as possible.
> >>> What frameworks will you use? What ORM? what databases?
> >>>
> >>> Why will it perform well?
> >>> How will you get new features into the hands of users quickly?
> >>> How will you avoid this:
> >>> http://steve-yegge.blogspot.ca/2007/12/codes-worst-enemy.html
> >>> and this: http://www.laputan.org/mud/
> >>>
> >>> Think big and go wild!
> >>>
> >>> Thanks,
> >>> Caleb
> >>>
> >>> _______________________________________________
> >>> devs mailing list
> >>> [hidden email]
> >>> http://lists.xwiki.org/mailman/listinfo/devs
> >>>
> >> _______________________________________________
> >> devs mailing list
> >> [hidden email]
> >> http://lists.xwiki.org/mailman/listinfo/devs
> >
> > _______________________________________________
> > devs mailing list
> > [hidden email]
> > http://lists.xwiki.org/mailman/listinfo/devs
> >
>
> _______________________________________________
> devs mailing list
> [hidden email]
> http://lists.xwiki.org/mailman/listinfo/devs
>
_______________________________________________
devs mailing list
[hidden email]
http://lists.xwiki.org/mailman/listinfo/devs