Helm Chart Xwiki - GSOC'19

classic Classic list List threaded Threaded
23 messages Options
12
Reply | Threaded
Open this post in threaded view
|

Helm Chart Xwiki - GSOC'19

Ashish Sharma
Hi everyone,
Thanks for having me here

About Me

I am Ashish Sharma, selected as a student for Google Summer of Code. I am
final year student enrolled in Guru Gobind Singh Indraprastha University,
Delhi. I am a resident of India.

Profiles
GitHub -  https://github.com/ashish932/xwiki-helm-chart/
LinkedIn - https://www.linkedin.com/in/ashish932/
Riot - @ashish932:matrix.org

I will be presenting my project "Helm Chart for XWiki" to all of you.
Following
are the relevant details.

Helm Chart for XWiki

Mentors: Shubham Jain, Neha Gupta

Technologies: Kubernetes, Docker, other if required

Overview
The proposed project is a helm chart that would deploy xwiki as highly
available and reliable. It should be configurable with different
databases(either a standalone database or a clustered one) that are
configurable with xwiki. It would give the option to either configure solr
externally (standalone or clustered) or managed within the container. It
should deploy the app on a shared file system like a rook. It should
support Istio virtual services, istio matrix, and istio distributed tracing
and should be a secured system with RBAC and security credential rotation.
The chart should be easily deployed on GKE and amazon EKS.

Features

->  Support for different Databases
->  Choice between using an external database, a single node DB or a
multi-cluster DB setup
-> Support for shared file system
-> Support for istio and it's services
-> RBAC, SSL and other security methods

If you have any features in mind that should be added please feel free to
reply to this mail.

Some Design Questions?
-> Which Databases should be supported?
-> As we have to detach solr out of the docker container(run it in an
independent container) would be there a requirement for a code change, and
we should approach it?
-> Apart from solr is there any other stateful service that could or should
be detached from the docker container?

Here is my current repository which deploys XWiki for MySQL database using
official XWiki docker container:-
 https://github.com/ashish932/xwiki-helm-chart/


Thank You
Ashish Sharma
Reply | Threaded
Open this post in threaded view
|

Re: Helm Chart Xwiki - GSOC'19

vmassol
Administrator
Hi Ashish,

> On 21 May 2019, at 19:37, Ashish Sharma <[hidden email]> wrote:
>
> Hi everyone,
> Thanks for having me here
>
> About Me
>
> I am Ashish Sharma, selected as a student for Google Summer of Code. I am
> final year student enrolled in Guru Gobind Singh Indraprastha University,
> Delhi. I am a resident of India.
>
> Profiles
> GitHub -  https://github.com/ashish932/xwiki-helm-chart/
> LinkedIn - https://www.linkedin.com/in/ashish932/
> Riot - @ashish932:matrix.org
>
> I will be presenting my project "Helm Chart for XWiki" to all of you.
> Following
> are the relevant details.
>
> Helm Chart for XWiki
>
> Mentors: Shubham Jain, Neha Gupta
>
> Technologies: Kubernetes, Docker, other if required
>
> Overview
> The proposed project is a helm chart that would deploy xwiki as highly
> available and reliable. It should be configurable with different
> databases(either a standalone database or a clustered one) that are
> configurable with xwiki. It would give the option to either configure solr
> externally (standalone or clustered) or managed within the container. It
> should deploy the app on a shared file system like a rook. It should
> support Istio virtual services, istio matrix, and istio distributed tracing
> and should be a secured system with RBAC and security credential rotation.
> The chart should be easily deployed on GKE and amazon EKS.
>
> Features
>
> ->  Support for different Databases
> ->  Choice between using an external database, a single node DB or a
> multi-cluster DB setup
> -> Support for shared file system
> -> Support for istio and it's services
> -> RBAC, SSL and other security methods
>
> If you have any features in mind that should be added please feel free to
> reply to this mail.
>
> Some Design Questions?
> -> Which Databases should be supported?
> -> As we have to detach solr out of the docker container(run it in an
> independent container) would be there a requirement for a code change, and
> we should approach it?
> -> Apart from solr is there any other stateful service that could or should
> be detached from the docker container?
>
> Here is my current repository which deploys XWiki for MySQL database using
> official XWiki docker container:-
> https://github.com/ashish932/xwiki-helm-chart/

Sounds good.

I’ll read your mail in detail a bit later and try to reply to some question. For now, here’s a small detail: we need a repo located inside the xwiki-contrib github organization. See https://contrib.xwiki.org/

Thanks!
-Vincent

> Thank You
> Ashish Sharma

Reply | Threaded
Open this post in threaded view
|

Re: Helm Chart Xwiki - GSOC'19

vmassol
Administrator
In reply to this post by Ashish Sharma
Hi Ashish,

> On 21 May 2019, at 19:37, Ashish Sharma <[hidden email]> wrote:
>
> Hi everyone,
> Thanks for having me here
>
> About Me
>
> I am Ashish Sharma, selected as a student for Google Summer of Code. I am
> final year student enrolled in Guru Gobind Singh Indraprastha University,
> Delhi. I am a resident of India.
>
> Profiles
> GitHub -  https://github.com/ashish932/xwiki-helm-chart/
> LinkedIn - https://www.linkedin.com/in/ashish932/
> Riot - @ashish932:matrix.org
>
> I will be presenting my project "Helm Chart for XWiki" to all of you.
> Following
> are the relevant details.
>
> Helm Chart for XWiki
>
> Mentors: Shubham Jain, Neha Gupta
>
> Technologies: Kubernetes, Docker, other if required
>
> Overview
> The proposed project is a helm chart that would deploy xwiki as highly
> available and reliable. It should be configurable with different
> databases(either a standalone database or a clustered one) that are
> configurable with xwiki. It would give the option to either configure solr
> externally (standalone or clustered) or managed within the container.

It also needs to be deployed on various Servlet Containers and with a LibreOffice server configured.

In short, it should support all the configurations supported by the official XWiki Docker images :)

> It
> should deploy the app on a shared file system like a rook. It should
> support Istio virtual services, istio matrix, and istio distributed tracing
> and should be a secured system with RBAC and security credential rotation.

Ok I have no knowledge about this… The shared file system sounds good to support clustering XWiki for the permanent directory sharing.

> The chart should be easily deployed on GKE and amazon EKS.

On locally on minikube for testing I guess?

> Features
>
> ->  Support for different Databases
> ->  Choice between using an external database, a single node DB or a
> multi-cluster DB setup
> -> Support for shared file system
> -> Support for istio and it's services
> -> RBAC, SSL and other security methods
>
> If you have any features in mind that should be added please feel free to
> reply to this mail.
>
> Some Design Questions?
> -> Which Databases should be supported?

See https://dev.xwiki.org/xwiki/bin/view/Community/SupportStrategy/DatabaseSupportStrategy

More generally see the XWiki Docker image documentation and https://dev.xwiki.org/xwiki/bin/view/Community/SupportStrategy

> -> As we have to detach solr out of the docker container(run it in an
> independent container) would be there a requirement for a code change, and
> we should approach it?

See the doc for Solr which explains how to configure an external SOLR. There are some XWiki code/config to deploy in the external SOLR AFAIR.

> -> Apart from solr is there any other stateful service that could or should
> be detached from the docker container?

Which docker container are you referring to? The Servlet Container one?

> Here is my current repository which deploys XWiki for MySQL database using
> official XWiki docker container:-
> https://github.com/ashish932/xwiki-helm-chart/

Will need to have a look. Some additional questions:

1) Is there an official place for publishing helm charts, similar to dockerhub for docker images? How do you provide the helm charts to XWiki users in term of packaging? In other words, what will we/you document on https://www.xwiki.org/xwiki/bin/view/Documentation/AdminGuide/Installation/ ?

2) How do you plan to have smoke tests for the XWiki Helm charts, as part of the build, to verify that they work fine? How can we execute XWik’s functional tests based on TestContainers using Helm Charts?

3) Do the helm charts for XWiki need to be built? If so, can it be done with Maven? Since we’ll be offering several I guess, how do you avoid code duplication and make maintenance of them as simple as possible?

4) Could you provide a roadmap with various milestones and dates? See the last roadmap email I sent on the devs list and please reply to it.

5) It seems you missed the bonding period so you’ll need to catch up on that. Could you pick some jira issue and work on them quickly? This is to make sure you use the proper xwiki communication channels, learn to work with the xwiki team and start understanding the xwiki architecture, and how extensions are developed for XWiki. Since you’re a bit late, you’ll need to catch up and achieve this very quickly now and be present full time on the xwiki chat :)

6) Idea: please review the issues on the XWiki Docker project’s jira. Maybe you can help with those too as part of onboarding/bonding. It should also give you additional use cases for the XWiki Helm Charts IMO.

Thanks! I’m eager to follow your work and see xwiki helm charts in action.

-Vincent

>
>
> Thank You
> Ashish Sharma

Reply | Threaded
Open this post in threaded view
|

Re: Helm Chart Xwiki - GSOC'19

Ashish Sharma
On Wed, May 22, 2019 at 12:42 PM Vincent Massol <[hidden email]> wrote:

> Hi Ashish,
>
> > On 21 May 2019, at 19:37, Ashish Sharma <[hidden email]>
> wrote:
> >
> > Hi everyone,
> > Thanks for having me here
> >
> > About Me
> >
> > I am Ashish Sharma, selected as a student for Google Summer of Code. I am
> > final year student enrolled in Guru Gobind Singh Indraprastha University,
> > Delhi. I am a resident of India.
> >
> > Profiles
> > GitHub -  https://github.com/ashish932/xwiki-helm-chart/
> > LinkedIn - https://www.linkedin.com/in/ashish932/
> > Riot - @ashish932:matrix.org
> >
> > I will be presenting my project "Helm Chart for XWiki" to all of you.
> > Following
> > are the relevant details.
> >
> > Helm Chart for XWiki
> >
> > Mentors: Shubham Jain, Neha Gupta
> >
> > Technologies: Kubernetes, Docker, other if required
> >
> > Overview
> > The proposed project is a helm chart that would deploy xwiki as highly
> > available and reliable. It should be configurable with different
> > databases(either a standalone database or a clustered one) that are
> > configurable with xwiki. It would give the option to either configure
> solr
> > externally (standalone or clustered) or managed within the container.
>
> It also needs to be deployed on various Servlet Containers and with a
> LibreOffice server configured.
>
> In short, it should support all the configurations supported by the
> official XWiki Docker images :)
>
> > It
> > should deploy the app on a shared file system like a rook. It should
> > support Istio virtual services, istio matrix, and istio distributed
> tracing
> > and should be a secured system with RBAC and security credential
> rotation.
>
> Ok I have no knowledge about this… The shared file system sounds good to
> support clustering XWiki for the permanent directory sharing.
>
> > The chart should be easily deployed on GKE and amazon EKS.
>
> On locally on minikube for testing I guess?
>

 Yes we can test using minikube one

>
> > Features
> >
> > ->  Support for different Databases
> > ->  Choice between using an external database, a single node DB or a
> > multi-cluster DB setup
> > -> Support for shared file system
> > -> Support for istio and it's services
> > -> RBAC, SSL and other security methods
> >
> > If you have any features in mind that should be added please feel free to
> > reply to this mail.
> >
> > Some Design Questions?
> > -> Which Databases should be supported?
>
> See
> https://dev.xwiki.org/xwiki/bin/view/Community/SupportStrategy/DatabaseSupportStrategy
>
> More generally see the XWiki Docker image documentation and
> https://dev.xwiki.org/xwiki/bin/view/Community/SupportStrategy
>
> > -> As we have to detach solr out of the docker container(run it in an
> > independent container) would be there a requirement for a code change,
> and
> > we should approach it?
>
> See the doc for Solr which explains how to configure an external SOLR.
> There are some XWiki code/config to deploy in the external SOLR AFAIR.
> comunity controls chart
> > -> Apart from solr is there any other stateful service that could or
> should
> > be detached from the docker container?
>
> Which docker container are you referring to? The Servlet Container one?
>
> xwiki:lts-mysql-tomcat


> > Here is my current repository which deploys XWiki for MySQL database
> using
> > official XWiki docker container:-
> > https://github.com/ashish932/xwiki-helm-chart/
>
> Will need to have a look. Some additional questions:
>
> 1) Is there an official place for publishing helm charts, similar to
> dockerhub for docker images? How do you provide the helm charts to XWiki
> users in term of packaging? In other words, what will we/you document on
> https://www.xwiki.org/xwiki/bin/view/Documentation/AdminGuide/Installation/
> ?
>
> Yes, There is a centralized community control chart management
repository. Helm charts are hosted there. Organizations like mysql,
wordpress uses it.
https://github.com/helm/charts
But in addition to that, we can build our own charts repository locally,
and host it. Similar to docker run command, we have helm update command


> 2) How do you plan to have smoke tests for the XWiki Helm charts, as part
> of the build, to verify that they work fine? How can we execute XWik’s
> functional tests based on TestContainers using Helm Charts?
>

There are some tools available
chart-testing(started by helm) -98 commits
terratest - 1039 commits (It looks a good tool to use)


> 3) Do the helm charts for XWiki need to be built? If so, can it be done
> with Maven? Since we’ll be offering several I guess, how do you avoid code
> duplication and make maintenance of them as simple as possible?
>

No, the charts don’t need to be built externally, though it gets compressed
when pushed f we build to chart repository(this is an automatic step in
case of official chart repo and we can automate using ci our own chart repo)

4) Could you provide a roadmap with various milestones and dates? See the
> last roadmap email I sent on the devs list and please reply to it.
>

Okay I would work on that and update the progress on the mail thread


> 5) It seems you missed the bonding period so you’ll need to catch up on
> that. Could you pick some jira issue and work on them quickly? This is to
> make sure you use the proper xwiki communication channels, learn to work
> with the xwiki team and start understanding the xwiki architecture, and how
> extensions are developed for XWiki. Since you’re a bit late, you’ll need to
> catch up and achieve this very quickly now and be present full time on the
> xwiki chat :)
>
I would start doing that

>
> 6) Idea: please review the issues on the XWiki Docker project’s jira.
> Maybe you can help with those too as part of onboarding/bonding. It should
> also give you additional use cases for the XWiki Helm Charts IMO.
>
> Thanks! I’m eager to follow your work and see xwiki helm charts in action.
>
> -Vincent
>
> >
> >
> > Thank You
> > Ashish Sharma
>
> Thank You
Ashish
Reply | Threaded
Open this post in threaded view
|

Re: Helm Chart Xwiki - GSOC'19

Ashish Sharma
Hi everyone,

The first week update of helm chart xwiki project. We currently have a
chart that would deploy xwiki mysql stateful service with a persistent
volume. The link to the repo:-

https://github.com/xwiki-contrib/xwiki-helm

Thank You
Ashish

On Thu, 23 May 2019 at 7:27 PM, Ashish Sharma <[hidden email]>
wrote:

>
>
> On Wed, May 22, 2019 at 12:42 PM Vincent Massol <[hidden email]>
> wrote:
>
>> Hi Ashish,
>>
>> > On 21 May 2019, at 19:37, Ashish Sharma <[hidden email]>
>> wrote:
>> >
>> > Hi everyone,
>> > Thanks for having me here
>> >
>> > About Me
>> >
>> > I am Ashish Sharma, selected as a student for Google Summer of Code. I
>> am
>> > final year student enrolled in Guru Gobind Singh Indraprastha
>> University,
>> > Delhi. I am a resident of India.
>> >
>> > Profiles
>> > GitHub -  https://github.com/ashish932/xwiki-helm-chart/
>> > LinkedIn - https://www.linkedin.com/in/ashish932/
>> > Riot - @ashish932:matrix.org
>> >
>> > I will be presenting my project "Helm Chart for XWiki" to all of you.
>> > Following
>> > are the relevant details.
>> >
>> > Helm Chart for XWiki
>> >
>> > Mentors: Shubham Jain, Neha Gupta
>> >
>> > Technologies: Kubernetes, Docker, other if required
>> >
>> > Overview
>> > The proposed project is a helm chart that would deploy xwiki as highly
>> > available and reliable. It should be configurable with different
>> > databases(either a standalone database or a clustered one) that are
>> > configurable with xwiki. It would give the option to either configure
>> solr
>> > externally (standalone or clustered) or managed within the container.
>>
>> It also needs to be deployed on various Servlet Containers and with a
>> LibreOffice server configured.
>>
>> In short, it should support all the configurations supported by the
>> official XWiki Docker images :)
>>
>> > It
>> > should deploy the app on a shared file system like a rook. It should
>> > support Istio virtual services, istio matrix, and istio distributed
>> tracing
>> > and should be a secured system with RBAC and security credential
>> rotation.
>>
>> Ok I have no knowledge about this… The shared file system sounds good to
>> support clustering XWiki for the permanent directory sharing.
>>
>> > The chart should be easily deployed on GKE and amazon EKS.
>>
>> On locally on minikube for testing I guess?
>>
>
>  Yes we can test using minikube one
>
>>
>> > Features
>> >
>> > ->  Support for different Databases
>> > ->  Choice between using an external database, a single node DB or a
>> > multi-cluster DB setup
>> > -> Support for shared file system
>> > -> Support for istio and it's services
>> > -> RBAC, SSL and other security methods
>> >
>> > If you have any features in mind that should be added please feel free
>> to
>> > reply to this mail.
>> >
>> > Some Design Questions?
>> > -> Which Databases should be supported?
>>
>> See
>> https://dev.xwiki.org/xwiki/bin/view/Community/SupportStrategy/DatabaseSupportStrategy
>>
>> More generally see the XWiki Docker image documentation and
>> https://dev.xwiki.org/xwiki/bin/view/Community/SupportStrategy
>>
>> > -> As we have to detach solr out of the docker container(run it in an
>> > independent container) would be there a requirement for a code change,
>> and
>> > we should approach it?
>>
>> See the doc for Solr which explains how to configure an external SOLR.
>> There are some XWiki code/config to deploy in the external SOLR AFAIR.
>> comunity controls chart
>> > -> Apart from solr is there any other stateful service that could or
>> should
>> > be detached from the docker container?
>>
>> Which docker container are you referring to? The Servlet Container one?
>>
>> xwiki:lts-mysql-tomcat
>
>
>> > Here is my current repository which deploys XWiki for MySQL database
>> using
>> > official XWiki docker container:-
>> > https://github.com/ashish932/xwiki-helm-chart/
>>
>> Will need to have a look. Some additional questions:
>>
>> 1) Is there an official place for publishing helm charts, similar to
>> dockerhub for docker images? How do you provide the helm charts to XWiki
>> users in term of packaging? In other words, what will we/you document on
>> https://www.xwiki.org/xwiki/bin/view/Documentation/AdminGuide/Installation/
>> ?
>>
>> Yes, There is a centralized community control chart management
> repository. Helm charts are hosted there. Organizations like mysql,
> wordpress uses it.
> https://github.com/helm/charts
> But in addition to that, we can build our own charts repository locally,
> and host it. Similar to docker run command, we have helm update command
>
>
>> 2) How do you plan to have smoke tests for the XWiki Helm charts, as part
>> of the build, to verify that they work fine? How can we execute XWik’s
>> functional tests based on TestContainers using Helm Charts?
>>
>
> There are some tools available
> chart-testing(started by helm) -98 commits
> terratest - 1039 commits (It looks a good tool to use)
>
>
>> 3) Do the helm charts for XWiki need to be built? If so, can it be done
>> with Maven? Since we’ll be offering several I guess, how do you avoid code
>> duplication and make maintenance of them as simple as possible?
>>
>
> No, the charts don’t need to be built externally, though it gets
> compressed when pushed f we build to chart repository(this is an automatic
> step in case of official chart repo and we can automate using ci our own
> chart repo)
>
> 4) Could you provide a roadmap with various milestones and dates? See the
>> last roadmap email I sent on the devs list and please reply to it.
>>
>
> Okay I would work on that and update the progress on the mail thread
>
>
>> 5) It seems you missed the bonding period so you’ll need to catch up on
>> that. Could you pick some jira issue and work on them quickly? This is to
>> make sure you use the proper xwiki communication channels, learn to work
>> with the xwiki team and start understanding the xwiki architecture, and how
>> extensions are developed for XWiki. Since you’re a bit late, you’ll need to
>> catch up and achieve this very quickly now and be present full time on the
>> xwiki chat :)
>>
> I would start doing that
>
>>
>> 6) Idea: please review the issues on the XWiki Docker project’s jira.
>> Maybe you can help with those too as part of onboarding/bonding. It should
>> also give you additional use cases for the XWiki Helm Charts IMO.
>>
>> Thanks! I’m eager to follow your work and see xwiki helm charts in action.
>>
>> -Vincent
>>
>> >
>> >
>> > Thank You
>> > Ashish Sharma
>>
>> Thank You
> Ashish
>
Reply | Threaded
Open this post in threaded view
|

Re: Helm Chart Xwiki - GSOC'19

vmassol
Administrator
Hi Ashish,

> On 5 Jun 2019, at 15:07, Ashish Sharma <[hidden email]> wrote:
>
> Hi everyone,
>
> The first week update of helm chart xwiki project. We currently have a
> chart that would deploy xwiki mysql stateful service with a persistent
> volume. The link to the repo:-
>
> https://github.com/xwiki-contrib/xwiki-helm

Thanks for the update.

Would be great if you could update the README file to explain who to use it.

Thanks
-Vincent


>
> Thank You
> Ashish
>
> On Thu, 23 May 2019 at 7:27 PM, Ashish Sharma <[hidden email]>
> wrote:
>
>>
>>
>> On Wed, May 22, 2019 at 12:42 PM Vincent Massol <[hidden email]>
>> wrote:
>>
>>> Hi Ashish,
>>>
>>>> On 21 May 2019, at 19:37, Ashish Sharma <[hidden email]>
>>> wrote:
>>>>
>>>> Hi everyone,
>>>> Thanks for having me here
>>>>
>>>> About Me
>>>>
>>>> I am Ashish Sharma, selected as a student for Google Summer of Code. I
>>> am
>>>> final year student enrolled in Guru Gobind Singh Indraprastha
>>> University,
>>>> Delhi. I am a resident of India.
>>>>
>>>> Profiles
>>>> GitHub -  https://github.com/ashish932/xwiki-helm-chart/
>>>> LinkedIn - https://www.linkedin.com/in/ashish932/
>>>> Riot - @ashish932:matrix.org
>>>>
>>>> I will be presenting my project "Helm Chart for XWiki" to all of you.
>>>> Following
>>>> are the relevant details.
>>>>
>>>> Helm Chart for XWiki
>>>>
>>>> Mentors: Shubham Jain, Neha Gupta
>>>>
>>>> Technologies: Kubernetes, Docker, other if required
>>>>
>>>> Overview
>>>> The proposed project is a helm chart that would deploy xwiki as highly
>>>> available and reliable. It should be configurable with different
>>>> databases(either a standalone database or a clustered one) that are
>>>> configurable with xwiki. It would give the option to either configure
>>> solr
>>>> externally (standalone or clustered) or managed within the container.
>>>
>>> It also needs to be deployed on various Servlet Containers and with a
>>> LibreOffice server configured.
>>>
>>> In short, it should support all the configurations supported by the
>>> official XWiki Docker images :)
>>>
>>>> It
>>>> should deploy the app on a shared file system like a rook. It should
>>>> support Istio virtual services, istio matrix, and istio distributed
>>> tracing
>>>> and should be a secured system with RBAC and security credential
>>> rotation.
>>>
>>> Ok I have no knowledge about this… The shared file system sounds good to
>>> support clustering XWiki for the permanent directory sharing.
>>>
>>>> The chart should be easily deployed on GKE and amazon EKS.
>>>
>>> On locally on minikube for testing I guess?
>>>
>>
>> Yes we can test using minikube one
>>
>>>
>>>> Features
>>>>
>>>> ->  Support for different Databases
>>>> ->  Choice between using an external database, a single node DB or a
>>>> multi-cluster DB setup
>>>> -> Support for shared file system
>>>> -> Support for istio and it's services
>>>> -> RBAC, SSL and other security methods
>>>>
>>>> If you have any features in mind that should be added please feel free
>>> to
>>>> reply to this mail.
>>>>
>>>> Some Design Questions?
>>>> -> Which Databases should be supported?
>>>
>>> See
>>> https://dev.xwiki.org/xwiki/bin/view/Community/SupportStrategy/DatabaseSupportStrategy
>>>
>>> More generally see the XWiki Docker image documentation and
>>> https://dev.xwiki.org/xwiki/bin/view/Community/SupportStrategy
>>>
>>>> -> As we have to detach solr out of the docker container(run it in an
>>>> independent container) would be there a requirement for a code change,
>>> and
>>>> we should approach it?
>>>
>>> See the doc for Solr which explains how to configure an external SOLR.
>>> There are some XWiki code/config to deploy in the external SOLR AFAIR.
>>> comunity controls chart
>>>> -> Apart from solr is there any other stateful service that could or
>>> should
>>>> be detached from the docker container?
>>>
>>> Which docker container are you referring to? The Servlet Container one?
>>>
>>> xwiki:lts-mysql-tomcat
>>
>>
>>>> Here is my current repository which deploys XWiki for MySQL database
>>> using
>>>> official XWiki docker container:-
>>>> https://github.com/ashish932/xwiki-helm-chart/
>>>
>>> Will need to have a look. Some additional questions:
>>>
>>> 1) Is there an official place for publishing helm charts, similar to
>>> dockerhub for docker images? How do you provide the helm charts to XWiki
>>> users in term of packaging? In other words, what will we/you document on
>>> https://www.xwiki.org/xwiki/bin/view/Documentation/AdminGuide/Installation/
>>> ?
>>>
>>> Yes, There is a centralized community control chart management
>> repository. Helm charts are hosted there. Organizations like mysql,
>> wordpress uses it.
>> https://github.com/helm/charts
>> But in addition to that, we can build our own charts repository locally,
>> and host it. Similar to docker run command, we have helm update command
>>
>>
>>> 2) How do you plan to have smoke tests for the XWiki Helm charts, as part
>>> of the build, to verify that they work fine? How can we execute XWik’s
>>> functional tests based on TestContainers using Helm Charts?
>>>
>>
>> There are some tools available
>> chart-testing(started by helm) -98 commits
>> terratest - 1039 commits (It looks a good tool to use)
>>
>>
>>> 3) Do the helm charts for XWiki need to be built? If so, can it be done
>>> with Maven? Since we’ll be offering several I guess, how do you avoid code
>>> duplication and make maintenance of them as simple as possible?
>>>
>>
>> No, the charts don’t need to be built externally, though it gets
>> compressed when pushed f we build to chart repository(this is an automatic
>> step in case of official chart repo and we can automate using ci our own
>> chart repo)
>>
>> 4) Could you provide a roadmap with various milestones and dates? See the
>>> last roadmap email I sent on the devs list and please reply to it.
>>>
>>
>> Okay I would work on that and update the progress on the mail thread
>>
>>
>>> 5) It seems you missed the bonding period so you’ll need to catch up on
>>> that. Could you pick some jira issue and work on them quickly? This is to
>>> make sure you use the proper xwiki communication channels, learn to work
>>> with the xwiki team and start understanding the xwiki architecture, and how
>>> extensions are developed for XWiki. Since you’re a bit late, you’ll need to
>>> catch up and achieve this very quickly now and be present full time on the
>>> xwiki chat :)
>>>
>> I would start doing that
>>
>>>
>>> 6) Idea: please review the issues on the XWiki Docker project’s jira.
>>> Maybe you can help with those too as part of onboarding/bonding. It should
>>> also give you additional use cases for the XWiki Helm Charts IMO.
>>>
>>> Thanks! I’m eager to follow your work and see xwiki helm charts in action.
>>>
>>> -Vincent
>>>
>>>>
>>>>
>>>> Thank You
>>>> Ashish Sharma
>>>
>>> Thank You
>> Ashish
>>

Reply | Threaded
Open this post in threaded view
|

Re: Helm Chart Xwiki - GSOC'19

Ashish Sharma
Hi everyone,

Update on HELM project:-

We have added some features to the chart that includes:-
* Making chart configurable with both mysql and postgres database
* We are now using configmaps and secrets for storing database credentials
* We use helm dependency to manage the dependencies of the chart

I think our chart is ready to be deployed in the incubator section of helm
community repo. <https://github.com/helm/charts> If you guys agree I would
make the necessary changes required to deploy it there.

Thanks
- Ashish

On Wed, Jun 5, 2019 at 7:40 PM Vincent Massol <[hidden email]> wrote:

> Hi Ashish,
>
> > On 5 Jun 2019, at 15:07, Ashish Sharma <[hidden email]>
> wrote:
> >
> > Hi everyone,
> >
> > The first week update of helm chart xwiki project. We currently have a
> > chart that would deploy xwiki mysql stateful service with a persistent
> > volume. The link to the repo:-
> >
> > https://github.com/xwiki-contrib/xwiki-helm
>
> Thanks for the update.
>
> Would be great if you could update the README file to explain who to use
> it.
>
> Thanks
> -Vincent
>
>
> >
> > Thank You
> > Ashish
> >
> > On Thu, 23 May 2019 at 7:27 PM, Ashish Sharma <
> [hidden email]>
> > wrote:
> >
> >>
> >>
> >> On Wed, May 22, 2019 at 12:42 PM Vincent Massol <[hidden email]>
> >> wrote:
> >>
> >>> Hi Ashish,
> >>>
> >>>> On 21 May 2019, at 19:37, Ashish Sharma <[hidden email]>
> >>> wrote:
> >>>>
> >>>> Hi everyone,
> >>>> Thanks for having me here
> >>>>
> >>>> About Me
> >>>>
> >>>> I am Ashish Sharma, selected as a student for Google Summer of Code. I
> >>> am
> >>>> final year student enrolled in Guru Gobind Singh Indraprastha
> >>> University,
> >>>> Delhi. I am a resident of India.
> >>>>
> >>>> Profiles
> >>>> GitHub -  https://github.com/ashish932/xwiki-helm-chart/
> >>>> LinkedIn - https://www.linkedin.com/in/ashish932/
> >>>> Riot - @ashish932:matrix.org
> >>>>
> >>>> I will be presenting my project "Helm Chart for XWiki" to all of you.
> >>>> Following
> >>>> are the relevant details.
> >>>>
> >>>> Helm Chart for XWiki
> >>>>
> >>>> Mentors: Shubham Jain, Neha Gupta
> >>>>
> >>>> Technologies: Kubernetes, Docker, other if required
> >>>>
> >>>> Overview
> >>>> The proposed project is a helm chart that would deploy xwiki as highly
> >>>> available and reliable. It should be configurable with different
> >>>> databases(either a standalone database or a clustered one) that are
> >>>> configurable with xwiki. It would give the option to either configure
> >>> solr
> >>>> externally (standalone or clustered) or managed within the container.
> >>>
> >>> It also needs to be deployed on various Servlet Containers and with a
> >>> LibreOffice server configured.
> >>>
> >>> In short, it should support all the configurations supported by the
> >>> official XWiki Docker images :)
> >>>
> >>>> It
> >>>> should deploy the app on a shared file system like a rook. It should
> >>>> support Istio virtual services, istio matrix, and istio distributed
> >>> tracing
> >>>> and should be a secured system with RBAC and security credential
> >>> rotation.
> >>>
> >>> Ok I have no knowledge about this… The shared file system sounds good
> to
> >>> support clustering XWiki for the permanent directory sharing.
> >>>
> >>>> The chart should be easily deployed on GKE and amazon EKS.
> >>>
> >>> On locally on minikube for testing I guess?
> >>>
> >>
> >> Yes we can test using minikube one
> >>
> >>>
> >>>> Features
> >>>>
> >>>> ->  Support for different Databases
> >>>> ->  Choice between using an external database, a single node DB or a
> >>>> multi-cluster DB setup
> >>>> -> Support for shared file system
> >>>> -> Support for istio and it's services
> >>>> -> RBAC, SSL and other security methods
> >>>>
> >>>> If you have any features in mind that should be added please feel free
> >>> to
> >>>> reply to this mail.
> >>>>
> >>>> Some Design Questions?
> >>>> -> Which Databases should be supported?
> >>>
> >>> See
> >>>
> https://dev.xwiki.org/xwiki/bin/view/Community/SupportStrategy/DatabaseSupportStrategy
> >>>
> >>> More generally see the XWiki Docker image documentation and
> >>> https://dev.xwiki.org/xwiki/bin/view/Community/SupportStrategy
> >>>
> >>>> -> As we have to detach solr out of the docker container(run it in an
> >>>> independent container) would be there a requirement for a code change,
> >>> and
> >>>> we should approach it?
> >>>
> >>> See the doc for Solr which explains how to configure an external SOLR.
> >>> There are some XWiki code/config to deploy in the external SOLR AFAIR.
> >>> comunity controls chart
> >>>> -> Apart from solr is there any other stateful service that could or
> >>> should
> >>>> be detached from the docker container?
> >>>
> >>> Which docker container are you referring to? The Servlet Container one?
> >>>
> >>> xwiki:lts-mysql-tomcat
> >>
> >>
> >>>> Here is my current repository which deploys XWiki for MySQL database
> >>> using
> >>>> official XWiki docker container:-
> >>>> https://github.com/ashish932/xwiki-helm-chart/
> >>>
> >>> Will need to have a look. Some additional questions:
> >>>
> >>> 1) Is there an official place for publishing helm charts, similar to
> >>> dockerhub for docker images? How do you provide the helm charts to
> XWiki
> >>> users in term of packaging? In other words, what will we/you document
> on
> >>>
> https://www.xwiki.org/xwiki/bin/view/Documentation/AdminGuide/Installation/
> >>> ?
> >>>
> >>> Yes, There is a centralized community control chart management
> >> repository. Helm charts are hosted there. Organizations like mysql,
> >> wordpress uses it.
> >> https://github.com/helm/charts
> >> But in addition to that, we can build our own charts repository locally,
> >> and host it. Similar to docker run command, we have helm update command
> >>
> >>
> >>> 2) How do you plan to have smoke tests for the XWiki Helm charts, as
> part
> >>> of the build, to verify that they work fine? How can we execute XWik’s
> >>> functional tests based on TestContainers using Helm Charts?
> >>>
> >>
> >> There are some tools available
> >> chart-testing(started by helm) -98 commits
> >> terratest - 1039 commits (It looks a good tool to use)
> >>
> >>
> >>> 3) Do the helm charts for XWiki need to be built? If so, can it be done
> >>> with Maven? Since we’ll be offering several I guess, how do you avoid
> code
> >>> duplication and make maintenance of them as simple as possible?
> >>>
> >>
> >> No, the charts don’t need to be built externally, though it gets
> >> compressed when pushed f we build to chart repository(this is an
> automatic
> >> step in case of official chart repo and we can automate using ci our own
> >> chart repo)
> >>
> >> 4) Could you provide a roadmap with various milestones and dates? See
> the
> >>> last roadmap email I sent on the devs list and please reply to it.
> >>>
> >>
> >> Okay I would work on that and update the progress on the mail thread
> >>
> >>
> >>> 5) It seems you missed the bonding period so you’ll need to catch up on
> >>> that. Could you pick some jira issue and work on them quickly? This is
> to
> >>> make sure you use the proper xwiki communication channels, learn to
> work
> >>> with the xwiki team and start understanding the xwiki architecture,
> and how
> >>> extensions are developed for XWiki. Since you’re a bit late, you’ll
> need to
> >>> catch up and achieve this very quickly now and be present full time on
> the
> >>> xwiki chat :)
> >>>
> >> I would start doing that
> >>
> >>>
> >>> 6) Idea: please review the issues on the XWiki Docker project’s jira.
> >>> Maybe you can help with those too as part of onboarding/bonding. It
> should
> >>> also give you additional use cases for the XWiki Helm Charts IMO.
> >>>
> >>> Thanks! I’m eager to follow your work and see xwiki helm charts in
> action.
> >>>
> >>> -Vincent
> >>>
> >>>>
> >>>>
> >>>> Thank You
> >>>> Ashish Sharma
> >>>
> >>> Thank You
> >> Ashish
> >>
>
>
Reply | Threaded
Open this post in threaded view
|

Re: Helm Chart Xwiki - GSOC'19

Shubham Jain
Hey Ashish,

Thanks for the update.

Adding our chart to incubator section of helm community charts repository sounds great. Although (unlike Dockerhub) this repo is for community charts and not really official charts of the respective installations, but supporting this chart in the community chart repo should help with the visibility.

Lets also work on adding tests to the helm chart to ensure things are going fine.

Best,
Shubham.  

> On 21-Jun-2019, at 4:37 AM, Ashish Sharma <[hidden email]> wrote:
>
> Hi everyone,
>
> Update on HELM project:-
>
> We have added some features to the chart that includes:-
> * Making chart configurable with both mysql and postgres database
> * We are now using configmaps and secrets for storing database credentials
> * We use helm dependency to manage the dependencies of the chart
>
> I think our chart is ready to be deployed in the incubator section of helm
> community repo. <https://github.com/helm/charts> If you guys agree I would
> make the necessary changes required to deploy it there.
>
> Thanks
> - Ashish
>
> On Wed, Jun 5, 2019 at 7:40 PM Vincent Massol <[hidden email]> wrote:
>
>> Hi Ashish,
>>
>>> On 5 Jun 2019, at 15:07, Ashish Sharma <[hidden email]>
>> wrote:
>>>
>>> Hi everyone,
>>>
>>> The first week update of helm chart xwiki project. We currently have a
>>> chart that would deploy xwiki mysql stateful service with a persistent
>>> volume. The link to the repo:-
>>>
>>> https://github.com/xwiki-contrib/xwiki-helm
>>
>> Thanks for the update.
>>
>> Would be great if you could update the README file to explain who to use
>> it.
>>
>> Thanks
>> -Vincent
>>
>>
>>>
>>> Thank You
>>> Ashish
>>>
>>> On Thu, 23 May 2019 at 7:27 PM, Ashish Sharma <
>> [hidden email]>
>>> wrote:
>>>
>>>>
>>>>
>>>> On Wed, May 22, 2019 at 12:42 PM Vincent Massol <[hidden email]>
>>>> wrote:
>>>>
>>>>> Hi Ashish,
>>>>>
>>>>>> On 21 May 2019, at 19:37, Ashish Sharma <[hidden email]>
>>>>> wrote:
>>>>>>
>>>>>> Hi everyone,
>>>>>> Thanks for having me here
>>>>>>
>>>>>> About Me
>>>>>>
>>>>>> I am Ashish Sharma, selected as a student for Google Summer of Code. I
>>>>> am
>>>>>> final year student enrolled in Guru Gobind Singh Indraprastha
>>>>> University,
>>>>>> Delhi. I am a resident of India.
>>>>>>
>>>>>> Profiles
>>>>>> GitHub -  https://github.com/ashish932/xwiki-helm-chart/
>>>>>> LinkedIn - https://www.linkedin.com/in/ashish932/
>>>>>> Riot - @ashish932:matrix.org
>>>>>>
>>>>>> I will be presenting my project "Helm Chart for XWiki" to all of you.
>>>>>> Following
>>>>>> are the relevant details.
>>>>>>
>>>>>> Helm Chart for XWiki
>>>>>>
>>>>>> Mentors: Shubham Jain, Neha Gupta
>>>>>>
>>>>>> Technologies: Kubernetes, Docker, other if required
>>>>>>
>>>>>> Overview
>>>>>> The proposed project is a helm chart that would deploy xwiki as highly
>>>>>> available and reliable. It should be configurable with different
>>>>>> databases(either a standalone database or a clustered one) that are
>>>>>> configurable with xwiki. It would give the option to either configure
>>>>> solr
>>>>>> externally (standalone or clustered) or managed within the container.
>>>>>
>>>>> It also needs to be deployed on various Servlet Containers and with a
>>>>> LibreOffice server configured.
>>>>>
>>>>> In short, it should support all the configurations supported by the
>>>>> official XWiki Docker images :)
>>>>>
>>>>>> It
>>>>>> should deploy the app on a shared file system like a rook. It should
>>>>>> support Istio virtual services, istio matrix, and istio distributed
>>>>> tracing
>>>>>> and should be a secured system with RBAC and security credential
>>>>> rotation.
>>>>>
>>>>> Ok I have no knowledge about this… The shared file system sounds good
>> to
>>>>> support clustering XWiki for the permanent directory sharing.
>>>>>
>>>>>> The chart should be easily deployed on GKE and amazon EKS.
>>>>>
>>>>> On locally on minikube for testing I guess?
>>>>>
>>>>
>>>> Yes we can test using minikube one
>>>>
>>>>>
>>>>>> Features
>>>>>>
>>>>>> ->  Support for different Databases
>>>>>> ->  Choice between using an external database, a single node DB or a
>>>>>> multi-cluster DB setup
>>>>>> -> Support for shared file system
>>>>>> -> Support for istio and it's services
>>>>>> -> RBAC, SSL and other security methods
>>>>>>
>>>>>> If you have any features in mind that should be added please feel free
>>>>> to
>>>>>> reply to this mail.
>>>>>>
>>>>>> Some Design Questions?
>>>>>> -> Which Databases should be supported?
>>>>>
>>>>> See
>>>>>
>> https://dev.xwiki.org/xwiki/bin/view/Community/SupportStrategy/DatabaseSupportStrategy
>>>>>
>>>>> More generally see the XWiki Docker image documentation and
>>>>> https://dev.xwiki.org/xwiki/bin/view/Community/SupportStrategy
>>>>>
>>>>>> -> As we have to detach solr out of the docker container(run it in an
>>>>>> independent container) would be there a requirement for a code change,
>>>>> and
>>>>>> we should approach it?
>>>>>
>>>>> See the doc for Solr which explains how to configure an external SOLR.
>>>>> There are some XWiki code/config to deploy in the external SOLR AFAIR.
>>>>> comunity controls chart
>>>>>> -> Apart from solr is there any other stateful service that could or
>>>>> should
>>>>>> be detached from the docker container?
>>>>>
>>>>> Which docker container are you referring to? The Servlet Container one?
>>>>>
>>>>> xwiki:lts-mysql-tomcat
>>>>
>>>>
>>>>>> Here is my current repository which deploys XWiki for MySQL database
>>>>> using
>>>>>> official XWiki docker container:-
>>>>>> https://github.com/ashish932/xwiki-helm-chart/
>>>>>
>>>>> Will need to have a look. Some additional questions:
>>>>>
>>>>> 1) Is there an official place for publishing helm charts, similar to
>>>>> dockerhub for docker images? How do you provide the helm charts to
>> XWiki
>>>>> users in term of packaging? In other words, what will we/you document
>> on
>>>>>
>> https://www.xwiki.org/xwiki/bin/view/Documentation/AdminGuide/Installation/
>>>>> ?
>>>>>
>>>>> Yes, There is a centralized community control chart management
>>>> repository. Helm charts are hosted there. Organizations like mysql,
>>>> wordpress uses it.
>>>> https://github.com/helm/charts
>>>> But in addition to that, we can build our own charts repository locally,
>>>> and host it. Similar to docker run command, we have helm update command
>>>>
>>>>
>>>>> 2) How do you plan to have smoke tests for the XWiki Helm charts, as
>> part
>>>>> of the build, to verify that they work fine? How can we execute XWik’s
>>>>> functional tests based on TestContainers using Helm Charts?
>>>>>
>>>>
>>>> There are some tools available
>>>> chart-testing(started by helm) -98 commits
>>>> terratest - 1039 commits (It looks a good tool to use)
>>>>
>>>>
>>>>> 3) Do the helm charts for XWiki need to be built? If so, can it be done
>>>>> with Maven? Since we’ll be offering several I guess, how do you avoid
>> code
>>>>> duplication and make maintenance of them as simple as possible?
>>>>>
>>>>
>>>> No, the charts don’t need to be built externally, though it gets
>>>> compressed when pushed f we build to chart repository(this is an
>> automatic
>>>> step in case of official chart repo and we can automate using ci our own
>>>> chart repo)
>>>>
>>>> 4) Could you provide a roadmap with various milestones and dates? See
>> the
>>>>> last roadmap email I sent on the devs list and please reply to it.
>>>>>
>>>>
>>>> Okay I would work on that and update the progress on the mail thread
>>>>
>>>>
>>>>> 5) It seems you missed the bonding period so you’ll need to catch up on
>>>>> that. Could you pick some jira issue and work on them quickly? This is
>> to
>>>>> make sure you use the proper xwiki communication channels, learn to
>> work
>>>>> with the xwiki team and start understanding the xwiki architecture,
>> and how
>>>>> extensions are developed for XWiki. Since you’re a bit late, you’ll
>> need to
>>>>> catch up and achieve this very quickly now and be present full time on
>> the
>>>>> xwiki chat :)
>>>>>
>>>> I would start doing that
>>>>
>>>>>
>>>>> 6) Idea: please review the issues on the XWiki Docker project’s jira.
>>>>> Maybe you can help with those too as part of onboarding/bonding. It
>> should
>>>>> also give you additional use cases for the XWiki Helm Charts IMO.
>>>>>
>>>>> Thanks! I’m eager to follow your work and see xwiki helm charts in
>> action.
>>>>>
>>>>> -Vincent
>>>>>
>>>>>>
>>>>>>
>>>>>> Thank You
>>>>>> Ashish Sharma
>>>>>
>>>>> Thank You
>>>> Ashish
>>>>
>>
>>

Reply | Threaded
Open this post in threaded view
|

Re: Helm Chart Xwiki - GSOC'19

Ashish Sharma
Hi Shubham,

Thank you Shubham, I will start working on adding test cases.

Thanks,
Ashish

On Mon, 24 Jun 2019 at 3:32 PM, Shubham Jain <[hidden email]>
wrote:

> Hey Ashish,
>
> Thanks for the update.
>
> Adding our chart to incubator section of helm community charts repository
> sounds great. Although (unlike Dockerhub) this repo is for community charts
> and not really official charts of the respective installations, but
> supporting this chart in the community chart repo should help with the
> visibility.
>
> Lets also work on adding tests to the helm chart to ensure things are
> going fine.
>
> Best,
> Shubham.
>
> > On 21-Jun-2019, at 4:37 AM, Ashish Sharma <[hidden email]>
> wrote:
> >
> > Hi everyone,
> >
> > Update on HELM project:-
> >
> > We have added some features to the chart that includes:-
> > * Making chart configurable with both mysql and postgres database
> > * We are now using configmaps and secrets for storing database
> credentials
> > * We use helm dependency to manage the dependencies of the chart
> >
> > I think our chart is ready to be deployed in the incubator section of
> helm
> > community repo. <https://github.com/helm/charts> If you guys agree I
> would
> > make the necessary changes required to deploy it there.
> >
> > Thanks
> > - Ashish
> >
> > On Wed, Jun 5, 2019 at 7:40 PM Vincent Massol <[hidden email]>
> wrote:
> >
> >> Hi Ashish,
> >>
> >>> On 5 Jun 2019, at 15:07, Ashish Sharma <[hidden email]>
> >> wrote:
> >>>
> >>> Hi everyone,
> >>>
> >>> The first week update of helm chart xwiki project. We currently have a
> >>> chart that would deploy xwiki mysql stateful service with a persistent
> >>> volume. The link to the repo:-
> >>>
> >>> https://github.com/xwiki-contrib/xwiki-helm
> >>
> >> Thanks for the update.
> >>
> >> Would be great if you could update the README file to explain who to use
> >> it.
> >>
> >> Thanks
> >> -Vincent
> >>
> >>
> >>>
> >>> Thank You
> >>> Ashish
> >>>
> >>> On Thu, 23 May 2019 at 7:27 PM, Ashish Sharma <
> >> [hidden email]>
> >>> wrote:
> >>>
> >>>>
> >>>>
> >>>> On Wed, May 22, 2019 at 12:42 PM Vincent Massol <[hidden email]>
> >>>> wrote:
> >>>>
> >>>>> Hi Ashish,
> >>>>>
> >>>>>> On 21 May 2019, at 19:37, Ashish Sharma <[hidden email]
> >
> >>>>> wrote:
> >>>>>>
> >>>>>> Hi everyone,
> >>>>>> Thanks for having me here
> >>>>>>
> >>>>>> About Me
> >>>>>>
> >>>>>> I am Ashish Sharma, selected as a student for Google Summer of
> Code. I
> >>>>> am
> >>>>>> final year student enrolled in Guru Gobind Singh Indraprastha
> >>>>> University,
> >>>>>> Delhi. I am a resident of India.
> >>>>>>
> >>>>>> Profiles
> >>>>>> GitHub -  https://github.com/ashish932/xwiki-helm-chart/
> >>>>>> LinkedIn - https://www.linkedin.com/in/ashish932/
> >>>>>> Riot - @ashish932:matrix.org
> >>>>>>
> >>>>>> I will be presenting my project "Helm Chart for XWiki" to all of
> you.
> >>>>>> Following
> >>>>>> are the relevant details.
> >>>>>>
> >>>>>> Helm Chart for XWiki
> >>>>>>
> >>>>>> Mentors: Shubham Jain, Neha Gupta
> >>>>>>
> >>>>>> Technologies: Kubernetes, Docker, other if required
> >>>>>>
> >>>>>> Overview
> >>>>>> The proposed project is a helm chart that would deploy xwiki as
> highly
> >>>>>> available and reliable. It should be configurable with different
> >>>>>> databases(either a standalone database or a clustered one) that are
> >>>>>> configurable with xwiki. It would give the option to either
> configure
> >>>>> solr
> >>>>>> externally (standalone or clustered) or managed within the
> container.
> >>>>>
> >>>>> It also needs to be deployed on various Servlet Containers and with a
> >>>>> LibreOffice server configured.
> >>>>>
> >>>>> In short, it should support all the configurations supported by the
> >>>>> official XWiki Docker images :)
> >>>>>
> >>>>>> It
> >>>>>> should deploy the app on a shared file system like a rook. It should
> >>>>>> support Istio virtual services, istio matrix, and istio distributed
> >>>>> tracing
> >>>>>> and should be a secured system with RBAC and security credential
> >>>>> rotation.
> >>>>>
> >>>>> Ok I have no knowledge about this… The shared file system sounds good
> >> to
> >>>>> support clustering XWiki for the permanent directory sharing.
> >>>>>
> >>>>>> The chart should be easily deployed on GKE and amazon EKS.
> >>>>>
> >>>>> On locally on minikube for testing I guess?
> >>>>>
> >>>>
> >>>> Yes we can test using minikube one
> >>>>
> >>>>>
> >>>>>> Features
> >>>>>>
> >>>>>> ->  Support for different Databases
> >>>>>> ->  Choice between using an external database, a single node DB or a
> >>>>>> multi-cluster DB setup
> >>>>>> -> Support for shared file system
> >>>>>> -> Support for istio and it's services
> >>>>>> -> RBAC, SSL and other security methods
> >>>>>>
> >>>>>> If you have any features in mind that should be added please feel
> free
> >>>>> to
> >>>>>> reply to this mail.
> >>>>>>
> >>>>>> Some Design Questions?
> >>>>>> -> Which Databases should be supported?
> >>>>>
> >>>>> See
> >>>>>
> >>
> https://dev.xwiki.org/xwiki/bin/view/Community/SupportStrategy/DatabaseSupportStrategy
> >>>>>
> >>>>> More generally see the XWiki Docker image documentation and
> >>>>> https://dev.xwiki.org/xwiki/bin/view/Community/SupportStrategy
> >>>>>
> >>>>>> -> As we have to detach solr out of the docker container(run it in
> an
> >>>>>> independent container) would be there a requirement for a code
> change,
> >>>>> and
> >>>>>> we should approach it?
> >>>>>
> >>>>> See the doc for Solr which explains how to configure an external
> SOLR.
> >>>>> There are some XWiki code/config to deploy in the external SOLR
> AFAIR.
> >>>>> comunity controls chart
> >>>>>> -> Apart from solr is there any other stateful service that could or
> >>>>> should
> >>>>>> be detached from the docker container?
> >>>>>
> >>>>> Which docker container are you referring to? The Servlet Container
> one?
> >>>>>
> >>>>> xwiki:lts-mysql-tomcat
> >>>>
> >>>>
> >>>>>> Here is my current repository which deploys XWiki for MySQL database
> >>>>> using
> >>>>>> official XWiki docker container:-
> >>>>>> https://github.com/ashish932/xwiki-helm-chart/
> >>>>>
> >>>>> Will need to have a look. Some additional questions:
> >>>>>
> >>>>> 1) Is there an official place for publishing helm charts, similar to
> >>>>> dockerhub for docker images? How do you provide the helm charts to
> >> XWiki
> >>>>> users in term of packaging? In other words, what will we/you document
> >> on
> >>>>>
> >>
> https://www.xwiki.org/xwiki/bin/view/Documentation/AdminGuide/Installation/
> >>>>> ?
> >>>>>
> >>>>> Yes, There is a centralized community control chart management
> >>>> repository. Helm charts are hosted there. Organizations like mysql,
> >>>> wordpress uses it.
> >>>> https://github.com/helm/charts
> >>>> But in addition to that, we can build our own charts repository
> locally,
> >>>> and host it. Similar to docker run command, we have helm update
> command
> >>>>
> >>>>
> >>>>> 2) How do you plan to have smoke tests for the XWiki Helm charts, as
> >> part
> >>>>> of the build, to verify that they work fine? How can we execute
> XWik’s
> >>>>> functional tests based on TestContainers using Helm Charts?
> >>>>>
> >>>>
> >>>> There are some tools available
> >>>> chart-testing(started by helm) -98 commits
> >>>> terratest - 1039 commits (It looks a good tool to use)
> >>>>
> >>>>
> >>>>> 3) Do the helm charts for XWiki need to be built? If so, can it be
> done
> >>>>> with Maven? Since we’ll be offering several I guess, how do you avoid
> >> code
> >>>>> duplication and make maintenance of them as simple as possible?
> >>>>>
> >>>>
> >>>> No, the charts don’t need to be built externally, though it gets
> >>>> compressed when pushed f we build to chart repository(this is an
> >> automatic
> >>>> step in case of official chart repo and we can automate using ci our
> own
> >>>> chart repo)
> >>>>
> >>>> 4) Could you provide a roadmap with various milestones and dates? See
> >> the
> >>>>> last roadmap email I sent on the devs list and please reply to it.
> >>>>>
> >>>>
> >>>> Okay I would work on that and update the progress on the mail thread
> >>>>
> >>>>
> >>>>> 5) It seems you missed the bonding period so you’ll need to catch up
> on
> >>>>> that. Could you pick some jira issue and work on them quickly? This
> is
> >> to
> >>>>> make sure you use the proper xwiki communication channels, learn to
> >> work
> >>>>> with the xwiki team and start understanding the xwiki architecture,
> >> and how
> >>>>> extensions are developed for XWiki. Since you’re a bit late, you’ll
> >> need to
> >>>>> catch up and achieve this very quickly now and be present full time
> on
> >> the
> >>>>> xwiki chat :)
> >>>>>
> >>>> I would start doing that
> >>>>
> >>>>>
> >>>>> 6) Idea: please review the issues on the XWiki Docker project’s jira.
> >>>>> Maybe you can help with those too as part of onboarding/bonding. It
> >> should
> >>>>> also give you additional use cases for the XWiki Helm Charts IMO.
> >>>>>
> >>>>> Thanks! I’m eager to follow your work and see xwiki helm charts in
> >> action.
> >>>>>
> >>>>> -Vincent
> >>>>>
> >>>>>>
> >>>>>>
> >>>>>> Thank You
> >>>>>> Ashish Sharma
> >>>>>
> >>>>> Thank You
> >>>> Ashish
> >>>>
> >>
> >>
>
>
Reply | Threaded
Open this post in threaded view
|

Re: Helm Chart Xwiki - GSOC'19

Ashish Sharma
Hello everyone,

I require the xwiki solr docker container to do progress with my work. I
have made a pull request
<https://github.com/xwiki-contrib/docker-xwiki/pull/22>, for the same.
Please look into it.

Moreover, I was working with Rook(Ceph Operator) for a shared file system
which would be a better approach to deploy our app on production. So I
think we should document it somewhere. Can anyone help me where should I do
that. It may contain the details to configure rook-ceph, and then
connecting it.

P.S. I am writing a blog at Medium on how to configure rook on Google
Kubernetes Engine. We could also use that?

Thanks,
Ashish

On Mon, Jun 24, 2019 at 8:31 PM Ashish Sharma <[hidden email]>
wrote:

> Hi Shubham,
>
> Thank you Shubham, I will start working on adding test cases.
>
> Thanks,
> Ashish
>
> On Mon, 24 Jun 2019 at 3:32 PM, Shubham Jain <[hidden email]>
> wrote:
>
>> Hey Ashish,
>>
>> Thanks for the update.
>>
>> Adding our chart to incubator section of helm community charts repository
>> sounds great. Although (unlike Dockerhub) this repo is for community charts
>> and not really official charts of the respective installations, but
>> supporting this chart in the community chart repo should help with the
>> visibility.
>>
>> Lets also work on adding tests to the helm chart to ensure things are
>> going fine.
>>
>> Best,
>> Shubham.
>>
>> > On 21-Jun-2019, at 4:37 AM, Ashish Sharma <[hidden email]>
>> wrote:
>> >
>> > Hi everyone,
>> >
>> > Update on HELM project:-
>> >
>> > We have added some features to the chart that includes:-
>> > * Making chart configurable with both mysql and postgres database
>> > * We are now using configmaps and secrets for storing database
>> credentials
>> > * We use helm dependency to manage the dependencies of the chart
>> >
>> > I think our chart is ready to be deployed in the incubator section of
>> helm
>> > community repo. <https://github.com/helm/charts> If you guys agree I
>> would
>> > make the necessary changes required to deploy it there.
>> >
>> > Thanks
>> > - Ashish
>> >
>> > On Wed, Jun 5, 2019 at 7:40 PM Vincent Massol <[hidden email]>
>> wrote:
>> >
>> >> Hi Ashish,
>> >>
>> >>> On 5 Jun 2019, at 15:07, Ashish Sharma <[hidden email]>
>> >> wrote:
>> >>>
>> >>> Hi everyone,
>> >>>
>> >>> The first week update of helm chart xwiki project. We currently have a
>> >>> chart that would deploy xwiki mysql stateful service with a persistent
>> >>> volume. The link to the repo:-
>> >>>
>> >>> https://github.com/xwiki-contrib/xwiki-helm
>> >>
>> >> Thanks for the update.
>> >>
>> >> Would be great if you could update the README file to explain who to
>> use
>> >> it.
>> >>
>> >> Thanks
>> >> -Vincent
>> >>
>> >>
>> >>>
>> >>> Thank You
>> >>> Ashish
>> >>>
>> >>> On Thu, 23 May 2019 at 7:27 PM, Ashish Sharma <
>> >> [hidden email]>
>> >>> wrote:
>> >>>
>> >>>>
>> >>>>
>> >>>> On Wed, May 22, 2019 at 12:42 PM Vincent Massol <[hidden email]>
>> >>>> wrote:
>> >>>>
>> >>>>> Hi Ashish,
>> >>>>>
>> >>>>>> On 21 May 2019, at 19:37, Ashish Sharma <
>> [hidden email]>
>> >>>>> wrote:
>> >>>>>>
>> >>>>>> Hi everyone,
>> >>>>>> Thanks for having me here
>> >>>>>>
>> >>>>>> About Me
>> >>>>>>
>> >>>>>> I am Ashish Sharma, selected as a student for Google Summer of
>> Code. I
>> >>>>> am
>> >>>>>> final year student enrolled in Guru Gobind Singh Indraprastha
>> >>>>> University,
>> >>>>>> Delhi. I am a resident of India.
>> >>>>>>
>> >>>>>> Profiles
>> >>>>>> GitHub -  https://github.com/ashish932/xwiki-helm-chart/
>> >>>>>> LinkedIn - https://www.linkedin.com/in/ashish932/
>> >>>>>> Riot - @ashish932:matrix.org
>> >>>>>>
>> >>>>>> I will be presenting my project "Helm Chart for XWiki" to all of
>> you.
>> >>>>>> Following
>> >>>>>> are the relevant details.
>> >>>>>>
>> >>>>>> Helm Chart for XWiki
>> >>>>>>
>> >>>>>> Mentors: Shubham Jain, Neha Gupta
>> >>>>>>
>> >>>>>> Technologies: Kubernetes, Docker, other if required
>> >>>>>>
>> >>>>>> Overview
>> >>>>>> The proposed project is a helm chart that would deploy xwiki as
>> highly
>> >>>>>> available and reliable. It should be configurable with different
>> >>>>>> databases(either a standalone database or a clustered one) that are
>> >>>>>> configurable with xwiki. It would give the option to either
>> configure
>> >>>>> solr
>> >>>>>> externally (standalone or clustered) or managed within the
>> container.
>> >>>>>
>> >>>>> It also needs to be deployed on various Servlet Containers and with
>> a
>> >>>>> LibreOffice server configured.
>> >>>>>
>> >>>>> In short, it should support all the configurations supported by the
>> >>>>> official XWiki Docker images :)
>> >>>>>
>> >>>>>> It
>> >>>>>> should deploy the app on a shared file system like a rook. It
>> should
>> >>>>>> support Istio virtual services, istio matrix, and istio distributed
>> >>>>> tracing
>> >>>>>> and should be a secured system with RBAC and security credential
>> >>>>> rotation.
>> >>>>>
>> >>>>> Ok I have no knowledge about this… The shared file system sounds
>> good
>> >> to
>> >>>>> support clustering XWiki for the permanent directory sharing.
>> >>>>>
>> >>>>>> The chart should be easily deployed on GKE and amazon EKS.
>> >>>>>
>> >>>>> On locally on minikube for testing I guess?
>> >>>>>
>> >>>>
>> >>>> Yes we can test using minikube one
>> >>>>
>> >>>>>
>> >>>>>> Features
>> >>>>>>
>> >>>>>> ->  Support for different Databases
>> >>>>>> ->  Choice between using an external database, a single node DB or
>> a
>> >>>>>> multi-cluster DB setup
>> >>>>>> -> Support for shared file system
>> >>>>>> -> Support for istio and it's services
>> >>>>>> -> RBAC, SSL and other security methods
>> >>>>>>
>> >>>>>> If you have any features in mind that should be added please feel
>> free
>> >>>>> to
>> >>>>>> reply to this mail.
>> >>>>>>
>> >>>>>> Some Design Questions?
>> >>>>>> -> Which Databases should be supported?
>> >>>>>
>> >>>>> See
>> >>>>>
>> >>
>> https://dev.xwiki.org/xwiki/bin/view/Community/SupportStrategy/DatabaseSupportStrategy
>> >>>>>
>> >>>>> More generally see the XWiki Docker image documentation and
>> >>>>> https://dev.xwiki.org/xwiki/bin/view/Community/SupportStrategy
>> >>>>>
>> >>>>>> -> As we have to detach solr out of the docker container(run it in
>> an
>> >>>>>> independent container) would be there a requirement for a code
>> change,
>> >>>>> and
>> >>>>>> we should approach it?
>> >>>>>
>> >>>>> See the doc for Solr which explains how to configure an external
>> SOLR.
>> >>>>> There are some XWiki code/config to deploy in the external SOLR
>> AFAIR.
>> >>>>> comunity controls chart
>> >>>>>> -> Apart from solr is there any other stateful service that could
>> or
>> >>>>> should
>> >>>>>> be detached from the docker container?
>> >>>>>
>> >>>>> Which docker container are you referring to? The Servlet Container
>> one?
>> >>>>>
>> >>>>> xwiki:lts-mysql-tomcat
>> >>>>
>> >>>>
>> >>>>>> Here is my current repository which deploys XWiki for MySQL
>> database
>> >>>>> using
>> >>>>>> official XWiki docker container:-
>> >>>>>> https://github.com/ashish932/xwiki-helm-chart/
>> >>>>>
>> >>>>> Will need to have a look. Some additional questions:
>> >>>>>
>> >>>>> 1) Is there an official place for publishing helm charts, similar to
>> >>>>> dockerhub for docker images? How do you provide the helm charts to
>> >> XWiki
>> >>>>> users in term of packaging? In other words, what will we/you
>> document
>> >> on
>> >>>>>
>> >>
>> https://www.xwiki.org/xwiki/bin/view/Documentation/AdminGuide/Installation/
>> >>>>> ?
>> >>>>>
>> >>>>> Yes, There is a centralized community control chart management
>> >>>> repository. Helm charts are hosted there. Organizations like mysql,
>> >>>> wordpress uses it.
>> >>>> https://github.com/helm/charts
>> >>>> But in addition to that, we can build our own charts repository
>> locally,
>> >>>> and host it. Similar to docker run command, we have helm update
>> command
>> >>>>
>> >>>>
>> >>>>> 2) How do you plan to have smoke tests for the XWiki Helm charts, as
>> >> part
>> >>>>> of the build, to verify that they work fine? How can we execute
>> XWik’s
>> >>>>> functional tests based on TestContainers using Helm Charts?
>> >>>>>
>> >>>>
>> >>>> There are some tools available
>> >>>> chart-testing(started by helm) -98 commits
>> >>>> terratest - 1039 commits (It looks a good tool to use)
>> >>>>
>> >>>>
>> >>>>> 3) Do the helm charts for XWiki need to be built? If so, can it be
>> done
>> >>>>> with Maven? Since we’ll be offering several I guess, how do you
>> avoid
>> >> code
>> >>>>> duplication and make maintenance of them as simple as possible?
>> >>>>>
>> >>>>
>> >>>> No, the charts don’t need to be built externally, though it gets
>> >>>> compressed when pushed f we build to chart repository(this is an
>> >> automatic
>> >>>> step in case of official chart repo and we can automate using ci our
>> own
>> >>>> chart repo)
>> >>>>
>> >>>> 4) Could you provide a roadmap with various milestones and dates? See
>> >> the
>> >>>>> last roadmap email I sent on the devs list and please reply to it.
>> >>>>>
>> >>>>
>> >>>> Okay I would work on that and update the progress on the mail thread
>> >>>>
>> >>>>
>> >>>>> 5) It seems you missed the bonding period so you’ll need to catch
>> up on
>> >>>>> that. Could you pick some jira issue and work on them quickly? This
>> is
>> >> to
>> >>>>> make sure you use the proper xwiki communication channels, learn to
>> >> work
>> >>>>> with the xwiki team and start understanding the xwiki architecture,
>> >> and how
>> >>>>> extensions are developed for XWiki. Since you’re a bit late, you’ll
>> >> need to
>> >>>>> catch up and achieve this very quickly now and be present full time
>> on
>> >> the
>> >>>>> xwiki chat :)
>> >>>>>
>> >>>> I would start doing that
>> >>>>
>> >>>>>
>> >>>>> 6) Idea: please review the issues on the XWiki Docker project’s
>> jira.
>> >>>>> Maybe you can help with those too as part of onboarding/bonding. It
>> >> should
>> >>>>> also give you additional use cases for the XWiki Helm Charts IMO.
>> >>>>>
>> >>>>> Thanks! I’m eager to follow your work and see xwiki helm charts in
>> >> action.
>> >>>>>
>> >>>>> -Vincent
>> >>>>>
>> >>>>>>
>> >>>>>>
>> >>>>>> Thank You
>> >>>>>> Ashish Sharma
>> >>>>>
>> >>>>> Thank You
>> >>>> Ashish
>> >>>>
>> >>
>> >>
>>
>>
Reply | Threaded
Open this post in threaded view
|

Re: Helm Chart Xwiki - GSOC'19

Shubham Jain
Hey Ashish,

I think Vincent/Thomas can help with the PR on the xwiki-docker project.

Regarding adding docs for High-Availability setup using shared data volumes, I think you can use could create a docs folder in the GitHub repo, and add the docs in Markdown format. I think you can also document it in XWiki blog <https://www.xwiki.org/xwiki/bin/view/Blog/>. The chart should support enabling HA, and then passing a storage class which supports shareable volumes to Pods, and then you can document examples for using this with something like Rook+GKE setup.

Best,
Shubham.  


> On 08-Jul-2019, at 1:57 PM, Ashish Sharma <[hidden email]> wrote:
>
> Hello everyone,
>
> I require the xwiki solr docker container to do progress with my work. I
> have made a pull request
> <https://github.com/xwiki-contrib/docker-xwiki/pull/22>, for the same.
> Please look into it.
>
> Moreover, I was working with Rook(Ceph Operator) for a shared file system
> which would be a better approach to deploy our app on production. So I
> think we should document it somewhere. Can anyone help me where should I do
> that. It may contain the details to configure rook-ceph, and then
> connecting it.
>
> P.S. I am writing a blog at Medium on how to configure rook on Google
> Kubernetes Engine. We could also use that?
>
> Thanks,
> Ashish
>
> On Mon, Jun 24, 2019 at 8:31 PM Ashish Sharma <[hidden email]>
> wrote:
>
>> Hi Shubham,
>>
>> Thank you Shubham, I will start working on adding test cases.
>>
>> Thanks,
>> Ashish
>>
>> On Mon, 24 Jun 2019 at 3:32 PM, Shubham Jain <[hidden email]>
>> wrote:
>>
>>> Hey Ashish,
>>>
>>> Thanks for the update.
>>>
>>> Adding our chart to incubator section of helm community charts repository
>>> sounds great. Although (unlike Dockerhub) this repo is for community charts
>>> and not really official charts of the respective installations, but
>>> supporting this chart in the community chart repo should help with the
>>> visibility.
>>>
>>> Lets also work on adding tests to the helm chart to ensure things are
>>> going fine.
>>>
>>> Best,
>>> Shubham.
>>>
>>>> On 21-Jun-2019, at 4:37 AM, Ashish Sharma <[hidden email]>
>>> wrote:
>>>>
>>>> Hi everyone,
>>>>
>>>> Update on HELM project:-
>>>>
>>>> We have added some features to the chart that includes:-
>>>> * Making chart configurable with both mysql and postgres database
>>>> * We are now using configmaps and secrets for storing database
>>> credentials
>>>> * We use helm dependency to manage the dependencies of the chart
>>>>
>>>> I think our chart is ready to be deployed in the incubator section of
>>> helm
>>>> community repo. <https://github.com/helm/charts> If you guys agree I
>>> would
>>>> make the necessary changes required to deploy it there.
>>>>
>>>> Thanks
>>>> - Ashish
>>>>
>>>> On Wed, Jun 5, 2019 at 7:40 PM Vincent Massol <[hidden email]>
>>> wrote:
>>>>
>>>>> Hi Ashish,
>>>>>
>>>>>> On 5 Jun 2019, at 15:07, Ashish Sharma <[hidden email]>
>>>>> wrote:
>>>>>>
>>>>>> Hi everyone,
>>>>>>
>>>>>> The first week update of helm chart xwiki project. We currently have a
>>>>>> chart that would deploy xwiki mysql stateful service with a persistent
>>>>>> volume. The link to the repo:-
>>>>>>
>>>>>> https://github.com/xwiki-contrib/xwiki-helm
>>>>>
>>>>> Thanks for the update.
>>>>>
>>>>> Would be great if you could update the README file to explain who to
>>> use
>>>>> it.
>>>>>
>>>>> Thanks
>>>>> -Vincent
>>>>>
>>>>>
>>>>>>
>>>>>> Thank You
>>>>>> Ashish
>>>>>>
>>>>>> On Thu, 23 May 2019 at 7:27 PM, Ashish Sharma <
>>>>> [hidden email]>
>>>>>> wrote:
>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> On Wed, May 22, 2019 at 12:42 PM Vincent Massol <[hidden email]>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> Hi Ashish,
>>>>>>>>
>>>>>>>>> On 21 May 2019, at 19:37, Ashish Sharma <
>>> [hidden email]>
>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>> Hi everyone,
>>>>>>>>> Thanks for having me here
>>>>>>>>>
>>>>>>>>> About Me
>>>>>>>>>
>>>>>>>>> I am Ashish Sharma, selected as a student for Google Summer of
>>> Code. I
>>>>>>>> am
>>>>>>>>> final year student enrolled in Guru Gobind Singh Indraprastha
>>>>>>>> University,
>>>>>>>>> Delhi. I am a resident of India.
>>>>>>>>>
>>>>>>>>> Profiles
>>>>>>>>> GitHub -  https://github.com/ashish932/xwiki-helm-chart/
>>>>>>>>> LinkedIn - https://www.linkedin.com/in/ashish932/
>>>>>>>>> Riot - @ashish932:matrix.org
>>>>>>>>>
>>>>>>>>> I will be presenting my project "Helm Chart for XWiki" to all of
>>> you.
>>>>>>>>> Following
>>>>>>>>> are the relevant details.
>>>>>>>>>
>>>>>>>>> Helm Chart for XWiki
>>>>>>>>>
>>>>>>>>> Mentors: Shubham Jain, Neha Gupta
>>>>>>>>>
>>>>>>>>> Technologies: Kubernetes, Docker, other if required
>>>>>>>>>
>>>>>>>>> Overview
>>>>>>>>> The proposed project is a helm chart that would deploy xwiki as
>>> highly
>>>>>>>>> available and reliable. It should be configurable with different
>>>>>>>>> databases(either a standalone database or a clustered one) that are
>>>>>>>>> configurable with xwiki. It would give the option to either
>>> configure
>>>>>>>> solr
>>>>>>>>> externally (standalone or clustered) or managed within the
>>> container.
>>>>>>>>
>>>>>>>> It also needs to be deployed on various Servlet Containers and with
>>> a
>>>>>>>> LibreOffice server configured.
>>>>>>>>
>>>>>>>> In short, it should support all the configurations supported by the
>>>>>>>> official XWiki Docker images :)
>>>>>>>>
>>>>>>>>> It
>>>>>>>>> should deploy the app on a shared file system like a rook. It
>>> should
>>>>>>>>> support Istio virtual services, istio matrix, and istio distributed
>>>>>>>> tracing
>>>>>>>>> and should be a secured system with RBAC and security credential
>>>>>>>> rotation.
>>>>>>>>
>>>>>>>> Ok I have no knowledge about this… The shared file system sounds
>>> good
>>>>> to
>>>>>>>> support clustering XWiki for the permanent directory sharing.
>>>>>>>>
>>>>>>>>> The chart should be easily deployed on GKE and amazon EKS.
>>>>>>>>
>>>>>>>> On locally on minikube for testing I guess?
>>>>>>>>
>>>>>>>
>>>>>>> Yes we can test using minikube one
>>>>>>>
>>>>>>>>
>>>>>>>>> Features
>>>>>>>>>
>>>>>>>>> ->  Support for different Databases
>>>>>>>>> ->  Choice between using an external database, a single node DB or
>>> a
>>>>>>>>> multi-cluster DB setup
>>>>>>>>> -> Support for shared file system
>>>>>>>>> -> Support for istio and it's services
>>>>>>>>> -> RBAC, SSL and other security methods
>>>>>>>>>
>>>>>>>>> If you have any features in mind that should be added please feel
>>> free
>>>>>>>> to
>>>>>>>>> reply to this mail.
>>>>>>>>>
>>>>>>>>> Some Design Questions?
>>>>>>>>> -> Which Databases should be supported?
>>>>>>>>
>>>>>>>> See
>>>>>>>>
>>>>>
>>> https://dev.xwiki.org/xwiki/bin/view/Community/SupportStrategy/DatabaseSupportStrategy
>>>>>>>>
>>>>>>>> More generally see the XWiki Docker image documentation and
>>>>>>>> https://dev.xwiki.org/xwiki/bin/view/Community/SupportStrategy
>>>>>>>>
>>>>>>>>> -> As we have to detach solr out of the docker container(run it in
>>> an
>>>>>>>>> independent container) would be there a requirement for a code
>>> change,
>>>>>>>> and
>>>>>>>>> we should approach it?
>>>>>>>>
>>>>>>>> See the doc for Solr which explains how to configure an external
>>> SOLR.
>>>>>>>> There are some XWiki code/config to deploy in the external SOLR
>>> AFAIR.
>>>>>>>> comunity controls chart
>>>>>>>>> -> Apart from solr is there any other stateful service that could
>>> or
>>>>>>>> should
>>>>>>>>> be detached from the docker container?
>>>>>>>>
>>>>>>>> Which docker container are you referring to? The Servlet Container
>>> one?
>>>>>>>>
>>>>>>>> xwiki:lts-mysql-tomcat
>>>>>>>
>>>>>>>
>>>>>>>>> Here is my current repository which deploys XWiki for MySQL
>>> database
>>>>>>>> using
>>>>>>>>> official XWiki docker container:-
>>>>>>>>> https://github.com/ashish932/xwiki-helm-chart/
>>>>>>>>
>>>>>>>> Will need to have a look. Some additional questions:
>>>>>>>>
>>>>>>>> 1) Is there an official place for publishing helm charts, similar to
>>>>>>>> dockerhub for docker images? How do you provide the helm charts to
>>>>> XWiki
>>>>>>>> users in term of packaging? In other words, what will we/you
>>> document
>>>>> on
>>>>>>>>
>>>>>
>>> https://www.xwiki.org/xwiki/bin/view/Documentation/AdminGuide/Installation/
>>>>>>>> ?
>>>>>>>>
>>>>>>>> Yes, There is a centralized community control chart management
>>>>>>> repository. Helm charts are hosted there. Organizations like mysql,
>>>>>>> wordpress uses it.
>>>>>>> https://github.com/helm/charts
>>>>>>> But in addition to that, we can build our own charts repository
>>> locally,
>>>>>>> and host it. Similar to docker run command, we have helm update
>>> command
>>>>>>>
>>>>>>>
>>>>>>>> 2) How do you plan to have smoke tests for the XWiki Helm charts, as
>>>>> part
>>>>>>>> of the build, to verify that they work fine? How can we execute
>>> XWik’s
>>>>>>>> functional tests based on TestContainers using Helm Charts?
>>>>>>>>
>>>>>>>
>>>>>>> There are some tools available
>>>>>>> chart-testing(started by helm) -98 commits
>>>>>>> terratest - 1039 commits (It looks a good tool to use)
>>>>>>>
>>>>>>>
>>>>>>>> 3) Do the helm charts for XWiki need to be built? If so, can it be
>>> done
>>>>>>>> with Maven? Since we’ll be offering several I guess, how do you
>>> avoid
>>>>> code
>>>>>>>> duplication and make maintenance of them as simple as possible?
>>>>>>>>
>>>>>>>
>>>>>>> No, the charts don’t need to be built externally, though it gets
>>>>>>> compressed when pushed f we build to chart repository(this is an
>>>>> automatic
>>>>>>> step in case of official chart repo and we can automate using ci our
>>> own
>>>>>>> chart repo)
>>>>>>>
>>>>>>> 4) Could you provide a roadmap with various milestones and dates? See
>>>>> the
>>>>>>>> last roadmap email I sent on the devs list and please reply to it.
>>>>>>>>
>>>>>>>
>>>>>>> Okay I would work on that and update the progress on the mail thread
>>>>>>>
>>>>>>>
>>>>>>>> 5) It seems you missed the bonding period so you’ll need to catch
>>> up on
>>>>>>>> that. Could you pick some jira issue and work on them quickly? This
>>> is
>>>>> to
>>>>>>>> make sure you use the proper xwiki communication channels, learn to
>>>>> work
>>>>>>>> with the xwiki team and start understanding the xwiki architecture,
>>>>> and how
>>>>>>>> extensions are developed for XWiki. Since you’re a bit late, you’ll
>>>>> need to
>>>>>>>> catch up and achieve this very quickly now and be present full time
>>> on
>>>>> the
>>>>>>>> xwiki chat :)
>>>>>>>>
>>>>>>> I would start doing that
>>>>>>>
>>>>>>>>
>>>>>>>> 6) Idea: please review the issues on the XWiki Docker project’s
>>> jira.
>>>>>>>> Maybe you can help with those too as part of onboarding/bonding. It
>>>>> should
>>>>>>>> also give you additional use cases for the XWiki Helm Charts IMO.
>>>>>>>>
>>>>>>>> Thanks! I’m eager to follow your work and see xwiki helm charts in
>>>>> action.
>>>>>>>>
>>>>>>>> -Vincent
>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Thank You
>>>>>>>>> Ashish Sharma
>>>>>>>>
>>>>>>>> Thank You
>>>>>>> Ashish
>>>>>>>
>>>>>
>>>>>
>>>
>>>

Reply | Threaded
Open this post in threaded view
|

Xwiki Helm HA

Ashish Sharma
Hello everyone,

I am trying to implement xwiki helm chart in High availability but facing
some problems.

How we can implement High Availability(HA) in Kubernetes:-

1. Through Deployment:-  In this approach we would use a shared file
storage, that would get attached to multiple pods(or containers)
2. Through StatefulSet:- In this approach every pod(or container) would
have it’s own different volume and state, which should be synchronised by
an consensus algorithm

Problem faced in xwiki case:-

1. In deployment case we need a locking mechanism for our filesystem,
moreover caching and states should also be maintained outside the container.
2. In StatefulSet the consensus algorithm is not present so it is currently
not possible.

Could anyone help me through it.

Thanks,
Ashish

On Wed, 10 Jul 2019 at 10:32 AM, Shubham Jain <[hidden email]>
wrote:

> Hey Ashish,
>
> I think Vincent/Thomas can help with the PR on the xwiki-docker project.
>
> Regarding adding docs for High-Availability setup using shared data
> volumes, I think you can use could create a docs folder in the GitHub repo,
> and add the docs in Markdown format. I think you can also document it in
> XWiki blog <https://www.xwiki.org/xwiki/bin/view/Blog/>. The chart should
> support enabling HA, and then passing a storage class which supports
> shareable volumes to Pods, and then you can document examples for using
> this with something like Rook+GKE setup.
>
> Best,
> Shubham.
>
>
> > On 08-Jul-2019, at 1:57 PM, Ashish Sharma <[hidden email]>
> wrote:
> >
> > Hello everyone,
> >
> > I require the xwiki solr docker container to do progress with my work. I
> > have made a pull request
> > <https://github.com/xwiki-contrib/docker-xwiki/pull/22>, for the same.
> > Please look into it.
> >
> > Moreover, I was working with Rook(Ceph Operator) for a shared file system
> > which would be a better approach to deploy our app on production. So I
> > think we should document it somewhere. Can anyone help me where should I
> do
> > that. It may contain the details to configure rook-ceph, and then
> > connecting it.
> >
> > P.S. I am writing a blog at Medium on how to configure rook on Google
> > Kubernetes Engine. We could also use that?
> >
> > Thanks,
> > Ashish
> >
> > On Mon, Jun 24, 2019 at 8:31 PM Ashish Sharma <
> [hidden email]>
> > wrote:
> >
> >> Hi Shubham,
> >>
> >> Thank you Shubham, I will start working on adding test cases.
> >>
> >> Thanks,
> >> Ashish
> >>
> >> On Mon, 24 Jun 2019 at 3:32 PM, Shubham Jain <[hidden email]>
> >> wrote:
> >>
> >>> Hey Ashish,
> >>>
> >>> Thanks for the update.
> >>>
> >>> Adding our chart to incubator section of helm community charts
> repository
> >>> sounds great. Although (unlike Dockerhub) this repo is for community
> charts
> >>> and not really official charts of the respective installations, but
> >>> supporting this chart in the community chart repo should help with the
> >>> visibility.
> >>>
> >>> Lets also work on adding tests to the helm chart to ensure things are
> >>> going fine.
> >>>
> >>> Best,
> >>> Shubham.
> >>>
> >>>> On 21-Jun-2019, at 4:37 AM, Ashish Sharma <[hidden email]
> >
> >>> wrote:
> >>>>
> >>>> Hi everyone,
> >>>>
> >>>> Update on HELM project:-
> >>>>
> >>>> We have added some features to the chart that includes:-
> >>>> * Making chart configurable with both mysql and postgres database
> >>>> * We are now using configmaps and secrets for storing database
> >>> credentials
> >>>> * We use helm dependency to manage the dependencies of the chart
> >>>>
> >>>> I think our chart is ready to be deployed in the incubator section of
> >>> helm
> >>>> community repo. <https://github.com/helm/charts> If you guys agree I
> >>> would
> >>>> make the necessary changes required to deploy it there.
> >>>>
> >>>> Thanks
> >>>> - Ashish
> >>>>
> >>>> On Wed, Jun 5, 2019 at 7:40 PM Vincent Massol <[hidden email]>
> >>> wrote:
> >>>>
> >>>>> Hi Ashish,
> >>>>>
> >>>>>> On 5 Jun 2019, at 15:07, Ashish Sharma <[hidden email]>
> >>>>> wrote:
> >>>>>>
> >>>>>> Hi everyone,
> >>>>>>
> >>>>>> The first week update of helm chart xwiki project. We currently
> have a
> >>>>>> chart that would deploy xwiki mysql stateful service with a
> persistent
> >>>>>> volume. The link to the repo:-
> >>>>>>
> >>>>>> https://github.com/xwiki-contrib/xwiki-helm
> >>>>>
> >>>>> Thanks for the update.
> >>>>>
> >>>>> Would be great if you could update the README file to explain who to
> >>> use
> >>>>> it.
> >>>>>
> >>>>> Thanks
> >>>>> -Vincent
> >>>>>
> >>>>>
> >>>>>>
> >>>>>> Thank You
> >>>>>> Ashish
> >>>>>>
> >>>>>> On Thu, 23 May 2019 at 7:27 PM, Ashish Sharma <
> >>>>> [hidden email]>
> >>>>>> wrote:
> >>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>> On Wed, May 22, 2019 at 12:42 PM Vincent Massol <
> [hidden email]>
> >>>>>>> wrote:
> >>>>>>>
> >>>>>>>> Hi Ashish,
> >>>>>>>>
> >>>>>>>>> On 21 May 2019, at 19:37, Ashish Sharma <
> >>> [hidden email]>
> >>>>>>>> wrote:
> >>>>>>>>>
> >>>>>>>>> Hi everyone,
> >>>>>>>>> Thanks for having me here
> >>>>>>>>>
> >>>>>>>>> About Me
> >>>>>>>>>
> >>>>>>>>> I am Ashish Sharma, selected as a student for Google Summer of
> >>> Code. I
> >>>>>>>> am
> >>>>>>>>> final year student enrolled in Guru Gobind Singh Indraprastha
> >>>>>>>> University,
> >>>>>>>>> Delhi. I am a resident of India.
> >>>>>>>>>
> >>>>>>>>> Profiles
> >>>>>>>>> GitHub -  https://github.com/ashish932/xwiki-helm-chart/
> >>>>>>>>> LinkedIn - https://www.linkedin.com/in/ashish932/
> >>>>>>>>> Riot - @ashish932:matrix.org
> >>>>>>>>>
> >>>>>>>>> I will be presenting my project "Helm Chart for XWiki" to all of
> >>> you.
> >>>>>>>>> Following
> >>>>>>>>> are the relevant details.
> >>>>>>>>>
> >>>>>>>>> Helm Chart for XWiki
> >>>>>>>>>
> >>>>>>>>> Mentors: Shubham Jain, Neha Gupta
> >>>>>>>>>
> >>>>>>>>> Technologies: Kubernetes, Docker, other if required
> >>>>>>>>>
> >>>>>>>>> Overview
> >>>>>>>>> The proposed project is a helm chart that would deploy xwiki as
> >>> highly
> >>>>>>>>> available and reliable. It should be configurable with different
> >>>>>>>>> databases(either a standalone database or a clustered one) that
> are
> >>>>>>>>> configurable with xwiki. It would give the option to either
> >>> configure
> >>>>>>>> solr
> >>>>>>>>> externally (standalone or clustered) or managed within the
> >>> container.
> >>>>>>>>
> >>>>>>>> It also needs to be deployed on various Servlet Containers and
> with
> >>> a
> >>>>>>>> LibreOffice server configured.
> >>>>>>>>
> >>>>>>>> In short, it should support all the configurations supported by
> the
> >>>>>>>> official XWiki Docker images :)
> >>>>>>>>
> >>>>>>>>> It
> >>>>>>>>> should deploy the app on a shared file system like a rook. It
> >>> should
> >>>>>>>>> support Istio virtual services, istio matrix, and istio
> distributed
> >>>>>>>> tracing
> >>>>>>>>> and should be a secured system with RBAC and security credential
> >>>>>>>> rotation.
> >>>>>>>>
> >>>>>>>> Ok I have no knowledge about this… The shared file system sounds
> >>> good
> >>>>> to
> >>>>>>>> support clustering XWiki for the permanent directory sharing.
> >>>>>>>>
> >>>>>>>>> The chart should be easily deployed on GKE and amazon EKS.
> >>>>>>>>
> >>>>>>>> On locally on minikube for testing I guess?
> >>>>>>>>
> >>>>>>>
> >>>>>>> Yes we can test using minikube one
> >>>>>>>
> >>>>>>>>
> >>>>>>>>> Features
> >>>>>>>>>
> >>>>>>>>> ->  Support for different Databases
> >>>>>>>>> ->  Choice between using an external database, a single node DB
> or
> >>> a
> >>>>>>>>> multi-cluster DB setup
> >>>>>>>>> -> Support for shared file system
> >>>>>>>>> -> Support for istio and it's services
> >>>>>>>>> -> RBAC, SSL and other security methods
> >>>>>>>>>
> >>>>>>>>> If you have any features in mind that should be added please feel
> >>> free
> >>>>>>>> to
> >>>>>>>>> reply to this mail.
> >>>>>>>>>
> >>>>>>>>> Some Design Questions?
> >>>>>>>>> -> Which Databases should be supported?
> >>>>>>>>
> >>>>>>>> See
> >>>>>>>>
> >>>>>
> >>>
> https://dev.xwiki.org/xwiki/bin/view/Community/SupportStrategy/DatabaseSupportStrategy
> >>>>>>>>
> >>>>>>>> More generally see the XWiki Docker image documentation and
> >>>>>>>> https://dev.xwiki.org/xwiki/bin/view/Community/SupportStrategy
> >>>>>>>>
> >>>>>>>>> -> As we have to detach solr out of the docker container(run it
> in
> >>> an
> >>>>>>>>> independent container) would be there a requirement for a code
> >>> change,
> >>>>>>>> and
> >>>>>>>>> we should approach it?
> >>>>>>>>
> >>>>>>>> See the doc for Solr which explains how to configure an external
> >>> SOLR.
> >>>>>>>> There are some XWiki code/config to deploy in the external SOLR
> >>> AFAIR.
> >>>>>>>> comunity controls chart
> >>>>>>>>> -> Apart from solr is there any other stateful service that could
> >>> or
> >>>>>>>> should
> >>>>>>>>> be detached from the docker container?
> >>>>>>>>
> >>>>>>>> Which docker container are you referring to? The Servlet Container
> >>> one?
> >>>>>>>>
> >>>>>>>> xwiki:lts-mysql-tomcat
> >>>>>>>
> >>>>>>>
> >>>>>>>>> Here is my current repository which deploys XWiki for MySQL
> >>> database
> >>>>>>>> using
> >>>>>>>>> official XWiki docker container:-
> >>>>>>>>> https://github.com/ashish932/xwiki-helm-chart/
> >>>>>>>>
> >>>>>>>> Will need to have a look. Some additional questions:
> >>>>>>>>
> >>>>>>>> 1) Is there an official place for publishing helm charts, similar
> to
> >>>>>>>> dockerhub for docker images? How do you provide the helm charts to
> >>>>> XWiki
> >>>>>>>> users in term of packaging? In other words, what will we/you
> >>> document
> >>>>> on
> >>>>>>>>
> >>>>>
> >>>
> https://www.xwiki.org/xwiki/bin/view/Documentation/AdminGuide/Installation/
> >>>>>>>> ?
> >>>>>>>>
> >>>>>>>> Yes, There is a centralized community control chart management
> >>>>>>> repository. Helm charts are hosted there. Organizations like mysql,
> >>>>>>> wordpress uses it.
> >>>>>>> https://github.com/helm/charts
> >>>>>>> But in addition to that, we can build our own charts repository
> >>> locally,
> >>>>>>> and host it. Similar to docker run command, we have helm update
> >>> command
> >>>>>>>
> >>>>>>>
> >>>>>>>> 2) How do you plan to have smoke tests for the XWiki Helm charts,
> as
> >>>>> part
> >>>>>>>> of the build, to verify that they work fine? How can we execute
> >>> XWik’s
> >>>>>>>> functional tests based on TestContainers using Helm Charts?
> >>>>>>>>
> >>>>>>>
> >>>>>>> There are some tools available
> >>>>>>> chart-testing(started by helm) -98 commits
> >>>>>>> terratest - 1039 commits (It looks a good tool to use)
> >>>>>>>
> >>>>>>>
> >>>>>>>> 3) Do the helm charts for XWiki need to be built? If so, can it be
> >>> done
> >>>>>>>> with Maven? Since we’ll be offering several I guess, how do you
> >>> avoid
> >>>>> code
> >>>>>>>> duplication and make maintenance of them as simple as possible?
> >>>>>>>>
> >>>>>>>
> >>>>>>> No, the charts don’t need to be built externally, though it gets
> >>>>>>> compressed when pushed f we build to chart repository(this is an
> >>>>> automatic
> >>>>>>> step in case of official chart repo and we can automate using ci
> our
> >>> own
> >>>>>>> chart repo)
> >>>>>>>
> >>>>>>> 4) Could you provide a roadmap with various milestones and dates?
> See
> >>>>> the
> >>>>>>>> last roadmap email I sent on the devs list and please reply to it.
> >>>>>>>>
> >>>>>>>
> >>>>>>> Okay I would work on that and update the progress on the mail
> thread
> >>>>>>>
> >>>>>>>
> >>>>>>>> 5) It seems you missed the bonding period so you’ll need to catch
> >>> up on
> >>>>>>>> that. Could you pick some jira issue and work on them quickly?
> This
> >>> is
> >>>>> to
> >>>>>>>> make sure you use the proper xwiki communication channels, learn
> to
> >>>>> work
> >>>>>>>> with the xwiki team and start understanding the xwiki
> architecture,
> >>>>> and how
> >>>>>>>> extensions are developed for XWiki. Since you’re a bit late,
> you’ll
> >>>>> need to
> >>>>>>>> catch up and achieve this very quickly now and be present full
> time
> >>> on
> >>>>> the
> >>>>>>>> xwiki chat :)
> >>>>>>>>
> >>>>>>> I would start doing that
> >>>>>>>
> >>>>>>>>
> >>>>>>>> 6) Idea: please review the issues on the XWiki Docker project’s
> >>> jira.
> >>>>>>>> Maybe you can help with those too as part of onboarding/bonding.
> It
> >>>>> should
> >>>>>>>> also give you additional use cases for the XWiki Helm Charts IMO.
> >>>>>>>>
> >>>>>>>> Thanks! I’m eager to follow your work and see xwiki helm charts in
> >>>>> action.
> >>>>>>>>
> >>>>>>>> -Vincent
> >>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>> Thank You
> >>>>>>>>> Ashish Sharma
> >>>>>>>>
> >>>>>>>> Thank You
> >>>>>>> Ashish
> >>>>>>>
> >>>>>
> >>>>>
> >>>
> >>>
>
>
Reply | Threaded
Open this post in threaded view
|

Re: Helm Chart Xwiki - GSOC'19

Ashish Sharma
In reply to this post by Shubham Jain
Hello everyone,

I am trying to implement the xwiki helm chart in High availability but
facing some problems.

How we can implement High Availability(HA) in Kubernetes:-

1. Through Deployment:-  In this approach, we would use shared file
storage, that would get attached to multiple pods(or containers)
2. Through StatefulSet:- In this approach, every pod(or container) would
have its own different volume and state, which should be synchronized by a
consensus algorithm

Problem faced in XWiki case:-

1. In deployment case we need a locking mechanism for our filesystem,
moreover caching and states should also be maintained outside the container.
2. In StatefulSet the consensus algorithm is not present so it is currently
not possible.

Could anyone help me through it?

Thanks,
Ashish
Reply | Threaded
Open this post in threaded view
|

Re: Xwiki Helm HA

Ashish Sharma
In reply to this post by Ashish Sharma
Resent the email at the mail thread.

On Fri, Jul 19, 2019 at 2:01 AM Ashish Sharma <[hidden email]>
wrote:

> Hello everyone,
>
> I am trying to implement xwiki helm chart in High availability but facing
> some problems.
>
> How we can implement High Availability(HA) in Kubernetes:-
>
> 1. Through Deployment:-  In this approach we would use a shared file
> storage, that would get attached to multiple pods(or containers)
> 2. Through StatefulSet:- In this approach every pod(or container) would
> have it’s own different volume and state, which should be synchronised by
> an consensus algorithm
>
> Problem faced in xwiki case:-
>
> 1. In deployment case we need a locking mechanism for our filesystem,
> moreover caching and states should also be maintained outside the container.
> 2. In StatefulSet the consensus algorithm is not present so it is
> currently not possible.
>
> Could anyone help me through it.
>
> Thanks,
> Ashish
>
> On Wed, 10 Jul 2019 at 10:32 AM, Shubham Jain <[hidden email]>
> wrote:
>
>> Hey Ashish,
>>
>> I think Vincent/Thomas can help with the PR on the xwiki-docker project.
>>
>> Regarding adding docs for High-Availability setup using shared data
>> volumes, I think you can use could create a docs folder in the GitHub repo,
>> and add the docs in Markdown format. I think you can also document it in
>> XWiki blog <https://www.xwiki.org/xwiki/bin/view/Blog/>. The chart
>> should support enabling HA, and then passing a storage class which supports
>> shareable volumes to Pods, and then you can document examples for using
>> this with something like Rook+GKE setup.
>>
>> Best,
>> Shubham.
>>
>>
>> > On 08-Jul-2019, at 1:57 PM, Ashish Sharma <[hidden email]>
>> wrote:
>> >
>> > Hello everyone,
>> >
>> > I require the xwiki solr docker container to do progress with my work. I
>> > have made a pull request
>> > <https://github.com/xwiki-contrib/docker-xwiki/pull/22>, for the same.
>> > Please look into it.
>> >
>> > Moreover, I was working with Rook(Ceph Operator) for a shared file
>> system
>> > which would be a better approach to deploy our app on production. So I
>> > think we should document it somewhere. Can anyone help me where should
>> I do
>> > that. It may contain the details to configure rook-ceph, and then
>> > connecting it.
>> >
>> > P.S. I am writing a blog at Medium on how to configure rook on Google
>> > Kubernetes Engine. We could also use that?
>> >
>> > Thanks,
>> > Ashish
>> >
>> > On Mon, Jun 24, 2019 at 8:31 PM Ashish Sharma <
>> [hidden email]>
>> > wrote:
>> >
>> >> Hi Shubham,
>> >>
>> >> Thank you Shubham, I will start working on adding test cases.
>> >>
>> >> Thanks,
>> >> Ashish
>> >>
>> >> On Mon, 24 Jun 2019 at 3:32 PM, Shubham Jain <[hidden email]>
>> >> wrote:
>> >>
>> >>> Hey Ashish,
>> >>>
>> >>> Thanks for the update.
>> >>>
>> >>> Adding our chart to incubator section of helm community charts
>> repository
>> >>> sounds great. Although (unlike Dockerhub) this repo is for community
>> charts
>> >>> and not really official charts of the respective installations, but
>> >>> supporting this chart in the community chart repo should help with the
>> >>> visibility.
>> >>>
>> >>> Lets also work on adding tests to the helm chart to ensure things are
>> >>> going fine.
>> >>>
>> >>> Best,
>> >>> Shubham.
>> >>>
>> >>>> On 21-Jun-2019, at 4:37 AM, Ashish Sharma <
>> [hidden email]>
>> >>> wrote:
>> >>>>
>> >>>> Hi everyone,
>> >>>>
>> >>>> Update on HELM project:-
>> >>>>
>> >>>> We have added some features to the chart that includes:-
>> >>>> * Making chart configurable with both mysql and postgres database
>> >>>> * We are now using configmaps and secrets for storing database
>> >>> credentials
>> >>>> * We use helm dependency to manage the dependencies of the chart
>> >>>>
>> >>>> I think our chart is ready to be deployed in the incubator section of
>> >>> helm
>> >>>> community repo. <https://github.com/helm/charts> If you guys agree I
>> >>> would
>> >>>> make the necessary changes required to deploy it there.
>> >>>>
>> >>>> Thanks
>> >>>> - Ashish
>> >>>>
>> >>>> On Wed, Jun 5, 2019 at 7:40 PM Vincent Massol <[hidden email]>
>> >>> wrote:
>> >>>>
>> >>>>> Hi Ashish,
>> >>>>>
>> >>>>>> On 5 Jun 2019, at 15:07, Ashish Sharma <[hidden email]
>> >
>> >>>>> wrote:
>> >>>>>>
>> >>>>>> Hi everyone,
>> >>>>>>
>> >>>>>> The first week update of helm chart xwiki project. We currently
>> have a
>> >>>>>> chart that would deploy xwiki mysql stateful service with a
>> persistent
>> >>>>>> volume. The link to the repo:-
>> >>>>>>
>> >>>>>> https://github.com/xwiki-contrib/xwiki-helm
>> >>>>>
>> >>>>> Thanks for the update.
>> >>>>>
>> >>>>> Would be great if you could update the README file to explain who to
>> >>> use
>> >>>>> it.
>> >>>>>
>> >>>>> Thanks
>> >>>>> -Vincent
>> >>>>>
>> >>>>>
>> >>>>>>
>> >>>>>> Thank You
>> >>>>>> Ashish
>> >>>>>>
>> >>>>>> On Thu, 23 May 2019 at 7:27 PM, Ashish Sharma <
>> >>>>> [hidden email]>
>> >>>>>> wrote:
>> >>>>>>
>> >>>>>>>
>> >>>>>>>
>> >>>>>>> On Wed, May 22, 2019 at 12:42 PM Vincent Massol <
>> [hidden email]>
>> >>>>>>> wrote:
>> >>>>>>>
>> >>>>>>>> Hi Ashish,
>> >>>>>>>>
>> >>>>>>>>> On 21 May 2019, at 19:37, Ashish Sharma <
>> >>> [hidden email]>
>> >>>>>>>> wrote:
>> >>>>>>>>>
>> >>>>>>>>> Hi everyone,
>> >>>>>>>>> Thanks for having me here
>> >>>>>>>>>
>> >>>>>>>>> About Me
>> >>>>>>>>>
>> >>>>>>>>> I am Ashish Sharma, selected as a student for Google Summer of
>> >>> Code. I
>> >>>>>>>> am
>> >>>>>>>>> final year student enrolled in Guru Gobind Singh Indraprastha
>> >>>>>>>> University,
>> >>>>>>>>> Delhi. I am a resident of India.
>> >>>>>>>>>
>> >>>>>>>>> Profiles
>> >>>>>>>>> GitHub -  https://github.com/ashish932/xwiki-helm-chart/
>> >>>>>>>>> LinkedIn - https://www.linkedin.com/in/ashish932/
>> >>>>>>>>> Riot - @ashish932:matrix.org
>> >>>>>>>>>
>> >>>>>>>>> I will be presenting my project "Helm Chart for XWiki" to all of
>> >>> you.
>> >>>>>>>>> Following
>> >>>>>>>>> are the relevant details.
>> >>>>>>>>>
>> >>>>>>>>> Helm Chart for XWiki
>> >>>>>>>>>
>> >>>>>>>>> Mentors: Shubham Jain, Neha Gupta
>> >>>>>>>>>
>> >>>>>>>>> Technologies: Kubernetes, Docker, other if required
>> >>>>>>>>>
>> >>>>>>>>> Overview
>> >>>>>>>>> The proposed project is a helm chart that would deploy xwiki as
>> >>> highly
>> >>>>>>>>> available and reliable. It should be configurable with different
>> >>>>>>>>> databases(either a standalone database or a clustered one) that
>> are
>> >>>>>>>>> configurable with xwiki. It would give the option to either
>> >>> configure
>> >>>>>>>> solr
>> >>>>>>>>> externally (standalone or clustered) or managed within the
>> >>> container.
>> >>>>>>>>
>> >>>>>>>> It also needs to be deployed on various Servlet Containers and
>> with
>> >>> a
>> >>>>>>>> LibreOffice server configured.
>> >>>>>>>>
>> >>>>>>>> In short, it should support all the configurations supported by
>> the
>> >>>>>>>> official XWiki Docker images :)
>> >>>>>>>>
>> >>>>>>>>> It
>> >>>>>>>>> should deploy the app on a shared file system like a rook. It
>> >>> should
>> >>>>>>>>> support Istio virtual services, istio matrix, and istio
>> distributed
>> >>>>>>>> tracing
>> >>>>>>>>> and should be a secured system with RBAC and security credential
>> >>>>>>>> rotation.
>> >>>>>>>>
>> >>>>>>>> Ok I have no knowledge about this… The shared file system sounds
>> >>> good
>> >>>>> to
>> >>>>>>>> support clustering XWiki for the permanent directory sharing.
>> >>>>>>>>
>> >>>>>>>>> The chart should be easily deployed on GKE and amazon EKS.
>> >>>>>>>>
>> >>>>>>>> On locally on minikube for testing I guess?
>> >>>>>>>>
>> >>>>>>>
>> >>>>>>> Yes we can test using minikube one
>> >>>>>>>
>> >>>>>>>>
>> >>>>>>>>> Features
>> >>>>>>>>>
>> >>>>>>>>> ->  Support for different Databases
>> >>>>>>>>> ->  Choice between using an external database, a single node DB
>> or
>> >>> a
>> >>>>>>>>> multi-cluster DB setup
>> >>>>>>>>> -> Support for shared file system
>> >>>>>>>>> -> Support for istio and it's services
>> >>>>>>>>> -> RBAC, SSL and other security methods
>> >>>>>>>>>
>> >>>>>>>>> If you have any features in mind that should be added please
>> feel
>> >>> free
>> >>>>>>>> to
>> >>>>>>>>> reply to this mail.
>> >>>>>>>>>
>> >>>>>>>>> Some Design Questions?
>> >>>>>>>>> -> Which Databases should be supported?
>> >>>>>>>>
>> >>>>>>>> See
>> >>>>>>>>
>> >>>>>
>> >>>
>> https://dev.xwiki.org/xwiki/bin/view/Community/SupportStrategy/DatabaseSupportStrategy
>> >>>>>>>>
>> >>>>>>>> More generally see the XWiki Docker image documentation and
>> >>>>>>>> https://dev.xwiki.org/xwiki/bin/view/Community/SupportStrategy
>> >>>>>>>>
>> >>>>>>>>> -> As we have to detach solr out of the docker container(run it
>> in
>> >>> an
>> >>>>>>>>> independent container) would be there a requirement for a code
>> >>> change,
>> >>>>>>>> and
>> >>>>>>>>> we should approach it?
>> >>>>>>>>
>> >>>>>>>> See the doc for Solr which explains how to configure an external
>> >>> SOLR.
>> >>>>>>>> There are some XWiki code/config to deploy in the external SOLR
>> >>> AFAIR.
>> >>>>>>>> comunity controls chart
>> >>>>>>>>> -> Apart from solr is there any other stateful service that
>> could
>> >>> or
>> >>>>>>>> should
>> >>>>>>>>> be detached from the docker container?
>> >>>>>>>>
>> >>>>>>>> Which docker container are you referring to? The Servlet
>> Container
>> >>> one?
>> >>>>>>>>
>> >>>>>>>> xwiki:lts-mysql-tomcat
>> >>>>>>>
>> >>>>>>>
>> >>>>>>>>> Here is my current repository which deploys XWiki for MySQL
>> >>> database
>> >>>>>>>> using
>> >>>>>>>>> official XWiki docker container:-
>> >>>>>>>>> https://github.com/ashish932/xwiki-helm-chart/
>> >>>>>>>>
>> >>>>>>>> Will need to have a look. Some additional questions:
>> >>>>>>>>
>> >>>>>>>> 1) Is there an official place for publishing helm charts,
>> similar to
>> >>>>>>>> dockerhub for docker images? How do you provide the helm charts
>> to
>> >>>>> XWiki
>> >>>>>>>> users in term of packaging? In other words, what will we/you
>> >>> document
>> >>>>> on
>> >>>>>>>>
>> >>>>>
>> >>>
>> https://www.xwiki.org/xwiki/bin/view/Documentation/AdminGuide/Installation/
>> >>>>>>>> ?
>> >>>>>>>>
>> >>>>>>>> Yes, There is a centralized community control chart management
>> >>>>>>> repository. Helm charts are hosted there. Organizations like
>> mysql,
>> >>>>>>> wordpress uses it.
>> >>>>>>> https://github.com/helm/charts
>> >>>>>>> But in addition to that, we can build our own charts repository
>> >>> locally,
>> >>>>>>> and host it. Similar to docker run command, we have helm update
>> >>> command
>> >>>>>>>
>> >>>>>>>
>> >>>>>>>> 2) How do you plan to have smoke tests for the XWiki Helm
>> charts, as
>> >>>>> part
>> >>>>>>>> of the build, to verify that they work fine? How can we execute
>> >>> XWik’s
>> >>>>>>>> functional tests based on TestContainers using Helm Charts?
>> >>>>>>>>
>> >>>>>>>
>> >>>>>>> There are some tools available
>> >>>>>>> chart-testing(started by helm) -98 commits
>> >>>>>>> terratest - 1039 commits (It looks a good tool to use)
>> >>>>>>>
>> >>>>>>>
>> >>>>>>>> 3) Do the helm charts for XWiki need to be built? If so, can it
>> be
>> >>> done
>> >>>>>>>> with Maven? Since we’ll be offering several I guess, how do you
>> >>> avoid
>> >>>>> code
>> >>>>>>>> duplication and make maintenance of them as simple as possible?
>> >>>>>>>>
>> >>>>>>>
>> >>>>>>> No, the charts don’t need to be built externally, though it gets
>> >>>>>>> compressed when pushed f we build to chart repository(this is an
>> >>>>> automatic
>> >>>>>>> step in case of official chart repo and we can automate using ci
>> our
>> >>> own
>> >>>>>>> chart repo)
>> >>>>>>>
>> >>>>>>> 4) Could you provide a roadmap with various milestones and dates?
>> See
>> >>>>> the
>> >>>>>>>> last roadmap email I sent on the devs list and please reply to
>> it.
>> >>>>>>>>
>> >>>>>>>
>> >>>>>>> Okay I would work on that and update the progress on the mail
>> thread
>> >>>>>>>
>> >>>>>>>
>> >>>>>>>> 5) It seems you missed the bonding period so you’ll need to catch
>> >>> up on
>> >>>>>>>> that. Could you pick some jira issue and work on them quickly?
>> This
>> >>> is
>> >>>>> to
>> >>>>>>>> make sure you use the proper xwiki communication channels, learn
>> to
>> >>>>> work
>> >>>>>>>> with the xwiki team and start understanding the xwiki
>> architecture,
>> >>>>> and how
>> >>>>>>>> extensions are developed for XWiki. Since you’re a bit late,
>> you’ll
>> >>>>> need to
>> >>>>>>>> catch up and achieve this very quickly now and be present full
>> time
>> >>> on
>> >>>>> the
>> >>>>>>>> xwiki chat :)
>> >>>>>>>>
>> >>>>>>> I would start doing that
>> >>>>>>>
>> >>>>>>>>
>> >>>>>>>> 6) Idea: please review the issues on the XWiki Docker project’s
>> >>> jira.
>> >>>>>>>> Maybe you can help with those too as part of onboarding/bonding.
>> It
>> >>>>> should
>> >>>>>>>> also give you additional use cases for the XWiki Helm Charts IMO.
>> >>>>>>>>
>> >>>>>>>> Thanks! I’m eager to follow your work and see xwiki helm charts
>> in
>> >>>>> action.
>> >>>>>>>>
>> >>>>>>>> -Vincent
>> >>>>>>>>
>> >>>>>>>>>
>> >>>>>>>>>
>> >>>>>>>>> Thank You
>> >>>>>>>>> Ashish Sharma
>> >>>>>>>>
>> >>>>>>>> Thank You
>> >>>>>>> Ashish
>> >>>>>>>
>> >>>>>
>> >>>>>
>> >>>
>> >>>
>>
>>
Reply | Threaded
Open this post in threaded view
|

Re: Xwiki Helm HA

Shubham Jain
Hey Ashish,

I think for now you could stick to deployment. Have you been ever to discover the data you need to move to the shared file system? Is the in-memory caching mechanism causing problems?

@Thomas/Vincent - Can you please help us understand the implications of running XWiki in a clustered mode by sharing data volumes?

Best,
Shubham.


Best,
Shubham?

> On 19-Jul-2019, at 5:35 AM, Ashish Sharma <[hidden email]> wrote:
>
> Resent the email at the mail thread.
>
> On Fri, Jul 19, 2019 at 2:01 AM Ashish Sharma <[hidden email]>
> wrote:
>
>> Hello everyone,
>>
>> I am trying to implement xwiki helm chart in High availability but facing
>> some problems.
>>
>> How we can implement High Availability(HA) in Kubernetes:-
>>
>> 1. Through Deployment:-  In this approach we would use a shared file
>> storage, that would get attached to multiple pods(or containers)
>> 2. Through StatefulSet:- In this approach every pod(or container) would
>> have it’s own different volume and state, which should be synchronised by
>> an consensus algorithm
>>
>> Problem faced in xwiki case:-
>>
>> 1. In deployment case we need a locking mechanism for our filesystem,
>> moreover caching and states should also be maintained outside the container.
>> 2. In StatefulSet the consensus algorithm is not present so it is
>> currently not possible.
>>
>> Could anyone help me through it.
>>
>> Thanks,
>> Ashish
>>
>> On Wed, 10 Jul 2019 at 10:32 AM, Shubham Jain <[hidden email]>
>> wrote:
>>
>>> Hey Ashish,
>>>
>>> I think Vincent/Thomas can help with the PR on the xwiki-docker project.
>>>
>>> Regarding adding docs for High-Availability setup using shared data
>>> volumes, I think you can use could create a docs folder in the GitHub repo,
>>> and add the docs in Markdown format. I think you can also document it in
>>> XWiki blog <https://www.xwiki.org/xwiki/bin/view/Blog/>. The chart
>>> should support enabling HA, and then passing a storage class which supports
>>> shareable volumes to Pods, and then you can document examples for using
>>> this with something like Rook+GKE setup.
>>>
>>> Best,
>>> Shubham.
>>>
>>>
>>>> On 08-Jul-2019, at 1:57 PM, Ashish Sharma <[hidden email]>
>>> wrote:
>>>>
>>>> Hello everyone,
>>>>
>>>> I require the xwiki solr docker container to do progress with my work. I
>>>> have made a pull request
>>>> <https://github.com/xwiki-contrib/docker-xwiki/pull/22>, for the same.
>>>> Please look into it.
>>>>
>>>> Moreover, I was working with Rook(Ceph Operator) for a shared file
>>> system
>>>> which would be a better approach to deploy our app on production. So I
>>>> think we should document it somewhere. Can anyone help me where should
>>> I do
>>>> that. It may contain the details to configure rook-ceph, and then
>>>> connecting it.
>>>>
>>>> P.S. I am writing a blog at Medium on how to configure rook on Google
>>>> Kubernetes Engine. We could also use that?
>>>>
>>>> Thanks,
>>>> Ashish
>>>>
>>>> On Mon, Jun 24, 2019 at 8:31 PM Ashish Sharma <
>>> [hidden email]>
>>>> wrote:
>>>>
>>>>> Hi Shubham,
>>>>>
>>>>> Thank you Shubham, I will start working on adding test cases.
>>>>>
>>>>> Thanks,
>>>>> Ashish
>>>>>
>>>>> On Mon, 24 Jun 2019 at 3:32 PM, Shubham Jain <[hidden email]>
>>>>> wrote:
>>>>>
>>>>>> Hey Ashish,
>>>>>>
>>>>>> Thanks for the update.
>>>>>>
>>>>>> Adding our chart to incubator section of helm community charts
>>> repository
>>>>>> sounds great. Although (unlike Dockerhub) this repo is for community
>>> charts
>>>>>> and not really official charts of the respective installations, but
>>>>>> supporting this chart in the community chart repo should help with the
>>>>>> visibility.
>>>>>>
>>>>>> Lets also work on adding tests to the helm chart to ensure things are
>>>>>> going fine.
>>>>>>
>>>>>> Best,
>>>>>> Shubham.
>>>>>>
>>>>>>> On 21-Jun-2019, at 4:37 AM, Ashish Sharma <
>>> [hidden email]>
>>>>>> wrote:
>>>>>>>
>>>>>>> Hi everyone,
>>>>>>>
>>>>>>> Update on HELM project:-
>>>>>>>
>>>>>>> We have added some features to the chart that includes:-
>>>>>>> * Making chart configurable with both mysql and postgres database
>>>>>>> * We are now using configmaps and secrets for storing database
>>>>>> credentials
>>>>>>> * We use helm dependency to manage the dependencies of the chart
>>>>>>>
>>>>>>> I think our chart is ready to be deployed in the incubator section of
>>>>>> helm
>>>>>>> community repo. <https://github.com/helm/charts> If you guys agree I
>>>>>> would
>>>>>>> make the necessary changes required to deploy it there.
>>>>>>>
>>>>>>> Thanks
>>>>>>> - Ashish
>>>>>>>
>>>>>>> On Wed, Jun 5, 2019 at 7:40 PM Vincent Massol <[hidden email]>
>>>>>> wrote:
>>>>>>>
>>>>>>>> Hi Ashish,
>>>>>>>>
>>>>>>>>> On 5 Jun 2019, at 15:07, Ashish Sharma <[hidden email]
>>>>
>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>> Hi everyone,
>>>>>>>>>
>>>>>>>>> The first week update of helm chart xwiki project. We currently
>>> have a
>>>>>>>>> chart that would deploy xwiki mysql stateful service with a
>>> persistent
>>>>>>>>> volume. The link to the repo:-
>>>>>>>>>
>>>>>>>>> https://github.com/xwiki-contrib/xwiki-helm
>>>>>>>>
>>>>>>>> Thanks for the update.
>>>>>>>>
>>>>>>>> Would be great if you could update the README file to explain who to
>>>>>> use
>>>>>>>> it.
>>>>>>>>
>>>>>>>> Thanks
>>>>>>>> -Vincent
>>>>>>>>
>>>>>>>>
>>>>>>>>>
>>>>>>>>> Thank You
>>>>>>>>> Ashish
>>>>>>>>>
>>>>>>>>> On Thu, 23 May 2019 at 7:27 PM, Ashish Sharma <
>>>>>>>> [hidden email]>
>>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On Wed, May 22, 2019 at 12:42 PM Vincent Massol <
>>> [hidden email]>
>>>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>>> Hi Ashish,
>>>>>>>>>>>
>>>>>>>>>>>> On 21 May 2019, at 19:37, Ashish Sharma <
>>>>>> [hidden email]>
>>>>>>>>>>> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>> Hi everyone,
>>>>>>>>>>>> Thanks for having me here
>>>>>>>>>>>>
>>>>>>>>>>>> About Me
>>>>>>>>>>>>
>>>>>>>>>>>> I am Ashish Sharma, selected as a student for Google Summer of
>>>>>> Code. I
>>>>>>>>>>> am
>>>>>>>>>>>> final year student enrolled in Guru Gobind Singh Indraprastha
>>>>>>>>>>> University,
>>>>>>>>>>>> Delhi. I am a resident of India.
>>>>>>>>>>>>
>>>>>>>>>>>> Profiles
>>>>>>>>>>>> GitHub -  https://github.com/ashish932/xwiki-helm-chart/
>>>>>>>>>>>> LinkedIn - https://www.linkedin.com/in/ashish932/
>>>>>>>>>>>> Riot - @ashish932:matrix.org
>>>>>>>>>>>>
>>>>>>>>>>>> I will be presenting my project "Helm Chart for XWiki" to all of
>>>>>> you.
>>>>>>>>>>>> Following
>>>>>>>>>>>> are the relevant details.
>>>>>>>>>>>>
>>>>>>>>>>>> Helm Chart for XWiki
>>>>>>>>>>>>
>>>>>>>>>>>> Mentors: Shubham Jain, Neha Gupta
>>>>>>>>>>>>
>>>>>>>>>>>> Technologies: Kubernetes, Docker, other if required
>>>>>>>>>>>>
>>>>>>>>>>>> Overview
>>>>>>>>>>>> The proposed project is a helm chart that would deploy xwiki as
>>>>>> highly
>>>>>>>>>>>> available and reliable. It should be configurable with different
>>>>>>>>>>>> databases(either a standalone database or a clustered one) that
>>> are
>>>>>>>>>>>> configurable with xwiki. It would give the option to either
>>>>>> configure
>>>>>>>>>>> solr
>>>>>>>>>>>> externally (standalone or clustered) or managed within the
>>>>>> container.
>>>>>>>>>>>
>>>>>>>>>>> It also needs to be deployed on various Servlet Containers and
>>> with
>>>>>> a
>>>>>>>>>>> LibreOffice server configured.
>>>>>>>>>>>
>>>>>>>>>>> In short, it should support all the configurations supported by
>>> the
>>>>>>>>>>> official XWiki Docker images :)
>>>>>>>>>>>
>>>>>>>>>>>> It
>>>>>>>>>>>> should deploy the app on a shared file system like a rook. It
>>>>>> should
>>>>>>>>>>>> support Istio virtual services, istio matrix, and istio
>>> distributed
>>>>>>>>>>> tracing
>>>>>>>>>>>> and should be a secured system with RBAC and security credential
>>>>>>>>>>> rotation.
>>>>>>>>>>>
>>>>>>>>>>> Ok I have no knowledge about this… The shared file system sounds
>>>>>> good
>>>>>>>> to
>>>>>>>>>>> support clustering XWiki for the permanent directory sharing.
>>>>>>>>>>>
>>>>>>>>>>>> The chart should be easily deployed on GKE and amazon EKS.
>>>>>>>>>>>
>>>>>>>>>>> On locally on minikube for testing I guess?
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Yes we can test using minikube one
>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>> Features
>>>>>>>>>>>>
>>>>>>>>>>>> ->  Support for different Databases
>>>>>>>>>>>> ->  Choice between using an external database, a single node DB
>>> or
>>>>>> a
>>>>>>>>>>>> multi-cluster DB setup
>>>>>>>>>>>> -> Support for shared file system
>>>>>>>>>>>> -> Support for istio and it's services
>>>>>>>>>>>> -> RBAC, SSL and other security methods
>>>>>>>>>>>>
>>>>>>>>>>>> If you have any features in mind that should be added please
>>> feel
>>>>>> free
>>>>>>>>>>> to
>>>>>>>>>>>> reply to this mail.
>>>>>>>>>>>>
>>>>>>>>>>>> Some Design Questions?
>>>>>>>>>>>> -> Which Databases should be supported?
>>>>>>>>>>>
>>>>>>>>>>> See
>>>>>>>>>>>
>>>>>>>>
>>>>>>
>>> https://dev.xwiki.org/xwiki/bin/view/Community/SupportStrategy/DatabaseSupportStrategy
>>>>>>>>>>>
>>>>>>>>>>> More generally see the XWiki Docker image documentation and
>>>>>>>>>>> https://dev.xwiki.org/xwiki/bin/view/Community/SupportStrategy
>>>>>>>>>>>
>>>>>>>>>>>> -> As we have to detach solr out of the docker container(run it
>>> in
>>>>>> an
>>>>>>>>>>>> independent container) would be there a requirement for a code
>>>>>> change,
>>>>>>>>>>> and
>>>>>>>>>>>> we should approach it?
>>>>>>>>>>>
>>>>>>>>>>> See the doc for Solr which explains how to configure an external
>>>>>> SOLR.
>>>>>>>>>>> There are some XWiki code/config to deploy in the external SOLR
>>>>>> AFAIR.
>>>>>>>>>>> comunity controls chart
>>>>>>>>>>>> -> Apart from solr is there any other stateful service that
>>> could
>>>>>> or
>>>>>>>>>>> should
>>>>>>>>>>>> be detached from the docker container?
>>>>>>>>>>>
>>>>>>>>>>> Which docker container are you referring to? The Servlet
>>> Container
>>>>>> one?
>>>>>>>>>>>
>>>>>>>>>>> xwiki:lts-mysql-tomcat
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>>> Here is my current repository which deploys XWiki for MySQL
>>>>>> database
>>>>>>>>>>> using
>>>>>>>>>>>> official XWiki docker container:-
>>>>>>>>>>>> https://github.com/ashish932/xwiki-helm-chart/
>>>>>>>>>>>
>>>>>>>>>>> Will need to have a look. Some additional questions:
>>>>>>>>>>>
>>>>>>>>>>> 1) Is there an official place for publishing helm charts,
>>> similar to
>>>>>>>>>>> dockerhub for docker images? How do you provide the helm charts
>>> to
>>>>>>>> XWiki
>>>>>>>>>>> users in term of packaging? In other words, what will we/you
>>>>>> document
>>>>>>>> on
>>>>>>>>>>>
>>>>>>>>
>>>>>>
>>> https://www.xwiki.org/xwiki/bin/view/Documentation/AdminGuide/Installation/
>>>>>>>>>>> ?
>>>>>>>>>>>
>>>>>>>>>>> Yes, There is a centralized community control chart management
>>>>>>>>>> repository. Helm charts are hosted there. Organizations like
>>> mysql,
>>>>>>>>>> wordpress uses it.
>>>>>>>>>> https://github.com/helm/charts
>>>>>>>>>> But in addition to that, we can build our own charts repository
>>>>>> locally,
>>>>>>>>>> and host it. Similar to docker run command, we have helm update
>>>>>> command
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>> 2) How do you plan to have smoke tests for the XWiki Helm
>>> charts, as
>>>>>>>> part
>>>>>>>>>>> of the build, to verify that they work fine? How can we execute
>>>>>> XWik’s
>>>>>>>>>>> functional tests based on TestContainers using Helm Charts?
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> There are some tools available
>>>>>>>>>> chart-testing(started by helm) -98 commits
>>>>>>>>>> terratest - 1039 commits (It looks a good tool to use)
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>> 3) Do the helm charts for XWiki need to be built? If so, can it
>>> be
>>>>>> done
>>>>>>>>>>> with Maven? Since we’ll be offering several I guess, how do you
>>>>>> avoid
>>>>>>>> code
>>>>>>>>>>> duplication and make maintenance of them as simple as possible?
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> No, the charts don’t need to be built externally, though it gets
>>>>>>>>>> compressed when pushed f we build to chart repository(this is an
>>>>>>>> automatic
>>>>>>>>>> step in case of official chart repo and we can automate using ci
>>> our
>>>>>> own
>>>>>>>>>> chart repo)
>>>>>>>>>>
>>>>>>>>>> 4) Could you provide a roadmap with various milestones and dates?
>>> See
>>>>>>>> the
>>>>>>>>>>> last roadmap email I sent on the devs list and please reply to
>>> it.
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Okay I would work on that and update the progress on the mail
>>> thread
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>> 5) It seems you missed the bonding period so you’ll need to catch
>>>>>> up on
>>>>>>>>>>> that. Could you pick some jira issue and work on them quickly?
>>> This
>>>>>> is
>>>>>>>> to
>>>>>>>>>>> make sure you use the proper xwiki communication channels, learn
>>> to
>>>>>>>> work
>>>>>>>>>>> with the xwiki team and start understanding the xwiki
>>> architecture,
>>>>>>>> and how
>>>>>>>>>>> extensions are developed for XWiki. Since you’re a bit late,
>>> you’ll
>>>>>>>> need to
>>>>>>>>>>> catch up and achieve this very quickly now and be present full
>>> time
>>>>>> on
>>>>>>>> the
>>>>>>>>>>> xwiki chat :)
>>>>>>>>>>>
>>>>>>>>>> I would start doing that
>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> 6) Idea: please review the issues on the XWiki Docker project’s
>>>>>> jira.
>>>>>>>>>>> Maybe you can help with those too as part of onboarding/bonding.
>>> It
>>>>>>>> should
>>>>>>>>>>> also give you additional use cases for the XWiki Helm Charts IMO.
>>>>>>>>>>>
>>>>>>>>>>> Thanks! I’m eager to follow your work and see xwiki helm charts
>>> in
>>>>>>>> action.
>>>>>>>>>>>
>>>>>>>>>>> -Vincent
>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> Thank You
>>>>>>>>>>>> Ashish Sharma
>>>>>>>>>>>
>>>>>>>>>>> Thank You
>>>>>>>>>> Ashish
>>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>
>>>>>>
>>>
>>>
Reply | Threaded
Open this post in threaded view
|

Re: Xwiki Helm HA

Ashish Sharma
Hey Shubham,

I have moved the solr folder as exposed by the docker container to the file
system. I think in-memory cache is creating problem.

I started 3 container of xwiki with shared file storage, but all three were
trying to initiate the postgress database, that  is causing problem.

I want to know few things about xwiki:-

1. Do we use java threads ? -> if we do then filesystem locking should have
been implemented for that?

2. Other stateful things in our application like caching or other
filestorage that should be externalise?


Thanks,
Ashish

On Fri, 19 Jul 2019 at 3:59 AM, Shubham Jain <[hidden email]>
wrote:

> Hey Ashish,
>
> I think for now you could stick to deployment. Have you been ever to
> discover the data you need to move to the shared file system? Is the
> in-memory caching mechanism causing problems?
>
> @Thomas/Vincent - Can you please help us understand the implications of
> running XWiki in a clustered mode by sharing data volumes?
>
> Best,
> Shubham.
>
>
> Best,
> Shubham?
>
> > On 19-Jul-2019, at 5:35 AM, Ashish Sharma <[hidden email]>
> wrote:
> >
> > Resent the email at the mail thread.
> >
> > On Fri, Jul 19, 2019 at 2:01 AM Ashish Sharma <
> [hidden email]>
> > wrote:
> >
> >> Hello everyone,
> >>
> >> I am trying to implement xwiki helm chart in High availability but
> facing
> >> some problems.
> >>
> >> How we can implement High Availability(HA) in Kubernetes:-
> >>
> >> 1. Through Deployment:-  In this approach we would use a shared file
> >> storage, that would get attached to multiple pods(or containers)
> >> 2. Through StatefulSet:- In this approach every pod(or container) would
> >> have it’s own different volume and state, which should be synchronised
> by
> >> an consensus algorithm
> >>
> >> Problem faced in xwiki case:-
> >>
> >> 1. In deployment case we need a locking mechanism for our filesystem,
> >> moreover caching and states should also be maintained outside the
> container.
> >> 2. In StatefulSet the consensus algorithm is not present so it is
> >> currently not possible.
> >>
> >> Could anyone help me through it.
> >>
> >> Thanks,
> >> Ashish
> >>
> >> On Wed, 10 Jul 2019 at 10:32 AM, Shubham Jain <[hidden email]>
> >> wrote:
> >>
> >>> Hey Ashish,
> >>>
> >>> I think Vincent/Thomas can help with the PR on the xwiki-docker
> project.
> >>>
> >>> Regarding adding docs for High-Availability setup using shared data
> >>> volumes, I think you can use could create a docs folder in the GitHub
> repo,
> >>> and add the docs in Markdown format. I think you can also document it
> in
> >>> XWiki blog <https://www.xwiki.org/xwiki/bin/view/Blog/>. The chart
> >>> should support enabling HA, and then passing a storage class which
> supports
> >>> shareable volumes to Pods, and then you can document examples for using
> >>> this with something like Rook+GKE setup.
> >>>
> >>> Best,
> >>> Shubham.
> >>>
> >>>
> >>>> On 08-Jul-2019, at 1:57 PM, Ashish Sharma <[hidden email]
> >
> >>> wrote:
> >>>>
> >>>> Hello everyone,
> >>>>
> >>>> I require the xwiki solr docker container to do progress with my
> work. I
> >>>> have made a pull request
> >>>> <https://github.com/xwiki-contrib/docker-xwiki/pull/22>, for the
> same.
> >>>> Please look into it.
> >>>>
> >>>> Moreover, I was working with Rook(Ceph Operator) for a shared file
> >>> system
> >>>> which would be a better approach to deploy our app on production. So I
> >>>> think we should document it somewhere. Can anyone help me where should
> >>> I do
> >>>> that. It may contain the details to configure rook-ceph, and then
> >>>> connecting it.
> >>>>
> >>>> P.S. I am writing a blog at Medium on how to configure rook on Google
> >>>> Kubernetes Engine. We could also use that?
> >>>>
> >>>> Thanks,
> >>>> Ashish
> >>>>
> >>>> On Mon, Jun 24, 2019 at 8:31 PM Ashish Sharma <
> >>> [hidden email]>
> >>>> wrote:
> >>>>
> >>>>> Hi Shubham,
> >>>>>
> >>>>> Thank you Shubham, I will start working on adding test cases.
> >>>>>
> >>>>> Thanks,
> >>>>> Ashish
> >>>>>
> >>>>> On Mon, 24 Jun 2019 at 3:32 PM, Shubham Jain <
> [hidden email]>
> >>>>> wrote:
> >>>>>
> >>>>>> Hey Ashish,
> >>>>>>
> >>>>>> Thanks for the update.
> >>>>>>
> >>>>>> Adding our chart to incubator section of helm community charts
> >>> repository
> >>>>>> sounds great. Although (unlike Dockerhub) this repo is for community
> >>> charts
> >>>>>> and not really official charts of the respective installations, but
> >>>>>> supporting this chart in the community chart repo should help with
> the
> >>>>>> visibility.
> >>>>>>
> >>>>>> Lets also work on adding tests to the helm chart to ensure things
> are
> >>>>>> going fine.
> >>>>>>
> >>>>>> Best,
> >>>>>> Shubham.
> >>>>>>
> >>>>>>> On 21-Jun-2019, at 4:37 AM, Ashish Sharma <
> >>> [hidden email]>
> >>>>>> wrote:
> >>>>>>>
> >>>>>>> Hi everyone,
> >>>>>>>
> >>>>>>> Update on HELM project:-
> >>>>>>>
> >>>>>>> We have added some features to the chart that includes:-
> >>>>>>> * Making chart configurable with both mysql and postgres database
> >>>>>>> * We are now using configmaps and secrets for storing database
> >>>>>> credentials
> >>>>>>> * We use helm dependency to manage the dependencies of the chart
> >>>>>>>
> >>>>>>> I think our chart is ready to be deployed in the incubator section
> of
> >>>>>> helm
> >>>>>>> community repo. <https://github.com/helm/charts> If you guys
> agree I
> >>>>>> would
> >>>>>>> make the necessary changes required to deploy it there.
> >>>>>>>
> >>>>>>> Thanks
> >>>>>>> - Ashish
> >>>>>>>
> >>>>>>> On Wed, Jun 5, 2019 at 7:40 PM Vincent Massol <[hidden email]>
> >>>>>> wrote:
> >>>>>>>
> >>>>>>>> Hi Ashish,
> >>>>>>>>
> >>>>>>>>> On 5 Jun 2019, at 15:07, Ashish Sharma <
> [hidden email]
> >>>>
> >>>>>>>> wrote:
> >>>>>>>>>
> >>>>>>>>> Hi everyone,
> >>>>>>>>>
> >>>>>>>>> The first week update of helm chart xwiki project. We currently
> >>> have a
> >>>>>>>>> chart that would deploy xwiki mysql stateful service with a
> >>> persistent
> >>>>>>>>> volume. The link to the repo:-
> >>>>>>>>>
> >>>>>>>>> https://github.com/xwiki-contrib/xwiki-helm
> >>>>>>>>
> >>>>>>>> Thanks for the update.
> >>>>>>>>
> >>>>>>>> Would be great if you could update the README file to explain who
> to
> >>>>>> use
> >>>>>>>> it.
> >>>>>>>>
> >>>>>>>> Thanks
> >>>>>>>> -Vincent
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>>
> >>>>>>>>> Thank You
> >>>>>>>>> Ashish
> >>>>>>>>>
> >>>>>>>>> On Thu, 23 May 2019 at 7:27 PM, Ashish Sharma <
> >>>>>>>> [hidden email]>
> >>>>>>>>> wrote:
> >>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>> On Wed, May 22, 2019 at 12:42 PM Vincent Massol <
> >>> [hidden email]>
> >>>>>>>>>> wrote:
> >>>>>>>>>>
> >>>>>>>>>>> Hi Ashish,
> >>>>>>>>>>>
> >>>>>>>>>>>> On 21 May 2019, at 19:37, Ashish Sharma <
> >>>>>> [hidden email]>
> >>>>>>>>>>> wrote:
> >>>>>>>>>>>>
> >>>>>>>>>>>> Hi everyone,
> >>>>>>>>>>>> Thanks for having me here
> >>>>>>>>>>>>
> >>>>>>>>>>>> About Me
> >>>>>>>>>>>>
> >>>>>>>>>>>> I am Ashish Sharma, selected as a student for Google Summer of
> >>>>>> Code. I
> >>>>>>>>>>> am
> >>>>>>>>>>>> final year student enrolled in Guru Gobind Singh Indraprastha
> >>>>>>>>>>> University,
> >>>>>>>>>>>> Delhi. I am a resident of India.
> >>>>>>>>>>>>
> >>>>>>>>>>>> Profiles
> >>>>>>>>>>>> GitHub -  https://github.com/ashish932/xwiki-helm-chart/
> >>>>>>>>>>>> LinkedIn - https://www.linkedin.com/in/ashish932/
> >>>>>>>>>>>> Riot - @ashish932:matrix.org
> >>>>>>>>>>>>
> >>>>>>>>>>>> I will be presenting my project "Helm Chart for XWiki" to all
> of
> >>>>>> you.
> >>>>>>>>>>>> Following
> >>>>>>>>>>>> are the relevant details.
> >>>>>>>>>>>>
> >>>>>>>>>>>> Helm Chart for XWiki
> >>>>>>>>>>>>
> >>>>>>>>>>>> Mentors: Shubham Jain, Neha Gupta
> >>>>>>>>>>>>
> >>>>>>>>>>>> Technologies: Kubernetes, Docker, other if required
> >>>>>>>>>>>>
> >>>>>>>>>>>> Overview
> >>>>>>>>>>>> The proposed project is a helm chart that would deploy xwiki
> as
> >>>>>> highly
> >>>>>>>>>>>> available and reliable. It should be configurable with
> different
> >>>>>>>>>>>> databases(either a standalone database or a clustered one)
> that
> >>> are
> >>>>>>>>>>>> configurable with xwiki. It would give the option to either
> >>>>>> configure
> >>>>>>>>>>> solr
> >>>>>>>>>>>> externally (standalone or clustered) or managed within the
> >>>>>> container.
> >>>>>>>>>>>
> >>>>>>>>>>> It also needs to be deployed on various Servlet Containers and
> >>> with
> >>>>>> a
> >>>>>>>>>>> LibreOffice server configured.
> >>>>>>>>>>>
> >>>>>>>>>>> In short, it should support all the configurations supported by
> >>> the
> >>>>>>>>>>> official XWiki Docker images :)
> >>>>>>>>>>>
> >>>>>>>>>>>> It
> >>>>>>>>>>>> should deploy the app on a shared file system like a rook. It
> >>>>>> should
> >>>>>>>>>>>> support Istio virtual services, istio matrix, and istio
> >>> distributed
> >>>>>>>>>>> tracing
> >>>>>>>>>>>> and should be a secured system with RBAC and security
> credential
> >>>>>>>>>>> rotation.
> >>>>>>>>>>>
> >>>>>>>>>>> Ok I have no knowledge about this… The shared file system
> sounds
> >>>>>> good
> >>>>>>>> to
> >>>>>>>>>>> support clustering XWiki for the permanent directory sharing.
> >>>>>>>>>>>
> >>>>>>>>>>>> The chart should be easily deployed on GKE and amazon EKS.
> >>>>>>>>>>>
> >>>>>>>>>>> On locally on minikube for testing I guess?
> >>>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>> Yes we can test using minikube one
> >>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>>> Features
> >>>>>>>>>>>>
> >>>>>>>>>>>> ->  Support for different Databases
> >>>>>>>>>>>> ->  Choice between using an external database, a single node
> DB
> >>> or
> >>>>>> a
> >>>>>>>>>>>> multi-cluster DB setup
> >>>>>>>>>>>> -> Support for shared file system
> >>>>>>>>>>>> -> Support for istio and it's services
> >>>>>>>>>>>> -> RBAC, SSL and other security methods
> >>>>>>>>>>>>
> >>>>>>>>>>>> If you have any features in mind that should be added please
> >>> feel
> >>>>>> free
> >>>>>>>>>>> to
> >>>>>>>>>>>> reply to this mail.
> >>>>>>>>>>>>
> >>>>>>>>>>>> Some Design Questions?
> >>>>>>>>>>>> -> Which Databases should be supported?
> >>>>>>>>>>>
> >>>>>>>>>>> See
> >>>>>>>>>>>
> >>>>>>>>
> >>>>>>
> >>>
> https://dev.xwiki.org/xwiki/bin/view/Community/SupportStrategy/DatabaseSupportStrategy
> >>>>>>>>>>>
> >>>>>>>>>>> More generally see the XWiki Docker image documentation and
> >>>>>>>>>>> https://dev.xwiki.org/xwiki/bin/view/Community/SupportStrategy
> >>>>>>>>>>>
> >>>>>>>>>>>> -> As we have to detach solr out of the docker container(run
> it
> >>> in
> >>>>>> an
> >>>>>>>>>>>> independent container) would be there a requirement for a code
> >>>>>> change,
> >>>>>>>>>>> and
> >>>>>>>>>>>> we should approach it?
> >>>>>>>>>>>
> >>>>>>>>>>> See the doc for Solr which explains how to configure an
> external
> >>>>>> SOLR.
> >>>>>>>>>>> There are some XWiki code/config to deploy in the external SOLR
> >>>>>> AFAIR.
> >>>>>>>>>>> comunity controls chart
> >>>>>>>>>>>> -> Apart from solr is there any other stateful service that
> >>> could
> >>>>>> or
> >>>>>>>>>>> should
> >>>>>>>>>>>> be detached from the docker container?
> >>>>>>>>>>>
> >>>>>>>>>>> Which docker container are you referring to? The Servlet
> >>> Container
> >>>>>> one?
> >>>>>>>>>>>
> >>>>>>>>>>> xwiki:lts-mysql-tomcat
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>>> Here is my current repository which deploys XWiki for MySQL
> >>>>>> database
> >>>>>>>>>>> using
> >>>>>>>>>>>> official XWiki docker container:-
> >>>>>>>>>>>> https://github.com/ashish932/xwiki-helm-chart/
> >>>>>>>>>>>
> >>>>>>>>>>> Will need to have a look. Some additional questions:
> >>>>>>>>>>>
> >>>>>>>>>>> 1) Is there an official place for publishing helm charts,
> >>> similar to
> >>>>>>>>>>> dockerhub for docker images? How do you provide the helm charts
> >>> to
> >>>>>>>> XWiki
> >>>>>>>>>>> users in term of packaging? In other words, what will we/you
> >>>>>> document
> >>>>>>>> on
> >>>>>>>>>>>
> >>>>>>>>
> >>>>>>
> >>>
> https://www.xwiki.org/xwiki/bin/view/Documentation/AdminGuide/Installation/
> >>>>>>>>>>> ?
> >>>>>>>>>>>
> >>>>>>>>>>> Yes, There is a centralized community control chart management
> >>>>>>>>>> repository. Helm charts are hosted there. Organizations like
> >>> mysql,
> >>>>>>>>>> wordpress uses it.
> >>>>>>>>>> https://github.com/helm/charts
> >>>>>>>>>> But in addition to that, we can build our own charts repository
> >>>>>> locally,
> >>>>>>>>>> and host it. Similar to docker run command, we have helm update
> >>>>>> command
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>> 2) How do you plan to have smoke tests for the XWiki Helm
> >>> charts, as
> >>>>>>>> part
> >>>>>>>>>>> of the build, to verify that they work fine? How can we execute
> >>>>>> XWik’s
> >>>>>>>>>>> functional tests based on TestContainers using Helm Charts?
> >>>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>> There are some tools available
> >>>>>>>>>> chart-testing(started by helm) -98 commits
> >>>>>>>>>> terratest - 1039 commits (It looks a good tool to use)
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>> 3) Do the helm charts for XWiki need to be built? If so, can it
> >>> be
> >>>>>> done
> >>>>>>>>>>> with Maven? Since we’ll be offering several I guess, how do you
> >>>>>> avoid
> >>>>>>>> code
> >>>>>>>>>>> duplication and make maintenance of them as simple as possible?
> >>>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>> No, the charts don’t need to be built externally, though it gets
> >>>>>>>>>> compressed when pushed f we build to chart repository(this is an
> >>>>>>>> automatic
> >>>>>>>>>> step in case of official chart repo and we can automate using ci
> >>> our
> >>>>>> own
> >>>>>>>>>> chart repo)
> >>>>>>>>>>
> >>>>>>>>>> 4) Could you provide a roadmap with various milestones and
> dates?
> >>> See
> >>>>>>>> the
> >>>>>>>>>>> last roadmap email I sent on the devs list and please reply to
> >>> it.
> >>>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>> Okay I would work on that and update the progress on the mail
> >>> thread
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>> 5) It seems you missed the bonding period so you’ll need to
> catch
> >>>>>> up on
> >>>>>>>>>>> that. Could you pick some jira issue and work on them quickly?
> >>> This
> >>>>>> is
> >>>>>>>> to
> >>>>>>>>>>> make sure you use the proper xwiki communication channels,
> learn
> >>> to
> >>>>>>>> work
> >>>>>>>>>>> with the xwiki team and start understanding the xwiki
> >>> architecture,
> >>>>>>>> and how
> >>>>>>>>>>> extensions are developed for XWiki. Since you’re a bit late,
> >>> you’ll
> >>>>>>>> need to
> >>>>>>>>>>> catch up and achieve this very quickly now and be present full
> >>> time
> >>>>>> on
> >>>>>>>> the
> >>>>>>>>>>> xwiki chat :)
> >>>>>>>>>>>
> >>>>>>>>>> I would start doing that
> >>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>> 6) Idea: please review the issues on the XWiki Docker project’s
> >>>>>> jira.
> >>>>>>>>>>> Maybe you can help with those too as part of
> onboarding/bonding.
> >>> It
> >>>>>>>> should
> >>>>>>>>>>> also give you additional use cases for the XWiki Helm Charts
> IMO.
> >>>>>>>>>>>
> >>>>>>>>>>> Thanks! I’m eager to follow your work and see xwiki helm charts
> >>> in
> >>>>>>>> action.
> >>>>>>>>>>>
> >>>>>>>>>>> -Vincent
> >>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>> Thank You
> >>>>>>>>>>>> Ashish Sharma
> >>>>>>>>>>>
> >>>>>>>>>>> Thank You
> >>>>>>>>>> Ashish
> >>>>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>
> >>>>>>
> >>>
> >>>
>
Reply | Threaded
Open this post in threaded view
|

Re: Xwiki Helm HA

Shubham Jain
Hey Ashish,

I don’t understand XWiki well enough to fully understand the implications of running a clustered setup by sharing file system.

Can you try keeping the deployment size initially 1, and then later try to increase the size? Does that also cause problems?

Best,
Shubham.

> On 19-Jul-2019, at 4:03 PM, Ashish Sharma <[hidden email]> wrote:
>
> Hey Shubham,
>
> I have moved the solr folder as exposed by the docker container to the file
> system. I think in-memory cache is creating problem.
>
> I started 3 container of xwiki with shared file storage, but all three were
> trying to initiate the postgress database, that  is causing problem.
>
> I want to know few things about xwiki:-
>
> 1. Do we use java threads ? -> if we do then filesystem locking should have
> been implemented for that?
>
> 2. Other stateful things in our application like caching or other
> filestorage that should be externalise?
>
>
> Thanks,
> Ashish
>
> On Fri, 19 Jul 2019 at 3:59 AM, Shubham Jain <[hidden email]>
> wrote:
>
>> Hey Ashish,
>>
>> I think for now you could stick to deployment. Have you been ever to
>> discover the data you need to move to the shared file system? Is the
>> in-memory caching mechanism causing problems?
>>
>> @Thomas/Vincent - Can you please help us understand the implications of
>> running XWiki in a clustered mode by sharing data volumes?
>>
>> Best,
>> Shubham.
>>
>>
>> Best,
>> Shubham?
>>
>>> On 19-Jul-2019, at 5:35 AM, Ashish Sharma <[hidden email]>
>> wrote:
>>>
>>> Resent the email at the mail thread.
>>>
>>> On Fri, Jul 19, 2019 at 2:01 AM Ashish Sharma <
>> [hidden email]>
>>> wrote:
>>>
>>>> Hello everyone,
>>>>
>>>> I am trying to implement xwiki helm chart in High availability but
>> facing
>>>> some problems.
>>>>
>>>> How we can implement High Availability(HA) in Kubernetes:-
>>>>
>>>> 1. Through Deployment:-  In this approach we would use a shared file
>>>> storage, that would get attached to multiple pods(or containers)
>>>> 2. Through StatefulSet:- In this approach every pod(or container) would
>>>> have it’s own different volume and state, which should be synchronised
>> by
>>>> an consensus algorithm
>>>>
>>>> Problem faced in xwiki case:-
>>>>
>>>> 1. In deployment case we need a locking mechanism for our filesystem,
>>>> moreover caching and states should also be maintained outside the
>> container.
>>>> 2. In StatefulSet the consensus algorithm is not present so it is
>>>> currently not possible.
>>>>
>>>> Could anyone help me through it.
>>>>
>>>> Thanks,
>>>> Ashish
>>>>
>>>> On Wed, 10 Jul 2019 at 10:32 AM, Shubham Jain <[hidden email]>
>>>> wrote:
>>>>
>>>>> Hey Ashish,
>>>>>
>>>>> I think Vincent/Thomas can help with the PR on the xwiki-docker
>> project.
>>>>>
>>>>> Regarding adding docs for High-Availability setup using shared data
>>>>> volumes, I think you can use could create a docs folder in the GitHub
>> repo,
>>>>> and add the docs in Markdown format. I think you can also document it
>> in
>>>>> XWiki blog <https://www.xwiki.org/xwiki/bin/view/Blog/>. The chart
>>>>> should support enabling HA, and then passing a storage class which
>> supports
>>>>> shareable volumes to Pods, and then you can document examples for using
>>>>> this with something like Rook+GKE setup.
>>>>>
>>>>> Best,
>>>>> Shubham.
>>>>>
>>>>>
>>>>>> On 08-Jul-2019, at 1:57 PM, Ashish Sharma <[hidden email]
>>>
>>>>> wrote:
>>>>>>
>>>>>> Hello everyone,
>>>>>>
>>>>>> I require the xwiki solr docker container to do progress with my
>> work. I
>>>>>> have made a pull request
>>>>>> <https://github.com/xwiki-contrib/docker-xwiki/pull/22>, for the
>> same.
>>>>>> Please look into it.
>>>>>>
>>>>>> Moreover, I was working with Rook(Ceph Operator) for a shared file
>>>>> system
>>>>>> which would be a better approach to deploy our app on production. So I
>>>>>> think we should document it somewhere. Can anyone help me where should
>>>>> I do
>>>>>> that. It may contain the details to configure rook-ceph, and then
>>>>>> connecting it.
>>>>>>
>>>>>> P.S. I am writing a blog at Medium on how to configure rook on Google
>>>>>> Kubernetes Engine. We could also use that?
>>>>>>
>>>>>> Thanks,
>>>>>> Ashish
>>>>>>
>>>>>> On Mon, Jun 24, 2019 at 8:31 PM Ashish Sharma <
>>>>> [hidden email]>
>>>>>> wrote:
>>>>>>
>>>>>>> Hi Shubham,
>>>>>>>
>>>>>>> Thank you Shubham, I will start working on adding test cases.
>>>>>>>
>>>>>>> Thanks,
>>>>>>> Ashish
>>>>>>>
>>>>>>> On Mon, 24 Jun 2019 at 3:32 PM, Shubham Jain <
>> [hidden email]>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> Hey Ashish,
>>>>>>>>
>>>>>>>> Thanks for the update.
>>>>>>>>
>>>>>>>> Adding our chart to incubator section of helm community charts
>>>>> repository
>>>>>>>> sounds great. Although (unlike Dockerhub) this repo is for community
>>>>> charts
>>>>>>>> and not really official charts of the respective installations, but
>>>>>>>> supporting this chart in the community chart repo should help with
>> the
>>>>>>>> visibility.
>>>>>>>>
>>>>>>>> Lets also work on adding tests to the helm chart to ensure things
>> are
>>>>>>>> going fine.
>>>>>>>>
>>>>>>>> Best,
>>>>>>>> Shubham.
>>>>>>>>
>>>>>>>>> On 21-Jun-2019, at 4:37 AM, Ashish Sharma <
>>>>> [hidden email]>
>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>> Hi everyone,
>>>>>>>>>
>>>>>>>>> Update on HELM project:-
>>>>>>>>>
>>>>>>>>> We have added some features to the chart that includes:-
>>>>>>>>> * Making chart configurable with both mysql and postgres database
>>>>>>>>> * We are now using configmaps and secrets for storing database
>>>>>>>> credentials
>>>>>>>>> * We use helm dependency to manage the dependencies of the chart
>>>>>>>>>
>>>>>>>>> I think our chart is ready to be deployed in the incubator section
>> of
>>>>>>>> helm
>>>>>>>>> community repo. <https://github.com/helm/charts> If you guys
>> agree I
>>>>>>>> would
>>>>>>>>> make the necessary changes required to deploy it there.
>>>>>>>>>
>>>>>>>>> Thanks
>>>>>>>>> - Ashish
>>>>>>>>>
>>>>>>>>> On Wed, Jun 5, 2019 at 7:40 PM Vincent Massol <[hidden email]>
>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>>> Hi Ashish,
>>>>>>>>>>
>>>>>>>>>>> On 5 Jun 2019, at 15:07, Ashish Sharma <
>> [hidden email]
>>>>>>
>>>>>>>>>> wrote:
>>>>>>>>>>>
>>>>>>>>>>> Hi everyone,
>>>>>>>>>>>
>>>>>>>>>>> The first week update of helm chart xwiki project. We currently
>>>>> have a
>>>>>>>>>>> chart that would deploy xwiki mysql stateful service with a
>>>>> persistent
>>>>>>>>>>> volume. The link to the repo:-
>>>>>>>>>>>
>>>>>>>>>>> https://github.com/xwiki-contrib/xwiki-helm
>>>>>>>>>>
>>>>>>>>>> Thanks for the update.
>>>>>>>>>>
>>>>>>>>>> Would be great if you could update the README file to explain who
>> to
>>>>>>>> use
>>>>>>>>>> it.
>>>>>>>>>>
>>>>>>>>>> Thanks
>>>>>>>>>> -Vincent
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> Thank You
>>>>>>>>>>> Ashish
>>>>>>>>>>>
>>>>>>>>>>> On Thu, 23 May 2019 at 7:27 PM, Ashish Sharma <
>>>>>>>>>> [hidden email]>
>>>>>>>>>>> wrote:
>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> On Wed, May 22, 2019 at 12:42 PM Vincent Massol <
>>>>> [hidden email]>
>>>>>>>>>>>> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> Hi Ashish,
>>>>>>>>>>>>>
>>>>>>>>>>>>>> On 21 May 2019, at 19:37, Ashish Sharma <
>>>>>>>> [hidden email]>
>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Hi everyone,
>>>>>>>>>>>>>> Thanks for having me here
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> About Me
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> I am Ashish Sharma, selected as a student for Google Summer of
>>>>>>>> Code. I
>>>>>>>>>>>>> am
>>>>>>>>>>>>>> final year student enrolled in Guru Gobind Singh Indraprastha
>>>>>>>>>>>>> University,
>>>>>>>>>>>>>> Delhi. I am a resident of India.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Profiles
>>>>>>>>>>>>>> GitHub -  https://github.com/ashish932/xwiki-helm-chart/
>>>>>>>>>>>>>> LinkedIn - https://www.linkedin.com/in/ashish932/
>>>>>>>>>>>>>> Riot - @ashish932:matrix.org
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> I will be presenting my project "Helm Chart for XWiki" to all
>> of
>>>>>>>> you.
>>>>>>>>>>>>>> Following
>>>>>>>>>>>>>> are the relevant details.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Helm Chart for XWiki
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Mentors: Shubham Jain, Neha Gupta
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Technologies: Kubernetes, Docker, other if required
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Overview
>>>>>>>>>>>>>> The proposed project is a helm chart that would deploy xwiki
>> as
>>>>>>>> highly
>>>>>>>>>>>>>> available and reliable. It should be configurable with
>> different
>>>>>>>>>>>>>> databases(either a standalone database or a clustered one)
>> that
>>>>> are
>>>>>>>>>>>>>> configurable with xwiki. It would give the option to either
>>>>>>>> configure
>>>>>>>>>>>>> solr
>>>>>>>>>>>>>> externally (standalone or clustered) or managed within the
>>>>>>>> container.
>>>>>>>>>>>>>
>>>>>>>>>>>>> It also needs to be deployed on various Servlet Containers and
>>>>> with
>>>>>>>> a
>>>>>>>>>>>>> LibreOffice server configured.
>>>>>>>>>>>>>
>>>>>>>>>>>>> In short, it should support all the configurations supported by
>>>>> the
>>>>>>>>>>>>> official XWiki Docker images :)
>>>>>>>>>>>>>
>>>>>>>>>>>>>> It
>>>>>>>>>>>>>> should deploy the app on a shared file system like a rook. It
>>>>>>>> should
>>>>>>>>>>>>>> support Istio virtual services, istio matrix, and istio
>>>>> distributed
>>>>>>>>>>>>> tracing
>>>>>>>>>>>>>> and should be a secured system with RBAC and security
>> credential
>>>>>>>>>>>>> rotation.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Ok I have no knowledge about this… The shared file system
>> sounds
>>>>>>>> good
>>>>>>>>>> to
>>>>>>>>>>>>> support clustering XWiki for the permanent directory sharing.
>>>>>>>>>>>>>
>>>>>>>>>>>>>> The chart should be easily deployed on GKE and amazon EKS.
>>>>>>>>>>>>>
>>>>>>>>>>>>> On locally on minikube for testing I guess?
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> Yes we can test using minikube one
>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>> Features
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> ->  Support for different Databases
>>>>>>>>>>>>>> ->  Choice between using an external database, a single node
>> DB
>>>>> or
>>>>>>>> a
>>>>>>>>>>>>>> multi-cluster DB setup
>>>>>>>>>>>>>> -> Support for shared file system
>>>>>>>>>>>>>> -> Support for istio and it's services
>>>>>>>>>>>>>> -> RBAC, SSL and other security methods
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> If you have any features in mind that should be added please
>>>>> feel
>>>>>>>> free
>>>>>>>>>>>>> to
>>>>>>>>>>>>>> reply to this mail.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Some Design Questions?
>>>>>>>>>>>>>> -> Which Databases should be supported?
>>>>>>>>>>>>>
>>>>>>>>>>>>> See
>>>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>
>>>>>
>> https://dev.xwiki.org/xwiki/bin/view/Community/SupportStrategy/DatabaseSupportStrategy
>>>>>>>>>>>>>
>>>>>>>>>>>>> More generally see the XWiki Docker image documentation and
>>>>>>>>>>>>> https://dev.xwiki.org/xwiki/bin/view/Community/SupportStrategy
>>>>>>>>>>>>>
>>>>>>>>>>>>>> -> As we have to detach solr out of the docker container(run
>> it
>>>>> in
>>>>>>>> an
>>>>>>>>>>>>>> independent container) would be there a requirement for a code
>>>>>>>> change,
>>>>>>>>>>>>> and
>>>>>>>>>>>>>> we should approach it?
>>>>>>>>>>>>>
>>>>>>>>>>>>> See the doc for Solr which explains how to configure an
>> external
>>>>>>>> SOLR.
>>>>>>>>>>>>> There are some XWiki code/config to deploy in the external SOLR
>>>>>>>> AFAIR.
>>>>>>>>>>>>> comunity controls chart
>>>>>>>>>>>>>> -> Apart from solr is there any other stateful service that
>>>>> could
>>>>>>>> or
>>>>>>>>>>>>> should
>>>>>>>>>>>>>> be detached from the docker container?
>>>>>>>>>>>>>
>>>>>>>>>>>>> Which docker container are you referring to? The Servlet
>>>>> Container
>>>>>>>> one?
>>>>>>>>>>>>>
>>>>>>>>>>>>> xwiki:lts-mysql-tomcat
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>>> Here is my current repository which deploys XWiki for MySQL
>>>>>>>> database
>>>>>>>>>>>>> using
>>>>>>>>>>>>>> official XWiki docker container:-
>>>>>>>>>>>>>> https://github.com/ashish932/xwiki-helm-chart/
>>>>>>>>>>>>>
>>>>>>>>>>>>> Will need to have a look. Some additional questions:
>>>>>>>>>>>>>
>>>>>>>>>>>>> 1) Is there an official place for publishing helm charts,
>>>>> similar to
>>>>>>>>>>>>> dockerhub for docker images? How do you provide the helm charts
>>>>> to
>>>>>>>>>> XWiki
>>>>>>>>>>>>> users in term of packaging? In other words, what will we/you
>>>>>>>> document
>>>>>>>>>> on
>>>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>
>>>>>
>> https://www.xwiki.org/xwiki/bin/view/Documentation/AdminGuide/Installation/
>>>>>>>>>>>>> ?
>>>>>>>>>>>>>
>>>>>>>>>>>>> Yes, There is a centralized community control chart management
>>>>>>>>>>>> repository. Helm charts are hosted there. Organizations like
>>>>> mysql,
>>>>>>>>>>>> wordpress uses it.
>>>>>>>>>>>> https://github.com/helm/charts
>>>>>>>>>>>> But in addition to that, we can build our own charts repository
>>>>>>>> locally,
>>>>>>>>>>>> and host it. Similar to docker run command, we have helm update
>>>>>>>> command
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>> 2) How do you plan to have smoke tests for the XWiki Helm
>>>>> charts, as
>>>>>>>>>> part
>>>>>>>>>>>>> of the build, to verify that they work fine? How can we execute
>>>>>>>> XWik’s
>>>>>>>>>>>>> functional tests based on TestContainers using Helm Charts?
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> There are some tools available
>>>>>>>>>>>> chart-testing(started by helm) -98 commits
>>>>>>>>>>>> terratest - 1039 commits (It looks a good tool to use)
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>> 3) Do the helm charts for XWiki need to be built? If so, can it
>>>>> be
>>>>>>>> done
>>>>>>>>>>>>> with Maven? Since we’ll be offering several I guess, how do you
>>>>>>>> avoid
>>>>>>>>>> code
>>>>>>>>>>>>> duplication and make maintenance of them as simple as possible?
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> No, the charts don’t need to be built externally, though it gets
>>>>>>>>>>>> compressed when pushed f we build to chart repository(this is an
>>>>>>>>>> automatic
>>>>>>>>>>>> step in case of official chart repo and we can automate using ci
>>>>> our
>>>>>>>> own
>>>>>>>>>>>> chart repo)
>>>>>>>>>>>>
>>>>>>>>>>>> 4) Could you provide a roadmap with various milestones and
>> dates?
>>>>> See
>>>>>>>>>> the
>>>>>>>>>>>>> last roadmap email I sent on the devs list and please reply to
>>>>> it.
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> Okay I would work on that and update the progress on the mail
>>>>> thread
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>> 5) It seems you missed the bonding period so you’ll need to
>> catch
>>>>>>>> up on
>>>>>>>>>>>>> that. Could you pick some jira issue and work on them quickly?
>>>>> This
>>>>>>>> is
>>>>>>>>>> to
>>>>>>>>>>>>> make sure you use the proper xwiki communication channels,
>> learn
>>>>> to
>>>>>>>>>> work
>>>>>>>>>>>>> with the xwiki team and start understanding the xwiki
>>>>> architecture,
>>>>>>>>>> and how
>>>>>>>>>>>>> extensions are developed for XWiki. Since you’re a bit late,
>>>>> you’ll
>>>>>>>>>> need to
>>>>>>>>>>>>> catch up and achieve this very quickly now and be present full
>>>>> time
>>>>>>>> on
>>>>>>>>>> the
>>>>>>>>>>>>> xwiki chat :)
>>>>>>>>>>>>>
>>>>>>>>>>>> I would start doing that
>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> 6) Idea: please review the issues on the XWiki Docker project’s
>>>>>>>> jira.
>>>>>>>>>>>>> Maybe you can help with those too as part of
>> onboarding/bonding.
>>>>> It
>>>>>>>>>> should
>>>>>>>>>>>>> also give you additional use cases for the XWiki Helm Charts
>> IMO.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Thanks! I’m eager to follow your work and see xwiki helm charts
>>>>> in
>>>>>>>>>> action.
>>>>>>>>>>>>>
>>>>>>>>>>>>> -Vincent
>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Thank You
>>>>>>>>>>>>>> Ashish Sharma
>>>>>>>>>>>>>
>>>>>>>>>>>>> Thank You
>>>>>>>>>>>> Ashish
>>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>
>>>>>
>>
Reply | Threaded
Open this post in threaded view
|

Re: Xwiki Helm HA

Ashish Sharma
Hey Shubham,

I have tried initialising with 1 pod and scaling it up to multiple pods,
also tried session affinity, and some other cases but none of them worked.

Thanks,
Ashish
On Fri, 19 Jul 2019 at 4:51 PM, Shubham Jain <[hidden email]>
wrote:

> Hey Ashish,
>
> I don’t understand XWiki well enough to fully understand the implications
> of running a clustered setup by sharing file system.
>
> Can you try keeping the deployment size initially 1, and then later try to
> increase the size? Does that also cause problems?
>
> Best,
> Shubham.
>
> > On 19-Jul-2019, at 4:03 PM, Ashish Sharma <[hidden email]>
> wrote:
> >
> > Hey Shubham,
> >
> > I have moved the solr folder as exposed by the docker container to the
> file
> > system. I think in-memory cache is creating problem.
> >
> > I started 3 container of xwiki with shared file storage, but all three
> were
> > trying to initiate the postgress database, that  is causing problem.
> >
> > I want to know few things about xwiki:-
> >
> > 1. Do we use java threads ? -> if we do then filesystem locking should
> have
> > been implemented for that?
> >
> > 2. Other stateful things in our application like caching or other
> > filestorage that should be externalise?
> >
> >
> > Thanks,
> > Ashish
> >
> > On Fri, 19 Jul 2019 at 3:59 AM, Shubham Jain <[hidden email]>
> > wrote:
> >
> >> Hey Ashish,
> >>
> >> I think for now you could stick to deployment. Have you been ever to
> >> discover the data you need to move to the shared file system? Is the
> >> in-memory caching mechanism causing problems?
> >>
> >> @Thomas/Vincent - Can you please help us understand the implications of
> >> running XWiki in a clustered mode by sharing data volumes?
> >>
> >> Best,
> >> Shubham.
> >>
> >>
> >> Best,
> >> Shubham?
> >>
> >>> On 19-Jul-2019, at 5:35 AM, Ashish Sharma <[hidden email]>
> >> wrote:
> >>>
> >>> Resent the email at the mail thread.
> >>>
> >>> On Fri, Jul 19, 2019 at 2:01 AM Ashish Sharma <
> >> [hidden email]>
> >>> wrote:
> >>>
> >>>> Hello everyone,
> >>>>
> >>>> I am trying to implement xwiki helm chart in High availability but
> >> facing
> >>>> some problems.
> >>>>
> >>>> How we can implement High Availability(HA) in Kubernetes:-
> >>>>
> >>>> 1. Through Deployment:-  In this approach we would use a shared file
> >>>> storage, that would get attached to multiple pods(or containers)
> >>>> 2. Through StatefulSet:- In this approach every pod(or container)
> would
> >>>> have it’s own different volume and state, which should be synchronised
> >> by
> >>>> an consensus algorithm
> >>>>
> >>>> Problem faced in xwiki case:-
> >>>>
> >>>> 1. In deployment case we need a locking mechanism for our filesystem,
> >>>> moreover caching and states should also be maintained outside the
> >> container.
> >>>> 2. In StatefulSet the consensus algorithm is not present so it is
> >>>> currently not possible.
> >>>>
> >>>> Could anyone help me through it.
> >>>>
> >>>> Thanks,
> >>>> Ashish
> >>>>
> >>>> On Wed, 10 Jul 2019 at 10:32 AM, Shubham Jain <
> [hidden email]>
> >>>> wrote:
> >>>>
> >>>>> Hey Ashish,
> >>>>>
> >>>>> I think Vincent/Thomas can help with the PR on the xwiki-docker
> >> project.
> >>>>>
> >>>>> Regarding adding docs for High-Availability setup using shared data
> >>>>> volumes, I think you can use could create a docs folder in the GitHub
> >> repo,
> >>>>> and add the docs in Markdown format. I think you can also document it
> >> in
> >>>>> XWiki blog <https://www.xwiki.org/xwiki/bin/view/Blog/>. The chart
> >>>>> should support enabling HA, and then passing a storage class which
> >> supports
> >>>>> shareable volumes to Pods, and then you can document examples for
> using
> >>>>> this with something like Rook+GKE setup.
> >>>>>
> >>>>> Best,
> >>>>> Shubham.
> >>>>>
> >>>>>
> >>>>>> On 08-Jul-2019, at 1:57 PM, Ashish Sharma <
> [hidden email]
> >>>
> >>>>> wrote:
> >>>>>>
> >>>>>> Hello everyone,
> >>>>>>
> >>>>>> I require the xwiki solr docker container to do progress with my
> >> work. I
> >>>>>> have made a pull request
> >>>>>> <https://github.com/xwiki-contrib/docker-xwiki/pull/22>, for the
> >> same.
> >>>>>> Please look into it.
> >>>>>>
> >>>>>> Moreover, I was working with Rook(Ceph Operator) for a shared file
> >>>>> system
> >>>>>> which would be a better approach to deploy our app on production.
> So I
> >>>>>> think we should document it somewhere. Can anyone help me where
> should
> >>>>> I do
> >>>>>> that. It may contain the details to configure rook-ceph, and then
> >>>>>> connecting it.
> >>>>>>
> >>>>>> P.S. I am writing a blog at Medium on how to configure rook on
> Google
> >>>>>> Kubernetes Engine. We could also use that?
> >>>>>>
> >>>>>> Thanks,
> >>>>>> Ashish
> >>>>>>
> >>>>>> On Mon, Jun 24, 2019 at 8:31 PM Ashish Sharma <
> >>>>> [hidden email]>
> >>>>>> wrote:
> >>>>>>
> >>>>>>> Hi Shubham,
> >>>>>>>
> >>>>>>> Thank you Shubham, I will start working on adding test cases.
> >>>>>>>
> >>>>>>> Thanks,
> >>>>>>> Ashish
> >>>>>>>
> >>>>>>> On Mon, 24 Jun 2019 at 3:32 PM, Shubham Jain <
> >> [hidden email]>
> >>>>>>> wrote:
> >>>>>>>
> >>>>>>>> Hey Ashish,
> >>>>>>>>
> >>>>>>>> Thanks for the update.
> >>>>>>>>
> >>>>>>>> Adding our chart to incubator section of helm community charts
> >>>>> repository
> >>>>>>>> sounds great. Although (unlike Dockerhub) this repo is for
> community
> >>>>> charts
> >>>>>>>> and not really official charts of the respective installations,
> but
> >>>>>>>> supporting this chart in the community chart repo should help with
> >> the
> >>>>>>>> visibility.
> >>>>>>>>
> >>>>>>>> Lets also work on adding tests to the helm chart to ensure things
> >> are
> >>>>>>>> going fine.
> >>>>>>>>
> >>>>>>>> Best,
> >>>>>>>> Shubham.
> >>>>>>>>
> >>>>>>>>> On 21-Jun-2019, at 4:37 AM, Ashish Sharma <
> >>>>> [hidden email]>
> >>>>>>>> wrote:
> >>>>>>>>>
> >>>>>>>>> Hi everyone,
> >>>>>>>>>
> >>>>>>>>> Update on HELM project:-
> >>>>>>>>>
> >>>>>>>>> We have added some features to the chart that includes:-
> >>>>>>>>> * Making chart configurable with both mysql and postgres database
> >>>>>>>>> * We are now using configmaps and secrets for storing database
> >>>>>>>> credentials
> >>>>>>>>> * We use helm dependency to manage the dependencies of the chart
> >>>>>>>>>
> >>>>>>>>> I think our chart is ready to be deployed in the incubator
> section
> >> of
> >>>>>>>> helm
> >>>>>>>>> community repo. <https://github.com/helm/charts> If you guys
> >> agree I
> >>>>>>>> would
> >>>>>>>>> make the necessary changes required to deploy it there.
> >>>>>>>>>
> >>>>>>>>> Thanks
> >>>>>>>>> - Ashish
> >>>>>>>>>
> >>>>>>>>> On Wed, Jun 5, 2019 at 7:40 PM Vincent Massol <
> [hidden email]>
> >>>>>>>> wrote:
> >>>>>>>>>
> >>>>>>>>>> Hi Ashish,
> >>>>>>>>>>
> >>>>>>>>>>> On 5 Jun 2019, at 15:07, Ashish Sharma <
> >> [hidden email]
> >>>>>>
> >>>>>>>>>> wrote:
> >>>>>>>>>>>
> >>>>>>>>>>> Hi everyone,
> >>>>>>>>>>>
> >>>>>>>>>>> The first week update of helm chart xwiki project. We currently
> >>>>> have a
> >>>>>>>>>>> chart that would deploy xwiki mysql stateful service with a
> >>>>> persistent
> >>>>>>>>>>> volume. The link to the repo:-
> >>>>>>>>>>>
> >>>>>>>>>>> https://github.com/xwiki-contrib/xwiki-helm
> >>>>>>>>>>
> >>>>>>>>>> Thanks for the update.
> >>>>>>>>>>
> >>>>>>>>>> Would be great if you could update the README file to explain
> who
> >> to
> >>>>>>>> use
> >>>>>>>>>> it.
> >>>>>>>>>>
> >>>>>>>>>> Thanks
> >>>>>>>>>> -Vincent
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>> Thank You
> >>>>>>>>>>> Ashish
> >>>>>>>>>>>
> >>>>>>>>>>> On Thu, 23 May 2019 at 7:27 PM, Ashish Sharma <
> >>>>>>>>>> [hidden email]>
> >>>>>>>>>>> wrote:
> >>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>> On Wed, May 22, 2019 at 12:42 PM Vincent Massol <
> >>>>> [hidden email]>
> >>>>>>>>>>>> wrote:
> >>>>>>>>>>>>
> >>>>>>>>>>>>> Hi Ashish,
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>> On 21 May 2019, at 19:37, Ashish Sharma <
> >>>>>>>> [hidden email]>
> >>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> Hi everyone,
> >>>>>>>>>>>>>> Thanks for having me here
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> About Me
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> I am Ashish Sharma, selected as a student for Google Summer
> of
> >>>>>>>> Code. I
> >>>>>>>>>>>>> am
> >>>>>>>>>>>>>> final year student enrolled in Guru Gobind Singh
> Indraprastha
> >>>>>>>>>>>>> University,
> >>>>>>>>>>>>>> Delhi. I am a resident of India.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> Profiles
> >>>>>>>>>>>>>> GitHub -  https://github.com/ashish932/xwiki-helm-chart/
> >>>>>>>>>>>>>> LinkedIn - https://www.linkedin.com/in/ashish932/
> >>>>>>>>>>>>>> Riot - @ashish932:matrix.org
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> I will be presenting my project "Helm Chart for XWiki" to
> all
> >> of
> >>>>>>>> you.
> >>>>>>>>>>>>>> Following
> >>>>>>>>>>>>>> are the relevant details.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> Helm Chart for XWiki
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> Mentors: Shubham Jain, Neha Gupta
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> Technologies: Kubernetes, Docker, other if required
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> Overview
> >>>>>>>>>>>>>> The proposed project is a helm chart that would deploy xwiki
> >> as
> >>>>>>>> highly
> >>>>>>>>>>>>>> available and reliable. It should be configurable with
> >> different
> >>>>>>>>>>>>>> databases(either a standalone database or a clustered one)
> >> that
> >>>>> are
> >>>>>>>>>>>>>> configurable with xwiki. It would give the option to either
> >>>>>>>> configure
> >>>>>>>>>>>>> solr
> >>>>>>>>>>>>>> externally (standalone or clustered) or managed within the
> >>>>>>>> container.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> It also needs to be deployed on various Servlet Containers
> and
> >>>>> with
> >>>>>>>> a
> >>>>>>>>>>>>> LibreOffice server configured.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> In short, it should support all the configurations supported
> by
> >>>>> the
> >>>>>>>>>>>>> official XWiki Docker images :)
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>> It
> >>>>>>>>>>>>>> should deploy the app on a shared file system like a rook.
> It
> >>>>>>>> should
> >>>>>>>>>>>>>> support Istio virtual services, istio matrix, and istio
> >>>>> distributed
> >>>>>>>>>>>>> tracing
> >>>>>>>>>>>>>> and should be a secured system with RBAC and security
> >> credential
> >>>>>>>>>>>>> rotation.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> Ok I have no knowledge about this… The shared file system
> >> sounds
> >>>>>>>> good
> >>>>>>>>>> to
> >>>>>>>>>>>>> support clustering XWiki for the permanent directory sharing.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>> The chart should be easily deployed on GKE and amazon EKS.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> On locally on minikube for testing I guess?
> >>>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>> Yes we can test using minikube one
> >>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>> Features
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> ->  Support for different Databases
> >>>>>>>>>>>>>> ->  Choice between using an external database, a single node
> >> DB
> >>>>> or
> >>>>>>>> a
> >>>>>>>>>>>>>> multi-cluster DB setup
> >>>>>>>>>>>>>> -> Support for shared file system
> >>>>>>>>>>>>>> -> Support for istio and it's services
> >>>>>>>>>>>>>> -> RBAC, SSL and other security methods
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> If you have any features in mind that should be added please
> >>>>> feel
> >>>>>>>> free
> >>>>>>>>>>>>> to
> >>>>>>>>>>>>>> reply to this mail.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> Some Design Questions?
> >>>>>>>>>>>>>> -> Which Databases should be supported?
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> See
> >>>>>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>
> >>>>>
> >>
> https://dev.xwiki.org/xwiki/bin/view/Community/SupportStrategy/DatabaseSupportStrategy
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> More generally see the XWiki Docker image documentation and
> >>>>>>>>>>>>>
> https://dev.xwiki.org/xwiki/bin/view/Community/SupportStrategy
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>> -> As we have to detach solr out of the docker container(run
> >> it
> >>>>> in
> >>>>>>>> an
> >>>>>>>>>>>>>> independent container) would be there a requirement for a
> code
> >>>>>>>> change,
> >>>>>>>>>>>>> and
> >>>>>>>>>>>>>> we should approach it?
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> See the doc for Solr which explains how to configure an
> >> external
> >>>>>>>> SOLR.
> >>>>>>>>>>>>> There are some XWiki code/config to deploy in the external
> SOLR
> >>>>>>>> AFAIR.
> >>>>>>>>>>>>> comunity controls chart
> >>>>>>>>>>>>>> -> Apart from solr is there any other stateful service that
> >>>>> could
> >>>>>>>> or
> >>>>>>>>>>>>> should
> >>>>>>>>>>>>>> be detached from the docker container?
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> Which docker container are you referring to? The Servlet
> >>>>> Container
> >>>>>>>> one?
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> xwiki:lts-mysql-tomcat
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>>> Here is my current repository which deploys XWiki for MySQL
> >>>>>>>> database
> >>>>>>>>>>>>> using
> >>>>>>>>>>>>>> official XWiki docker container:-
> >>>>>>>>>>>>>> https://github.com/ashish932/xwiki-helm-chart/
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> Will need to have a look. Some additional questions:
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> 1) Is there an official place for publishing helm charts,
> >>>>> similar to
> >>>>>>>>>>>>> dockerhub for docker images? How do you provide the helm
> charts
> >>>>> to
> >>>>>>>>>> XWiki
> >>>>>>>>>>>>> users in term of packaging? In other words, what will we/you
> >>>>>>>> document
> >>>>>>>>>> on
> >>>>>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>
> >>>>>
> >>
> https://www.xwiki.org/xwiki/bin/view/Documentation/AdminGuide/Installation/
> >>>>>>>>>>>>> ?
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> Yes, There is a centralized community control chart
> management
> >>>>>>>>>>>> repository. Helm charts are hosted there. Organizations like
> >>>>> mysql,
> >>>>>>>>>>>> wordpress uses it.
> >>>>>>>>>>>> https://github.com/helm/charts
> >>>>>>>>>>>> But in addition to that, we can build our own charts
> repository
> >>>>>>>> locally,
> >>>>>>>>>>>> and host it. Similar to docker run command, we have helm
> update
> >>>>>>>> command
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>> 2) How do you plan to have smoke tests for the XWiki Helm
> >>>>> charts, as
> >>>>>>>>>> part
> >>>>>>>>>>>>> of the build, to verify that they work fine? How can we
> execute
> >>>>>>>> XWik’s
> >>>>>>>>>>>>> functional tests based on TestContainers using Helm Charts?
> >>>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>> There are some tools available
> >>>>>>>>>>>> chart-testing(started by helm) -98 commits
> >>>>>>>>>>>> terratest - 1039 commits (It looks a good tool to use)
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>> 3) Do the helm charts for XWiki need to be built? If so, can
> it
> >>>>> be
> >>>>>>>> done
> >>>>>>>>>>>>> with Maven? Since we’ll be offering several I guess, how do
> you
> >>>>>>>> avoid
> >>>>>>>>>> code
> >>>>>>>>>>>>> duplication and make maintenance of them as simple as
> possible?
> >>>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>> No, the charts don’t need to be built externally, though it
> gets
> >>>>>>>>>>>> compressed when pushed f we build to chart repository(this is
> an
> >>>>>>>>>> automatic
> >>>>>>>>>>>> step in case of official chart repo and we can automate using
> ci
> >>>>> our
> >>>>>>>> own
> >>>>>>>>>>>> chart repo)
> >>>>>>>>>>>>
> >>>>>>>>>>>> 4) Could you provide a roadmap with various milestones and
> >> dates?
> >>>>> See
> >>>>>>>>>> the
> >>>>>>>>>>>>> last roadmap email I sent on the devs list and please reply
> to
> >>>>> it.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>> Okay I would work on that and update the progress on the mail
> >>>>> thread
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>> 5) It seems you missed the bonding period so you’ll need to
> >> catch
> >>>>>>>> up on
> >>>>>>>>>>>>> that. Could you pick some jira issue and work on them
> quickly?
> >>>>> This
> >>>>>>>> is
> >>>>>>>>>> to
> >>>>>>>>>>>>> make sure you use the proper xwiki communication channels,
> >> learn
> >>>>> to
> >>>>>>>>>> work
> >>>>>>>>>>>>> with the xwiki team and start understanding the xwiki
> >>>>> architecture,
> >>>>>>>>>> and how
> >>>>>>>>>>>>> extensions are developed for XWiki. Since you’re a bit late,
> >>>>> you’ll
> >>>>>>>>>> need to
> >>>>>>>>>>>>> catch up and achieve this very quickly now and be present
> full
> >>>>> time
> >>>>>>>> on
> >>>>>>>>>> the
> >>>>>>>>>>>>> xwiki chat :)
> >>>>>>>>>>>>>
> >>>>>>>>>>>> I would start doing that
> >>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> 6) Idea: please review the issues on the XWiki Docker
> project’s
> >>>>>>>> jira.
> >>>>>>>>>>>>> Maybe you can help with those too as part of
> >> onboarding/bonding.
> >>>>> It
> >>>>>>>>>> should
> >>>>>>>>>>>>> also give you additional use cases for the XWiki Helm Charts
> >> IMO.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> Thanks! I’m eager to follow your work and see xwiki helm
> charts
> >>>>> in
> >>>>>>>>>> action.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> -Vincent
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> Thank You
> >>>>>>>>>>>>>> Ashish Sharma
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> Thank You
> >>>>>>>>>>>> Ashish
> >>>>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>
> >>>>>
> >>
>
Reply | Threaded
Open this post in threaded view
|

Re: Xwiki Helm HA

Shubham Jain
That’s interesting.

Were you able to figure out the problems when you scale up from 1 Pod? If not, share the logs. Let’s try to figure out problem.

Best,
Shubham.

> On 20-Jul-2019, at 6:36 AM, Ashish Sharma <[hidden email]> wrote:
>
> Hey Shubham,
>
> I have tried initialising with 1 pod and scaling it up to multiple pods,
> also tried session affinity, and some other cases but none of them worked.
>
> Thanks,
> Ashish
> On Fri, 19 Jul 2019 at 4:51 PM, Shubham Jain <[hidden email]>
> wrote:
>
>> Hey Ashish,
>>
>> I don’t understand XWiki well enough to fully understand the implications
>> of running a clustered setup by sharing file system.
>>
>> Can you try keeping the deployment size initially 1, and then later try to
>> increase the size? Does that also cause problems?
>>
>> Best,
>> Shubham.
>>
>>> On 19-Jul-2019, at 4:03 PM, Ashish Sharma <[hidden email]>
>> wrote:
>>>
>>> Hey Shubham,
>>>
>>> I have moved the solr folder as exposed by the docker container to the
>> file
>>> system. I think in-memory cache is creating problem.
>>>
>>> I started 3 container of xwiki with shared file storage, but all three
>> were
>>> trying to initiate the postgress database, that  is causing problem.
>>>
>>> I want to know few things about xwiki:-
>>>
>>> 1. Do we use java threads ? -> if we do then filesystem locking should
>> have
>>> been implemented for that?
>>>
>>> 2. Other stateful things in our application like caching or other
>>> filestorage that should be externalise?
>>>
>>>
>>> Thanks,
>>> Ashish
>>>
>>> On Fri, 19 Jul 2019 at 3:59 AM, Shubham Jain <[hidden email]>
>>> wrote:
>>>
>>>> Hey Ashish,
>>>>
>>>> I think for now you could stick to deployment. Have you been ever to
>>>> discover the data you need to move to the shared file system? Is the
>>>> in-memory caching mechanism causing problems?
>>>>
>>>> @Thomas/Vincent - Can you please help us understand the implications of
>>>> running XWiki in a clustered mode by sharing data volumes?
>>>>
>>>> Best,
>>>> Shubham.
>>>>
>>>>
>>>> Best,
>>>> Shubham?
>>>>
>>>>> On 19-Jul-2019, at 5:35 AM, Ashish Sharma <[hidden email]>
>>>> wrote:
>>>>>
>>>>> Resent the email at the mail thread.
>>>>>
>>>>> On Fri, Jul 19, 2019 at 2:01 AM Ashish Sharma <
>>>> [hidden email]>
>>>>> wrote:
>>>>>
>>>>>> Hello everyone,
>>>>>>
>>>>>> I am trying to implement xwiki helm chart in High availability but
>>>> facing
>>>>>> some problems.
>>>>>>
>>>>>> How we can implement High Availability(HA) in Kubernetes:-
>>>>>>
>>>>>> 1. Through Deployment:-  In this approach we would use a shared file
>>>>>> storage, that would get attached to multiple pods(or containers)
>>>>>> 2. Through StatefulSet:- In this approach every pod(or container)
>> would
>>>>>> have it’s own different volume and state, which should be synchronised
>>>> by
>>>>>> an consensus algorithm
>>>>>>
>>>>>> Problem faced in xwiki case:-
>>>>>>
>>>>>> 1. In deployment case we need a locking mechanism for our filesystem,
>>>>>> moreover caching and states should also be maintained outside the
>>>> container.
>>>>>> 2. In StatefulSet the consensus algorithm is not present so it is
>>>>>> currently not possible.
>>>>>>
>>>>>> Could anyone help me through it.
>>>>>>
>>>>>> Thanks,
>>>>>> Ashish
>>>>>>
>>>>>> On Wed, 10 Jul 2019 at 10:32 AM, Shubham Jain <
>> [hidden email]>
>>>>>> wrote:
>>>>>>
>>>>>>> Hey Ashish,
>>>>>>>
>>>>>>> I think Vincent/Thomas can help with the PR on the xwiki-docker
>>>> project.
>>>>>>>
>>>>>>> Regarding adding docs for High-Availability setup using shared data
>>>>>>> volumes, I think you can use could create a docs folder in the GitHub
>>>> repo,
>>>>>>> and add the docs in Markdown format. I think you can also document it
>>>> in
>>>>>>> XWiki blog <https://www.xwiki.org/xwiki/bin/view/Blog/>. The chart
>>>>>>> should support enabling HA, and then passing a storage class which
>>>> supports
>>>>>>> shareable volumes to Pods, and then you can document examples for
>> using
>>>>>>> this with something like Rook+GKE setup.
>>>>>>>
>>>>>>> Best,
>>>>>>> Shubham.
>>>>>>>
>>>>>>>
>>>>>>>> On 08-Jul-2019, at 1:57 PM, Ashish Sharma <
>> [hidden email]
>>>>>
>>>>>>> wrote:
>>>>>>>>
>>>>>>>> Hello everyone,
>>>>>>>>
>>>>>>>> I require the xwiki solr docker container to do progress with my
>>>> work. I
>>>>>>>> have made a pull request
>>>>>>>> <https://github.com/xwiki-contrib/docker-xwiki/pull/22>, for the
>>>> same.
>>>>>>>> Please look into it.
>>>>>>>>
>>>>>>>> Moreover, I was working with Rook(Ceph Operator) for a shared file
>>>>>>> system
>>>>>>>> which would be a better approach to deploy our app on production.
>> So I
>>>>>>>> think we should document it somewhere. Can anyone help me where
>> should
>>>>>>> I do
>>>>>>>> that. It may contain the details to configure rook-ceph, and then
>>>>>>>> connecting it.
>>>>>>>>
>>>>>>>> P.S. I am writing a blog at Medium on how to configure rook on
>> Google
>>>>>>>> Kubernetes Engine. We could also use that?
>>>>>>>>
>>>>>>>> Thanks,
>>>>>>>> Ashish
>>>>>>>>
>>>>>>>> On Mon, Jun 24, 2019 at 8:31 PM Ashish Sharma <
>>>>>>> [hidden email]>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> Hi Shubham,
>>>>>>>>>
>>>>>>>>> Thank you Shubham, I will start working on adding test cases.
>>>>>>>>>
>>>>>>>>> Thanks,
>>>>>>>>> Ashish
>>>>>>>>>
>>>>>>>>> On Mon, 24 Jun 2019 at 3:32 PM, Shubham Jain <
>>>> [hidden email]>
>>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>>> Hey Ashish,
>>>>>>>>>>
>>>>>>>>>> Thanks for the update.
>>>>>>>>>>
>>>>>>>>>> Adding our chart to incubator section of helm community charts
>>>>>>> repository
>>>>>>>>>> sounds great. Although (unlike Dockerhub) this repo is for
>> community
>>>>>>> charts
>>>>>>>>>> and not really official charts of the respective installations,
>> but
>>>>>>>>>> supporting this chart in the community chart repo should help with
>>>> the
>>>>>>>>>> visibility.
>>>>>>>>>>
>>>>>>>>>> Lets also work on adding tests to the helm chart to ensure things
>>>> are
>>>>>>>>>> going fine.
>>>>>>>>>>
>>>>>>>>>> Best,
>>>>>>>>>> Shubham.
>>>>>>>>>>
>>>>>>>>>>> On 21-Jun-2019, at 4:37 AM, Ashish Sharma <
>>>>>>> [hidden email]>
>>>>>>>>>> wrote:
>>>>>>>>>>>
>>>>>>>>>>> Hi everyone,
>>>>>>>>>>>
>>>>>>>>>>> Update on HELM project:-
>>>>>>>>>>>
>>>>>>>>>>> We have added some features to the chart that includes:-
>>>>>>>>>>> * Making chart configurable with both mysql and postgres database
>>>>>>>>>>> * We are now using configmaps and secrets for storing database
>>>>>>>>>> credentials
>>>>>>>>>>> * We use helm dependency to manage the dependencies of the chart
>>>>>>>>>>>
>>>>>>>>>>> I think our chart is ready to be deployed in the incubator
>> section
>>>> of
>>>>>>>>>> helm
>>>>>>>>>>> community repo. <https://github.com/helm/charts> If you guys
>>>> agree I
>>>>>>>>>> would
>>>>>>>>>>> make the necessary changes required to deploy it there.
>>>>>>>>>>>
>>>>>>>>>>> Thanks
>>>>>>>>>>> - Ashish
>>>>>>>>>>>
>>>>>>>>>>> On Wed, Jun 5, 2019 at 7:40 PM Vincent Massol <
>> [hidden email]>
>>>>>>>>>> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> Hi Ashish,
>>>>>>>>>>>>
>>>>>>>>>>>>> On 5 Jun 2019, at 15:07, Ashish Sharma <
>>>> [hidden email]
>>>>>>>>
>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>> Hi everyone,
>>>>>>>>>>>>>
>>>>>>>>>>>>> The first week update of helm chart xwiki project. We currently
>>>>>>> have a
>>>>>>>>>>>>> chart that would deploy xwiki mysql stateful service with a
>>>>>>> persistent
>>>>>>>>>>>>> volume. The link to the repo:-
>>>>>>>>>>>>>
>>>>>>>>>>>>> https://github.com/xwiki-contrib/xwiki-helm
>>>>>>>>>>>>
>>>>>>>>>>>> Thanks for the update.
>>>>>>>>>>>>
>>>>>>>>>>>> Would be great if you could update the README file to explain
>> who
>>>> to
>>>>>>>>>> use
>>>>>>>>>>>> it.
>>>>>>>>>>>>
>>>>>>>>>>>> Thanks
>>>>>>>>>>>> -Vincent
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> Thank You
>>>>>>>>>>>>> Ashish
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Thu, 23 May 2019 at 7:27 PM, Ashish Sharma <
>>>>>>>>>>>> [hidden email]>
>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On Wed, May 22, 2019 at 12:42 PM Vincent Massol <
>>>>>>> [hidden email]>
>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Hi Ashish,
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> On 21 May 2019, at 19:37, Ashish Sharma <
>>>>>>>>>> [hidden email]>
>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Hi everyone,
>>>>>>>>>>>>>>>> Thanks for having me here
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> About Me
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> I am Ashish Sharma, selected as a student for Google Summer
>> of
>>>>>>>>>> Code. I
>>>>>>>>>>>>>>> am
>>>>>>>>>>>>>>>> final year student enrolled in Guru Gobind Singh
>> Indraprastha
>>>>>>>>>>>>>>> University,
>>>>>>>>>>>>>>>> Delhi. I am a resident of India.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Profiles
>>>>>>>>>>>>>>>> GitHub -  https://github.com/ashish932/xwiki-helm-chart/
>>>>>>>>>>>>>>>> LinkedIn - https://www.linkedin.com/in/ashish932/
>>>>>>>>>>>>>>>> Riot - @ashish932:matrix.org
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> I will be presenting my project "Helm Chart for XWiki" to
>> all
>>>> of
>>>>>>>>>> you.
>>>>>>>>>>>>>>>> Following
>>>>>>>>>>>>>>>> are the relevant details.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Helm Chart for XWiki
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Mentors: Shubham Jain, Neha Gupta
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Technologies: Kubernetes, Docker, other if required
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Overview
>>>>>>>>>>>>>>>> The proposed project is a helm chart that would deploy xwiki
>>>> as
>>>>>>>>>> highly
>>>>>>>>>>>>>>>> available and reliable. It should be configurable with
>>>> different
>>>>>>>>>>>>>>>> databases(either a standalone database or a clustered one)
>>>> that
>>>>>>> are
>>>>>>>>>>>>>>>> configurable with xwiki. It would give the option to either
>>>>>>>>>> configure
>>>>>>>>>>>>>>> solr
>>>>>>>>>>>>>>>> externally (standalone or clustered) or managed within the
>>>>>>>>>> container.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> It also needs to be deployed on various Servlet Containers
>> and
>>>>>>> with
>>>>>>>>>> a
>>>>>>>>>>>>>>> LibreOffice server configured.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> In short, it should support all the configurations supported
>> by
>>>>>>> the
>>>>>>>>>>>>>>> official XWiki Docker images :)
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> It
>>>>>>>>>>>>>>>> should deploy the app on a shared file system like a rook.
>> It
>>>>>>>>>> should
>>>>>>>>>>>>>>>> support Istio virtual services, istio matrix, and istio
>>>>>>> distributed
>>>>>>>>>>>>>>> tracing
>>>>>>>>>>>>>>>> and should be a secured system with RBAC and security
>>>> credential
>>>>>>>>>>>>>>> rotation.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Ok I have no knowledge about this… The shared file system
>>>> sounds
>>>>>>>>>> good
>>>>>>>>>>>> to
>>>>>>>>>>>>>>> support clustering XWiki for the permanent directory sharing.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> The chart should be easily deployed on GKE and amazon EKS.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> On locally on minikube for testing I guess?
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Yes we can test using minikube one
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Features
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> ->  Support for different Databases
>>>>>>>>>>>>>>>> ->  Choice between using an external database, a single node
>>>> DB
>>>>>>> or
>>>>>>>>>> a
>>>>>>>>>>>>>>>> multi-cluster DB setup
>>>>>>>>>>>>>>>> -> Support for shared file system
>>>>>>>>>>>>>>>> -> Support for istio and it's services
>>>>>>>>>>>>>>>> -> RBAC, SSL and other security methods
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> If you have any features in mind that should be added please
>>>>>>> feel
>>>>>>>>>> free
>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>> reply to this mail.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Some Design Questions?
>>>>>>>>>>>>>>>> -> Which Databases should be supported?
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> See
>>>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>
>>>>
>> https://dev.xwiki.org/xwiki/bin/view/Community/SupportStrategy/DatabaseSupportStrategy
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> More generally see the XWiki Docker image documentation and
>>>>>>>>>>>>>>>
>> https://dev.xwiki.org/xwiki/bin/view/Community/SupportStrategy
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> -> As we have to detach solr out of the docker container(run
>>>> it
>>>>>>> in
>>>>>>>>>> an
>>>>>>>>>>>>>>>> independent container) would be there a requirement for a
>> code
>>>>>>>>>> change,
>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>> we should approach it?
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> See the doc for Solr which explains how to configure an
>>>> external
>>>>>>>>>> SOLR.
>>>>>>>>>>>>>>> There are some XWiki code/config to deploy in the external
>> SOLR
>>>>>>>>>> AFAIR.
>>>>>>>>>>>>>>> comunity controls chart
>>>>>>>>>>>>>>>> -> Apart from solr is there any other stateful service that
>>>>>>> could
>>>>>>>>>> or
>>>>>>>>>>>>>>> should
>>>>>>>>>>>>>>>> be detached from the docker container?
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Which docker container are you referring to? The Servlet
>>>>>>> Container
>>>>>>>>>> one?
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> xwiki:lts-mysql-tomcat
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Here is my current repository which deploys XWiki for MySQL
>>>>>>>>>> database
>>>>>>>>>>>>>>> using
>>>>>>>>>>>>>>>> official XWiki docker container:-
>>>>>>>>>>>>>>>> https://github.com/ashish932/xwiki-helm-chart/
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Will need to have a look. Some additional questions:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> 1) Is there an official place for publishing helm charts,
>>>>>>> similar to
>>>>>>>>>>>>>>> dockerhub for docker images? How do you provide the helm
>> charts
>>>>>>> to
>>>>>>>>>>>> XWiki
>>>>>>>>>>>>>>> users in term of packaging? In other words, what will we/you
>>>>>>>>>> document
>>>>>>>>>>>> on
>>>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>
>>>>
>> https://www.xwiki.org/xwiki/bin/view/Documentation/AdminGuide/Installation/
>>>>>>>>>>>>>>> ?
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Yes, There is a centralized community control chart
>> management
>>>>>>>>>>>>>> repository. Helm charts are hosted there. Organizations like
>>>>>>> mysql,
>>>>>>>>>>>>>> wordpress uses it.
>>>>>>>>>>>>>> https://github.com/helm/charts
>>>>>>>>>>>>>> But in addition to that, we can build our own charts
>> repository
>>>>>>>>>> locally,
>>>>>>>>>>>>>> and host it. Similar to docker run command, we have helm
>> update
>>>>>>>>>> command
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> 2) How do you plan to have smoke tests for the XWiki Helm
>>>>>>> charts, as
>>>>>>>>>>>> part
>>>>>>>>>>>>>>> of the build, to verify that they work fine? How can we
>> execute
>>>>>>>>>> XWik’s
>>>>>>>>>>>>>>> functional tests based on TestContainers using Helm Charts?
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> There are some tools available
>>>>>>>>>>>>>> chart-testing(started by helm) -98 commits
>>>>>>>>>>>>>> terratest - 1039 commits (It looks a good tool to use)
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> 3) Do the helm charts for XWiki need to be built? If so, can
>> it
>>>>>>> be
>>>>>>>>>> done
>>>>>>>>>>>>>>> with Maven? Since we’ll be offering several I guess, how do
>> you
>>>>>>>>>> avoid
>>>>>>>>>>>> code
>>>>>>>>>>>>>>> duplication and make maintenance of them as simple as
>> possible?
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> No, the charts don’t need to be built externally, though it
>> gets
>>>>>>>>>>>>>> compressed when pushed f we build to chart repository(this is
>> an
>>>>>>>>>>>> automatic
>>>>>>>>>>>>>> step in case of official chart repo and we can automate using
>> ci
>>>>>>> our
>>>>>>>>>> own
>>>>>>>>>>>>>> chart repo)
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> 4) Could you provide a roadmap with various milestones and
>>>> dates?
>>>>>>> See
>>>>>>>>>>>> the
>>>>>>>>>>>>>>> last roadmap email I sent on the devs list and please reply
>> to
>>>>>>> it.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Okay I would work on that and update the progress on the mail
>>>>>>> thread
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> 5) It seems you missed the bonding period so you’ll need to
>>>> catch
>>>>>>>>>> up on
>>>>>>>>>>>>>>> that. Could you pick some jira issue and work on them
>> quickly?
>>>>>>> This
>>>>>>>>>> is
>>>>>>>>>>>> to
>>>>>>>>>>>>>>> make sure you use the proper xwiki communication channels,
>>>> learn
>>>>>>> to
>>>>>>>>>>>> work
>>>>>>>>>>>>>>> with the xwiki team and start understanding the xwiki
>>>>>>> architecture,
>>>>>>>>>>>> and how
>>>>>>>>>>>>>>> extensions are developed for XWiki. Since you’re a bit late,
>>>>>>> you’ll
>>>>>>>>>>>> need to
>>>>>>>>>>>>>>> catch up and achieve this very quickly now and be present
>> full
>>>>>>> time
>>>>>>>>>> on
>>>>>>>>>>>> the
>>>>>>>>>>>>>>> xwiki chat :)
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>> I would start doing that
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> 6) Idea: please review the issues on the XWiki Docker
>> project’s
>>>>>>>>>> jira.
>>>>>>>>>>>>>>> Maybe you can help with those too as part of
>>>> onboarding/bonding.
>>>>>>> It
>>>>>>>>>>>> should
>>>>>>>>>>>>>>> also give you additional use cases for the XWiki Helm Charts
>>>> IMO.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Thanks! I’m eager to follow your work and see xwiki helm
>> charts
>>>>>>> in
>>>>>>>>>>>> action.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> -Vincent
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Thank You
>>>>>>>>>>>>>>>> Ashish Sharma
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Thank You
>>>>>>>>>>>>>> Ashish
>>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>
>>>>>>>
>>>>
>>
Reply | Threaded
Open this post in threaded view
|

Re: Xwiki Helm HA

Ashish Sharma
Hey Shubham,

Yes I did that and it did not work, once I scale it by starting it from 1.
The website no longer loads.

On Sat, 20 Jul 2019 at 10:00 AM, Shubham Jain <[hidden email]>
wrote:

> That’s interesting.
>
> Were you able to figure out the problems when you scale up from 1 Pod? If
> not, share the logs. Let’s try to figure out problem.
>
> Best,
> Shubham.
>
> > On 20-Jul-2019, at 6:36 AM, Ashish Sharma <[hidden email]>
> wrote:
> >
> > Hey Shubham,
> >
> > I have tried initialising with 1 pod and scaling it up to multiple pods,
> > also tried session affinity, and some other cases but none of them
> worked.
> >
> > Thanks,
> > Ashish
> > On Fri, 19 Jul 2019 at 4:51 PM, Shubham Jain <[hidden email]>
> > wrote:
> >
> >> Hey Ashish,
> >>
> >> I don’t understand XWiki well enough to fully understand the
> implications
> >> of running a clustered setup by sharing file system.
> >>
> >> Can you try keeping the deployment size initially 1, and then later try
> to
> >> increase the size? Does that also cause problems?
> >>
> >> Best,
> >> Shubham.
> >>
> >>> On 19-Jul-2019, at 4:03 PM, Ashish Sharma <[hidden email]>
> >> wrote:
> >>>
> >>> Hey Shubham,
> >>>
> >>> I have moved the solr folder as exposed by the docker container to the
> >> file
> >>> system. I think in-memory cache is creating problem.
> >>>
> >>> I started 3 container of xwiki with shared file storage, but all three
> >> were
> >>> trying to initiate the postgress database, that  is causing problem.
> >>>
> >>> I want to know few things about xwiki:-
> >>>
> >>> 1. Do we use java threads ? -> if we do then filesystem locking should
> >> have
> >>> been implemented for that?
> >>>
> >>> 2. Other stateful things in our application like caching or other
> >>> filestorage that should be externalise?
> >>>
> >>>
> >>> Thanks,
> >>> Ashish
> >>>
> >>> On Fri, 19 Jul 2019 at 3:59 AM, Shubham Jain <[hidden email]>
> >>> wrote:
> >>>
> >>>> Hey Ashish,
> >>>>
> >>>> I think for now you could stick to deployment. Have you been ever to
> >>>> discover the data you need to move to the shared file system? Is the
> >>>> in-memory caching mechanism causing problems?
> >>>>
> >>>> @Thomas/Vincent - Can you please help us understand the implications
> of
> >>>> running XWiki in a clustered mode by sharing data volumes?
> >>>>
> >>>> Best,
> >>>> Shubham.
> >>>>
> >>>>
> >>>> Best,
> >>>> Shubham?
> >>>>
> >>>>> On 19-Jul-2019, at 5:35 AM, Ashish Sharma <
> [hidden email]>
> >>>> wrote:
> >>>>>
> >>>>> Resent the email at the mail thread.
> >>>>>
> >>>>> On Fri, Jul 19, 2019 at 2:01 AM Ashish Sharma <
> >>>> [hidden email]>
> >>>>> wrote:
> >>>>>
> >>>>>> Hello everyone,
> >>>>>>
> >>>>>> I am trying to implement xwiki helm chart in High availability but
> >>>> facing
> >>>>>> some problems.
> >>>>>>
> >>>>>> How we can implement High Availability(HA) in Kubernetes:-
> >>>>>>
> >>>>>> 1. Through Deployment:-  In this approach we would use a shared file
> >>>>>> storage, that would get attached to multiple pods(or containers)
> >>>>>> 2. Through StatefulSet:- In this approach every pod(or container)
> >> would
> >>>>>> have it’s own different volume and state, which should be
> synchronised
> >>>> by
> >>>>>> an consensus algorithm
> >>>>>>
> >>>>>> Problem faced in xwiki case:-
> >>>>>>
> >>>>>> 1. In deployment case we need a locking mechanism for our
> filesystem,
> >>>>>> moreover caching and states should also be maintained outside the
> >>>> container.
> >>>>>> 2. In StatefulSet the consensus algorithm is not present so it is
> >>>>>> currently not possible.
> >>>>>>
> >>>>>> Could anyone help me through it.
> >>>>>>
> >>>>>> Thanks,
> >>>>>> Ashish
> >>>>>>
> >>>>>> On Wed, 10 Jul 2019 at 10:32 AM, Shubham Jain <
> >> [hidden email]>
> >>>>>> wrote:
> >>>>>>
> >>>>>>> Hey Ashish,
> >>>>>>>
> >>>>>>> I think Vincent/Thomas can help with the PR on the xwiki-docker
> >>>> project.
> >>>>>>>
> >>>>>>> Regarding adding docs for High-Availability setup using shared data
> >>>>>>> volumes, I think you can use could create a docs folder in the
> GitHub
> >>>> repo,
> >>>>>>> and add the docs in Markdown format. I think you can also document
> it
> >>>> in
> >>>>>>> XWiki blog <https://www.xwiki.org/xwiki/bin/view/Blog/>. The chart
> >>>>>>> should support enabling HA, and then passing a storage class which
> >>>> supports
> >>>>>>> shareable volumes to Pods, and then you can document examples for
> >> using
> >>>>>>> this with something like Rook+GKE setup.
> >>>>>>>
> >>>>>>> Best,
> >>>>>>> Shubham.
> >>>>>>>
> >>>>>>>
> >>>>>>>> On 08-Jul-2019, at 1:57 PM, Ashish Sharma <
> >> [hidden email]
> >>>>>
> >>>>>>> wrote:
> >>>>>>>>
> >>>>>>>> Hello everyone,
> >>>>>>>>
> >>>>>>>> I require the xwiki solr docker container to do progress with my
> >>>> work. I
> >>>>>>>> have made a pull request
> >>>>>>>> <https://github.com/xwiki-contrib/docker-xwiki/pull/22>, for the
> >>>> same.
> >>>>>>>> Please look into it.
> >>>>>>>>
> >>>>>>>> Moreover, I was working with Rook(Ceph Operator) for a shared file
> >>>>>>> system
> >>>>>>>> which would be a better approach to deploy our app on production.
> >> So I
> >>>>>>>> think we should document it somewhere. Can anyone help me where
> >> should
> >>>>>>> I do
> >>>>>>>> that. It may contain the details to configure rook-ceph, and then
> >>>>>>>> connecting it.
> >>>>>>>>
> >>>>>>>> P.S. I am writing a blog at Medium on how to configure rook on
> >> Google
> >>>>>>>> Kubernetes Engine. We could also use that?
> >>>>>>>>
> >>>>>>>> Thanks,
> >>>>>>>> Ashish
> >>>>>>>>
> >>>>>>>> On Mon, Jun 24, 2019 at 8:31 PM Ashish Sharma <
> >>>>>>> [hidden email]>
> >>>>>>>> wrote:
> >>>>>>>>
> >>>>>>>>> Hi Shubham,
> >>>>>>>>>
> >>>>>>>>> Thank you Shubham, I will start working on adding test cases.
> >>>>>>>>>
> >>>>>>>>> Thanks,
> >>>>>>>>> Ashish
> >>>>>>>>>
> >>>>>>>>> On Mon, 24 Jun 2019 at 3:32 PM, Shubham Jain <
> >>>> [hidden email]>
> >>>>>>>>> wrote:
> >>>>>>>>>
> >>>>>>>>>> Hey Ashish,
> >>>>>>>>>>
> >>>>>>>>>> Thanks for the update.
> >>>>>>>>>>
> >>>>>>>>>> Adding our chart to incubator section of helm community charts
> >>>>>>> repository
> >>>>>>>>>> sounds great. Although (unlike Dockerhub) this repo is for
> >> community
> >>>>>>> charts
> >>>>>>>>>> and not really official charts of the respective installations,
> >> but
> >>>>>>>>>> supporting this chart in the community chart repo should help
> with
> >>>> the
> >>>>>>>>>> visibility.
> >>>>>>>>>>
> >>>>>>>>>> Lets also work on adding tests to the helm chart to ensure
> things
> >>>> are
> >>>>>>>>>> going fine.
> >>>>>>>>>>
> >>>>>>>>>> Best,
> >>>>>>>>>> Shubham.
> >>>>>>>>>>
> >>>>>>>>>>> On 21-Jun-2019, at 4:37 AM, Ashish Sharma <
> >>>>>>> [hidden email]>
> >>>>>>>>>> wrote:
> >>>>>>>>>>>
> >>>>>>>>>>> Hi everyone,
> >>>>>>>>>>>
> >>>>>>>>>>> Update on HELM project:-
> >>>>>>>>>>>
> >>>>>>>>>>> We have added some features to the chart that includes:-
> >>>>>>>>>>> * Making chart configurable with both mysql and postgres
> database
> >>>>>>>>>>> * We are now using configmaps and secrets for storing database
> >>>>>>>>>> credentials
> >>>>>>>>>>> * We use helm dependency to manage the dependencies of the
> chart
> >>>>>>>>>>>
> >>>>>>>>>>> I think our chart is ready to be deployed in the incubator
> >> section
> >>>> of
> >>>>>>>>>> helm
> >>>>>>>>>>> community repo. <https://github.com/helm/charts> If you guys
> >>>> agree I
> >>>>>>>>>> would
> >>>>>>>>>>> make the necessary changes required to deploy it there.
> >>>>>>>>>>>
> >>>>>>>>>>> Thanks
> >>>>>>>>>>> - Ashish
> >>>>>>>>>>>
> >>>>>>>>>>> On Wed, Jun 5, 2019 at 7:40 PM Vincent Massol <
> >> [hidden email]>
> >>>>>>>>>> wrote:
> >>>>>>>>>>>
> >>>>>>>>>>>> Hi Ashish,
> >>>>>>>>>>>>
> >>>>>>>>>>>>> On 5 Jun 2019, at 15:07, Ashish Sharma <
> >>>> [hidden email]
> >>>>>>>>
> >>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> Hi everyone,
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> The first week update of helm chart xwiki project. We
> currently
> >>>>>>> have a
> >>>>>>>>>>>>> chart that would deploy xwiki mysql stateful service with a
> >>>>>>> persistent
> >>>>>>>>>>>>> volume. The link to the repo:-
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> https://github.com/xwiki-contrib/xwiki-helm
> >>>>>>>>>>>>
> >>>>>>>>>>>> Thanks for the update.
> >>>>>>>>>>>>
> >>>>>>>>>>>> Would be great if you could update the README file to explain
> >> who
> >>>> to
> >>>>>>>>>> use
> >>>>>>>>>>>> it.
> >>>>>>>>>>>>
> >>>>>>>>>>>> Thanks
> >>>>>>>>>>>> -Vincent
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> Thank You
> >>>>>>>>>>>>> Ashish
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> On Thu, 23 May 2019 at 7:27 PM, Ashish Sharma <
> >>>>>>>>>>>> [hidden email]>
> >>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> On Wed, May 22, 2019 at 12:42 PM Vincent Massol <
> >>>>>>> [hidden email]>
> >>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> Hi Ashish,
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> On 21 May 2019, at 19:37, Ashish Sharma <
> >>>>>>>>>> [hidden email]>
> >>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> Hi everyone,
> >>>>>>>>>>>>>>>> Thanks for having me here
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> About Me
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> I am Ashish Sharma, selected as a student for Google
> Summer
> >> of
> >>>>>>>>>> Code. I
> >>>>>>>>>>>>>>> am
> >>>>>>>>>>>>>>>> final year student enrolled in Guru Gobind Singh
> >> Indraprastha
> >>>>>>>>>>>>>>> University,
> >>>>>>>>>>>>>>>> Delhi. I am a resident of India.
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> Profiles
> >>>>>>>>>>>>>>>> GitHub -  https://github.com/ashish932/xwiki-helm-chart/
> >>>>>>>>>>>>>>>> LinkedIn - https://www.linkedin.com/in/ashish932/
> >>>>>>>>>>>>>>>> Riot - @ashish932:matrix.org
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> I will be presenting my project "Helm Chart for XWiki" to
> >> all
> >>>> of
> >>>>>>>>>> you.
> >>>>>>>>>>>>>>>> Following
> >>>>>>>>>>>>>>>> are the relevant details.
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> Helm Chart for XWiki
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> Mentors: Shubham Jain, Neha Gupta
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> Technologies: Kubernetes, Docker, other if required
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> Overview
> >>>>>>>>>>>>>>>> The proposed project is a helm chart that would deploy
> xwiki
> >>>> as
> >>>>>>>>>> highly
> >>>>>>>>>>>>>>>> available and reliable. It should be configurable with
> >>>> different
> >>>>>>>>>>>>>>>> databases(either a standalone database or a clustered one)
> >>>> that
> >>>>>>> are
> >>>>>>>>>>>>>>>> configurable with xwiki. It would give the option to
> either
> >>>>>>>>>> configure
> >>>>>>>>>>>>>>> solr
> >>>>>>>>>>>>>>>> externally (standalone or clustered) or managed within the
> >>>>>>>>>> container.
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> It also needs to be deployed on various Servlet Containers
> >> and
> >>>>>>> with
> >>>>>>>>>> a
> >>>>>>>>>>>>>>> LibreOffice server configured.
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> In short, it should support all the configurations
> supported
> >> by
> >>>>>>> the
> >>>>>>>>>>>>>>> official XWiki Docker images :)
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> It
> >>>>>>>>>>>>>>>> should deploy the app on a shared file system like a rook.
> >> It
> >>>>>>>>>> should
> >>>>>>>>>>>>>>>> support Istio virtual services, istio matrix, and istio
> >>>>>>> distributed
> >>>>>>>>>>>>>>> tracing
> >>>>>>>>>>>>>>>> and should be a secured system with RBAC and security
> >>>> credential
> >>>>>>>>>>>>>>> rotation.
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> Ok I have no knowledge about this… The shared file system
> >>>> sounds
> >>>>>>>>>> good
> >>>>>>>>>>>> to
> >>>>>>>>>>>>>>> support clustering XWiki for the permanent directory
> sharing.
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> The chart should be easily deployed on GKE and amazon EKS.
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> On locally on minikube for testing I guess?
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> Yes we can test using minikube one
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> Features
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> ->  Support for different Databases
> >>>>>>>>>>>>>>>> ->  Choice between using an external database, a single
> node
> >>>> DB
> >>>>>>> or
> >>>>>>>>>> a
> >>>>>>>>>>>>>>>> multi-cluster DB setup
> >>>>>>>>>>>>>>>> -> Support for shared file system
> >>>>>>>>>>>>>>>> -> Support for istio and it's services
> >>>>>>>>>>>>>>>> -> RBAC, SSL and other security methods
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> If you have any features in mind that should be added
> please
> >>>>>>> feel
> >>>>>>>>>> free
> >>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>> reply to this mail.
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> Some Design Questions?
> >>>>>>>>>>>>>>>> -> Which Databases should be supported?
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> See
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>
> >>>>
> >>
> https://dev.xwiki.org/xwiki/bin/view/Community/SupportStrategy/DatabaseSupportStrategy
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> More generally see the XWiki Docker image documentation and
> >>>>>>>>>>>>>>>
> >> https://dev.xwiki.org/xwiki/bin/view/Community/SupportStrategy
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> -> As we have to detach solr out of the docker
> container(run
> >>>> it
> >>>>>>> in
> >>>>>>>>>> an
> >>>>>>>>>>>>>>>> independent container) would be there a requirement for a
> >> code
> >>>>>>>>>> change,
> >>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>> we should approach it?
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> See the doc for Solr which explains how to configure an
> >>>> external
> >>>>>>>>>> SOLR.
> >>>>>>>>>>>>>>> There are some XWiki code/config to deploy in the external
> >> SOLR
> >>>>>>>>>> AFAIR.
> >>>>>>>>>>>>>>> comunity controls chart
> >>>>>>>>>>>>>>>> -> Apart from solr is there any other stateful service
> that
> >>>>>>> could
> >>>>>>>>>> or
> >>>>>>>>>>>>>>> should
> >>>>>>>>>>>>>>>> be detached from the docker container?
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> Which docker container are you referring to? The Servlet
> >>>>>>> Container
> >>>>>>>>>> one?
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> xwiki:lts-mysql-tomcat
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> Here is my current repository which deploys XWiki for
> MySQL
> >>>>>>>>>> database
> >>>>>>>>>>>>>>> using
> >>>>>>>>>>>>>>>> official XWiki docker container:-
> >>>>>>>>>>>>>>>> https://github.com/ashish932/xwiki-helm-chart/
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> Will need to have a look. Some additional questions:
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> 1) Is there an official place for publishing helm charts,
> >>>>>>> similar to
> >>>>>>>>>>>>>>> dockerhub for docker images? How do you provide the helm
> >> charts
> >>>>>>> to
> >>>>>>>>>>>> XWiki
> >>>>>>>>>>>>>>> users in term of packaging? In other words, what will
> we/you
> >>>>>>>>>> document
> >>>>>>>>>>>> on
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>
> >>>>
> >>
> https://www.xwiki.org/xwiki/bin/view/Documentation/AdminGuide/Installation/
> >>>>>>>>>>>>>>> ?
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> Yes, There is a centralized community control chart
> >> management
> >>>>>>>>>>>>>> repository. Helm charts are hosted there. Organizations like
> >>>>>>> mysql,
> >>>>>>>>>>>>>> wordpress uses it.
> >>>>>>>>>>>>>> https://github.com/helm/charts
> >>>>>>>>>>>>>> But in addition to that, we can build our own charts
> >> repository
> >>>>>>>>>> locally,
> >>>>>>>>>>>>>> and host it. Similar to docker run command, we have helm
> >> update
> >>>>>>>>>> command
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> 2) How do you plan to have smoke tests for the XWiki Helm
> >>>>>>> charts, as
> >>>>>>>>>>>> part
> >>>>>>>>>>>>>>> of the build, to verify that they work fine? How can we
> >> execute
> >>>>>>>>>> XWik’s
> >>>>>>>>>>>>>>> functional tests based on TestContainers using Helm Charts?
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> There are some tools available
> >>>>>>>>>>>>>> chart-testing(started by helm) -98 commits
> >>>>>>>>>>>>>> terratest - 1039 commits (It looks a good tool to use)
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> 3) Do the helm charts for XWiki need to be built? If so,
> can
> >> it
> >>>>>>> be
> >>>>>>>>>> done
> >>>>>>>>>>>>>>> with Maven? Since we’ll be offering several I guess, how do
> >> you
> >>>>>>>>>> avoid
> >>>>>>>>>>>> code
> >>>>>>>>>>>>>>> duplication and make maintenance of them as simple as
> >> possible?
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> No, the charts don’t need to be built externally, though it
> >> gets
> >>>>>>>>>>>>>> compressed when pushed f we build to chart repository(this
> is
> >> an
> >>>>>>>>>>>> automatic
> >>>>>>>>>>>>>> step in case of official chart repo and we can automate
> using
> >> ci
> >>>>>>> our
> >>>>>>>>>> own
> >>>>>>>>>>>>>> chart repo)
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> 4) Could you provide a roadmap with various milestones and
> >>>> dates?
> >>>>>>> See
> >>>>>>>>>>>> the
> >>>>>>>>>>>>>>> last roadmap email I sent on the devs list and please reply
> >> to
> >>>>>>> it.
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> Okay I would work on that and update the progress on the
> mail
> >>>>>>> thread
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> 5) It seems you missed the bonding period so you’ll need to
> >>>> catch
> >>>>>>>>>> up on
> >>>>>>>>>>>>>>> that. Could you pick some jira issue and work on them
> >> quickly?
> >>>>>>> This
> >>>>>>>>>> is
> >>>>>>>>>>>> to
> >>>>>>>>>>>>>>> make sure you use the proper xwiki communication channels,
> >>>> learn
> >>>>>>> to
> >>>>>>>>>>>> work
> >>>>>>>>>>>>>>> with the xwiki team and start understanding the xwiki
> >>>>>>> architecture,
> >>>>>>>>>>>> and how
> >>>>>>>>>>>>>>> extensions are developed for XWiki. Since you’re a bit
> late,
> >>>>>>> you’ll
> >>>>>>>>>>>> need to
> >>>>>>>>>>>>>>> catch up and achieve this very quickly now and be present
> >> full
> >>>>>>> time
> >>>>>>>>>> on
> >>>>>>>>>>>> the
> >>>>>>>>>>>>>>> xwiki chat :)
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>> I would start doing that
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> 6) Idea: please review the issues on the XWiki Docker
> >> project’s
> >>>>>>>>>> jira.
> >>>>>>>>>>>>>>> Maybe you can help with those too as part of
> >>>> onboarding/bonding.
> >>>>>>> It
> >>>>>>>>>>>> should
> >>>>>>>>>>>>>>> also give you additional use cases for the XWiki Helm
> Charts
> >>>> IMO.
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> Thanks! I’m eager to follow your work and see xwiki helm
> >> charts
> >>>>>>> in
> >>>>>>>>>>>> action.
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> -Vincent
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> Thank You
> >>>>>>>>>>>>>>>> Ashish Sharma
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> Thank You
> >>>>>>>>>>>>>> Ashish
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>
> >>
>
12