How to switch off Lucene/Solr indexers.

classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|

How to switch off Lucene/Solr indexers.

kgardas

Hello,

I'm attempting to import Wikipedia dump into XWiki using mediawiki/1.0
syntax. Since this process is quite slow and dealing with a lot of data
I'm trying to strip everything I don't need now.

While connecting visualvm to the xwiki JVM I've noticed that majority of
both CPU and memory resource is consumed by Solr index thread and Lucene
index thread. Solr leads here. To switch this off, I've switched off
completely search suggestions in Home->Administer Wiki->Search Sugest
and I've also switched Search system to be based on Database instead of
Solr. I've restarted XWiki and to my surprise from console log I see
Solr is started again and while profiling with visual vm I see again
those two index threads running and consuming majority of resources.

The platform is JDK 8, Solaris 11.1, XWiki 6.0.1 on top of PostgreSQL.

Is there any hint how to switch those index/search engines off completely?

Thanks!
Karel

_______________________________________________
devs mailing list
[hidden email]
http://lists.xwiki.org/mailman/listinfo/devs
Reply | Threaded
Open this post in threaded view
|

Re: How to switch off Lucene/Solr indexers.

Thomas Mortagne
Administrator
On Fri, Sep 19, 2014 at 3:02 PM, Karel Gardas <[hidden email]> wrote:

>
> Hello,
>
> I'm attempting to import Wikipedia dump into XWiki using mediawiki/1.0
> syntax. Since this process is quite slow and dealing with a lot of data I'm
> trying to strip everything I don't need now.
>
> While connecting visualvm to the xwiki JVM I've noticed that majority of
> both CPU and memory resource is consumed by Solr index thread and Lucene
> index thread. Solr leads here. To switch this off, I've switched off
> completely search suggestions in Home->Administer Wiki->Search Sugest and
> I've also switched Search system to be based on Database instead of Solr.

This only control what is the one that is going to be displayed by
default but there is nothing in the UI to disabled SOLR.

> I've restarted XWiki and to my surprise from console log I see Solr is
> started again and while profiling with visual vm I see again those two index
> threads running and consuming majority of resources.
>
> The platform is JDK 8, Solaris 11.1, XWiki 6.0.1 on top of PostgreSQL.
>
> Is there any hint how to switch those index/search engines off completely?

The simplest is probably to move all the solr and lucene related jar
files out of WEB-INF/lib/ (and put them back later to index al that).

>
> Thanks!
> Karel
>
> _______________________________________________
> devs mailing list
> [hidden email]
> http://lists.xwiki.org/mailman/listinfo/devs



--
Thomas Mortagne
_______________________________________________
devs mailing list
[hidden email]
http://lists.xwiki.org/mailman/listinfo/devs
Reply | Threaded
Open this post in threaded view
|

Re: How to switch off Lucene/Solr indexers.

vmassol
Administrator
 



On 19 Sep 2014 at 15:10:30, Thomas Mortagne ([hidden email](mailto:[hidden email])) wrote:

> On Fri, Sep 19, 2014 at 3:02 PM, Karel Gardas wrote:
> >
> > Hello,
> >
> > I'm attempting to import Wikipedia dump into XWiki using mediawiki/1.0
> > syntax. Since this process is quite slow and dealing with a lot of data I'm
> > trying to strip everything I don't need now.
> >
> > While connecting visualvm to the xwiki JVM I've noticed that majority of
> > both CPU and memory resource is consumed by Solr index thread and Lucene
> > index thread. Solr leads here. To switch this off, I've switched off
> > completely search suggestions in Home->Administer Wiki->Search Sugest and
> > I've also switched Search system to be based on Database instead of Solr.
>  
> This only control what is the one that is going to be displayed by
> default but there is nothing in the UI to disabled SOLR.
>  
> > I've restarted XWiki and to my surprise from console log I see Solr is
> > started again and while profiling with visual vm I see again those two index
> > threads running and consuming majority of resources.
> >
> > The platform is JDK 8, Solaris 11.1, XWiki 6.0.1 on top of PostgreSQL.
> >
> > Is there any hint how to switch those index/search engines off completely?
>  
> The simplest is probably to move all the solr and lucene related jar
> files out of WEB-INF/lib/ (and put them back later to index al that).

It’s not the first time I’ve heard people asking how to turn off SOLR. Maybe we should do something about it to make it simpler to turn off (people will need to understand they’ll need to reindex when they turn it on again though).

Thanks
-Vincent

_______________________________________________
devs mailing list
[hidden email]
http://lists.xwiki.org/mailman/listinfo/devs
Reply | Threaded
Open this post in threaded view
|

Re: How to switch off Lucene/Solr indexers.

kgardas
In reply to this post by Thomas Mortagne
On 09/19/14 03:09 PM, Thomas Mortagne wrote:
>> Is there any hint how to switch those index/search engines off completely?
>
> The simplest is probably to move all the solr and lucene related jar
> files out of WEB-INF/lib/ (and put them back later to index al that).

Thanks a lot Thomas for quick answer, this really helped here.

Karel

_______________________________________________
devs mailing list
[hidden email]
http://lists.xwiki.org/mailman/listinfo/devs
Reply | Threaded
Open this post in threaded view
|

Re: How to switch off Lucene/Solr indexers.

kgardas
In reply to this post by vmassol
On 09/19/14 03:12 PM, [hidden email] wrote:
>>> Is there any hint how to switch those index/search engines off completely?
>>
>> The simplest is probably to move all the solr and lucene related jar
>> files out of WEB-INF/lib/ (and put them back later to index al that).
>
> It’s not the first time I’ve heard people asking how to turn off SOLR. Maybe we should do something about it to make it simpler to turn off (people will need to understand they’ll need to reindex when they turn it on again though).

For me particular was a little bit confusing seeing Solr running even if
I choose DB-based search. So at least a hint on this preference page
would be great to have. Having a separate page for configuring Solr
internal engine would be even nicer indeed...

Thanks!
Karel
_______________________________________________
devs mailing list
[hidden email]
http://lists.xwiki.org/mailman/listinfo/devs