[Feedback] Search Requirements

classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

[Feedback] Search Requirements

Ecaterina Moraru (Valica)
Hi,

We introduced in XWiki 4.3 a new experimental search based on Apache Solr
[A] .

Right now our Solr Search [B] is marked as experimental and I don't know
how many of you got a chance to play a bit with it.

In order for it to become default we need to make sure we are covering the
major use cases. So in this mail we should provide some feedback on what is
really needed from a relevance, user experience, performance, etc. point of
view.

I am very interested also in some general feedback on the Lucene Search:
major problems you had with it and limitations.

The following questions are mostly related to Solr Search:

1) The Solr implementation adds a 'Filtered Search'. Are all filters that
are available now needed (wiki, space, type, filetype)? Are we missing
some? What is the most important one? Should we have multiple select?

From a technical point of view we could add a lot of things (object
properties, query boost, etc.) but I'm more interested, in production, what
are some advanced or common use cases of search usage. IMO our Lucene
implementation is too simplistic, but it would be a shame to add useless
complexity to the Solr one if we really don't need it. On the other hand
having a customizable search is a really nice thing.

The customization part is kind of complex, but also the most powerful. I
want to know how much would we need the ability to explicitly mention the
types of results we want vs. excluding things, etc.

2) The Solr search adds a 'Sort' feature. What do you think about that?

3) Search result metadatas: should we display the relevance? What other
information should be displayed besides name, location, type?

4) Other mentions.

Having a generic search is more complex, than knowing exactly what your
search needs to return. We need to make sure we don't add too many niche
things while keeping the functionality satisfiable for general use cases.

Thanks,
Caty

---------------
[A]
http://www.xwiki.org/xwiki/bin/view/ReleaseNotes/ReleaseNotesXWiki43#HExperimentalSolrbasedsearchengine
[B]
http://extensions.xwiki.org/xwiki/bin/view/Extension/Solr+Search+Application
_______________________________________________
devs mailing list
[hidden email]
http://lists.xwiki.org/mailman/listinfo/devs
Reply | Threaded
Open this post in threaded view
|

Re: [Feedback] Search Requirements

Guillaume Lerouge
Hi Caty,

thanks for your email. Here is a list of the main complaints I have heard
recently concerning the existing search:

   - We enter a word and it does not find similar words: for instance
   searching for "café" does not return docs containing "cafe" or "cafés"
   - An attached document is ranked before the XWiki document for a given
   search
   - We cannot control the weight of various page elements: how can we give
   more weight to the title and tags versus the content of a page or an
   attachment?
   - We search for a word that we know is in a page and yet that page
   doesn't show up -> usually after an indexing issue. Making sure the
   indexing is always up and doesn't break for new or updated documents is a
   priority
   - We want to get results only for a subset of pages: only XWiki
   documents, only attachments, everything but images, only in a given space
   => this means the ability to use and mix filters
   - The search is returning docs with a lot of noise: maybe we remove the
   results with a very low relevance score (under 5% for instance)?
   - About relevance: I don't think displaying it is really required.
   Intuitively, users know that the ranking of results is a measure of
   relevance (most relevant docs are on the first page, near the top)

That's it for a first list. What would be needed to investigate more are
real-life examples of queries that users were expecting to work but that
didn't. I'll try to find some and provide them to you.

Thanks,

Guillaume

On Wed, Apr 24, 2013 at 10:52 AM, Ecaterina Moraru (Valica) <
[hidden email]> wrote:

> Hi,
>
> We introduced in XWiki 4.3 a new experimental search based on Apache Solr
> [A] .
>
> Right now our Solr Search [B] is marked as experimental and I don't know
> how many of you got a chance to play a bit with it.
>
> In order for it to become default we need to make sure we are covering the
> major use cases. So in this mail we should provide some feedback on what is
> really needed from a relevance, user experience, performance, etc. point of
> view.
>
> I am very interested also in some general feedback on the Lucene Search:
> major problems you had with it and limitations.
>
> The following questions are mostly related to Solr Search:
>
> 1) The Solr implementation adds a 'Filtered Search'. Are all filters that
> are available now needed (wiki, space, type, filetype)? Are we missing
> some? What is the most important one? Should we have multiple select?
>
> From a technical point of view we could add a lot of things (object
> properties, query boost, etc.) but I'm more interested, in production, what
> are some advanced or common use cases of search usage. IMO our Lucene
> implementation is too simplistic, but it would be a shame to add useless
> complexity to the Solr one if we really don't need it. On the other hand
> having a customizable search is a really nice thing.
>
> The customization part is kind of complex, but also the most powerful. I
> want to know how much would we need the ability to explicitly mention the
> types of results we want vs. excluding things, etc.
>
> 2) The Solr search adds a 'Sort' feature. What do you think about that?
>
> 3) Search result metadatas: should we display the relevance? What other
> information should be displayed besides name, location, type?
>
> 4) Other mentions.
>
> Having a generic search is more complex, than knowing exactly what your
> search needs to return. We need to make sure we don't add too many niche
> things while keeping the functionality satisfiable for general use cases.
>
> Thanks,
> Caty
>
> ---------------
> [A]
>
> http://www.xwiki.org/xwiki/bin/view/ReleaseNotes/ReleaseNotesXWiki43#HExperimentalSolrbasedsearchengine
> [B]
>
> http://extensions.xwiki.org/xwiki/bin/view/Extension/Solr+Search+Application
> _______________________________________________
> devs mailing list
> [hidden email]
> http://lists.xwiki.org/mailman/listinfo/devs
_______________________________________________
devs mailing list
[hidden email]
http://lists.xwiki.org/mailman/listinfo/devs
Reply | Threaded
Open this post in threaded view
|

Re: [Feedback] Search Requirements

xrichard
Guillaume write a perfect synthesis of the mains complains / improvments
suggestions I have personnaly experimented while using XWiki / Lucene search in
an Wiki running XWiki 3.1 .

regards,


Le 24/04/2013 15:16, Guillaume Lerouge a écrit :

> Hi Caty,
>
> thanks for your email. Here is a list of the main complaints I have heard
> recently concerning the existing search:
>
>     - We enter a word and it does not find similar words: for instance
>     searching for "café" does not return docs containing "cafe" or "cafés"
>     - An attached document is ranked before the XWiki document for a given
>     search
>     - We cannot control the weight of various page elements: how can we give
>     more weight to the title and tags versus the content of a page or an
>     attachment?
>     - We search for a word that we know is in a page and yet that page
>     doesn't show up -> usually after an indexing issue. Making sure the
>     indexing is always up and doesn't break for new or updated documents is a
>     priority
>     - We want to get results only for a subset of pages: only XWiki
>     documents, only attachments, everything but images, only in a given space
>     => this means the ability to use and mix filters
>     - The search is returning docs with a lot of noise: maybe we remove the
>     results with a very low relevance score (under 5% for instance)?
>     - About relevance: I don't think displaying it is really required.
>     Intuitively, users know that the ranking of results is a measure of
>     relevance (most relevant docs are on the first page, near the top)
>
> That's it for a first list. What would be needed to investigate more are
> real-life examples of queries that users were expecting to work but that
> didn't. I'll try to find some and provide them to you.
>
> Thanks,
>
> Guillaume
>
> On Wed, Apr 24, 2013 at 10:52 AM, Ecaterina Moraru (Valica) <
> [hidden email]> wrote:
>
>> Hi,
>>
>> We introduced in XWiki 4.3 a new experimental search based on Apache Solr
>> [A] .
>>
>> Right now our Solr Search [B] is marked as experimental and I don't know
>> how many of you got a chance to play a bit with it.
>>
>> In order for it to become default we need to make sure we are covering the
>> major use cases. So in this mail we should provide some feedback on what is
>> really needed from a relevance, user experience, performance, etc. point of
>> view.
>>
>> I am very interested also in some general feedback on the Lucene Search:
>> major problems you had with it and limitations.
>>
>> The following questions are mostly related to Solr Search:
>>
>> 1) The Solr implementation adds a 'Filtered Search'. Are all filters that
>> are available now needed (wiki, space, type, filetype)? Are we missing
>> some? What is the most important one? Should we have multiple select?
>>
>>  From a technical point of view we could add a lot of things (object
>> properties, query boost, etc.) but I'm more interested, in production, what
>> are some advanced or common use cases of search usage. IMO our Lucene
>> implementation is too simplistic, but it would be a shame to add useless
>> complexity to the Solr one if we really don't need it. On the other hand
>> having a customizable search is a really nice thing.
>>
>> The customization part is kind of complex, but also the most powerful. I
>> want to know how much would we need the ability to explicitly mention the
>> types of results we want vs. excluding things, etc.
>>
>> 2) The Solr search adds a 'Sort' feature. What do you think about that?
>>
>> 3) Search result metadatas: should we display the relevance? What other
>> information should be displayed besides name, location, type?
>>
>> 4) Other mentions.
>>
>> Having a generic search is more complex, than knowing exactly what your
>> search needs to return. We need to make sure we don't add too many niche
>> things while keeping the functionality satisfiable for general use cases.
>>
>> Thanks,
>> Caty
>>
>> ---------------
>> [A]
>>
>> http://www.xwiki.org/xwiki/bin/view/ReleaseNotes/ReleaseNotesXWiki43#HExperimentalSolrbasedsearchengine
>> [B]
>>
>> http://extensions.xwiki.org/xwiki/bin/view/Extension/Solr+Search+Application
>> _______________________________________________
>> devs mailing list
>> [hidden email]
>> http://lists.xwiki.org/mailman/listinfo/devs
> _______________________________________________
> devs mailing list
> [hidden email]
> http://lists.xwiki.org/mailman/listinfo/devs

_______________________________________________
devs mailing list
[hidden email]
http://lists.xwiki.org/mailman/listinfo/devs
Reply | Threaded
Open this post in threaded view
|

Re: [Feedback] Search Requirements

Silvia Rusu
Hi,

I generally agree with Guillaume. I've heard this too a few times recently:
"We search for a word that we know is in a page and yet that page doesn't
show up".

Additionally it would be great to be able to filter results by language.
People have asked me before why the search results were mixed up and how
they could perform a search only for a certain language.

One other thing I'd find useful is the ability to filter results by author
/ person who edited the document. This way if I am particularly interested
in a document created / modified by someone I can easily find it.

Thanks,
Silvia


On Fri, May 10, 2013 at 10:24 AM, Ldm Public <[hidden email]> wrote:

> Guillaume write a perfect synthesis of the mains complains / improvments
> suggestions I have personnaly experimented while using XWiki / Lucene
> search in an Wiki running XWiki 3.1 .
>
> regards,
>
>
> Le 24/04/2013 15:16, Guillaume Lerouge a écrit :
>
>  Hi Caty,
>>
>> thanks for your email. Here is a list of the main complaints I have heard
>> recently concerning the existing search:
>>
>>     - We enter a word and it does not find similar words: for instance
>>     searching for "café" does not return docs containing "cafe" or "cafés"
>>     - An attached document is ranked before the XWiki document for a given
>>     search
>>     - We cannot control the weight of various page elements: how can we
>> give
>>     more weight to the title and tags versus the content of a page or an
>>     attachment?
>>     - We search for a word that we know is in a page and yet that page
>>     doesn't show up -> usually after an indexing issue. Making sure the
>>     indexing is always up and doesn't break for new or updated documents
>> is a
>>     priority
>>     - We want to get results only for a subset of pages: only XWiki
>>     documents, only attachments, everything but images, only in a given
>> space
>>     => this means the ability to use and mix filters
>>     - The search is returning docs with a lot of noise: maybe we remove
>> the
>>     results with a very low relevance score (under 5% for instance)?
>>     - About relevance: I don't think displaying it is really required.
>>     Intuitively, users know that the ranking of results is a measure of
>>     relevance (most relevant docs are on the first page, near the top)
>>
>> That's it for a first list. What would be needed to investigate more are
>> real-life examples of queries that users were expecting to work but that
>> didn't. I'll try to find some and provide them to you.
>>
>> Thanks,
>>
>> Guillaume
>>
>> On Wed, Apr 24, 2013 at 10:52 AM, Ecaterina Moraru (Valica) <
>> [hidden email]> wrote:
>>
>>  Hi,
>>>
>>> We introduced in XWiki 4.3 a new experimental search based on Apache Solr
>>> [A] .
>>>
>>> Right now our Solr Search [B] is marked as experimental and I don't know
>>> how many of you got a chance to play a bit with it.
>>>
>>> In order for it to become default we need to make sure we are covering
>>> the
>>> major use cases. So in this mail we should provide some feedback on what
>>> is
>>> really needed from a relevance, user experience, performance, etc. point
>>> of
>>> view.
>>>
>>> I am very interested also in some general feedback on the Lucene Search:
>>> major problems you had with it and limitations.
>>>
>>> The following questions are mostly related to Solr Search:
>>>
>>> 1) The Solr implementation adds a 'Filtered Search'. Are all filters that
>>> are available now needed (wiki, space, type, filetype)? Are we missing
>>> some? What is the most important one? Should we have multiple select?
>>>
>>>  From a technical point of view we could add a lot of things (object
>>> properties, query boost, etc.) but I'm more interested, in production,
>>> what
>>> are some advanced or common use cases of search usage. IMO our Lucene
>>> implementation is too simplistic, but it would be a shame to add useless
>>> complexity to the Solr one if we really don't need it. On the other hand
>>> having a customizable search is a really nice thing.
>>>
>>> The customization part is kind of complex, but also the most powerful. I
>>> want to know how much would we need the ability to explicitly mention the
>>> types of results we want vs. excluding things, etc.
>>>
>>> 2) The Solr search adds a 'Sort' feature. What do you think about that?
>>>
>>> 3) Search result metadatas: should we display the relevance? What other
>>> information should be displayed besides name, location, type?
>>>
>>> 4) Other mentions.
>>>
>>> Having a generic search is more complex, than knowing exactly what your
>>> search needs to return. We need to make sure we don't add too many niche
>>> things while keeping the functionality satisfiable for general use cases.
>>>
>>> Thanks,
>>> Caty
>>>
>>> ---------------
>>> [A]
>>>
>>> http://www.xwiki.org/xwiki/**bin/view/ReleaseNotes/**
>>> ReleaseNotesXWiki43#**HExperimentalSolrbasedsearchen**gine<http://www.xwiki.org/xwiki/bin/view/ReleaseNotes/ReleaseNotesXWiki43#HExperimentalSolrbasedsearchengine>
>>> [B]
>>>
>>> http://extensions.xwiki.org/**xwiki/bin/view/Extension/Solr+**
>>> Search+Application<http://extensions.xwiki.org/xwiki/bin/view/Extension/Solr+Search+Application>
>>> ______________________________**_________________
>>> devs mailing list
>>> [hidden email]
>>> http://lists.xwiki.org/**mailman/listinfo/devs<http://lists.xwiki.org/mailman/listinfo/devs>
>>>
>> ______________________________**_________________
>> devs mailing list
>> [hidden email]
>> http://lists.xwiki.org/**mailman/listinfo/devs<http://lists.xwiki.org/mailman/listinfo/devs>
>>
>
> ______________________________**_________________
> devs mailing list
> [hidden email]
> http://lists.xwiki.org/**mailman/listinfo/devs<http://lists.xwiki.org/mailman/listinfo/devs>
>
_______________________________________________
devs mailing list
[hidden email]
http://lists.xwiki.org/mailman/listinfo/devs
Silvia Rusu
Tester & Documentation Writer - XWiki
http://twitter.com/silviarusu