severe scalability issue with adding users

classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

severe scalability issue with adding users

alg0000
My team is integrating xwiki into a heavily trafficed commercial site (over a million uniques a day).

We have hundreds of thousands of registered users. Our current tests indicate that the tie to add a new user is linear with respect to the number of users already added. We went from less than 1/2 second per user to over a second per new user as our test data grew from 3k to 10k users.

The bottleneck seems to be a database query that has to iterate over all existing users in order to add an additional user.

Any feedback would be appreciated. As far as I can see, we are using standard xwiki functions to add users. Let me know what information you need from me in order to help provide accurate feedback.

Thank you,
Andy



--
You receive this message as a subscriber of the [hidden email] mailing list.
To unsubscribe: mailto:[hidden email]
For general help: mailto:[hidden email]?subject=help
ObjectWeb mailing lists service home page: http://www.objectweb.org/wws
Reply | Threaded
Open this post in threaded view
|

Re: severe scalability issue with adding users

Ludovic Dubost

Hi,

There are a few improvements needs to handle that amount of users.

1/ The storage of users would be more performant with a Custom Mapping
for users and groups. The custom mapping technology is ready. It has
been used for the preferences in the latest version. Users and groups
should use it. However to implement it queries that are looking for
users a different places in the code would need to be looked at and
potentially changed.

2/ The XWiki.XWikiAllGroup should be made implicit. Currently it's a
real group which is highly inefficient. We have hacked the storage in
order to read it faster and the function which ads the user to the
XWiki.XWikiAllGroup also contains a shortcut. However if you get to
write the full document it would take ages with the current storage.

I'm not sure exactly where the iteration on all users is ?

One thing you should check is database indexes (all columns that are
searched should be properly indexed). xws_value in xwikistrings is an
important one.

Mandriva has made a specific users and groups implementation in order to
handle more than 25000 users.
You could ask for this implementation or help us to 1/ and 2/ as this is
what we would like to do.

Ludovic

[hidden email] a écrit :

> My team is integrating xwiki into a heavily trafficed commercial site (over a million uniques a day).
>
> We have hundreds of thousands of registered users. Our current tests indicate that the tie to add a new user is linear with respect to the number of users already added. We went from less than 1/2 second per user to over a second per new user as our test data grew from 3k to 10k users.
>
> The bottleneck seems to be a database query that has to iterate over all existing users in order to add an additional user.
>
> Any feedback would be appreciated. As far as I can see, we are using standard xwiki functions to add users. Let me know what information you need from me in order to help provide accurate feedback.
>
> Thank you,
> Andy
>
>  
> ------------------------------------------------------------------------
>
>
> --
> You receive this message as a subscriber of the [hidden email] mailing list.
> To unsubscribe: mailto:[hidden email]
> For general help: mailto:[hidden email]?subject=help
> ObjectWeb mailing lists service home page: http://www.objectweb.org/wws
>  

--
Ludovic Dubost
XPertNet: http://www.xpertnet.fr/
Blog: http://www.ludovic.org/blog/
XWiki: http://www.xwiki.com
Skype: ldubost AIM: nvludo Yahoo: ludovic




--
You receive this message as a subscriber of the [hidden email] mailing list.
To unsubscribe: mailto:[hidden email]
For general help: mailto:[hidden email]?subject=help
ObjectWeb mailing lists service home page: http://www.objectweb.org/wws