Fwd: .xar Extension NOT Used Properly

classic Classic list List threaded Threaded
23 messages Options
12
Reply | Threaded
Open this post in threaded view
|

Fwd: .xar Extension NOT Used Properly

Catalin Hritcu
Hello all,

What would you think of using the JAR extension instead of (mis-)
using XAR for XWiki applications, and using the JAR manifest for
storing the information that is now stored in package.xml ? Any other
way this can be fixed ?

Already created a JIRA task about this

http://jira.xwiki.org/jira/browse/XWIKI-802

but I would like to hear your opinions before spending the time to
implement a change you might actually not want :)

Catalin

---------- Forwarded message ----------
From: Vincent Massol <[hidden email]>
Date: Feb 2, 2007 7:56 PM
Subject: Re: [xwiki-dev] .xar Extension NOT Used Properly
To: Catalin Hritcu <[hidden email]>


Hi Catalin,

I think it's best to discuss all this on the list.

Also, I've provided some answer in JIRA.

Thanks
-Vincent


On Feb 1, 2007, at 11:22 PM, Catalin Hritcu wrote:

> Hi Vincent,
>
> I created a JIRA issue for this and assigned it to myself:
> http://jira.xwiki.org/jira/browse/XWIKI-802
>
> At the time I don't really know much about what needs to be changed to
> make it happen, but I'll look into it. Suggestions welcome :)
>
>> I guess we could use a JAR format and use information in the manifest
>> to describe the content of an archive
>
> Can you also be more precise about the information you want stored in
> the manifest?
> http://java.sun.com/j2se/1.5.0/docs/guide/jar/jar.html#JAR%20Manifest
> According to this it seems that a manifest can store many things.
>
> Regards,
> Catalin
>
> On 1/31/07, Vincent Massol <[hidden email]> wrote:
>> Hi Catalin,
>>
>> See my other email on this topic. Basically I think I agree with
>> you but
>> we're currently focusing on releasing XWiki 1.0 and fixing bugs.
>> this is an
>> improvement request that will have to wait for quite long unless
>> someone
>> contributes a patch for it (including patches for the
>> documentation ;-)).
>>
>> Thanks
>> -Vincent
>>
>>
>> On Jan 8, 2007, at 11:32 AM, Catalin Hritcu wrote:
>>
>>
>> The new way to import/export wiki pages is very useful. However, I
>>  think that using the .xar extension for the zip(!) archive is highly
>>  confusing. This because there is already a XAR(eXtensible ARchiver)
>>  archiving utility which uses the .xar extension for quite a
>> while. And
>>  while now XAR is standalone, it will be quite probable used by the
>>  MacOSX package system in the future. Then download your archive will
>>  yield an error like this (safari automatically expands archives):
>>
>>  "Error opening xar archive: xwiki-1.0-beta-1.xar"
>>
>>  Links:
>>  http://www.opendarwin.org/projects/xar/
>>  http://www.nabble.com/xar-in-Mac-OS-X-t2081148.html
>>
>>
>> <messagefooter.txt>
>>




___________________________________________________________________________
Yahoo! Mail réinvente le mail ! Découvrez le nouveau Yahoo! Mail et
son interface révolutionnaire.
http://fr.mail.yahoo.com



--
You receive this message as a subscriber of the [hidden email] mailing list.
To unsubscribe: mailto:[hidden email]
For general help: mailto:[hidden email]?subject=help
ObjectWeb mailing lists service home page: http://www.objectweb.org/wws
Reply | Threaded
Open this post in threaded view
|

Re: .xar Extension NOT Used Properly

jeremi joslin-2
Hi,

 -1

I'm not for changing to jar. it's not a Java archive, it's an XWiki
Archive. I don't see the point to change. People will think it's
javacode.

if you look at this page :
http://filext.com/detaillist.php?extdetail=XAR there is some other
software who are using this extension. So why XWiki can't use it?


jeremi

On 2/3/07, Catalin Hritcu <[hidden email]> wrote:

> Hello all,
>
> What would you think of using the JAR extension instead of (mis-)
> using XAR for XWiki applications, and using the JAR manifest for
> storing the information that is now stored in package.xml ? Any other
> way this can be fixed ?
>
> Already created a JIRA task about this
>
> http://jira.xwiki.org/jira/browse/XWIKI-802
>
> but I would like to hear your opinions before spending the time to
> implement a change you might actually not want :)
>
> Catalin
>
> ---------- Forwarded message ----------
> From: Vincent Massol <[hidden email]>
> Date: Feb 2, 2007 7:56 PM
> Subject: Re: [xwiki-dev] .xar Extension NOT Used Properly
> To: Catalin Hritcu <[hidden email]>
>
>
> Hi Catalin,
>
> I think it's best to discuss all this on the list.
>
> Also, I've provided some answer in JIRA.
>
> Thanks
> -Vincent
>
>
> On Feb 1, 2007, at 11:22 PM, Catalin Hritcu wrote:
>
> > Hi Vincent,
> >
> > I created a JIRA issue for this and assigned it to myself:
> > http://jira.xwiki.org/jira/browse/XWIKI-802
> >
> > At the time I don't really know much about what needs to be changed to
> > make it happen, but I'll look into it. Suggestions welcome :)
> >
> >> I guess we could use a JAR format and use information in the manifest
> >> to describe the content of an archive
> >
> > Can you also be more precise about the information you want stored in
> > the manifest?
> > http://java.sun.com/j2se/1.5.0/docs/guide/jar/jar.html#JAR%20Manifest
> > According to this it seems that a manifest can store many things.
> >
> > Regards,
> > Catalin
> >
> > On 1/31/07, Vincent Massol <[hidden email]> wrote:
> >> Hi Catalin,
> >>
> >> See my other email on this topic. Basically I think I agree with
> >> you but
> >> we're currently focusing on releasing XWiki 1.0 and fixing bugs.
> >> this is an
> >> improvement request that will have to wait for quite long unless
> >> someone
> >> contributes a patch for it (including patches for the
> >> documentation ;-)).
> >>
> >> Thanks
> >> -Vincent
> >>
> >>
> >> On Jan 8, 2007, at 11:32 AM, Catalin Hritcu wrote:
> >>
> >>
> >> The new way to import/export wiki pages is very useful. However, I
> >>  think that using the .xar extension for the zip(!) archive is highly
> >>  confusing. This because there is already a XAR(eXtensible ARchiver)
> >>  archiving utility which uses the .xar extension for quite a
> >> while. And
> >>  while now XAR is standalone, it will be quite probable used by the
> >>  MacOSX package system in the future. Then download your archive will
> >>  yield an error like this (safari automatically expands archives):
> >>
> >>  "Error opening xar archive: xwiki-1.0-beta-1.xar"
> >>
> >>  Links:
> >>  http://www.opendarwin.org/projects/xar/
> >>  http://www.nabble.com/xar-in-Mac-OS-X-t2081148.html
> >>
> >>
> >> <messagefooter.txt>
> >>
>
>
>
>
>
> ___________________________________________________________________________
> Yahoo! Mail réinvente le mail ! Découvrez le nouveau Yahoo! Mail et
> son interface révolutionnaire.
> http://fr.mail.yahoo.com
>
>
>
> --
> You receive this message as a subscriber of the [hidden email] mailing list.
> To unsubscribe: mailto:[hidden email]
> For general help: mailto:[hidden email]?subject=help
> ObjectWeb mailing lists service home page: http://www.objectweb.org/wws
>
>

--
jeremi


--
You receive this message as a subscriber of the [hidden email] mailing list.
To unsubscribe: mailto:[hidden email]
For general help: mailto:[hidden email]?subject=help
ObjectWeb mailing lists service home page: http://www.objectweb.org/wws
Reply | Threaded
Open this post in threaded view
|

Re: .xar Extension NOT Used Properly

vmassol
Administrator
Hi Jeremi,

I'm +1 on my side but as you voted -1 we'll need to convince you to  
change your vote ;-) Let me give it a try below...

On Feb 3, 2007, at 9:41 AM, jeremi joslin wrote:

> Hi,
>
> -1
>
> I'm not for changing to jar. it's not a Java archive, it's an XWiki
> Archive. I don't see the point to change. People will think it's
> javacode.

* JAR is an archive format. It can be used for lots of different  
things. There are even software that use JARs for their installation.
* I really believe that in the future our applications will not be  
made only of XML files but will also contain other libraries (jar  
files) and java classes. For example imagine the IRC bot application.  
There are some xwiki pages but we also need the picoirc jar. We'll  
need to bundle all that together.
* XWiki is a Java application so all users are familiar with the JAR  
format.

Of course we can use the XAR format for that. XAR looks nice because  
there's XWiki inside but there's a cost of inventing a new archive  
format:
* It needs to be described
* There needs to be tools for it (or people need to rename it to  
zip...). BTW if it's a zip we might as well name it zip... The  
problem with Zip is that it does not support meta data information  
such as the jar format does. And we need these... And there are APIs  
to get these metadata so we don't have to reimplement all the code  
(as we've done for the package.xml file).
* It makes it difficult for the build (this is the same argument as  
above with tools) as all the existing plugins do not understand the  
XAR format so we have to reinvent all the tools... This is why I had  
to implement the xar handler and the xar plugin in our repository. It  
would make it much easier if it were an existing format such as JAR.

I think it would be much better to use an existing format (which we  
do except we're using Zip instead of JAR), already documented rather  
than invent a new one. There's no point in us hiding the underlying  
format we use by changing the extension. We might as well use that  
format's extension...

> if you look at this page :
> http://filext.com/detaillist.php?extdetail=XAR there is some other
> software who are using this extension. So why XWiki can't use it?

Some people have thrown nuclear bombs in the past. Is that a good  
reason for others to do it? Probably not.

It's not because people are doing bad things that we have to follow  
them ;-)

Thanks
-Vincent

> On 2/3/07, Catalin Hritcu <[hidden email]> wrote:
>> Hello all,
>>
>> What would you think of using the JAR extension instead of (mis-)
>> using XAR for XWiki applications, and using the JAR manifest for
>> storing the information that is now stored in package.xml ? Any other
>> way this can be fixed ?
>>
>> Already created a JIRA task about this
>>
>> http://jira.xwiki.org/jira/browse/XWIKI-802
>>
>> but I would like to hear your opinions before spending the time to
>> implement a change you might actually not want :)
>>
>> Catalin
>>
>> ---------- Forwarded message ----------
>> From: Vincent Massol <[hidden email]>
>> Date: Feb 2, 2007 7:56 PM
>> Subject: Re: [xwiki-dev] .xar Extension NOT Used Properly
>> To: Catalin Hritcu <[hidden email]>
>>
>>
>> Hi Catalin,
>>
>> I think it's best to discuss all this on the list.
>>
>> Also, I've provided some answer in JIRA.
>>
>> Thanks
>> -Vincent
>>
>>
>> On Feb 1, 2007, at 11:22 PM, Catalin Hritcu wrote:
>>
>> > Hi Vincent,
>> >
>> > I created a JIRA issue for this and assigned it to myself:
>> > http://jira.xwiki.org/jira/browse/XWIKI-802
>> >
>> > At the time I don't really know much about what needs to be  
>> changed to
>> > make it happen, but I'll look into it. Suggestions welcome :)
>> >
>> >> I guess we could use a JAR format and use information in the  
>> manifest
>> >> to describe the content of an archive
>> >
>> > Can you also be more precise about the information you want  
>> stored in
>> > the manifest?
>> > <a href="http://java.sun.com/j2se/1.5.0/docs/guide/jar/jar.html#JAR%">http://java.sun.com/j2se/1.5.0/docs/guide/jar/jar.html#JAR% 
>> 20Manifest
>> > According to this it seems that a manifest can store many things.
>> >
>> > Regards,
>> > Catalin
>> >
>> > On 1/31/07, Vincent Massol <[hidden email]> wrote:
>> >> Hi Catalin,
>> >>
>> >> See my other email on this topic. Basically I think I agree with
>> >> you but
>> >> we're currently focusing on releasing XWiki 1.0 and fixing bugs.
>> >> this is an
>> >> improvement request that will have to wait for quite long unless
>> >> someone
>> >> contributes a patch for it (including patches for the
>> >> documentation ;-)).
>> >>
>> >> Thanks
>> >> -Vincent
>> >>
>> >>
>> >> On Jan 8, 2007, at 11:32 AM, Catalin Hritcu wrote:
>> >>
>> >>
>> >> The new way to import/export wiki pages is very useful. However, I
>> >>  think that using the .xar extension for the zip(!) archive is  
>> highly
>> >>  confusing. This because there is already a XAR(eXtensible  
>> ARchiver)
>> >>  archiving utility which uses the .xar extension for quite a
>> >> while. And
>> >>  while now XAR is standalone, it will be quite probable used by  
>> the
>> >>  MacOSX package system in the future. Then download your  
>> archive will
>> >>  yield an error like this (safari automatically expands archives):
>> >>
>> >>  "Error opening xar archive: xwiki-1.0-beta-1.xar"
>> >>
>> >>  Links:
>> >>  http://www.opendarwin.org/projects/xar/
>> >>  http://www.nabble.com/xar-in-Mac-OS-X-t2081148.html
>> >>
>> >>
>> >> <messagefooter.txt>
>> >>
>>
>>
>>
>>
>>
>> _____________________________________________________________________
>> ______
>> Yahoo! Mail réinvente le mail ! Découvrez le nouveau Yahoo! Mail et
>> son interface révolutionnaire.
>> http://fr.mail.yahoo.com
>>
>>
>>
>> --
>> You receive this message as a subscriber of the xwiki-
>> [hidden email] mailing list.
>> To unsubscribe: mailto:[hidden email]
>> For general help: mailto:[hidden email]?subject=help
>> ObjectWeb mailing lists service home page: http://
>> www.objectweb.org/wws
>>
>>
>
>
> --
> jeremi
>
> --
> You receive this message as a subscriber of the xwiki-
> [hidden email] mailing list.
> To unsubscribe: mailto:[hidden email]
> For general help: mailto:[hidden email]?subject=help
> ObjectWeb mailing lists service home page: http://www.objectweb.org/ 
> wws





___________________________________________________________________________
Yahoo! Mail réinvente le mail ! Découvrez le nouveau Yahoo! Mail et son interface révolutionnaire.
http://fr.mail.yahoo.com



--
You receive this message as a subscriber of the [hidden email] mailing list.
To unsubscribe: mailto:[hidden email]
For general help: mailto:[hidden email]?subject=help
ObjectWeb mailing lists service home page: http://www.objectweb.org/wws
Reply | Threaded
Open this post in threaded view
|

Re: .xar Extension NOT Used Properly

Catalin Hritcu
In reply to this post by jeremi joslin-2
Hello,

Why not use .xwiki then? Or something that does not lead to so much
confusion? The times when file extensions were limited by the file
system to 3 characters are long gone.

Using .xar is not just using an extension that is already used for
something else, it's using one that is used for exactly the same
thing. It's like using .rar for archiving zip files, it's like
OpenOffice.org was saving open document files with the .doc extension.

Anyway, if you think that xar is just OK, and not causing confusion
then no problem with me, I'll retract my proposal.

Catalin

On 2/3/07, jeremi joslin <[hidden email]> wrote:

> Hi,
>
>  -1
>
> I'm not for changing to jar. it's not a Java archive, it's an XWiki
> Archive. I don't see the point to change. People will think it's
> javacode.
>
> if you look at this page :
> http://filext.com/detaillist.php?extdetail=XAR there is some other
> software who are using this extension. So why XWiki can't use it?
>
>
> jeremi
>
> On 2/3/07, Catalin Hritcu <[hidden email]> wrote:
> > Hello all,
> >
> > What would you think of using the JAR extension instead of (mis-)
> > using XAR for XWiki applications, and using the JAR manifest for
> > storing the information that is now stored in package.xml ? Any other
> > way this can be fixed ?
> >
> > Already created a JIRA task about this
> >
> > http://jira.xwiki.org/jira/browse/XWIKI-802
> >
> > but I would like to hear your opinions before spending the time to
> > implement a change you might actually not want :)
> >
> > Catalin
> >
> > ---------- Forwarded message ----------
> > From: Vincent Massol <[hidden email]>
> > Date: Feb 2, 2007 7:56 PM
> > Subject: Re: [xwiki-dev] .xar Extension NOT Used Properly
> > To: Catalin Hritcu <[hidden email]>
> >
> >
> > Hi Catalin,
> >
> > I think it's best to discuss all this on the list.
> >
> > Also, I've provided some answer in JIRA.
> >
> > Thanks
> > -Vincent
> >
> >
> > On Feb 1, 2007, at 11:22 PM, Catalin Hritcu wrote:
> >
> > > Hi Vincent,
> > >
> > > I created a JIRA issue for this and assigned it to myself:
> > > http://jira.xwiki.org/jira/browse/XWIKI-802
> > >
> > > At the time I don't really know much about what needs to be changed to
> > > make it happen, but I'll look into it. Suggestions welcome :)
> > >
> > >> I guess we could use a JAR format and use information in the manifest
> > >> to describe the content of an archive
> > >
> > > Can you also be more precise about the information you want stored in
> > > the manifest?
> > > http://java.sun.com/j2se/1.5.0/docs/guide/jar/jar.html#JAR%20Manifest
> > > According to this it seems that a manifest can store many things.
> > >
> > > Regards,
> > > Catalin
> > >
> > > On 1/31/07, Vincent Massol <[hidden email]> wrote:
> > >> Hi Catalin,
> > >>
> > >> See my other email on this topic. Basically I think I agree with
> > >> you but
> > >> we're currently focusing on releasing XWiki 1.0 and fixing bugs.
> > >> this is an
> > >> improvement request that will have to wait for quite long unless
> > >> someone
> > >> contributes a patch for it (including patches for the
> > >> documentation ;-)).
> > >>
> > >> Thanks
> > >> -Vincent
> > >>
> > >>
> > >> On Jan 8, 2007, at 11:32 AM, Catalin Hritcu wrote:
> > >>
> > >>
> > >> The new way to import/export wiki pages is very useful. However, I
> > >>  think that using the .xar extension for the zip(!) archive is highly
> > >>  confusing. This because there is already a XAR(eXtensible ARchiver)
> > >>  archiving utility which uses the .xar extension for quite a
> > >> while. And
> > >>  while now XAR is standalone, it will be quite probable used by the
> > >>  MacOSX package system in the future. Then download your archive will
> > >>  yield an error like this (safari automatically expands archives):
> > >>
> > >>  "Error opening xar archive: xwiki-1.0-beta-1.xar"
> > >>
> > >>  Links:
> > >>  http://www.opendarwin.org/projects/xar/
> > >>  http://www.nabble.com/xar-in-Mac-OS-X-t2081148.html
> > >>
> > >>
> > >> <messagefooter.txt>
> > >>
> >
> >
> >
> >
> >
> > ___________________________________________________________________________
> > Yahoo! Mail réinvente le mail ! Découvrez le nouveau Yahoo! Mail et
> > son interface révolutionnaire.
> > http://fr.mail.yahoo.com
> >
> >
> >
> > --
> > You receive this message as a subscriber of the [hidden email] mailing list.
> > To unsubscribe: mailto:[hidden email]
> > For general help: mailto:[hidden email]?subject=help
> > ObjectWeb mailing lists service home page: http://www.objectweb.org/wws
> >
> >
>
>
> --
> jeremi
>
>
> --
> You receive this message as a subscriber of the [hidden email] mailing list.
> To unsubscribe: mailto:[hidden email]
> For general help: mailto:[hidden email]?subject=help
> ObjectWeb mailing lists service home page: http://www.objectweb.org/wws
>
>


--
You receive this message as a subscriber of the [hidden email] mailing list.
To unsubscribe: mailto:[hidden email]
For general help: mailto:[hidden email]?subject=help
ObjectWeb mailing lists service home page: http://www.objectweb.org/wws
Reply | Threaded
Open this post in threaded view
|

Re: .xar Extension NOT Used Properly

Catalin Hritcu
In reply to this post by vmassol
Hello Vincent,

To add to the alternatives you presented, you could even use proper
xar (http://code.google.com/p/xar/). It is actually a quite nice
archive format, and it does have support for metadata, checksums,
table of contents etc. On the other hand, it is not an established
archive format, and I don't think there is a Java implementation for
it, so the overhead of implementing this would be a lot more
substantial than using zip or jar.

Catalin

On 2/3/07, Vincent Massol <[hidden email]> wrote:

> Hi Jeremi,
>
> I'm +1 on my side but as you voted -1 we'll need to convince you to
> change your vote ;-) Let me give it a try below...
>
> On Feb 3, 2007, at 9:41 AM, jeremi joslin wrote:
>
> > Hi,
> >
> > -1
> >
> > I'm not for changing to jar. it's not a Java archive, it's an XWiki
> > Archive. I don't see the point to change. People will think it's
> > javacode.
>
> * JAR is an archive format. It can be used for lots of different
> things. There are even software that use JARs for their installation.
> * I really believe that in the future our applications will not be
> made only of XML files but will also contain other libraries (jar
> files) and java classes. For example imagine the IRC bot application.
> There are some xwiki pages but we also need the picoirc jar. We'll
> need to bundle all that together.
> * XWiki is a Java application so all users are familiar with the JAR
> format.
>
> Of course we can use the XAR format for that. XAR looks nice because
> there's XWiki inside but there's a cost of inventing a new archive
> format:
> * It needs to be described
> * There needs to be tools for it (or people need to rename it to
> zip...). BTW if it's a zip we might as well name it zip... The
> problem with Zip is that it does not support meta data information
> such as the jar format does. And we need these... And there are APIs
> to get these metadata so we don't have to reimplement all the code
> (as we've done for the package.xml file).
> * It makes it difficult for the build (this is the same argument as
> above with tools) as all the existing plugins do not understand the
> XAR format so we have to reinvent all the tools... This is why I had
> to implement the xar handler and the xar plugin in our repository. It
> would make it much easier if it were an existing format such as JAR.
>
> I think it would be much better to use an existing format (which we
> do except we're using Zip instead of JAR), already documented rather
> than invent a new one. There's no point in us hiding the underlying
> format we use by changing the extension. We might as well use that
> format's extension...
>
> > if you look at this page :
> > http://filext.com/detaillist.php?extdetail=XAR there is some other
> > software who are using this extension. So why XWiki can't use it?
>
> Some people have thrown nuclear bombs in the past. Is that a good
> reason for others to do it? Probably not.
>
> It's not because people are doing bad things that we have to follow
> them ;-)
>
> Thanks
> -Vincent
>
> > On 2/3/07, Catalin Hritcu <[hidden email]> wrote:
> >> Hello all,
> >>
> >> What would you think of using the JAR extension instead of (mis-)
> >> using XAR for XWiki applications, and using the JAR manifest for
> >> storing the information that is now stored in package.xml ? Any other
> >> way this can be fixed ?
> >>
> >> Already created a JIRA task about this
> >>
> >> http://jira.xwiki.org/jira/browse/XWIKI-802
> >>
> >> but I would like to hear your opinions before spending the time to
> >> implement a change you might actually not want :)
> >>
> >> Catalin
> >>
> >> ---------- Forwarded message ----------
> >> From: Vincent Massol <[hidden email]>
> >> Date: Feb 2, 2007 7:56 PM
> >> Subject: Re: [xwiki-dev] .xar Extension NOT Used Properly
> >> To: Catalin Hritcu <[hidden email]>
> >>
> >>
> >> Hi Catalin,
> >>
> >> I think it's best to discuss all this on the list.
> >>
> >> Also, I've provided some answer in JIRA.
> >>
> >> Thanks
> >> -Vincent
> >>
> >>
> >> On Feb 1, 2007, at 11:22 PM, Catalin Hritcu wrote:
> >>
> >> > Hi Vincent,
> >> >
> >> > I created a JIRA issue for this and assigned it to myself:
> >> > http://jira.xwiki.org/jira/browse/XWIKI-802
> >> >
> >> > At the time I don't really know much about what needs to be
> >> changed to
> >> > make it happen, but I'll look into it. Suggestions welcome :)
> >> >
> >> >> I guess we could use a JAR format and use information in the
> >> manifest
> >> >> to describe the content of an archive
> >> >
> >> > Can you also be more precise about the information you want
> >> stored in
> >> > the manifest?
> >> > <a href="http://java.sun.com/j2se/1.5.0/docs/guide/jar/jar.html#JAR%">http://java.sun.com/j2se/1.5.0/docs/guide/jar/jar.html#JAR%
> >> 20Manifest
> >> > According to this it seems that a manifest can store many things.
> >> >
> >> > Regards,
> >> > Catalin
> >> >
> >> > On 1/31/07, Vincent Massol <[hidden email]> wrote:
> >> >> Hi Catalin,
> >> >>
> >> >> See my other email on this topic. Basically I think I agree with
> >> >> you but
> >> >> we're currently focusing on releasing XWiki 1.0 and fixing bugs.
> >> >> this is an
> >> >> improvement request that will have to wait for quite long unless
> >> >> someone
> >> >> contributes a patch for it (including patches for the
> >> >> documentation ;-)).
> >> >>
> >> >> Thanks
> >> >> -Vincent
> >> >>
> >> >>
> >> >> On Jan 8, 2007, at 11:32 AM, Catalin Hritcu wrote:
> >> >>
> >> >>
> >> >> The new way to import/export wiki pages is very useful. However, I
> >> >>  think that using the .xar extension for the zip(!) archive is
> >> highly
> >> >>  confusing. This because there is already a XAR(eXtensible
> >> ARchiver)
> >> >>  archiving utility which uses the .xar extension for quite a
> >> >> while. And
> >> >>  while now XAR is standalone, it will be quite probable used by
> >> the
> >> >>  MacOSX package system in the future. Then download your
> >> archive will
> >> >>  yield an error like this (safari automatically expands archives):
> >> >>
> >> >>  "Error opening xar archive: xwiki-1.0-beta-1.xar"
> >> >>
> >> >>  Links:
> >> >>  http://www.opendarwin.org/projects/xar/
> >> >>  http://www.nabble.com/xar-in-Mac-OS-X-t2081148.html
> >> >>
> >> >>
> >> >> <messagefooter.txt>
> >> >>
> >>
> >>
> >>
> >>
> >>
> >> _____________________________________________________________________
> >> ______
> >> Yahoo! Mail réinvente le mail ! Découvrez le nouveau Yahoo! Mail et
> >> son interface révolutionnaire.
> >> http://fr.mail.yahoo.com
> >>
> >>
> >>
> >> --
> >> You receive this message as a subscriber of the xwiki-
> >> [hidden email] mailing list.
> >> To unsubscribe: mailto:[hidden email]
> >> For general help: mailto:[hidden email]?subject=help
> >> ObjectWeb mailing lists service home page: http://
> >> www.objectweb.org/wws
> >>
> >>
> >
> >
> > --
> > jeremi
> >
> > --
> > You receive this message as a subscriber of the xwiki-
> > [hidden email] mailing list.
> > To unsubscribe: mailto:[hidden email]
> > For general help: mailto:[hidden email]?subject=help
> > ObjectWeb mailing lists service home page: http://www.objectweb.org/
> > wws
>
>
>
>
>
>
> ___________________________________________________________________________
> Yahoo! Mail réinvente le mail ! Découvrez le nouveau Yahoo! Mail et son interface révolutionnaire.
> http://fr.mail.yahoo.com
>
>
>
> --
> You receive this message as a subscriber of the [hidden email] mailing list.
> To unsubscribe: mailto:[hidden email]
> For general help: mailto:[hidden email]?subject=help
> ObjectWeb mailing lists service home page: http://www.objectweb.org/wws
>
>


--
You receive this message as a subscriber of the [hidden email] mailing list.
To unsubscribe: mailto:[hidden email]
For general help: mailto:[hidden email]?subject=help
ObjectWeb mailing lists service home page: http://www.objectweb.org/wws
Reply | Threaded
Open this post in threaded view
|

Re: .xar Extension NOT Used Properly

jeremi joslin-2
In reply to this post by vmassol
Hi,

On 2/3/07, Vincent Massol <[hidden email]> wrote:

> Hi Jeremi,
>
> I'm +1 on my side but as you voted -1 we'll need to convince you to
> change your vote ;-) Let me give it a try below...
>
> On Feb 3, 2007, at 9:41 AM, jeremi joslin wrote:
>
> > Hi,
> >
> > -1
> >
> > I'm not for changing to jar. it's not a Java archive, it's an XWiki
> > Archive. I don't see the point to change. People will think it's
> > javacode.
>
> * JAR is an archive format. It can be used for lots of different
> things. There are even software that use JARs for their installation.
yes, but a JAR in java term is a zip. There is also JAR compression
method. so if you go this way, it's the same problem.

> * I really believe that in the future our applications will not be
> made only of XML files but will also contain other libraries (jar
> files) and java classes. For example imagine the IRC bot application.
> There are some xwiki pages but we also need the picoirc jar. We'll
> need to bundle all that together.
And what's wrong with putting a jar inside a xar? even if we use a
jar, it will only be readable by XWiki.


> * XWiki is a Java application so all users are familiar with the JAR
> format.
But it's not a normal jar. That will lead to misunderstanding.

>
> Of course we can use the XAR format for that. XAR looks nice because
> there's XWiki inside but there's a cost of inventing a new archive
> format:
> * It needs to be described
> * There needs to be tools for it (or people need to rename it to
> zip...). BTW if it's a zip we might as well name it zip... The
> problem with Zip is that it does not support meta data information
> such as the jar format does. And we need these... And there are APIs
> to get these metadata so we don't have to reimplement all the code
> (as we've done for the package.xml file).
> * It makes it difficult for the build (this is the same argument as
> above with tools) as all the existing plugins do not understand the
> XAR format so we have to reinvent all the tools... This is why I had
> to implement the xar handler and the xar plugin in our repository. It
> would make it much easier if it were an existing format such as JAR.
Why not renaming the xar to the zip. it's just that people don't need
to know what is inside, but if they want they can.

I dont see the point, we are using an API for reading and writing the
metadata: XML. We will also need to reimplement the writing of
metadata in the manifest file.

So it's already done isn't it?



>
> I think it would be much better to use an existing format (which we
> do except we're using Zip instead of JAR), already documented rather
> than invent a new one. There's no point in us hiding the underlying
> format we use by changing the extension. We might as well use that
> format's extension...
Ok, rename it to zip if you prefer, that's only a problem the
extension to be xar  for openning it under windows. But i prefer
people to think it's a file where they have all there pages, and don't
need to read what is inside. But if they want, they can.


>
> > if you look at this page :
> > http://filext.com/detaillist.php?extdetail=XAR there is some other
> > software who are using this extension. So why XWiki can't use it?
>
> Some people have thrown nuclear bombs in the past. Is that a good
> reason for others to do it? Probably not.
>
> It's not because people are doing bad things that we have to follow
> them ;-)
>
So why changing to the jar format? ;-)
And even the xar format is not so used. I ask to my friends, no one
know it. Did you know it before? I thing changing to the jar format
will bring more misunderstanding.

jeremi



--
Jeremi Joslin (http://www.jeremi.info)
skype: jeremi23 - jabber: [hidden email]
http://www.xwiki.com - http://www.pengyou-project.info



--
You receive this message as a subscriber of the [hidden email] mailing list.
To unsubscribe: mailto:[hidden email]
For general help: mailto:[hidden email]?subject=help
ObjectWeb mailing lists service home page: http://www.objectweb.org/wws
Reply | Threaded
Open this post in threaded view
|

Re: .xar Extension NOT Used Properly

vmassol
Administrator
Hi Jeremi,

On Feb 4, 2007, at 4:44 PM, jeremi joslin wrote:

> On 2/3/07, Vincent Massol <[hidden email]> wrote:
>> Hi Jeremi,
>>
>> I'm +1 on my side but as you voted -1 we'll need to convince you to
>> change your vote ;-) Let me give it a try below...
>>
>> On Feb 3, 2007, at 9:41 AM, jeremi joslin wrote:
>>
>> > Hi,
>> >
>> > -1
>> >
>> > I'm not for changing to jar. it's not a Java archive, it's an XWiki
>> > Archive. I don't see the point to change. People will think it's
>> > javacode.
>>
>> * JAR is an archive format. It can be used for lots of different
>> things. There are even software that use JARs for their installation.
> yes, but a JAR in java term is a zip. There is also JAR compression
> method. so if you go this way, it's the same problem.
A JAR isn't a Zip in Java term. It's using the same compression  
algorithm. It's very different in term of metadata. I won't go in  
more details but you can check the JAR specification to know more  
about this.
>
>> * I really believe that in the future our applications will not be
>> made only of XML files but will also contain other libraries (jar
>> files) and java classes. For example imagine the IRC bot application.
>> There are some xwiki pages but we also need the picoirc jar. We'll
>> need to bundle all that together.
> And what's wrong with putting a jar inside a xar? even if we use a
> jar, it will only be readable by XWiki.

It's possible but harder. For example to read a JAR inside a XAR  
you'll probably need to write and register special handlers against  
various classes (such as URL factories, etc).

No, xwiki isn't the only one. All tools will be able to read a JAR or  
a ZIP (be them IDE, build tools, desktop tools, etc).

>> * XWiki is a Java application so all users are familiar with the JAR
>> format.
> But it's not a normal jar. That will lead to misunderstanding.

I'm not sure what you call a normal jar :-) You probably means  
classes. However a JAR is not meant to contain only classes. It's a  
generic format to contain any other kind of resources. The nice thing  
it has though is support for dependencies and class-paths.

>> Of course we can use the XAR format for that. XAR looks nice because
>> there's XWiki inside but there's a cost of inventing a new archive
>> format:
>> * It needs to be described
>> * There needs to be tools for it (or people need to rename it to
>> zip...). BTW if it's a zip we might as well name it zip... The
>> problem with Zip is that it does not support meta data information
>> such as the jar format does. And we need these... And there are APIs
>> to get these metadata so we don't have to reimplement all the code
>> (as we've done for the package.xml file).
>> * It makes it difficult for the build (this is the same argument as
>> above with tools) as all the existing plugins do not understand the
>> XAR format so we have to reinvent all the tools... This is why I had
>> to implement the xar handler and the xar plugin in our repository. It
>> would make it much easier if it were an existing format such as JAR.
>
> Why not renaming the xar to the zip. it's just that people don't need
> to know what is inside, but if they want they can.
>
As I said renaming to zip is a solution. Probably a better one than  
the current XAR. However the pity is that a zip doesn't support  
metadata by default. You need to define them, document them, teach  
them, write parsers for them, etc. JARs have this already (this is  
called the Manifest).

> I dont see the point, we are using an API for reading and writing the
> metadata: XML. We will also need to reimplement the writing of
> metadata in the manifest file.

No. The JDK provides an API for reading information from the manifest.

>
> So it's already done isn't it?

And it's possibly buggy :-) And you're the only one who knows about  
it. And it isn't documented. And you haven't implemented it for  
Maven2 (it's not easy and has some issues) nor any of the tool that  
will crop up.

The question is not that it's done (it's not finished BTW - No doc,  
no tests). It's about removing code and using standards (to reduce  
cost even further). Any line of code that is removed saves a lot  
(fixing bug, documentation, test, teaching, etc).

>>
>> I think it would be much better to use an existing format (which we
>> do except we're using Zip instead of JAR), already documented rather
>> than invent a new one. There's no point in us hiding the underlying
>> format we use by changing the extension. We might as well use that
>> format's extension...
> Ok, rename it to zip if you prefer, that's only a problem the
> extension to be xar  for openning it under windows

No that's not true. It's a problem for a lot of tools (Maven2  
included). BTW I'm on Mac OS X (which isn't windows) and I can't open  
a XAR either. I have to rename it to ZIP before... Unfortunately  
there are very few tools that look inside a file to know what format  
it is. Most will simply look at the file extension.

> . But i prefer
> people to think it's a file where they have all there pages, and don't
> need to read what is inside. But if they want, they can.

Sure that's ideal. Problem is that you always have to do this... For  
example there's no selective export so you have to expand it as a  
zip, remove files, edit the package.xml file and repackage.

>> > if you look at this page :
>> > http://filext.com/detaillist.php?extdetail=XAR there is some other
>> > software who are using this extension. So why XWiki can't use it?
>>
>> Some people have thrown nuclear bombs in the past. Is that a good
>> reason for others to do it? Probably not.
>>
>> It's not because people are doing bad things that we have to follow
>> them ;-)
>>
> So why changing to the jar format? ;-)
Oh, because I think it's an improvement (it certainly is at least for  
issues I have with Maven2 and it'll win us hundreds of lines of  
code). See below for explanations on lightweight container support.

> And even the xar format is not so used. I ask to my friends, no one
> know it. Did you know it before? I thing changing to the jar format
> will bring more misunderstanding.

I didn't know about the other XAR format and it wouldn't be a good  
choice (not known enough and thus not enough tool support).

I'm not going to push for the format change because we have a lot of  
work to be done right now and I'm not proposing to do it myself  
either. What I was just saying here is that I agree that inventing a  
format is probably not the best solution and reusing an existing well-
known format with metadata support is probably a better choice. JAR  
is such a choice. It looks to me like a good choice, especially  
considering that we have a need to put java classes in applications.

I see you're against it. That's fine. It means it's not a clear cut  
decision and we should think more about it. In any case as I said  
above we have more urgent problems. We'll need to revisit this when  
we move to full fledge application support. These will contain:

- pages
- java classes (plugins, macros, etc)
- external libs
- resources (HTML, js files, images, etc).

For example a GWT XWiki application will contain java and resources +  
the necessary pages and possibly external libs (we'll probably bundle  
them though). The IRC Bot will require an external lib for sure.

We need to revisit this in view with the new architecture (component-
based architecture based on a microkernel, aka lightweight  
container). You'll see that most of these containers use the JAR  
format (OSGi, Plexus, etc) so we'll probably have to use the JAR  
format anyway... :-)

Thanks
-Vincent


       

       
               
___________________________________________________________________________
Yahoo! Mail réinvente le mail ! Découvrez le nouveau Yahoo! Mail et son interface révolutionnaire.
http://fr.mail.yahoo.com



--
You receive this message as a subscriber of the [hidden email] mailing list.
To unsubscribe: mailto:[hidden email]
For general help: mailto:[hidden email]?subject=help
ObjectWeb mailing lists service home page: http://www.objectweb.org/wws
Reply | Threaded
Open this post in threaded view
|

Re: .xar Extension NOT Used Properly

Catalin Hritcu
In reply to this post by jeremi joslin-2
Hi Jeremi,

On 2/4/07, jeremi joslin <[hidden email]> wrote:

> Hi,
>
> On 2/3/07, Vincent Massol <[hidden email]> wrote:
> > Hi Jeremi,
> >
> > I'm +1 on my side but as you voted -1 we'll need to convince you to
> > change your vote ;-) Let me give it a try below...
> >
> > On Feb 3, 2007, at 9:41 AM, jeremi joslin wrote:
> >
> > > Hi,
> > >
> > > -1
> > >
> > > I'm not for changing to jar. it's not a Java archive, it's an XWiki
> > > Archive. I don't see the point to change. People will think it's
> > > javacode.
> >
> > * JAR is an archive format. It can be used for lots of different
> > things. There are even software that use JARs for their installation.
> yes, but a JAR in java term is a zip. There is also JAR compression
> method. so if you go this way, it's the same problem.
>
According to the JAR File Specification "A JAR file is essentially a
zip file that contains an optional META-INF directory". So can you
please explain what other compression format there is for JARs ? I
never heard of any others.


> > * I really believe that in the future our applications will not be
> > made only of XML files but will also contain other libraries (jar
> > files) and java classes. For example imagine the IRC bot application.
> > There are some xwiki pages but we also need the picoirc jar. We'll
> > need to bundle all that together.
> And what's wrong with putting a jar inside a xar? even if we use a
> jar, it will only be readable by XWiki.
>
>
> > * XWiki is a Java application so all users are familiar with the JAR
> > format.
> But it's not a normal jar. That will lead to misunderstanding.
>
Why would it be ... abnormal ? I don't understand.

> >
> > Of course we can use the XAR format for that. XAR looks nice because
> > there's XWiki inside but there's a cost of inventing a new archive
> > format:
> > * It needs to be described
> > * There needs to be tools for it (or people need to rename it to
> > zip...). BTW if it's a zip we might as well name it zip... The
> > problem with Zip is that it does not support meta data information
> > such as the jar format does. And we need these... And there are APIs
> > to get these metadata so we don't have to reimplement all the code
> > (as we've done for the package.xml file).
> > * It makes it difficult for the build (this is the same argument as
> > above with tools) as all the existing plugins do not understand the
> > XAR format so we have to reinvent all the tools... This is why I had
> > to implement the xar handler and the xar plugin in our repository. It
> > would make it much easier if it were an existing format such as JAR.
>
> Why not renaming the xar to the zip. it's just that people don't need
> to know what is inside, but if they want they can.
>
Vincent already provided the answer above. Because you need metadata
and zip does not directly have support for it.

> I dont see the point, we are using an API for reading and writing the
> metadata: XML. We will also need to reimplement the writing of
> metadata in the manifest file.
>
> So it's already done isn't it?
>
It is part of the standard library:
http://java.sun.com/j2se/1.5.0/docs/api/java/util/jar/Manifest.html

> >
> > I think it would be much better to use an existing format (which we
> > do except we're using Zip instead of JAR), already documented rather
> > than invent a new one. There's no point in us hiding the underlying
> > format we use by changing the extension. We might as well use that
> > format's extension...
> Ok, rename it to zip if you prefer, that's only a problem the
> extension to be xar  for openning it under windows. But i prefer
> people to think it's a file where they have all there pages, and don't
> need to read what is inside. But if they want, they can.
>
Renaming it to zip might be an option. I think not the best one for
the long term, but surely it would not be causing any confusion now.

>
> >
> > > if you look at this page :
> > > http://filext.com/detaillist.php?extdetail=XAR there is some other
> > > software who are using this extension. So why XWiki can't use it?
> >
> > Some people have thrown nuclear bombs in the past. Is that a good
> > reason for others to do it? Probably not.
> >
> > It's not because people are doing bad things that we have to follow
> > them ;-)
> >
> So why changing to the jar format? ;-)
So I see you don't really like JAR :)  Nevertheless, JAR is a widely
used, established format, with a stable specification
(http://java.sun.com/j2se/1.5.0/docs/guide/jar/jar.html) -- as opposed
to your Xwiki ARchive, which is neither established, nor documented,
and to make things worse the extension XAR is already used for
different kind of archives.

Standalone tools for extracting and creating JAR's are part of the
standard JDK -- there are no standalone tools for Xwiki ARchives,
unless you build some. If you just use some zip archiver then you will
have first to realise that the zip compression algorithm is used (XAR
files are binary, and have a strange extension which if you Google for
leeds to eXtensible ARchiver -- so quite confusing for your users).
Then still you need to know how to create the package.xml if you want
to create a XAR file. Sure, it is possible to create XARs using the
Export facility in XWiki, but that's the only easy way and is limited
to pages in your wiki, so no XWiki Applications ... just collection of
pages.

Also the API for accessing and modifying metadata in JAR files is part
of the Java standard library, is stable and well understood.

To sum up, JAR is an established format which is very well suited for
your problem. Do you really want to reinvent the wheel?

> And even the xar format is not so used. I ask to my friends, no one
> know it. Did you know it before?
>
You are right, eXtensible ARchiver is not widely used at this time.
But it will be quite probable used by the MacOSX package system in the
future (http://www.nabble.com/xar-in-Mac-OS-X-t2081148.html). So
almost all software written for the Mac might come one day in this
format.

> I thing changing to the jar format
> will bring more misunderstanding.
>
Can you be more specific here. Where exactly would confusion arise ?

Regards,
Catalin



--
You receive this message as a subscriber of the [hidden email] mailing list.
To unsubscribe: mailto:[hidden email]
For general help: mailto:[hidden email]?subject=help
ObjectWeb mailing lists service home page: http://www.objectweb.org/wws
Reply | Threaded
Open this post in threaded view
|

Re: .xar Extension NOT Used Properly

jeremi joslin-2
ok 0, but just not to block the decision.

do as you want.

but  I don't see the difference between writing in an XML file or a
manifest. You will both have to document it. I don't like manifest
because AFAIK it's a plain text file and not an xml. You will have
other problems I think.



> A JAR isn't a Zip in Java term. It's using the same compression
> algorithm. It's very different in term of metadata. I won't go in
> more details but you can check the JAR specification to know more
> about this.
A xar is not a zip in XWiki term :-)

> It's possible but harder. For example to read a JAR inside a XAR
> you'll probably need to write and register special handlers against
> various classes (such as URL factories, etc).
XAR is just an IMPORT/EXPORT format so you will not need this. you
just need to extract the jar from it!

> And it's possibly buggy :-) And you're the only one who knows about
> it. And it isn't documented. And you haven't implemented it for
> Maven2 (it's not easy and has some issues) nor any of the tool that
> will crop up.
No, xwiki isn't the only one. All tools will be able to read a JAR or
a ZIP (be them IDE, build tools, desktop tools, etc).
When I've done we were not working with maven, but we were with ant.
and it's implemented in ant (by ludovic).






I'm not he only one who know how the xar is working.

> For example there's no selective export so you have to expand it as a
> zip, remove files, edit the package.xml file and repackage.
No, I've done one, and Vincent you know it.

...
that can be a long discussion. Do as you want, i will not argue
anymore on this.

jeremi


--
Jeremi Joslin (http://www.jeremi.info)
skype: jeremi23 - jabber: [hidden email]
http://www.xwiki.com - http://www.pengyou-project.info



--
You receive this message as a subscriber of the [hidden email] mailing list.
To unsubscribe: mailto:[hidden email]
For general help: mailto:[hidden email]?subject=help
ObjectWeb mailing lists service home page: http://www.objectweb.org/wws
Reply | Threaded
Open this post in threaded view
|

Re: .xar Extension NOT Used Properly

vmassol
Administrator

On Feb 4, 2007, at 7:36 PM, jeremi joslin wrote:

[snip]

>> For example there's no selective export so you have to expand it as a
>> zip, remove files, edit the package.xml file and repackage.
> No, I've done one, and Vincent you know it.

Oh yes I know but Ludovic has removed it when we moved to using  
templates so it needs to be done again. Hopefully it can be done  
quickly but we still need to re-implement it.

And the process I've described above is exactly what I do at each  
release since Beta 1.

> ...
> that can be a long discussion. Do as you want, i will not argue
> anymore on this.

Arguing is good and healthy. As I said in my previous email we've  
reached a point where we *all* need to step back and think more about  
this. IMO seen the work we have to do for XWiki 1.0 it's better to  
postpone this post 1.0 release and think about it in the context of  
the new architecture.

Thanks for the discussion Jeremi
-Vincent

       

       
               
___________________________________________________________________________
Yahoo! Mail réinvente le mail ! Découvrez le nouveau Yahoo! Mail et son interface révolutionnaire.
http://fr.mail.yahoo.com



--
You receive this message as a subscriber of the [hidden email] mailing list.
To unsubscribe: mailto:[hidden email]
For general help: mailto:[hidden email]?subject=help
ObjectWeb mailing lists service home page: http://www.objectweb.org/wws
Reply | Threaded
Open this post in threaded view
|

Re: .xar Extension NOT Used Properly

jeremi joslin-2
On 2/4/07, Vincent Massol <[hidden email]> wrote:

>
> On Feb 4, 2007, at 7:36 PM, jeremi joslin wrote:
>
> [snip]
>
> >> For example there's no selective export so you have to expand it as a
> >> zip, remove files, edit the package.xml file and repackage.
> > No, I've done one, and Vincent you know it.
>
> Oh yes I know but Ludovic has removed it when we moved to using
> templates so it needs to be done again. Hopefully it can be done
> quickly but we still need to re-implement it.
>
> And the process I've described above is exactly what I do at each
> release since Beta 1.
>
> > ...
> > that can be a long discussion. Do as you want, i will not argue
> > anymore on this.
>
> Arguing is good and healthy. As I said in my previous email we've
> reached a point where we *all* need to step back and think more about
> this. IMO seen the work we have to do for XWiki 1.0 it's better to
> postpone this post 1.0 release and think about it in the context of
> the new architecture.
Yes, but sometime it's lose of time. We have more important things for
now to work on. I agree we will have to step back as you say after
1.0.

jeremi

--
Jeremi Joslin (http://www.jeremi.info)
skype: jeremi23 - jabber: [hidden email]
http://www.xwiki.com - http://www.pengyou-project.info



--
You receive this message as a subscriber of the [hidden email] mailing list.
To unsubscribe: mailto:[hidden email]
For general help: mailto:[hidden email]?subject=help
ObjectWeb mailing lists service home page: http://www.objectweb.org/wws
Reply | Threaded
Open this post in threaded view
|

Number of document decrease XWiki performance

Youcef Bey-2
In reply to this post by vmassol
Hi all,

I have until now not solved this problem (since more than 10 month):

- using this program to create automatically a document work well for few
documents:

   XW = xwiki.XWiki;
   XWContext = context.context;
   XWikiDocument NewEntry = new XWikiDocument(request.getParameter("web"),
SourceValue);
   NewEntry.setContent(EntryContent);
  XW.saveDocument(NewEntry, XWContext);


When putting these instructions in a LOOP for 10000 entries to transfer from
another TABLE in the same XWiki database cause a problem. The probleme is
not an error but the XWiki crash! and without finishing the LOOP, the xwiki
display error "out of memory!"

But now, taking the same program and run it in the shell (javac, java) to
transfert these 10000 entries to TABLE in the XWiki database, it's done in
less 1 minute?

Strange for me ...

- More ...and this is the more importante thing :

  One time I success to transfer 25 000 documents in the XWiki database
(using the same script above) the XWiki become very heavy and cannot really
work with it at all.

MySql tables are supposed to contain millions of records, and when using the
database adminstrator and check with SQL statment, I get resultat in few
seconds, that means that the problem is in XWiki.

My Question : Why with 25 000 documents XWiki don't work as usually

Anyone faced this problem before? Do you have made testing of XWiki for 1
millions documents for exemple?

I have XP, 1.2 MHz and 500 Mo but the following experience on MySQL gives
more arguments that not MySQL is causing this problem :

(Web link :
http://www.guilde.asso.fr/lurker/message/20050817.183426.a1b8e267.en.html)

-----------------------------------------------------------------
 Il y a quelques années, j'ai utilisé MySQL pour gérér 1.7 millions
d'enregistrements dans une table. La table faisait environ 500Mo, plus
400Mo d'indexes pour la recherche de textes dans un des champs.

    Coté perfs, sur un ordinateur portable à 500Mhz et 128Mo de ram qui
faisait tourner un Windows NT4 bourré d'applis en mémoire, il fallait
environ 5s à 10s pour recherche les tuples contenant certains mots-clefs
stokés dans le fameux champs de texte.

    Bref, des performances tout à fait acceptable, sur une machine qui
n'était pourtant pas du tout fait pour faire serveur de base de données
(et je ne parle pas que de l'OS...).
--------------------------------------------------------------

Many thanks for any suggestion.

Best wishes
----------------------------------
Youcef Bey




--
You receive this message as a subscriber of the [hidden email] mailing list.
To unsubscribe: mailto:[hidden email]
For general help: mailto:[hidden email]?subject=help
ObjectWeb mailing lists service home page: http://www.objectweb.org/wws
Reply | Threaded
Open this post in threaded view
|

Re: Number of document decrease XWiki performance

vmassol
Administrator
Hi Youcef,

(Please do not hijack existing thread, this is bad practice... :-)).

Regarding your issue you have to set up indexes if you're working  
with big tables (these are not set automatically as of now).

I'm not sure exactly what indexes need to be set but here's some idea  
of what should be done (people in the know please confirm):

create index xwd_name on xwikidoc (xwd_name);
create index xwd_fullname on xwikidoc (xwd_fullname);
create index xwd_web on xwikidoc (xwd_web);
create index xwd_name on xwikiobjects (xwo_name);
create index xwl_value on xwikilongs (xwl_value);
create index xwi_value on xwikiintegers (xwi_value);
create index xws_value on xwikistrings (xws_value);
create index xwl_value on xwikilargestrings (xwl_value(50));
create index xwo_classname on xwikiobjects (xwo_classname);
create index xwd_creation_date on xwikidoc (xwd_creation_date);
create index xwd_date on xwikidoc (xwd_date);
create index xwd_content_update_date on xwikidoc  
(xwd_content_update_date);
create index xwd_content_author on xwikidoc (xwd_content_author);
create index xwd_author on xwikidoc (xwd_author);
create index xwd_creator on xwikidoc (xwd_creator);
create index xwd_language on xwikidoc (xwd_language);
create index xwd_default_language on xwikidoc (xwd_default_language);
create index xwd_title on xwikidoc (xwd_title);
create index xwd_parent on xwikidoc (xwd_parent(50));

Note that we aready have a jira issue for this (http://jira.xwiki.org/ 
jira/browse/XWIKI-605). We just don't know yet if we can have a  
generic script that'll work for all databases.

There might be other issues but I'll let others with experience with  
big XWiki instances answer...

Thanks
-Vincent

On Feb 7, 2007, at 3:00 AM, Youcef BEY wrote:

> Hi all,
>
> I have until now not solved this problem (since more than 10 month):
>
> - using this program to create automatically a document work well  
> for few documents:
>
>   XW = xwiki.XWiki;
>   XWContext = context.context;
>   XWikiDocument NewEntry = new XWikiDocument(request.getParameter
> ("web"), SourceValue);
>   NewEntry.setContent(EntryContent);
>  XW.saveDocument(NewEntry, XWContext);
>
>
> When putting these instructions in a LOOP for 10000 entries to  
> transfer from another TABLE in the same XWiki database cause a  
> problem. The probleme is not an error but the XWiki crash! and  
> without finishing the LOOP, the xwiki display error "out of memory!"
>
> But now, taking the same program and run it in the shell (javac,  
> java) to transfert these 10000 entries to TABLE in the XWiki  
> database, it's done in less 1 minute?
>
> Strange for me ...
>
> - More ...and this is the more importante thing :
>
>  One time I success to transfer 25 000 documents in the XWiki  
> database (using the same script above) the XWiki become very heavy  
> and cannot really work with it at all.
>
> MySql tables are supposed to contain millions of records, and when  
> using the database adminstrator and check with SQL statment, I get  
> resultat in few seconds, that means that the problem is in XWiki.
>
> My Question : Why with 25 000 documents XWiki don't work as usually
>
> Anyone faced this problem before? Do you have made testing of XWiki  
> for 1 millions documents for exemple?
>
> I have XP, 1.2 MHz and 500 Mo but the following experience on MySQL  
> gives more arguments that not MySQL is causing this problem :
>
> (Web link : http://www.guilde.asso.fr/lurker/message/ 
> 20050817.183426.a1b8e267.en.html)
>
> -----------------------------------------------------------------
> Il y a quelques années, j'ai utilisé MySQL pour gérér 1.7 millions
> d'enregistrements dans une table. La table faisait environ 500Mo, plus
> 400Mo d'indexes pour la recherche de textes dans un des champs.
>
>    Coté perfs, sur un ordinateur portable à 500Mhz et 128Mo de ram qui
> faisait tourner un Windows NT4 bourré d'applis en mémoire, il fallait
> environ 5s à 10s pour recherche les tuples contenant certains mots-
> clefs
> stokés dans le fameux champs de texte.
>
>    Bref, des performances tout à fait acceptable, sur une machine qui
> n'était pourtant pas du tout fait pour faire serveur de base de  
> données
> (et je ne parle pas que de l'OS...).
> --------------------------------------------------------------
>
> Many thanks for any suggestion.
>
> Best wishes
> ----------------------------------
> Youcef Bey
>
>
> --
> You receive this message as a subscriber of the xwiki-
> [hidden email] mailing list.
> To unsubscribe: mailto:[hidden email]
> For general help: mailto:[hidden email]?subject=help
> ObjectWeb mailing lists service home page: http://www.objectweb.org/ 
> wws





___________________________________________________________________________
Yahoo! Mail réinvente le mail ! Découvrez le nouveau Yahoo! Mail et son interface révolutionnaire.
http://fr.mail.yahoo.com



--
You receive this message as a subscriber of the [hidden email] mailing list.
To unsubscribe: mailto:[hidden email]
For general help: mailto:[hidden email]?subject=help
ObjectWeb mailing lists service home page: http://www.objectweb.org/wws
Reply | Threaded
Open this post in threaded view
|

Re: Number of document decrease XWiki performance

jeremi joslin-2
In reply to this post by Youcef Bey-2
Hi Youcef,
Do you have objects in your pages?

jeremi

On 2/7/07, Youcef BEY <[hidden email]> wrote:

> Hi all,
>
> I have until now not solved this problem (since more than 10 month):
>
> - using this program to create automatically a document work well for few
> documents:
>
>    XW = xwiki.XWiki;
>    XWContext = context.context;
>    XWikiDocument NewEntry = new XWikiDocument(request.getParameter("web"),
> SourceValue);
>    NewEntry.setContent(EntryContent);
>   XW.saveDocument(NewEntry, XWContext);
>
>
> When putting these instructions in a LOOP for 10000 entries to transfer from
> another TABLE in the same XWiki database cause a problem. The probleme is
> not an error but the XWiki crash! and without finishing the LOOP, the xwiki
> display error "out of memory!"
>
> But now, taking the same program and run it in the shell (javac, java) to
> transfert these 10000 entries to TABLE in the XWiki database, it's done in
> less 1 minute?
>
> Strange for me ...
>
> - More ...and this is the more importante thing :
>
>   One time I success to transfer 25 000 documents in the XWiki database
> (using the same script above) the XWiki become very heavy and cannot really
> work with it at all.
>
> MySql tables are supposed to contain millions of records, and when using the
> database adminstrator and check with SQL statment, I get resultat in few
> seconds, that means that the problem is in XWiki.
>
> My Question : Why with 25 000 documents XWiki don't work as usually
>
> Anyone faced this problem before? Do you have made testing of XWiki for 1
> millions documents for exemple?
>
> I have XP, 1.2 MHz and 500 Mo but the following experience on MySQL gives
> more arguments that not MySQL is causing this problem :
>
> (Web link :
> http://www.guilde.asso.fr/lurker/message/20050817.183426.a1b8e267.en.html)
>
> -----------------------------------------------------------------
>  Il y a quelques années, j'ai utilisé MySQL pour gérér 1.7 millions
> d'enregistrements dans une table. La table faisait environ 500Mo, plus
> 400Mo d'indexes pour la recherche de textes dans un des champs.
>
>     Coté perfs, sur un ordinateur portable à 500Mhz et 128Mo de ram qui
> faisait tourner un Windows NT4 bourré d'applis en mémoire, il fallait
> environ 5s à 10s pour recherche les tuples contenant certains mots-clefs
> stokés dans le fameux champs de texte.
>
>     Bref, des performances tout à fait acceptable, sur une machine qui
> n'était pourtant pas du tout fait pour faire serveur de base de données
> (et je ne parle pas que de l'OS...).
> --------------------------------------------------------------
>
> Many thanks for any suggestion.
>
> Best wishes
> ----------------------------------
> Youcef Bey
>
>
>
>
> --
> You receive this message as a subscriber of the [hidden email] mailing list.
> To unsubscribe: mailto:[hidden email]
> For general help: mailto:[hidden email]?subject=help
> ObjectWeb mailing lists service home page: http://www.objectweb.org/wws
>
>

--
Jeremi Joslin (http://www.jeremi.info)
skype: jeremi23 - jabber: [hidden email]
http://www.xwiki.com - http://www.pengyou-project.info


--
You receive this message as a subscriber of the [hidden email] mailing list.
To unsubscribe: mailto:[hidden email]
For general help: mailto:[hidden email]?subject=help
ObjectWeb mailing lists service home page: http://www.objectweb.org/wws
Reply | Threaded
Open this post in threaded view
|

Re: Number of document decrease XWiki performance

Sébastien Gaïde
In reply to this post by Youcef Bey-2
Youcef BEY a écrit :
> Hi all,

Hi Youcef,

> - using this program to create automatically a document work well for
> few documents:
>
>   XW = xwiki.XWiki;
>   XWContext = context.context;
>   XWikiDocument NewEntry = new
> XWikiDocument(request.getParameter("web"), SourceValue);
>   NewEntry.setContent(EntryContent);
>  XW.saveDocument(NewEntry, XWContext);

could you please send us the real script you are using ? (I mean as much
'real' as possible).

> LOOP, the xwiki display error "out of memory!"

have you got a stack trace ?

> But now, taking the same program and run it in the shell (javac, java)
> to transfert these 10000 entries to TABLE in the XWiki database, it's
> done in less 1 minute?

could you please send us the source of this ?

I'm generating wikis automatically using XWiki apis, the biggest wiki
created hold 165 000 pages (each page containing between one to 5
objects). the only problem I got was about the archive handling that I
had to disable (which is not a problem since the pages are generated and
not edited), this was with an old XWiki version (svn 1226), since then
the archive system relies on another framework, so this may be not an
issue anymore (we are still using the same XWiki version).

I'm using only 4 indexes (I will post a reply tomorrow with the indexed
columns, I don't remember exactly and I can't access this information
right now). These indexes have greatly improved response time (at first
the biggest page (2,5 MB!) took 25 minutes to render ;-) after indexes
creation, and some other tweaking, it took only a few seconds).

indexes will improve response time, but it will not protect you from an
out of memory exception.

if you can send the scripts you are using to the list, I will be able to
run them on our wikis to see the result ...

Seb.




--
You receive this message as a subscriber of the [hidden email] mailing list.
To unsubscribe: mailto:[hidden email]
For general help: mailto:[hidden email]?subject=help
ObjectWeb mailing lists service home page: http://www.objectweb.org/wws
Reply | Threaded
Open this post in threaded view
|

Re: Number of document decrease XWiki performance

Sébastien Gaïde
Here are the indexes I'm using :

create index doc_fullname on xwikidoc (XWD_FULLNAME);
create index doc_web_name on xwikidoc (XWD_WEB, XWD_NAME);
create index object_name on xwikiobjects (XWO_NAME);
create index object_classname on xwikiobjects (XWO_CLASSNAME);

Seb.



--
You receive this message as a subscriber of the [hidden email] mailing list.
To unsubscribe: mailto:[hidden email]
For general help: mailto:[hidden email]?subject=help
ObjectWeb mailing lists service home page: http://www.objectweb.org/wws
Reply | Threaded
Open this post in threaded view
|

Re: Number of document decrease XWiki performance

Youcef Bey-2
In reply to this post by vmassol
Hi Vincent, Jeremi and Sebastien

Thank you very much for your reply and I apologize for late reply.

Your solution is not really enough for sloving the problem. Sorry for that.

 In fact, I created all the index so it seems that xwiki increase a little
the speed but not really enough.

I have compilted for testing it 10000 but xwiki become unusable!

I did not use class in my document. The format of document is textual and
small size (<2 Ko)

Sebastien, I will send you by tomorrow a file containing 350000 entries and
all programs. As now I am transfering all my data and XWiki to Vine Linux
(Debian) and also decided to leave Win XP for ever, I really enjoyed Linux
:-)

When I installed XWiki on my Linux machine (Vine Linux) I have got an error.
Even I have an experience with XWiki but this error can't find any issues
for solving it. I prefer to send it on another email

Thank again.

Youcef

From: "Vincent Massol" <[hidden email]>
To: <[hidden email]>
Sent: Tuesday, February 06, 2007 1:22 AM
Subject: Re: [xwiki-dev] Number of document decrease XWiki performance


Hi Youcef,

(Please do not hijack existing thread, this is bad practice... :-)).

Regarding your issue you have to set up indexes if you're working
with big tables (these are not set automatically as of now).

I'm not sure exactly what indexes need to be set but here's some idea
of what should be done (people in the know please confirm):

create index xwd_name on xwikidoc (xwd_name);
create index xwd_fullname on xwikidoc (xwd_fullname);
create index xwd_web on xwikidoc (xwd_web);
create index xwd_name on xwikiobjects (xwo_name);
create index xwl_value on xwikilongs (xwl_value);
create index xwi_value on xwikiintegers (xwi_value);
create index xws_value on xwikistrings (xws_value);
create index xwl_value on xwikilargestrings (xwl_value(50));
create index xwo_classname on xwikiobjects (xwo_classname);
create index xwd_creation_date on xwikidoc (xwd_creation_date);
create index xwd_date on xwikidoc (xwd_date);
create index xwd_content_update_date on xwikidoc
(xwd_content_update_date);
create index xwd_content_author on xwikidoc (xwd_content_author);
create index xwd_author on xwikidoc (xwd_author);
create index xwd_creator on xwikidoc (xwd_creator);
create index xwd_language on xwikidoc (xwd_language);
create index xwd_default_language on xwikidoc (xwd_default_language);
create index xwd_title on xwikidoc (xwd_title);
create index xwd_parent on xwikidoc (xwd_parent(50));

Note that we aready have a jira issue for this (http://jira.xwiki.org/
jira/browse/XWIKI-605). We just don't know yet if we can have a
generic script that'll work for all databases.

There might be other issues but I'll let others with experience with
big XWiki instances answer...

Thanks
-Vincent

On Feb 7, 2007, at 3:00 AM, Youcef BEY wrote:

> Hi all,
>
> I have until now not solved this problem (since more than 10 month):
>
> - using this program to create automatically a document work well
> for few documents:
>
>   XW = xwiki.XWiki;
>   XWContext = context.context;
>   XWikiDocument NewEntry = new XWikiDocument(request.getParameter
> ("web"), SourceValue);
>   NewEntry.setContent(EntryContent);
>  XW.saveDocument(NewEntry, XWContext);
>
>
> When putting these instructions in a LOOP for 10000 entries to
> transfer from another TABLE in the same XWiki database cause a
> problem. The probleme is not an error but the XWiki crash! and
> without finishing the LOOP, the xwiki display error "out of memory!"
>
> But now, taking the same program and run it in the shell (javac,
> java) to transfert these 10000 entries to TABLE in the XWiki
> database, it's done in less 1 minute?
>
> Strange for me ...
>
> - More ...and this is the more importante thing :
>
>  One time I success to transfer 25 000 documents in the XWiki
> database (using the same script above) the XWiki become very heavy
> and cannot really work with it at all.
>
> MySql tables are supposed to contain millions of records, and when
> using the database adminstrator and check with SQL statment, I get
> resultat in few seconds, that means that the problem is in XWiki.
>
> My Question : Why with 25 000 documents XWiki don't work as usually
>
> Anyone faced this problem before? Do you have made testing of XWiki
> for 1 millions documents for exemple?
>
> I have XP, 1.2 MHz and 500 Mo but the following experience on MySQL
> gives more arguments that not MySQL is causing this problem :
>
> (Web link : http://www.guilde.asso.fr/lurker/message/
> 20050817.183426.a1b8e267.en.html)
>
> -----------------------------------------------------------------
> Il y a quelques années, j'ai utilisé MySQL pour gérér 1.7 millions
> d'enregistrements dans une table. La table faisait environ 500Mo, plus
> 400Mo d'indexes pour la recherche de textes dans un des champs.
>
>    Coté perfs, sur un ordinateur portable à 500Mhz et 128Mo de ram qui
> faisait tourner un Windows NT4 bourré d'applis en mémoire, il fallait
> environ 5s à 10s pour recherche les tuples contenant certains mots-
> clefs
> stokés dans le fameux champs de texte.
>
>    Bref, des performances tout à fait acceptable, sur une machine qui
> n'était pourtant pas du tout fait pour faire serveur de base de
> données
> (et je ne parle pas que de l'OS...).
> --------------------------------------------------------------
>
> Many thanks for any suggestion.
>
> Best wishes
> ----------------------------------
> Youcef Bey
>
>
> --
> You receive this message as a subscriber of the xwiki-
> [hidden email] mailing list.
> To unsubscribe: mailto:[hidden email]
> For general help: mailto:[hidden email]?subject=help
> ObjectWeb mailing lists service home page: http://www.objectweb.org/
> wws





___________________________________________________________________________
Yahoo! Mail réinvente le mail ! Découvrez le nouveau Yahoo! Mail et son
interface révolutionnaire.
http://fr.mail.yahoo.com




--------------------------------------------------------------------------------


>
> --
> You receive this message as a subscriber of the [hidden email]
> mailing list.
> To unsubscribe: mailto:[hidden email]
> For general help: mailto:[hidden email]?subject=help
> ObjectWeb mailing lists service home page: http://www.objectweb.org/wws
>




--
You receive this message as a subscriber of the [hidden email] mailing list.
To unsubscribe: mailto:[hidden email]
For general help: mailto:[hidden email]?subject=help
ObjectWeb mailing lists service home page: http://www.objectweb.org/wws
Reply | Threaded
Open this post in threaded view
|

Re: Number of document decrease XWiki performance

Ludovic Dubost

Hi Youcef,

Have you modified the configuration for memory in Java from the default
of 64M. You might want to setup at least 256M.
If you have a lot of documents you might want to increase the number of
cached docs in xwiki.cfg also

Ludovic

Youcef BEY a écrit :

> Hi Vincent, Jeremi and Sebastien
>
> Thank you very much for your reply and I apologize for late reply.
>
> Your solution is not really enough for sloving the problem. Sorry for
> that.
>
> In fact, I created all the index so it seems that xwiki increase a
> little the speed but not really enough.
>
> I have compilted for testing it 10000 but xwiki become unusable!
>
> I did not use class in my document. The format of document is textual
> and small size (<2 Ko)
>
> Sebastien, I will send you by tomorrow a file containing 350000
> entries and all programs. As now I am transfering all my data and
> XWiki to Vine Linux (Debian) and also decided to leave Win XP for
> ever, I really enjoyed Linux :-)
>
> When I installed XWiki on my Linux machine (Vine Linux) I have got an
> error. Even I have an experience with XWiki but this error can't find
> any issues for solving it. I prefer to send it on another email
>
> Thank again.
>
> Youcef
>
> From: "Vincent Massol" <[hidden email]>
> To: <[hidden email]>
> Sent: Tuesday, February 06, 2007 1:22 AM
> Subject: Re: [xwiki-dev] Number of document decrease XWiki performance
>
>
> Hi Youcef,
>
> (Please do not hijack existing thread, this is bad practice... :-)).
>
> Regarding your issue you have to set up indexes if you're working
> with big tables (these are not set automatically as of now).
>
> I'm not sure exactly what indexes need to be set but here's some idea
> of what should be done (people in the know please confirm):
>
> create index xwd_name on xwikidoc (xwd_name);
> create index xwd_fullname on xwikidoc (xwd_fullname);
> create index xwd_web on xwikidoc (xwd_web);
> create index xwd_name on xwikiobjects (xwo_name);
> create index xwl_value on xwikilongs (xwl_value);
> create index xwi_value on xwikiintegers (xwi_value);
> create index xws_value on xwikistrings (xws_value);
> create index xwl_value on xwikilargestrings (xwl_value(50));
> create index xwo_classname on xwikiobjects (xwo_classname);
> create index xwd_creation_date on xwikidoc (xwd_creation_date);
> create index xwd_date on xwikidoc (xwd_date);
> create index xwd_content_update_date on xwikidoc
> (xwd_content_update_date);
> create index xwd_content_author on xwikidoc (xwd_content_author);
> create index xwd_author on xwikidoc (xwd_author);
> create index xwd_creator on xwikidoc (xwd_creator);
> create index xwd_language on xwikidoc (xwd_language);
> create index xwd_default_language on xwikidoc (xwd_default_language);
> create index xwd_title on xwikidoc (xwd_title);
> create index xwd_parent on xwikidoc (xwd_parent(50));
>
> Note that we aready have a jira issue for this (http://jira.xwiki.org/
> jira/browse/XWIKI-605). We just don't know yet if we can have a
> generic script that'll work for all databases.
>
> There might be other issues but I'll let others with experience with
> big XWiki instances answer...
>
> Thanks
> -Vincent
>
> On Feb 7, 2007, at 3:00 AM, Youcef BEY wrote:
>
>> Hi all,
>>
>> I have until now not solved this problem (since more than 10 month):
>>
>> - using this program to create automatically a document work well
>> for few documents:
>>
>>   XW = xwiki.XWiki;
>>   XWContext = context.context;
>>   XWikiDocument NewEntry = new XWikiDocument(request.getParameter
>> ("web"), SourceValue);
>>   NewEntry.setContent(EntryContent);
>>  XW.saveDocument(NewEntry, XWContext);
>>
>>
>> When putting these instructions in a LOOP for 10000 entries to
>> transfer from another TABLE in the same XWiki database cause a
>> problem. The probleme is not an error but the XWiki crash! and
>> without finishing the LOOP, the xwiki display error "out of memory!"
>>
>> But now, taking the same program and run it in the shell (javac,
>> java) to transfert these 10000 entries to TABLE in the XWiki
>> database, it's done in less 1 minute?
>>
>> Strange for me ...
>>
>> - More ...and this is the more importante thing :
>>
>>  One time I success to transfer 25 000 documents in the XWiki
>> database (using the same script above) the XWiki become very heavy
>> and cannot really work with it at all.
>>
>> MySql tables are supposed to contain millions of records, and when
>> using the database adminstrator and check with SQL statment, I get
>> resultat in few seconds, that means that the problem is in XWiki.
>>
>> My Question : Why with 25 000 documents XWiki don't work as usually
>>
>> Anyone faced this problem before? Do you have made testing of XWiki
>> for 1 millions documents for exemple?
>>
>> I have XP, 1.2 MHz and 500 Mo but the following experience on MySQL
>> gives more arguments that not MySQL is causing this problem :
>>
>> (Web link : http://www.guilde.asso.fr/lurker/message/
>> 20050817.183426.a1b8e267.en.html)
>>
>> -----------------------------------------------------------------
>> Il y a quelques années, j'ai utilisé MySQL pour gérér 1.7 millions
>> d'enregistrements dans une table. La table faisait environ 500Mo, plus
>> 400Mo d'indexes pour la recherche de textes dans un des champs.
>>
>>    Coté perfs, sur un ordinateur portable à 500Mhz et 128Mo de ram qui
>> faisait tourner un Windows NT4 bourré d'applis en mémoire, il fallait
>> environ 5s à 10s pour recherche les tuples contenant certains mots-
>> clefs
>> stokés dans le fameux champs de texte.
>>
>>    Bref, des performances tout à fait acceptable, sur une machine qui
>> n'était pourtant pas du tout fait pour faire serveur de base de
>> données
>> (et je ne parle pas que de l'OS...).
>> --------------------------------------------------------------
>>
>> Many thanks for any suggestion.
>>
>> Best wishes
>> ----------------------------------
>> Youcef Bey
>>
>>
>> --
>> You receive this message as a subscriber of the xwiki-
>> [hidden email] mailing list.
>> To unsubscribe: mailto:[hidden email]
>> For general help: mailto:[hidden email]?subject=help
>> ObjectWeb mailing lists service home page: http://www.objectweb.org/
>> wws
>
>
>
>
>
>
> ___________________________________________________________________________
>
> Yahoo! Mail réinvente le mail ! Découvrez le nouveau Yahoo! Mail et
> son interface révolutionnaire.
> http://fr.mail.yahoo.com
>
>
>
>
> --------------------------------------------------------------------------------
>
>
>
>>
>> --
>> You receive this message as a subscriber of the
>> [hidden email] mailing list.
>> To unsubscribe: mailto:[hidden email]
>> For general help: mailto:[hidden email]?subject=help
>> ObjectWeb mailing lists service home page: http://www.objectweb.org/wws
>>
>
>
> ------------------------------------------------------------------------
>
>
> --
> You receive this message as a subscriber of the [hidden email] mailing list.
> To unsubscribe: mailto:[hidden email]
> For general help: mailto:[hidden email]?subject=help
> ObjectWeb mailing lists service home page: http://www.objectweb.org/wws
>  

--
Ludovic Dubost
Blog: http://www.ludovic.org/blog/
XWiki: http://www.xwiki.com
Skype: ldubost GTalk: ldubost
AIM: nvludo Yahoo: ludovic




--
You receive this message as a subscriber of the [hidden email] mailing list.
To unsubscribe: mailto:[hidden email]
For general help: mailto:[hidden email]?subject=help
ObjectWeb mailing lists service home page: http://www.objectweb.org/wws
Reply | Threaded
Open this post in threaded view
|

Re: Number of document decrease XWiki performance

Youcef Bey-2
Hi Luduvic,

Thank you very much for your email.

Yes, I checked these options.

- I set the temporary java memory to 1 Go and increase the number of files to 1000000 in xml.cfg. Unfortunately, this configuration do not solve the problem.


- With the MySql administrator : a simple request for displaying around 10000 entries is done in few seconds and when deleting them it takes 5.0459 sec.

- Deleting one document in XWiki containing 10000 documents is done in 10 minutes, Strange...!

- The importation of a XWiki database with 13369 entries takes a few seconds from my second XWiki using an external database administrator.

- I created the indexes that Vincent suggested me to do. I remarked an increasing of speed but not really significant.

ps. I deal with data using UTF-8 encoding : Jetty CharSet = UTF-8, SQL DB with collation "utf8_general_ci", XWiki CharSet = UTF-8, etc.

I run such kind of tasks on WinXP (1 Ghz, 500 Mo RAM, 20 Go HD) and  also  with  last version of Debian (2,5 Ghz, 2 Go RAM and 100 Go HD).


This is what I got in the Shell during the work of XWiki:

-----------------------------------------------------------

WARN P1-16 http://localhost:8080/xwiki/bin/delete/A.H/WebHome?confirm=1& RegexTokenFilter:filter:99 - <span class="error">Error</span>: com.xpn.xwiki.render.filter.XWikiHeadingFilter@3bc63: java.lang.OutOfMemoryError: Java heap space
java.lang.OutOfMemoryError: Java heap space

18:15:26.764 INFO   [P1-14] org.mortbay.jetty.servlet.ServletHandler$Context.log(ServletHandler.java:1134) >55>  Velocity  [error] Left side ($!hidecolumns) of '==' operation has null value. If a reference, it may not be in the context. Operation not possible.  [line 9, column 21]
18:15:26.766 INFO   [P1-14] org.mortbay.jetty.servlet.ServletHandler$Context.log(ServletHandler.java:1134) >39>  Velocity  [error] RHS of #set statement is null. Context will not be modified.  [line 11, column 1]

-----------------------------------------------------------

Any suggestion will be deeply appreciated?

Thanks in advances.



Ludovic Dubost a écrit :

Hi Youcef,

Have you modified the configuration for memory in Java from the default of 64M. You might want to setup at least 256M.
If you have a lot of documents you might want to increase the number of cached docs in xwiki.cfg also

Ludovic




Youcef BEY a écrit :
Hi Vincent, Jeremi and Sebastien

Thank you very much for your reply and I apologize for late reply.

Your solution is not really enough for sloving the problem. Sorry for that.

In fact, I created all the index so it seems that xwiki increase a little the speed but not really enough.

I have compilted for testing it 10000 but xwiki become unusable!

I did not use class in my document. The format of document is textual and small size (<2 Ko)

Sebastien, I will send you by tomorrow a file containing 350000 entries and all programs. As now I am transfering all my data and XWiki to Vine Linux (Debian) and also decided to leave Win XP for ever, I really enjoyed Linux :-)

When I installed XWiki on my Linux machine (Vine Linux) I have got an error. Even I have an experience with XWiki but this error can't find any issues for solving it. I prefer to send it on another email

Thank again.

Youcef

From: "Vincent Massol" [hidden email]
To: [hidden email]
Sent: Tuesday, February 06, 2007 1:22 AM
Subject: Re: [xwiki-dev] Number of document decrease XWiki performance


Hi Youcef,

(Please do not hijack existing thread, this is bad practice... :-)).

Regarding your issue you have to set up indexes if you're working
with big tables (these are not set automatically as of now).

I'm not sure exactly what indexes need to be set but here's some idea
of what should be done (people in the know please confirm):

create index xwd_name on xwikidoc (xwd_name);
create index xwd_fullname on xwikidoc (xwd_fullname);
create index xwd_web on xwikidoc (xwd_web);
create index xwd_name on xwikiobjects (xwo_name);
create index xwl_value on xwikilongs (xwl_value);
create index xwi_value on xwikiintegers (xwi_value);
create index xws_value on xwikistrings (xws_value);
create index xwl_value on xwikilargestrings (xwl_value(50));
create index xwo_classname on xwikiobjects (xwo_classname);
create index xwd_creation_date on xwikidoc (xwd_creation_date);
create index xwd_date on xwikidoc (xwd_date);
create index xwd_content_update_date on xwikidoc
(xwd_content_update_date);
create index xwd_content_author on xwikidoc (xwd_content_author);
create index xwd_author on xwikidoc (xwd_author);
create index xwd_creator on xwikidoc (xwd_creator);
create index xwd_language on xwikidoc (xwd_language);
create index xwd_default_language on xwikidoc (xwd_default_language);
create index xwd_title on xwikidoc (xwd_title);
create index xwd_parent on xwikidoc (xwd_parent(50));

Note that we aready have a jira issue for this (http://jira.xwiki.org/
jira/browse/XWIKI-605). We just don't know yet if we can have a
generic script that'll work for all databases.

There might be other issues but I'll let others with experience with
big XWiki instances answer...

Thanks
-Vincent

On Feb 7, 2007, at 3:00 AM, Youcef BEY wrote:

Hi all,

I have until now not solved this problem (since more than 10 month):

- using this program to create automatically a document work well
for few documents:

  XW = xwiki.XWiki;
  XWContext = context.context;
  XWikiDocument NewEntry = new XWikiDocument(request.getParameter
("web"), SourceValue);
  NewEntry.setContent(EntryContent);
 XW.saveDocument(NewEntry, XWContext);


When putting these instructions in a LOOP for 10000 entries to
transfer from another TABLE in the same XWiki database cause a
problem. The probleme is not an error but the XWiki crash! and
without finishing the LOOP, the xwiki display error "out of memory!"

But now, taking the same program and run it in the shell (javac,
java) to transfert these 10000 entries to TABLE in the XWiki
database, it's done in less 1 minute?

Strange for me ...

- More ...and this is the more importante thing :

 One time I success to transfer 25 000 documents in the XWiki
database (using the same script above) the XWiki become very heavy
and cannot really work with it at all.

MySql tables are supposed to contain millions of records, and when
using the database adminstrator and check with SQL statment, I get
resultat in few seconds, that means that the problem is in XWiki.

My Question : Why with 25 000 documents XWiki don't work as usually

Anyone faced this problem before? Do you have made testing of XWiki
for 1 millions documents for exemple?

I have XP, 1.2 MHz and 500 Mo but the following experience on MySQL
gives more arguments that not MySQL is causing this problem :

(Web link : http://www.guilde.asso.fr/lurker/message/
20050817.183426.a1b8e267.en.html)

-----------------------------------------------------------------
Il y a quelques années, j'ai utilisé MySQL pour gérér 1.7 millions
d'enregistrements dans une table. La table faisait environ 500Mo, plus
400Mo d'indexes pour la recherche de textes dans un des champs.

   Coté perfs, sur un ordinateur portable à 500Mhz et 128Mo de ram qui
faisait tourner un Windows NT4 bourré d'applis en mémoire, il fallait
environ 5s à 10s pour recherche les tuples contenant certains mots-
clefs
stokés dans le fameux champs de texte.

   Bref, des performances tout à fait acceptable, sur une machine qui
n'était pourtant pas du tout fait pour faire serveur de base de
données
(et je ne parle pas que de l'OS...).
--------------------------------------------------------------

Many thanks for any suggestion.

Best wishes
----------------------------------
Youcef Bey


-- 
You receive this message as a subscriber of the xwiki-
[hidden email] mailing list.
To unsubscribe: [hidden email]
For general help: [hidden email]
ObjectWeb mailing lists service home page: http://www.objectweb.org/
wws






___________________________________________________________________________
Yahoo! Mail réinvente le mail ! Découvrez le nouveau Yahoo! Mail et son interface révolutionnaire.
http://fr.mail.yahoo.com




--------------------------------------------------------------------------------



-- 
You receive this message as a subscriber of the [hidden email] mailing list.
To unsubscribe: [hidden email]
For general help: [hidden email]
ObjectWeb mailing lists service home page: http://www.objectweb.org/wws



------------------------------------------------------------------------


--
You receive this message as a subscriber of the [hidden email] mailing list.
To unsubscribe: [hidden email]
For general help: [hidden email]
ObjectWeb mailing lists service home page: http://www.objectweb.org/wws
 



-- You receive this message as a subscriber of the [hidden email] mailing list. To unsubscribe: [hidden email] For general help: [hidden email] ObjectWeb mailing lists service home page: http://www.objectweb.org/wws



--
You receive this message as a subscriber of the [hidden email] mailing list.
To unsubscribe: mailto:[hidden email]
For general help: mailto:[hidden email]?subject=help
ObjectWeb mailing lists service home page: http://www.objectweb.org/wws
Reply | Threaded
Open this post in threaded view
|

Re: Number of document decrease XWiki performance

jeremi joslin-2
On 2/9/07, bey <[hidden email]> wrote:

>
>  Hi Luduvic,
>
>  Thank you very much for your email.
>
>  Yes, I checked these options.
>
>  - I set the temporary java memory to 1 Go and increase the number of files
> to 1000000 in xml.cfg. Unfortunately, this configuration do not solve the
> problem.
>
>
>  - With the MySql administrator : a simple request for displaying around
> 10000 entries is done in few seconds and when deleting them it takes 5.0459
> sec.
>
>  - Deleting one document in XWiki containing 10000 documents is done in 10
> minutes, Strange...!
>
>  - The importation of a XWiki database with 13369 entries takes a few
> seconds from my second XWiki using an external database administrator.
>
>  - I created the indexes that Vincent suggested me to do. I remarked an
> increasing of speed but not really significant.
>
>  ps. I deal with data using UTF-8 encoding : Jetty CharSet = UTF-8, SQL DB
> with collation "utf8_general_ci", XWiki CharSet = UTF-8, etc.
>
>  I run such kind of tasks on WinXP (1 Ghz, 500 Mo RAM, 20 Go HD) and  also
> with  last version of Debian (2,5 Ghz, 2 Go RAM and 100 Go HD).
>
>
>  This is what I got in the Shell during the work of XWiki:
>
> -----------------------------------------------------------
>
>  WARN P1-16
> http://localhost:8080/xwiki/bin/delete/A.H/WebHome?confirm=1&
> RegexTokenFilter:filter:99 - <span class="error">Error</span>:
> com.xpn.xwiki.render.filter.XWikiHeadingFilter@3bc63:
> java.lang.OutOfMemoryError: Java heap space
>  java.lang.OutOfMemoryError: Java heap space
>  18:15:26.764 INFO   [P1-14]
> org.mortbay.jetty.servlet.ServletHandler$Context.log(ServletHandler.java:1134)
> >55>  Velocity  [error] Left side ($!hidecolumns) of '==' operation has null
> value. If a reference, it may not be in the context. Operation not possible.
>  [line 9, column 21]
>  18:15:26.766 INFO   [P1-14]
> org.mortbay.jetty.servlet.ServletHandler$Context.log(ServletHandler.java:1134)
> >39>  Velocity  [error] RHS of #set statement is null. Context will not be
> modified.  [line 11, column 1]
>
> -----------------------------------------------------------
>
>  Any suggestion will be deeply appreciated?
ok, so the problem is what ludovic says. increase the memory of your
java.Look inside the catalina.sh to change the memory size, and
increase it to at least 256M. it's a parameter (looks like this i
think : "-X64m") to java.

jeremi

--
Jeremi Joslin (http://www.jeremi.info)
skype: jeremi23 - jabber: [hidden email]
http://www.xwiki.com - http://www.pengyou-project.info



--
You receive this message as a subscriber of the [hidden email] mailing list.
To unsubscribe: mailto:[hidden email]
For general help: mailto:[hidden email]?subject=help
ObjectWeb mailing lists service home page: http://www.objectweb.org/wws
12