Add pugixml to VTK

classic Classic list List threaded Threaded
20 messages Options
Reply | Threaded
Open this post in threaded view
|

Add pugixml to VTK

Dan Lipsa-2
Hi all,
I am working on a CityGML reader for VTK. For this I need a fast XML parser. pugixml is already used by ParaView and seems to be much faster than available choices already present in VTK: libxml2 and expat. pugixml has also a very clean API.


Are there any comments or concerns about moving pugixml from ParaView to VTK?

Thanks,
Dan


_______________________________________________
Powered by www.kitware.com

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Search the list archives at: http://markmail.org/search/?q=vtk-developers

Follow this link to subscribe/unsubscribe:
https://public.kitware.com/mailman/listinfo/vtk-developers

Reply | Threaded
Open this post in threaded view
|

Re: Add pugixml to VTK

Utkarsh Ayachit
+1
On Fri, Jun 15, 2018 at 10:59 AM Dan Lipsa <[hidden email]> wrote:

>
> Hi all,
> I am working on a CityGML reader for VTK. For this I need a fast XML parser. pugixml is already used by ParaView and seems to be much faster than available choices already present in VTK: libxml2 and expat. pugixml has also a very clean API.
>
> https://pugixml.org/benchmark.html
>
> Are there any comments or concerns about moving pugixml from ParaView to VTK?
>
> Thanks,
> Dan
>
> _______________________________________________
> Powered by www.kitware.com
>
> Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html
>
> Search the list archives at: http://markmail.org/search/?q=vtk-developers
>
> Follow this link to subscribe/unsubscribe:
> https://public.kitware.com/mailman/listinfo/vtk-developers
>
_______________________________________________
Powered by www.kitware.com

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Search the list archives at: http://markmail.org/search/?q=vtk-developers

Follow this link to subscribe/unsubscribe:
https://public.kitware.com/mailman/listinfo/vtk-developers

Reply | Threaded
Open this post in threaded view
|

Re: Add pugixml to VTK

David Thompson-2
>> I am working on a CityGML reader for VTK. For this I need a fast XML parser. pugixml is already used by ParaView and seems to be much faster than available choices already present in VTK: libxml2 and expat. pugixml has also a very clean API.
>>
>> https://pugixml.org/benchmark.html
>>
>> Are there any comments or concerns about moving pugixml from ParaView to VTK?

I thought Ben Boeckel was working on a removing as many things from ThirdParty as possible (and making them parts of a superbuild). So perhaps the CityGML reader should live in a VTK module that requires an external pugi? I know pugi is small, but...

        David
_______________________________________________
Powered by www.kitware.com

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Search the list archives at: http://markmail.org/search/?q=vtk-developers

Follow this link to subscribe/unsubscribe:
https://public.kitware.com/mailman/listinfo/vtk-developers

Reply | Threaded
Open this post in threaded view
|

Re: Add pugixml to VTK

jbor
Last time I checked, ParaView was using an older version of PugiXML (1.4) while there are now at version 1.9.

I think the benchmarks are considering the version 1.9. If you just port the current ParaView implementation of PugiXML to VTK, you might be disappointed.  

Best,

Jonathan

-----Original Message-----
From: vtk-developers <[hidden email]> On Behalf Of David Thompson
Sent: Friday, June 15, 2018 1:19 PM
To: Utkarsh Ayachit <[hidden email]>
Cc: [hidden email]
Subject: Re: [vtk-developers] Add pugixml to VTK

>> I am working on a CityGML reader for VTK. For this I need a fast XML parser. pugixml is already used by ParaView and seems to be much faster than available choices already present in VTK: libxml2 and expat. pugixml has also a very clean API.
>>
>> https://pugixml.org/benchmark.html
>>
>> Are there any comments or concerns about moving pugixml from ParaView to VTK?

I thought Ben Boeckel was working on a removing as many things from ThirdParty as possible (and making them parts of a superbuild). So perhaps the CityGML reader should live in a VTK module that requires an external pugi? I know pugi is small, but...

        David
_______________________________________________
Powered by www.kitware.com

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Search the list archives at: http://markmail.org/search/?q=vtk-developers

Follow this link to subscribe/unsubscribe:
https://public.kitware.com/mailman/listinfo/vtk-developers

_______________________________________________
Powered by www.kitware.com

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Search the list archives at: http://markmail.org/search/?q=vtk-developers

Follow this link to subscribe/unsubscribe:
https://public.kitware.com/mailman/listinfo/vtk-developers

Reply | Threaded
Open this post in threaded view
|

Re: Add pugixml to VTK

Dan Lipsa-2
In reply to this post by David Thompson-2
David,
This is certainly an option. There are many readers that require external libraries.
Maybe this is the better choice. Moving pugixml would be a future step when/if other modules start using it.

Dan


On Fri, Jun 15, 2018 at 1:19 PM David Thompson <[hidden email]> wrote:
>> I am working on a CityGML reader for VTK. For this I need a fast XML parser. pugixml is already used by ParaView and seems to be much faster than available choices already present in VTK: libxml2 and expat. pugixml has also a very clean API.
>>
>> https://pugixml.org/benchmark.html
>>
>> Are there any comments or concerns about moving pugixml from ParaView to VTK?

I thought Ben Boeckel was working on a removing as many things from ThirdParty as possible (and making them parts of a superbuild). So perhaps the CityGML reader should live in a VTK module that requires an external pugi? I know pugi is small, but...

        David

_______________________________________________
Powered by www.kitware.com

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Search the list archives at: http://markmail.org/search/?q=vtk-developers

Follow this link to subscribe/unsubscribe:
https://public.kitware.com/mailman/listinfo/vtk-developers

Reply | Threaded
Open this post in threaded view
|

Re: Add pugixml to VTK

Utkarsh Ayachit
In reply to this post by David Thompson-2
> I thought Ben Boeckel was working on a removing as many things from ThirdParty as possible (and making them parts of a superbuild). So perhaps the CityGML reader should live in a VTK module that requires an external pugi? I know pugi is small, but...

I am not sure that's correct. He was doing that for Python 3rd party
modules. Are you referring to that?
_______________________________________________
Powered by www.kitware.com

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Search the list archives at: http://markmail.org/search/?q=vtk-developers

Follow this link to subscribe/unsubscribe:
https://public.kitware.com/mailman/listinfo/vtk-developers

Reply | Threaded
Open this post in threaded view
|

Re: Add pugixml to VTK

chiranjibsur
My few cents... The disadvantage with pugixml is that it's a non-validating parser. If I want to make full use of XML schema for any custom developed plugin, it's a very poor choice then.

Any thoughts?

Thanks and regards,
Chiranjib

------
Using hand held device. Sorry for the typo, if any.

On Fri, Jun 15, 2018, 11:59 PM Utkarsh Ayachit <[hidden email]> wrote:
> I thought Ben Boeckel was working on a removing as many things from ThirdParty as possible (and making them parts of a superbuild). So perhaps the CityGML reader should live in a VTK module that requires an external pugi? I know pugi is small, but...

I am not sure that's correct. He was doing that for Python 3rd party
modules. Are you referring to that?
_______________________________________________
Powered by www.kitware.com

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Search the list archives at: http://markmail.org/search/?q=vtk-developers

Follow this link to subscribe/unsubscribe:
https://public.kitware.com/mailman/listinfo/vtk-developers


_______________________________________________
Powered by www.kitware.com

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Search the list archives at: http://markmail.org/search/?q=vtk-developers

Follow this link to subscribe/unsubscribe:
https://public.kitware.com/mailman/listinfo/vtk-developers

Reply | Threaded
Open this post in threaded view
|

Re: Add pugixml to VTK

Elvis Stansvik
2018-06-16 8:37 GMT+02:00 Chiranjib Sur <[hidden email]>:
My few cents... The disadvantage with pugixml is that it's a non-validating parser. If I want to make full use of XML schema for any custom developed plugin, it's a very poor choice then.

Any thoughts?

Also worth noting that it's a DOM-like parser only AFAIK, so if you want to do streaming parsing in a push paradigm à la SAX or a pull paradigm à la libxml2's reader interface, you'd need to turn to another library. Note that may not be a problem if VTK already has libxml2 as a dep, since then you could use that. I was thinking more if there should be a push towards one single lib, then it might make sense to pick one that has both DOM and streaming interfaces (don't know if that's a goal/consideration though).

Elvis


Thanks and regards,
Chiranjib

------
Using hand held device. Sorry for the typo, if any.

On Fri, Jun 15, 2018, 11:59 PM Utkarsh Ayachit <[hidden email]> wrote:
> I thought Ben Boeckel was working on a removing as many things from ThirdParty as possible (and making them parts of a superbuild). So perhaps the CityGML reader should live in a VTK module that requires an external pugi? I know pugi is small, but...

I am not sure that's correct. He was doing that for Python 3rd party
modules. Are you referring to that?
_______________________________________________
Powered by www.kitware.com

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Search the list archives at: http://markmail.org/search/?q=vtk-developers

Follow this link to subscribe/unsubscribe:
https://public.kitware.com/mailman/listinfo/vtk-developers


_______________________________________________
Powered by www.kitware.com

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Search the list archives at: http://markmail.org/search/?q=vtk-developers

Follow this link to subscribe/unsubscribe:
https://public.kitware.com/mailman/listinfo/vtk-developers




_______________________________________________
Powered by www.kitware.com

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Search the list archives at: http://markmail.org/search/?q=vtk-developers

Follow this link to subscribe/unsubscribe:
https://public.kitware.com/mailman/listinfo/vtk-developers

Reply | Threaded
Open this post in threaded view
|

Re: Add pugixml to VTK

David E DeMarle
 I was thinking more if there should be a push towards one single lib, ...

I for one would really like us to get down to just one xml parser in VTK. The more the less merrier in terms of maintenance effort and project complexity eh?

David E DeMarle
Kitware, Inc.
Principal Engineer
21 Corporate Drive
Clifton Park, NY 12065-8662
Phone: 518-881-4909


On Tue, Jun 19, 2018 at 3:36 PM Elvis Stansvik <[hidden email]> wrote:
2018-06-16 8:37 GMT+02:00 Chiranjib Sur <[hidden email]>:
My few cents... The disadvantage with pugixml is that it's a non-validating parser. If I want to make full use of XML schema for any custom developed plugin, it's a very poor choice then.

Any thoughts?

Also worth noting that it's a DOM-like parser only AFAIK, so if you want to do streaming parsing in a push paradigm à la SAX or a pull paradigm à la libxml2's reader interface, you'd need to turn to another library. Note that may not be a problem if VTK already has libxml2 as a dep, since then you could use that. I was thinking more if there should be a push towards one single lib, then it might make sense to pick one that has both DOM and streaming interfaces (don't know if that's a goal/consideration though).

Elvis


Thanks and regards,
Chiranjib

------
Using hand held device. Sorry for the typo, if any.

On Fri, Jun 15, 2018, 11:59 PM Utkarsh Ayachit <[hidden email]> wrote:
> I thought Ben Boeckel was working on a removing as many things from ThirdParty as possible (and making them parts of a superbuild). So perhaps the CityGML reader should live in a VTK module that requires an external pugi? I know pugi is small, but...

I am not sure that's correct. He was doing that for Python 3rd party
modules. Are you referring to that?
_______________________________________________
Powered by www.kitware.com

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Search the list archives at: http://markmail.org/search/?q=vtk-developers

Follow this link to subscribe/unsubscribe:
https://public.kitware.com/mailman/listinfo/vtk-developers


_______________________________________________
Powered by www.kitware.com

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Search the list archives at: http://markmail.org/search/?q=vtk-developers

Follow this link to subscribe/unsubscribe:
https://public.kitware.com/mailman/listinfo/vtk-developers



_______________________________________________
Powered by www.kitware.com

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Search the list archives at: http://markmail.org/search/?q=vtk-developers

Follow this link to subscribe/unsubscribe:
https://public.kitware.com/mailman/listinfo/vtk-developers


_______________________________________________
Powered by www.kitware.com

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Search the list archives at: http://markmail.org/search/?q=vtk-developers

Follow this link to subscribe/unsubscribe:
https://public.kitware.com/mailman/listinfo/vtk-developers

Reply | Threaded
Open this post in threaded view
|

Re: Add pugixml to VTK

jbor

My 2 cents here is that speed is of the essence. Validating XML are good for debugging, however I found that most don’t take the time to create an XSD file to enable the validation.

 

In most of our cases at Caboma, XML and meant to be human readable and editable, but it is rare that we manually edit the structure of an XML file.

However, we do wait after .xml parsing when the files are 20mb+. We hope that the community would move to the latest version of pugixml (1.9): by looking at the changelogs, there was a lot of performance improvements since version 1.4 (the current version used in ParaView).
https://pugixml.org/docs/manual.html#v1.5

 

Best

 

Jonathan Borduas

CTO, Caboma Inc.

 

 

From: vtk-developers <[hidden email]> On Behalf Of David E DeMarle
Sent: Wednesday, June 20, 2018 11:49 AM
To: Elvis Stansvik <[hidden email]>
Cc: vtkdev <[hidden email]>
Subject: Re: [vtk-developers] Add pugixml to VTK

 

 I was thinking more if there should be a push towards one single lib, ...

 

I for one would really like us to get down to just one xml parser in VTK. The more the less merrier in terms of maintenance effort and project complexity eh?

 

David E DeMarle
Kitware, Inc.
Principal Engineer
21 Corporate Drive
Clifton Park, NY 12065-8662
Phone: 518-881-4909

 

 

On Tue, Jun 19, 2018 at 3:36 PM Elvis Stansvik <[hidden email]> wrote:

2018-06-16 8:37 GMT+02:00 Chiranjib Sur <[hidden email]>:

My few cents... The disadvantage with pugixml is that it's a non-validating parser. If I want to make full use of XML schema for any custom developed plugin, it's a very poor choice then.

 

Any thoughts?

 

Also worth noting that it's a DOM-like parser only AFAIK, so if you want to do streaming parsing in a push paradigm à la SAX or a pull paradigm à la libxml2's reader interface, you'd need to turn to another library. Note that may not be a problem if VTK already has libxml2 as a dep, since then you could use that. I was thinking more if there should be a push towards one single lib, then it might make sense to pick one that has both DOM and streaming interfaces (don't know if that's a goal/consideration though).

 

Elvis

 

 

Thanks and regards,
Chiranjib

------
Using hand held device. Sorry for the typo, if any.

 

On Fri, Jun 15, 2018, 11:59 PM Utkarsh Ayachit <[hidden email]> wrote:

> I thought Ben Boeckel was working on a removing as many things from ThirdParty as possible (and making them parts of a superbuild). So perhaps the CityGML reader should live in a VTK module that requires an external pugi? I know pugi is small, but...

I am not sure that's correct. He was doing that for Python 3rd party
modules. Are you referring to that?
_______________________________________________
Powered by www.kitware.com

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Search the list archives at: http://markmail.org/search/?q=vtk-developers

Follow this link to subscribe/unsubscribe:
https://public.kitware.com/mailman/listinfo/vtk-developers


_______________________________________________
Powered by www.kitware.com

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Search the list archives at: http://markmail.org/search/?q=vtk-developers

Follow this link to subscribe/unsubscribe:
https://public.kitware.com/mailman/listinfo/vtk-developers

 

_______________________________________________
Powered by www.kitware.com

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Search the list archives at: http://markmail.org/search/?q=vtk-developers

Follow this link to subscribe/unsubscribe:
https://public.kitware.com/mailman/listinfo/vtk-developers


_______________________________________________
Powered by www.kitware.com

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Search the list archives at: http://markmail.org/search/?q=vtk-developers

Follow this link to subscribe/unsubscribe:
https://public.kitware.com/mailman/listinfo/vtk-developers

Reply | Threaded
Open this post in threaded view
|

Re: Add pugixml to VTK

Bob Obara
SMTK and CMB use pugixml for years and have been very happy with it.

Bob

Sent from my iPad

On Jun 20, 2018, at 12:07 PM, Jonathan Borduas <[hidden email]> wrote:

My 2 cents here is that speed is of the essence. Validating XML are good for debugging, however I found that most don’t take the time to create an XSD file to enable the validation.

 

In most of our cases at Caboma, XML and meant to be human readable and editable, but it is rare that we manually edit the structure of an XML file.

However, we do wait after .xml parsing when the files are 20mb+. We hope that the community would move to the latest version of pugixml (1.9): by looking at the changelogs, there was a lot of performance improvements since version 1.4 (the current version used in ParaView).
https://pugixml.org/docs/manual.html#v1.5

 

Best

 

Jonathan Borduas

CTO, Caboma Inc.

 

 

From: vtk-developers <[hidden email]> On Behalf Of David E DeMarle
Sent: Wednesday, June 20, 2018 11:49 AM
To: Elvis Stansvik <[hidden email]>
Cc: vtkdev <[hidden email]>
Subject: Re: [vtk-developers] Add pugixml to VTK

 

 I was thinking more if there should be a push towards one single lib, ...

 

I for one would really like us to get down to just one xml parser in VTK. The more the less merrier in terms of maintenance effort and project complexity eh?

 

David E DeMarle
Kitware, Inc.
Principal Engineer
21 Corporate Drive
Clifton Park, NY 12065-8662
Phone: 518-881-4909

 

 

On Tue, Jun 19, 2018 at 3:36 PM Elvis Stansvik <[hidden email]> wrote:

2018-06-16 8:37 GMT+02:00 Chiranjib Sur <[hidden email]>:

My few cents... The disadvantage with pugixml is that it's a non-validating parser. If I want to make full use of XML schema for any custom developed plugin, it's a very poor choice then.

 

Any thoughts?

 

Also worth noting that it's a DOM-like parser only AFAIK, so if you want to do streaming parsing in a push paradigm à la SAX or a pull paradigm à la libxml2's reader interface, you'd need to turn to another library. Note that may not be a problem if VTK already has libxml2 as a dep, since then you could use that. I was thinking more if there should be a push towards one single lib, then it might make sense to pick one that has both DOM and streaming interfaces (don't know if that's a goal/consideration though).

 

Elvis

 

 

Thanks and regards,
Chiranjib

------
Using hand held device. Sorry for the typo, if any.

 

On Fri, Jun 15, 2018, 11:59 PM Utkarsh Ayachit <[hidden email]> wrote:

> I thought Ben Boeckel was working on a removing as many things from ThirdParty as possible (and making them parts of a superbuild). So perhaps the CityGML reader should live in a VTK module that requires an external pugi? I know pugi is small, but...

I am not sure that's correct. He was doing that for Python 3rd party
modules. Are you referring to that?
_______________________________________________
Powered by www.kitware.com

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Search the list archives at: http://markmail.org/search/?q=vtk-developers

Follow this link to subscribe/unsubscribe:
https://public.kitware.com/mailman/listinfo/vtk-developers


_______________________________________________
Powered by www.kitware.com

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Search the list archives at: http://markmail.org/search/?q=vtk-developers

Follow this link to subscribe/unsubscribe:
https://public.kitware.com/mailman/listinfo/vtk-developers

 

_______________________________________________
Powered by www.kitware.com

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Search the list archives at: http://markmail.org/search/?q=vtk-developers

Follow this link to subscribe/unsubscribe:
https://public.kitware.com/mailman/listinfo/vtk-developers

_______________________________________________
Powered by www.kitware.com

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Search the list archives at: http://markmail.org/search/?q=vtk-developers

Follow this link to subscribe/unsubscribe:
https://public.kitware.com/mailman/listinfo/vtk-developers


_______________________________________________
Powered by www.kitware.com

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Search the list archives at: http://markmail.org/search/?q=vtk-developers

Follow this link to subscribe/unsubscribe:
https://public.kitware.com/mailman/listinfo/vtk-developers

Reply | Threaded
Open this post in threaded view
|

Re: Add pugixml to VTK

Dan Lipsa-2
Thank you all for your comments.  I am trying to summarize them next:

Advantages
1. ParaView, SMTK and CMB already use pugixml, so instead of maintaining this library in three projects we'll do it in one VTK (for those of us that work on more than one of these projects)
2. pugixml has a clean interface and it is (much) faster than the alternatives available in VTK. (only the latest pugixml version?)

Drawbacks:
1. no validation
2. no streaming (SAX interface)
3. an additional library to maintain in VTK.
4. Three different XML libraries in VTK.

Looking at the drawbacks: The features named in drawbacks 1. and 2. are not important if you don't want to use them. If you do, you have alternatives (libxml2 and expat).
I think drawback 3 is mitigated by advantage 1.

4. Seems to be the case that different XML libraries have different strengths (completeness versus speed). Given that, we either use the most complete library and give up speed or we have several libraries available and use them based on the feature we need most. VTK already seems to have taken this path with libxml2 and expat. I also suspect that in certain cases we just use the XML library available at the time, so we could convert the code to use a different XML library - Of course this requires time and work!

I setup a survey to better assess the community feeling on this: 




On Wed, Jun 20, 2018 at 12:13 PM Robert Michael O'Bara <[hidden email]> wrote:
SMTK and CMB use pugixml for years and have been very happy with it.

Bob

Sent from my iPad

On Jun 20, 2018, at 12:07 PM, Jonathan Borduas <[hidden email]> wrote:

My 2 cents here is that speed is of the essence. Validating XML are good for debugging, however I found that most don’t take the time to create an XSD file to enable the validation.

 

In most of our cases at Caboma, XML and meant to be human readable and editable, but it is rare that we manually edit the structure of an XML file.

However, we do wait after .xml parsing when the files are 20mb+. We hope that the community would move to the latest version of pugixml (1.9): by looking at the changelogs, there was a lot of performance improvements since version 1.4 (the current version used in ParaView).
https://pugixml.org/docs/manual.html#v1.5

 

Best

 

Jonathan Borduas

CTO, Caboma Inc.

 

 

From: vtk-developers <[hidden email]> On Behalf Of David E DeMarle
Sent: Wednesday, June 20, 2018 11:49 AM
To: Elvis Stansvik <[hidden email]>
Cc: vtkdev <[hidden email]>
Subject: Re: [vtk-developers] Add pugixml to VTK

 

 I was thinking more if there should be a push towards one single lib, ...

 

I for one would really like us to get down to just one xml parser in VTK. The more the less merrier in terms of maintenance effort and project complexity eh?

 

David E DeMarle
Kitware, Inc.
Principal Engineer
21 Corporate Drive
Clifton Park, NY 12065-8662
Phone: 518-881-4909

 

 

On Tue, Jun 19, 2018 at 3:36 PM Elvis Stansvik <[hidden email]> wrote:

2018-06-16 8:37 GMT+02:00 Chiranjib Sur <[hidden email]>:

My few cents... The disadvantage with pugixml is that it's a non-validating parser. If I want to make full use of XML schema for any custom developed plugin, it's a very poor choice then.

 

Any thoughts?

 

Also worth noting that it's a DOM-like parser only AFAIK, so if you want to do streaming parsing in a push paradigm à la SAX or a pull paradigm à la libxml2's reader interface, you'd need to turn to another library. Note that may not be a problem if VTK already has libxml2 as a dep, since then you could use that. I was thinking more if there should be a push towards one single lib, then it might make sense to pick one that has both DOM and streaming interfaces (don't know if that's a goal/consideration though).

 

Elvis

 

 

Thanks and regards,
Chiranjib

------
Using hand held device. Sorry for the typo, if any.

 

On Fri, Jun 15, 2018, 11:59 PM Utkarsh Ayachit <[hidden email]> wrote:

> I thought Ben Boeckel was working on a removing as many things from ThirdParty as possible (and making them parts of a superbuild). So perhaps the CityGML reader should live in a VTK module that requires an external pugi? I know pugi is small, but...

I am not sure that's correct. He was doing that for Python 3rd party
modules. Are you referring to that?
_______________________________________________
Powered by www.kitware.com

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Search the list archives at: http://markmail.org/search/?q=vtk-developers

Follow this link to subscribe/unsubscribe:
https://public.kitware.com/mailman/listinfo/vtk-developers


_______________________________________________
Powered by www.kitware.com

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Search the list archives at: http://markmail.org/search/?q=vtk-developers

Follow this link to subscribe/unsubscribe:
https://public.kitware.com/mailman/listinfo/vtk-developers

 

_______________________________________________
Powered by www.kitware.com

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Search the list archives at: http://markmail.org/search/?q=vtk-developers

Follow this link to subscribe/unsubscribe:
https://public.kitware.com/mailman/listinfo/vtk-developers

_______________________________________________
Powered by www.kitware.com

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Search the list archives at: http://markmail.org/search/?q=vtk-developers

Follow this link to subscribe/unsubscribe:
https://public.kitware.com/mailman/listinfo/vtk-developers

_______________________________________________
Powered by www.kitware.com

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Search the list archives at: http://markmail.org/search/?q=vtk-developers

Follow this link to subscribe/unsubscribe:
https://public.kitware.com/mailman/listinfo/vtk-developers


_______________________________________________
Powered by www.kitware.com

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Search the list archives at: http://markmail.org/search/?q=vtk-developers

Follow this link to subscribe/unsubscribe:
https://public.kitware.com/mailman/listinfo/vtk-developers

Reply | Threaded
Open this post in threaded view
|

Re: Add pugixml to VTK

Elvis Stansvik
Den fre 22 juni 2018 20:28Dan Lipsa <[hidden email]> skrev:
Thank you all for your comments.  I am trying to summarize them next:

Advantages
1. ParaView, SMTK and CMB already use pugixml, so instead of maintaining this library in three projects we'll do it in one VTK (for those of us that work on more than one of these projects)
2. pugixml has a clean interface and it is (much) faster than the alternatives available in VTK. (only the latest pugixml version?)

Drawbacks:
1. no validation
2. no streaming (SAX interface)
3. an additional library to maintain in VTK.
4. Three different XML libraries in VTK.

Looking at the drawbacks: The features named in drawbacks 1. and 2. are not important if you don't want to use them. If you do, you have alternatives (libxml2 and expat).
I think drawback 3 is mitigated by advantage 1.

4. Seems to be the case that different XML libraries have different strengths (completeness versus speed). Given that, we either use the most complete library and give up speed or we have several libraries available and use them based on the feature we need most. VTK already seems to have taken this path with libxml2 and expat. I also suspect that in certain cases we just use the XML library available at the time, so we could convert the code to use a different XML library - Of course this requires time and work!

I setup a survey to better assess the community feeling on this: 



I'm agnostic :)

An idea would be to maybe try to eliminate expat at some point afterwards, to bring it down to just two libs again? I don't think the speed advantage of expat over libxml2 could really matter that much? (libxml2 has both a SAX-like and a pull-style reader API).

libxml2 can then be the power horse for those who want 100% conformance and possibility of validation, while pugi for those who need speedy DOM.

I wonder what are the speed-requiring use cases for DOM parsing though? Are you parsing many small docs or a few big ones? Is the XML parsing a true bottleneck?

Elvis



On Wed, Jun 20, 2018 at 12:13 PM Robert Michael O'Bara <[hidden email]> wrote:
SMTK and CMB use pugixml for years and have been very happy with it.

Bob

Sent from my iPad

On Jun 20, 2018, at 12:07 PM, Jonathan Borduas <[hidden email]> wrote:

My 2 cents here is that speed is of the essence. Validating XML are good for debugging, however I found that most don’t take the time to create an XSD file to enable the validation.

 

In most of our cases at Caboma, XML and meant to be human readable and editable, but it is rare that we manually edit the structure of an XML file.

However, we do wait after .xml parsing when the files are 20mb+. We hope that the community would move to the latest version of pugixml (1.9): by looking at the changelogs, there was a lot of performance improvements since version 1.4 (the current version used in ParaView).
https://pugixml.org/docs/manual.html#v1.5

 

Best

 

Jonathan Borduas

CTO, Caboma Inc.

 

 

From: vtk-developers <[hidden email]> On Behalf Of David E DeMarle
Sent: Wednesday, June 20, 2018 11:49 AM
To: Elvis Stansvik <[hidden email]>
Cc: vtkdev <[hidden email]>
Subject: Re: [vtk-developers] Add pugixml to VTK

 

 I was thinking more if there should be a push towards one single lib, ...

 

I for one would really like us to get down to just one xml parser in VTK. The more the less merrier in terms of maintenance effort and project complexity eh?

 

David E DeMarle
Kitware, Inc.
Principal Engineer
21 Corporate Drive
Clifton Park, NY 12065-8662
Phone: 518-881-4909

 

 

On Tue, Jun 19, 2018 at 3:36 PM Elvis Stansvik <[hidden email]> wrote:

2018-06-16 8:37 GMT+02:00 Chiranjib Sur <[hidden email]>:

My few cents... The disadvantage with pugixml is that it's a non-validating parser. If I want to make full use of XML schema for any custom developed plugin, it's a very poor choice then.

 

Any thoughts?

 

Also worth noting that it's a DOM-like parser only AFAIK, so if you want to do streaming parsing in a push paradigm à la SAX or a pull paradigm à la libxml2's reader interface, you'd need to turn to another library. Note that may not be a problem if VTK already has libxml2 as a dep, since then you could use that. I was thinking more if there should be a push towards one single lib, then it might make sense to pick one that has both DOM and streaming interfaces (don't know if that's a goal/consideration though).

 

Elvis

 

 

Thanks and regards,
Chiranjib

------
Using hand held device. Sorry for the typo, if any.

 

On Fri, Jun 15, 2018, 11:59 PM Utkarsh Ayachit <[hidden email]> wrote:

> I thought Ben Boeckel was working on a removing as many things from ThirdParty as possible (and making them parts of a superbuild). So perhaps the CityGML reader should live in a VTK module that requires an external pugi? I know pugi is small, but...

I am not sure that's correct. He was doing that for Python 3rd party
modules. Are you referring to that?
_______________________________________________
Powered by www.kitware.com

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Search the list archives at: http://markmail.org/search/?q=vtk-developers

Follow this link to subscribe/unsubscribe:
https://public.kitware.com/mailman/listinfo/vtk-developers


_______________________________________________
Powered by www.kitware.com

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Search the list archives at: http://markmail.org/search/?q=vtk-developers

Follow this link to subscribe/unsubscribe:
https://public.kitware.com/mailman/listinfo/vtk-developers

 

_______________________________________________
Powered by www.kitware.com

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Search the list archives at: http://markmail.org/search/?q=vtk-developers

Follow this link to subscribe/unsubscribe:
https://public.kitware.com/mailman/listinfo/vtk-developers

_______________________________________________
Powered by www.kitware.com

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Search the list archives at: http://markmail.org/search/?q=vtk-developers

Follow this link to subscribe/unsubscribe:
https://public.kitware.com/mailman/listinfo/vtk-developers

_______________________________________________
Powered by www.kitware.com

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Search the list archives at: http://markmail.org/search/?q=vtk-developers

Follow this link to subscribe/unsubscribe:
https://public.kitware.com/mailman/listinfo/vtk-developers

_______________________________________________
Powered by www.kitware.com

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Search the list archives at: http://markmail.org/search/?q=vtk-developers

Follow this link to subscribe/unsubscribe:
https://public.kitware.com/mailman/listinfo/vtk-developers


_______________________________________________
Powered by www.kitware.com

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Search the list archives at: http://markmail.org/search/?q=vtk-developers

Follow this link to subscribe/unsubscribe:
https://public.kitware.com/mailman/listinfo/vtk-developers

Reply | Threaded
Open this post in threaded view
|

Re: Add pugixml to VTK

Dan Lipsa-2

I'm agnostic :) 

An idea would be to maybe try to eliminate expat at some point afterwards, to bring it down to just two libs again? I don't think the speed advantage of expat over libxml2 could really matter that much? (libxml2 has both a SAX-like and a pull-style reader API).
Sounds good. I think the problem is finding the time to do it.

libxml2 can then be the power horse for those who want 100% conformance and possibility of validation, while pugi for those who need speedy DOM.

This was my thought about this as well.
 
 
I wonder what are the speed-requiring use cases for DOM parsing though? Are you parsing many small docs or a few big ones? Is the XML parsing a true bottleneck?
 
For citygml I am parsing a few large files. I am not sure where the bottleneck is as I have not implemented it. The processing, after parsing, seems pretty simple though - traversing the tree and moving the data into a VTK polydata.

 
Elvis



On Wed, Jun 20, 2018 at 12:13 PM Robert Michael O'Bara <[hidden email]> wrote:
SMTK and CMB use pugixml for years and have been very happy with it.

Bob

Sent from my iPad

On Jun 20, 2018, at 12:07 PM, Jonathan Borduas <[hidden email]> wrote:

My 2 cents here is that speed is of the essence. Validating XML are good for debugging, however I found that most don’t take the time to create an XSD file to enable the validation.

 

In most of our cases at Caboma, XML and meant to be human readable and editable, but it is rare that we manually edit the structure of an XML file.

However, we do wait after .xml parsing when the files are 20mb+. We hope that the community would move to the latest version of pugixml (1.9): by looking at the changelogs, there was a lot of performance improvements since version 1.4 (the current version used in ParaView).
https://pugixml.org/docs/manual.html#v1.5

 

Best

 

Jonathan Borduas

CTO, Caboma Inc.

 

 

From: vtk-developers <[hidden email]> On Behalf Of David E DeMarle
Sent: Wednesday, June 20, 2018 11:49 AM
To: Elvis Stansvik <[hidden email]>
Cc: vtkdev <[hidden email]>
Subject: Re: [vtk-developers] Add pugixml to VTK

 

 I was thinking more if there should be a push towards one single lib, ...

 

I for one would really like us to get down to just one xml parser in VTK. The more the less merrier in terms of maintenance effort and project complexity eh?

 

David E DeMarle
Kitware, Inc.
Principal Engineer
21 Corporate Drive
Clifton Park, NY 12065-8662
Phone: 518-881-4909

 

 

On Tue, Jun 19, 2018 at 3:36 PM Elvis Stansvik <[hidden email]> wrote:

2018-06-16 8:37 GMT+02:00 Chiranjib Sur <[hidden email]>:

My few cents... The disadvantage with pugixml is that it's a non-validating parser. If I want to make full use of XML schema for any custom developed plugin, it's a very poor choice then.

 

Any thoughts?

 

Also worth noting that it's a DOM-like parser only AFAIK, so if you want to do streaming parsing in a push paradigm à la SAX or a pull paradigm à la libxml2's reader interface, you'd need to turn to another library. Note that may not be a problem if VTK already has libxml2 as a dep, since then you could use that. I was thinking more if there should be a push towards one single lib, then it might make sense to pick one that has both DOM and streaming interfaces (don't know if that's a goal/consideration though).

 

Elvis

 

 

Thanks and regards,
Chiranjib

------
Using hand held device. Sorry for the typo, if any.

 

On Fri, Jun 15, 2018, 11:59 PM Utkarsh Ayachit <[hidden email]> wrote:

> I thought Ben Boeckel was working on a removing as many things from ThirdParty as possible (and making them parts of a superbuild). So perhaps the CityGML reader should live in a VTK module that requires an external pugi? I know pugi is small, but...

I am not sure that's correct. He was doing that for Python 3rd party
modules. Are you referring to that?
_______________________________________________
Powered by www.kitware.com

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Search the list archives at: http://markmail.org/search/?q=vtk-developers

Follow this link to subscribe/unsubscribe:
https://public.kitware.com/mailman/listinfo/vtk-developers


_______________________________________________
Powered by www.kitware.com

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Search the list archives at: http://markmail.org/search/?q=vtk-developers

Follow this link to subscribe/unsubscribe:
https://public.kitware.com/mailman/listinfo/vtk-developers

 

_______________________________________________
Powered by www.kitware.com

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Search the list archives at: http://markmail.org/search/?q=vtk-developers

Follow this link to subscribe/unsubscribe:
https://public.kitware.com/mailman/listinfo/vtk-developers

_______________________________________________
Powered by www.kitware.com

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Search the list archives at: http://markmail.org/search/?q=vtk-developers

Follow this link to subscribe/unsubscribe:
https://public.kitware.com/mailman/listinfo/vtk-developers

_______________________________________________
Powered by www.kitware.com

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Search the list archives at: http://markmail.org/search/?q=vtk-developers

Follow this link to subscribe/unsubscribe:
https://public.kitware.com/mailman/listinfo/vtk-developers

_______________________________________________
Powered by www.kitware.com

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Search the list archives at: http://markmail.org/search/?q=vtk-developers

Follow this link to subscribe/unsubscribe:
https://public.kitware.com/mailman/listinfo/vtk-developers


_______________________________________________
Powered by www.kitware.com

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Search the list archives at: http://markmail.org/search/?q=vtk-developers

Follow this link to subscribe/unsubscribe:
https://public.kitware.com/mailman/listinfo/vtk-developers

Reply | Threaded
Open this post in threaded view
|

Re: Add pugixml to VTK

Elvis Stansvik
2018-07-05 18:40 GMT+02:00 Dan Lipsa <[hidden email]>:

>>
>> I'm agnostic :)
>>
>>
>> An idea would be to maybe try to eliminate expat at some point afterwards,
>> to bring it down to just two libs again? I don't think the speed advantage
>> of expat over libxml2 could really matter that much? (libxml2 has both a
>> SAX-like and a pull-style reader API).
>
> Sounds good. I think the problem is finding the time to do it.

Yes, this was more of a suggestion for something to aspire towards.
Didn't mean it should block the addition of pugixml.

>
>> libxml2 can then be the power horse for those who want 100% conformance
>> and possibility of validation, while pugi for those who need speedy DOM.
>
>
> This was my thought about this as well.
>
>
>>
>> I wonder what are the speed-requiring use cases for DOM parsing though?
>> Are you parsing many small docs or a few big ones? Is the XML parsing a true
>> bottleneck?
>
>
> For citygml I am parsing a few large files. I am not sure where the
> bottleneck is as I have not implemented it. The processing, after parsing,
> seems pretty simple though - traversing the tree and moving the data into a
> VTK polydata.

Alright. Maybe this could actually benefit from being done in a
streaming fashion (à la libxml2's reader API)? Or will it be
beneficial/easier to have access to the whole DOM in memory?

Elvis

>
>
>>
>> Elvis
>>
>>>
>>>
>>> On Wed, Jun 20, 2018 at 12:13 PM Robert Michael O'Bara
>>> <[hidden email]> wrote:
>>>>
>>>> SMTK and CMB use pugixml for years and have been very happy with it.
>>>>
>>>> Bob
>>>>
>>>> Sent from my iPad
>>>>
>>>> On Jun 20, 2018, at 12:07 PM, Jonathan Borduas
>>>> <[hidden email]> wrote:
>>>>
>>>> My 2 cents here is that speed is of the essence. Validating XML are good
>>>> for debugging, however I found that most don’t take the time to create an
>>>> XSD file to enable the validation.
>>>>
>>>>
>>>>
>>>> In most of our cases at Caboma, XML and meant to be human readable and
>>>> editable, but it is rare that we manually edit the structure of an XML file.
>>>>
>>>> However, we do wait after .xml parsing when the files are 20mb+. We hope
>>>> that the community would move to the latest version of pugixml (1.9): by
>>>> looking at the changelogs, there was a lot of performance improvements since
>>>> version 1.4 (the current version used in ParaView).
>>>> https://pugixml.org/docs/manual.html#v1.5
>>>>
>>>>
>>>>
>>>> Best
>>>>
>>>>
>>>>
>>>> Jonathan Borduas
>>>>
>>>> CTO, Caboma Inc.
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> From: vtk-developers <[hidden email]> On
>>>> Behalf Of David E DeMarle
>>>> Sent: Wednesday, June 20, 2018 11:49 AM
>>>> To: Elvis Stansvik <[hidden email]>
>>>> Cc: vtkdev <[hidden email]>
>>>> Subject: Re: [vtk-developers] Add pugixml to VTK
>>>>
>>>>
>>>>
>>>>  I was thinking more if there should be a push towards one single lib,
>>>> ...
>>>>
>>>>
>>>>
>>>> I for one would really like us to get down to just one xml parser in
>>>> VTK. The more the less merrier in terms of maintenance effort and project
>>>> complexity eh?
>>>>
>>>>
>>>>
>>>> David E DeMarle
>>>> Kitware, Inc.
>>>> Principal Engineer
>>>> 21 Corporate Drive
>>>> Clifton Park, NY 12065-8662
>>>> Phone: 518-881-4909
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> On Tue, Jun 19, 2018 at 3:36 PM Elvis Stansvik
>>>> <[hidden email]> wrote:
>>>>
>>>> 2018-06-16 8:37 GMT+02:00 Chiranjib Sur <[hidden email]>:
>>>>
>>>> My few cents... The disadvantage with pugixml is that it's a
>>>> non-validating parser. If I want to make full use of XML schema for any
>>>> custom developed plugin, it's a very poor choice then.
>>>>
>>>>
>>>>
>>>> Any thoughts?
>>>>
>>>>
>>>>
>>>> Also worth noting that it's a DOM-like parser only AFAIK, so if you want
>>>> to do streaming parsing in a push paradigm à la SAX or a pull paradigm à la
>>>> libxml2's reader interface, you'd need to turn to another library. Note that
>>>> may not be a problem if VTK already has libxml2 as a dep, since then you
>>>> could use that. I was thinking more if there should be a push towards one
>>>> single lib, then it might make sense to pick one that has both DOM and
>>>> streaming interfaces (don't know if that's a goal/consideration though).
>>>>
>>>>
>>>>
>>>> Elvis
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> Thanks and regards,
>>>> Chiranjib
>>>>
>>>> ------
>>>> Using hand held device. Sorry for the typo, if any.
>>>>
>>>>
>>>>
>>>> On Fri, Jun 15, 2018, 11:59 PM Utkarsh Ayachit
>>>> <[hidden email]> wrote:
>>>>
>>>> > I thought Ben Boeckel was working on a removing as many things from
>>>> > ThirdParty as possible (and making them parts of a superbuild). So perhaps
>>>> > the CityGML reader should live in a VTK module that requires an external
>>>> > pugi? I know pugi is small, but...
>>>>
>>>> I am not sure that's correct. He was doing that for Python 3rd party
>>>> modules. Are you referring to that?
>>>> _______________________________________________
>>>> Powered by www.kitware.com
>>>>
>>>> Visit other Kitware open-source projects at
>>>> http://www.kitware.com/opensource/opensource.html
>>>>
>>>> Search the list archives at:
>>>> http://markmail.org/search/?q=vtk-developers
>>>>
>>>> Follow this link to subscribe/unsubscribe:
>>>> https://public.kitware.com/mailman/listinfo/vtk-developers
>>>>
>>>>
>>>> _______________________________________________
>>>> Powered by www.kitware.com
>>>>
>>>> Visit other Kitware open-source projects at
>>>> http://www.kitware.com/opensource/opensource.html
>>>>
>>>> Search the list archives at:
>>>> http://markmail.org/search/?q=vtk-developers
>>>>
>>>> Follow this link to subscribe/unsubscribe:
>>>> https://public.kitware.com/mailman/listinfo/vtk-developers
>>>>
>>>>
>>>>
>>>> _______________________________________________
>>>> Powered by www.kitware.com
>>>>
>>>> Visit other Kitware open-source projects at
>>>> http://www.kitware.com/opensource/opensource.html
>>>>
>>>> Search the list archives at:
>>>> http://markmail.org/search/?q=vtk-developers
>>>>
>>>> Follow this link to subscribe/unsubscribe:
>>>> https://public.kitware.com/mailman/listinfo/vtk-developers
>>>>
>>>> _______________________________________________
>>>> Powered by www.kitware.com
>>>>
>>>> Visit other Kitware open-source projects at
>>>> http://www.kitware.com/opensource/opensource.html
>>>>
>>>> Search the list archives at:
>>>> http://markmail.org/search/?q=vtk-developers
>>>>
>>>> Follow this link to subscribe/unsubscribe:
>>>> https://public.kitware.com/mailman/listinfo/vtk-developers
>>>>
>>>> _______________________________________________
>>>> Powered by www.kitware.com
>>>>
>>>> Visit other Kitware open-source projects at
>>>> http://www.kitware.com/opensource/opensource.html
>>>>
>>>> Search the list archives at:
>>>> http://markmail.org/search/?q=vtk-developers
>>>>
>>>> Follow this link to subscribe/unsubscribe:
>>>> https://public.kitware.com/mailman/listinfo/vtk-developers
>>>>
>>> _______________________________________________
>>> Powered by www.kitware.com
>>>
>>> Visit other Kitware open-source projects at
>>> http://www.kitware.com/opensource/opensource.html
>>>
>>> Search the list archives at: http://markmail.org/search/?q=vtk-developers
>>>
>>> Follow this link to subscribe/unsubscribe:
>>> https://public.kitware.com/mailman/listinfo/vtk-developers
>>>
>
_______________________________________________
Powered by www.kitware.com

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Search the list archives at: http://markmail.org/search/?q=vtk-developers

Follow this link to subscribe/unsubscribe:
https://public.kitware.com/mailman/listinfo/vtk-developers

Reply | Threaded
Open this post in threaded view
|

Re: Add pugixml to VTK

Dan Lipsa-2


Alright. Maybe this could actually benefit from being done in a
streaming fashion (à la libxml2's reader API)? Or will it be
beneficial/easier to have access to the whole DOM in memory?

You are right - it could be implemented in streaming fashion.
I think the trade off is
- streaming: slower than dom, less memory than dom, possibly more complicated as you may need to store data for later.
- dom: quickest, more memory than streaming, simplest way to access citygml data.

For now I lean toward dom as our input data is tiled to start with so I don't expect really large files. I realize I have to revise the specs of the problem:
I'll need to parse many medium size XML files.
Dan
 

_______________________________________________
Powered by www.kitware.com

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Search the list archives at: http://markmail.org/search/?q=vtk-developers

Follow this link to subscribe/unsubscribe:
https://public.kitware.com/mailman/listinfo/vtk-developers

Reply | Threaded
Open this post in threaded view
|

Re: Add pugixml to VTK

Elvis Stansvik
2018-07-06 23:26 GMT+02:00 Jonathan Borduas <[hidden email]>:
> Hi,
>
> Here’s a large xml (paraview statefile with 200+ sphere). It weighs 17.5 MB.
> It takes 1-2 minutes to load with pugixml 1.4 on ParaView.

Alright. That shows the time of XML parsing is minuscule compared to
other things in that use case. That file parses in < 400 ms with
libxml2 here, and I guess pugixml would be even faster.

Elvis

>
>
>
> Best,
>
>
>
> Jonathan Borduas
>
>
>
> From: vtk-developers <[hidden email]> On Behalf
> Of Dan Lipsa
> Sent: Friday, July 06, 2018 4:59 PM
> To: [hidden email]
> Cc: VTK Developers <[hidden email]>
> Subject: Re: [vtk-developers] Add pugixml to VTK
>
>
>
>
>
>
> Alright. Maybe this could actually benefit from being done in a
> streaming fashion (à la libxml2's reader API)? Or will it be
> beneficial/easier to have access to the whole DOM in memory?
>
>
>
> You are right - it could be implemented in streaming fashion.
>
> I think the trade off is
>
> - streaming: slower than dom, less memory than dom, possibly more
> complicated as you may need to store data for later.
>
> - dom: quickest, more memory than streaming, simplest way to access citygml
> data.
>
>
>
> For now I lean toward dom as our input data is tiled to start with so I
> don't expect really large files. I realize I have to revise the specs of the
> problem:
>
> I'll need to parse many medium size XML files.
>
> Dan
>
>
_______________________________________________
Powered by www.kitware.com

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Search the list archives at: http://markmail.org/search/?q=vtk-developers

Follow this link to subscribe/unsubscribe:
https://public.kitware.com/mailman/listinfo/vtk-developers

Reply | Threaded
Open this post in threaded view
|

Re: Add pugixml to VTK

drjustice
Have you guys tried profiling what happens during those 2 minutes?

> On Jul 6, 2018, at 17:56, Elvis Stansvik <[hidden email]> wrote:
>
> 2018-07-06 23:26 GMT+02:00 Jonathan Borduas <[hidden email]>:
>> Hi,
>>
>> Here’s a large xml (paraview statefile with 200+ sphere). It weighs 17.5 MB.
>> It takes 1-2 minutes to load with pugixml 1.4 on ParaView.
>
> Alright. That shows the time of XML parsing is minuscule compared to
> other things in that use case. That file parses in < 400 ms with
> libxml2 here, and I guess pugixml would be even faster.
>
> Elvis
>
>>
>>
>>
>> Best,
>>
>>
>>
>> Jonathan Borduas
>>
>>
>>
>> From: vtk-developers <[hidden email]> On Behalf
>> Of Dan Lipsa
>> Sent: Friday, July 06, 2018 4:59 PM
>> To: [hidden email]
>> Cc: VTK Developers <[hidden email]>
>> Subject: Re: [vtk-developers] Add pugixml to VTK
>>
>>
>>
>>
>>
>>
>> Alright. Maybe this could actually benefit from being done in a
>> streaming fashion (à la libxml2's reader API)? Or will it be
>> beneficial/easier to have access to the whole DOM in memory?
>>
>>
>>
>> You are right - it could be implemented in streaming fashion.
>>
>> I think the trade off is
>>
>> - streaming: slower than dom, less memory than dom, possibly more
>> complicated as you may need to store data for later.
>>
>> - dom: quickest, more memory than streaming, simplest way to access citygml
>> data.
>>
>>
>>
>> For now I lean toward dom as our input data is tiled to start with so I
>> don't expect really large files. I realize I have to revise the specs of the
>> problem:
>>
>> I'll need to parse many medium size XML files.
>>
>> Dan
>>
>>
> _______________________________________________
> Powered by www.kitware.com
>
> Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html
>
> Search the list archives at: http://markmail.org/search/?q=vtk-developers
>
> Follow this link to subscribe/unsubscribe:
> https://public.kitware.com/mailman/listinfo/vtk-developers
>
_______________________________________________
Powered by www.kitware.com

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Search the list archives at: http://markmail.org/search/?q=vtk-developers

Follow this link to subscribe/unsubscribe:
https://public.kitware.com/mailman/listinfo/vtk-developers

Reply | Threaded
Open this post in threaded view
|

Re: Add pugixml to VTK

Dan Lipsa-2
Hi all,
I have 4 responses so far, all for adding pugixml to VTK. I'll go ahead and do it. ParaView, CMB and SMTK would benefit
as they'll have less maintenance cost and VTK will have access to pugi.

On Fri, Jul 6, 2018 at 9:20 PM Patrick Bergeron <[hidden email]> wrote:
Have you guys tried profiling what happens during those 2 minutes?

> On Jul 6, 2018, at 17:56, Elvis Stansvik <[hidden email]> wrote:
>
> 2018-07-06 23:26 GMT+02:00 Jonathan Borduas <[hidden email]>:
>> Hi,
>>
>> Here’s a large xml (paraview statefile with 200+ sphere). It weighs 17.5 MB.
>> It takes 1-2 minutes to load with pugixml 1.4 on ParaView.
>
Indeed, this seem like a long time. Do you mind filing a bug report and attaching the state file. Not sure what is the reason for this.
 
> Alright. That shows the time of XML parsing is minuscule compared to
> other things in that use case. That file parses in < 400 ms with
> libxml2 here, and I guess pugixml would be even faster. 
>
> Elvis
>
>>
>>
>>
>> Best,
>>
>>
>>
>> Jonathan Borduas
>>
>>
>>
>> From: vtk-developers <[hidden email]> On Behalf
>> Of Dan Lipsa
>> Sent: Friday, July 06, 2018 4:59 PM
>> To: [hidden email]
>> Cc: VTK Developers <[hidden email]>
>> Subject: Re: [vtk-developers] Add pugixml to VTK
>>
>>
>>
>>
>>
>>
>> Alright. Maybe this could actually benefit from being done in a
>> streaming fashion (à la libxml2's reader API)? Or will it be
>> beneficial/easier to have access to the whole DOM in memory?
>>
>>
>>
>> You are right - it could be implemented in streaming fashion.
>>
>> I think the trade off is
>>
>> - streaming: slower than dom, less memory than dom, possibly more
>> complicated as you may need to store data for later.
>>
>> - dom: quickest, more memory than streaming, simplest way to access citygml
>> data.
>>
>>
>>
>> For now I lean toward dom as our input data is tiled to start with so I
>> don't expect really large files. I realize I have to revise the specs of the
>> problem:
>>
>> I'll need to parse many medium size XML files.
>>
>> Dan
>>
>>
> _______________________________________________
> Powered by www.kitware.com
>
> Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html
>
> Search the list archives at: http://markmail.org/search/?q=vtk-developers
>
> Follow this link to subscribe/unsubscribe:
> https://public.kitware.com/mailman/listinfo/vtk-developers
>
_______________________________________________
Powered by www.kitware.com

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Search the list archives at: http://markmail.org/search/?q=vtk-developers

Follow this link to subscribe/unsubscribe:
https://public.kitware.com/mailman/listinfo/vtk-developers


_______________________________________________
Powered by www.kitware.com

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Search the list archives at: http://markmail.org/search/?q=vtk-developers

Follow this link to subscribe/unsubscribe:
https://public.kitware.com/mailman/listinfo/vtk-developers

Reply | Threaded
Open this post in threaded view
|

Re: Add pugixml to VTK

Dan Lipsa-2
Hi all,
Pugixml has been added to VTK master.


Dan



On Tue, Jul 17, 2018 at 9:54 AM Dan Lipsa <[hidden email]> wrote:
Hi all,
I have 4 responses so far, all for adding pugixml to VTK. I'll go ahead and do it. ParaView, CMB and SMTK would benefit
as they'll have less maintenance cost and VTK will have access to pugi.

On Fri, Jul 6, 2018 at 9:20 PM Patrick Bergeron <[hidden email]> wrote:
Have you guys tried profiling what happens during those 2 minutes?

> On Jul 6, 2018, at 17:56, Elvis Stansvik <[hidden email]> wrote:
>
> 2018-07-06 23:26 GMT+02:00 Jonathan Borduas <[hidden email]>:
>> Hi,
>>
>> Here’s a large xml (paraview statefile with 200+ sphere). It weighs 17.5 MB.
>> It takes 1-2 minutes to load with pugixml 1.4 on ParaView.
>
Indeed, this seem like a long time. Do you mind filing a bug report and attaching the state file. Not sure what is the reason for this.
 
> Alright. That shows the time of XML parsing is minuscule compared to
> other things in that use case. That file parses in < 400 ms with
> libxml2 here, and I guess pugixml would be even faster. 
>
> Elvis
>
>>
>>
>>
>> Best,
>>
>>
>>
>> Jonathan Borduas
>>
>>
>>
>> From: vtk-developers <[hidden email]> On Behalf
>> Of Dan Lipsa
>> Sent: Friday, July 06, 2018 4:59 PM
>> To: [hidden email]
>> Cc: VTK Developers <[hidden email]>
>> Subject: Re: [vtk-developers] Add pugixml to VTK
>>
>>
>>
>>
>>
>>
>> Alright. Maybe this could actually benefit from being done in a
>> streaming fashion (à la libxml2's reader API)? Or will it be
>> beneficial/easier to have access to the whole DOM in memory?
>>
>>
>>
>> You are right - it could be implemented in streaming fashion.
>>
>> I think the trade off is
>>
>> - streaming: slower than dom, less memory than dom, possibly more
>> complicated as you may need to store data for later.
>>
>> - dom: quickest, more memory than streaming, simplest way to access citygml
>> data.
>>
>>
>>
>> For now I lean toward dom as our input data is tiled to start with so I
>> don't expect really large files. I realize I have to revise the specs of the
>> problem:
>>
>> I'll need to parse many medium size XML files.
>>
>> Dan
>>
>>
> _______________________________________________
> Powered by www.kitware.com
>
> Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html
>
> Search the list archives at: http://markmail.org/search/?q=vtk-developers
>
> Follow this link to subscribe/unsubscribe:
> https://public.kitware.com/mailman/listinfo/vtk-developers
>
_______________________________________________
Powered by www.kitware.com

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Search the list archives at: http://markmail.org/search/?q=vtk-developers

Follow this link to subscribe/unsubscribe:
https://public.kitware.com/mailman/listinfo/vtk-developers


_______________________________________________
Powered by www.kitware.com

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Search the list archives at: http://markmail.org/search/?q=vtk-developers

Follow this link to subscribe/unsubscribe:
https://public.kitware.com/mailman/listinfo/vtk-developers