Profiling VTK/Performance improvements (with Python)

classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

Profiling VTK/Performance improvements (with Python)

Felix Mayr
Dear vtkusers,

I'm currently working on volumetric visualization of
StructuredGrid-data. I'm using python to create the grids from numpy
arrays. My basic pipeline is:
grid (1.2GiB, dimensions: 241x241x721) → vtkIsoContourFilter (setting a
nonexistent value, as I want to change this dynamically) →
vtkClipPolyData (using an ImplicitFunction generated from PolyData) →
vtkPolyDataNormals (for nice shading) → vtkPolyDataMapper → vtkActor
(which is stored and added to the scene at a later date).

Now, I have constructed a Tk-GUI (using the vtkTkRenderWindowInteractor)
and update the IsoContourFilter with new contour values. And well, this
process is slow (as well as pipeline creation) and I would seriously
like to improve performance and find the bottleneck if there's any.
Specifically (VTK 8.1.2):
     - I don't have a graphics card available on my dev machine.
Basically I'm just using the Haswell-era integrated-graphics. On another
machine I am running with LLVM-accelerated mesa and it's behaving the
same. I don't think this is the problem.
     - can I somehow time the execution of pipeline parts on updates?
(to find, what takes so long)
     - do the filters I use support SMP (in the python-wrapping?). I
compiled it with all parallelization-related variables enabled, but CPU
still doesn't go beyond 100%.
or is it just an insane dataset and I should start with a sensible VOI
of it?

Best Regards

Felix Mayr
<[hidden email]>
Technical University of Munich, Simulation of Nanosystems for Energy
Conversion [www.sne.ei.tum.de]
_______________________________________________
Powered by www.kitware.com

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Please keep messages on-topic and check the VTK FAQ at: http://www.vtk.org/Wiki/VTK_FAQ

Search the list archives at: http://markmail.org/search/?q=vtkusers

Follow this link to subscribe/unsubscribe:
https://vtk.org/mailman/listinfo/vtkusers
Fcs
Reply | Threaded
Open this post in threaded view
|

Re: Profiling VTK/Performance improvements (with Python)

Fcs
Not much of a VTK optimizer myself, but a few ideas to move the conversation
forward:

- Did you set the VTK_SMP_IMPLEMENTATION_TYPE flag to the correct backend
for your compiler? (If you're using LLVM: OpenMP). And does your version of
LLVM support OpenMP? Do you have all the libraries installed?  As a side
note, maybe this document will help:
https://vtk.org/Wiki/images/3/3b/VTK_SMP_Guide.pdf

- For timing, you can possibly start by measuring the time elapsed when
calling Update(). See for instance the timer.StartTimer() calls in this
python code:
https://vtk.org/gitweb?p=VTK.git;a=blob;f=Filters/Points/Testing/Python/TestPointInterpolator.py

- What do you call slow? 41 million cells is not a particularly big test
case, so no need to worry about that. What you do need to keep an eye on,
though, is your memory usage. If you run out of memory and your system
starts swapping to disk, then VTK will be unbearably slow for sure..



--
Sent from: http://vtk.1045678.n5.nabble.com/VTK-Users-f1224199.html
_______________________________________________
Powered by www.kitware.com

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Please keep messages on-topic and check the VTK FAQ at: http://www.vtk.org/Wiki/VTK_FAQ

Search the list archives at: http://markmail.org/search/?q=vtkusers

Follow this link to subscribe/unsubscribe:
https://vtk.org/mailman/listinfo/vtkusers
Reply | Threaded
Open this post in threaded view
|

Re: Profiling VTK/Performance improvements (with Python)

Felix Mayr
Hey, thanks for your reply

> - Did you set the VTK_SMP_IMPLEMENTATION_TYPE flag to the correct backend
> for your compiler?

I did so, though (I'm using LLVM for the MESA-acceleration only). For
GCC it should be the same though. Probably this has something to do with
the python-wrappers, are they releasing the GIL?
> - For timing, you can possibly start by measuring the time elapsed when
> calling Update(). See for instance the timer.StartTimer() calls in this
> python code:
> https://vtk.org/gitweb?p=VTK.git;a=blob;f=Filters/Points/Testing/Python/TestPointInterpolator.py
Ok, this is not what I was after, as I hoped for something which could
profile the whole pipeline in one go (preferably also breaking down on
memory usage, etc). But apparently the best is to measure how much time
the Update-call is blocking (which I did with time.time())

> - What do you call slow? 41 million cells is not a particularly big test
> case, so no need to worry about that. What you do need to keep an eye on,
> though, is your memory usage. If you run out of memory and your system
> starts swapping to disk, then VTK will be unbearably slow for sure..

Memory is no problem, there's plenty of it available. And also with a
smaller grid, the pipeline takes long. Anyways, I just "instrumented"
all the Update()-calls one by one and found this:

After isocontours update 4.545
After isonormalizer update 11.738
After isoclipper update 146.398
After filter update 146.399

So basically the vtkClipPolyData ist taking enormously long (the other
steps might be slow, but are bearable. Especially as they are twice as
fast on the server-nodes). Is there anything which is faster? I tried
vtkExtractPolyDataGeometry (don't need cell subdivision), but this was
not significantly faster and for the vtkBooleanOperationPolyDataFilter I
didn't find any examples... Maybe the clipFunction (135 points) is to
complicated, I will try to first cut the bounding box, then apply the
vtkClipPolyData (tough this is not really improving to a point where I
could get interactive performance, but a constant 30s would be better I
guess).

Felix Mayr
<[hidden email]>
Technical University of Munich, Simulation of Nanosystems for Energy
Conversion [www.sne.ei.tum.de]




_______________________________________________
Powered by www.kitware.com

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Please keep messages on-topic and check the VTK FAQ at: http://www.vtk.org/Wiki/VTK_FAQ

Search the list archives at: http://markmail.org/search/?q=vtkusers

Follow this link to subscribe/unsubscribe:
https://vtk.org/mailman/listinfo/vtkusers
Fcs
Reply | Threaded
Open this post in threaded view
|

Re: Profiling VTK/Performance improvements (with Python)

Fcs
Some filters can be quite slow indeed.

I played around a bit to compare the performance of a vtkClipDataSet and a
vtkTableBasedClipDataSet to extract an isosurface on an unstructured grid,
and this is what I get:

Total number of cells : 41.16 million.

Using vtkClipDataSet:
 -> Clip Time : 83.096 [s]

Using vtkTableBasedClipDataSet:
 -> Clip Time : 6.173 [s]

Is that a filter you could consider?  They also have some new, fast,
isosurface generators which should parallelize nicely:
https://vtk.org/doc/nightly/html/classvtkFlyingEdges3D.html#details

I haven't worked with them myself, but the claim is that they offer a
significant speed-up.

I would definitely be interested in hearing if you found a suitable
solution,

Kind regards,
Francois.





--
Sent from: http://vtk.1045678.n5.nabble.com/VTK-Users-f1224199.html
_______________________________________________
Powered by www.kitware.com

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Please keep messages on-topic and check the VTK FAQ at: http://www.vtk.org/Wiki/VTK_FAQ

Search the list archives at: http://markmail.org/search/?q=vtkusers

Follow this link to subscribe/unsubscribe:
https://vtk.org/mailman/listinfo/vtkusers