Horrible performance partitioning an UnstructuredGrid into connected regions with vtkConnectivityFilter

classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

Horrible performance partitioning an UnstructuredGrid into connected regions with vtkConnectivityFilter

scotsman60
Hello!!!

I am partitioning a single large vtkUnstructuredGrid with around 15 million
Points and Cells into contiguous cell regions using the
vtkConnectivityFilter.

I'm using VTK 7.1 with Python 3.6.2

The filter finds around 4,700 regions quite quickly - in about 10 seconds.

But then it takes almost the same amount of time to extract each region into
it's own grid and add to a MultiBlockDataSet - this is my end goal at the
moment.

Here is the code,

    def extractConnectedRegions(self):
        start = timer()
        connect = vtk.vtkConnectivityFilter()
        connect.SetInputData(self.mesh)
        connect.SetExtractionModeToSpecifiedRegions()
        connect.ColorRegionsOn();
        connect.Update()
        num_regions = connect.GetNumberOfExtractedRegions()
        end = timer()
        print("Extracted " + str(num_regions) + " regions in " +
str(end-start) + " seconds")
               
        self.mb = vtk.vtkMultiBlockDataSet()
               
        i=0
        while True:
            start = timer()
            extractGrid = vtk.vtkUnstructuredGrid()
            connect.AddSpecifiedRegion(i)
            connect.Update()
            extractGrid.DeepCopy(connect.GetOutput())
            if extractGrid.GetNumberOfCells() <= 0:
                break
            print(i,extractGrid.GetNumberOfCells())
            self.mb.SetBlock(i, extractGrid)
            connect.DeleteSpecifiedRegion(i)
            end = timer()
            print("Processed region " + str(i) + " in " + str(end-start) + "
seconds")
            i+=1
           
        return num_regions

 
The output from this fragment is shown below,

Time to add 15811239 points to vtkUnstructuredGrid - 0.19109638981581245
seconds
Time to size array for 15685372 elements is 8.62148243552241e-06 seconds
Time to create connectivity array for 15685372 elements is
0.6341917319162356 seconds
Time to populate vtkUnstructuredGrid with 15685372 cells is
1.5787703795422896 seconds
Extracted 4739 regions in 10.1558246735332 seconds
0 1
Processed region 0 in 7.074993349142119 seconds
1 1
Processed region 1 in 7.067869541369976 seconds
2 1
Processed region 2 in 6.956592478126069 seconds
3 1
Processed region 3 in 6.968502850836856 seconds


I have to believe that there's a more efficient way to do this than the one
I'm using, but I haven't been able to find one.

Any suggestions, hints or solutions will be gratefully appreciated.

Doug





--
Sent from: http://vtk.1045678.n5.nabble.com/VTK-Users-f1224199.html
_______________________________________________
Powered by www.kitware.com

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Please keep messages on-topic and check the VTK FAQ at: http://www.vtk.org/Wiki/VTK_FAQ

Search the list archives at: http://markmail.org/search/?q=vtkusers

Follow this link to subscribe/unsubscribe:
https://public.kitware.com/mailman/listinfo/vtkusers
Reply | Threaded
Open this post in threaded view
|

Re: Horrible performance partitioning an UnstructuredGrid into connected regions with vtkConnectivityFilter

kenichiro yoshimi
Hi Doug,

It may be faster to use a SetExtractionModeToAllRegions method in
vtkConnectivityFilter and then extract each region by vtkThreshold one
by one.

-----
    connect.SetExtractionModeToAllRegions()
    ...

    threshold = vtk.vtkThreshold()
    threshold.SetInputConnection(connect.GetOutputPort())
    threshold.SetInputArrayToProcess(0, 0, 0,
vtk.vtkDataObject.FIELD_ASSOCIATION_CELLS, 'RegionId')

    i = 0
    while True:
        start = timer()
        extractGrid = vtk.vtkUnstructuredGrid()
        threshold.ThresholdBetween(i, i)
        threshold.Update()
        extractGrid.DeepCopy(threshold.GetOutput())
        if extractGrid.GetNumberOfCells() <= 0:
            break
        print(i, extractGrid.GetNumberOfCells())
        mb.SetBlock(i, extractGrid)
        end = timer()
        print("Processed region " + str(i) + " in " + str(end-start) +
"seconds")
        i += 1
-----

Best

2018-06-17 18:44 GMT+09:00 scotsman60 <[hidden email]>:

> Hello!!!
>
> I am partitioning a single large vtkUnstructuredGrid with around 15 million
> Points and Cells into contiguous cell regions using the
> vtkConnectivityFilter.
>
> I'm using VTK 7.1 with Python 3.6.2
>
> The filter finds around 4,700 regions quite quickly - in about 10 seconds.
>
> But then it takes almost the same amount of time to extract each region into
> it's own grid and add to a MultiBlockDataSet - this is my end goal at the
> moment.
>
> Here is the code,
>
>     def extractConnectedRegions(self):
>         start = timer()
>         connect = vtk.vtkConnectivityFilter()
>         connect.SetInputData(self.mesh)
>         connect.SetExtractionModeToSpecifiedRegions()
>         connect.ColorRegionsOn();
>         connect.Update()
>         num_regions = connect.GetNumberOfExtractedRegions()
>         end = timer()
>         print("Extracted " + str(num_regions) + " regions in " +
> str(end-start) + " seconds")
>
>         self.mb = vtk.vtkMultiBlockDataSet()
>
>         i=0
>         while True:
>             start = timer()
>             extractGrid = vtk.vtkUnstructuredGrid()
>             connect.AddSpecifiedRegion(i)
>             connect.Update()
>             extractGrid.DeepCopy(connect.GetOutput())
>             if extractGrid.GetNumberOfCells() <= 0:
>                 break
>             print(i,extractGrid.GetNumberOfCells())
>             self.mb.SetBlock(i, extractGrid)
>             connect.DeleteSpecifiedRegion(i)
>             end = timer()
>             print("Processed region " + str(i) + " in " + str(end-start) + "
> seconds")
>             i+=1
>
>         return num_regions
>
>
> The output from this fragment is shown below,
>
> Time to add 15811239 points to vtkUnstructuredGrid - 0.19109638981581245
> seconds
> Time to size array for 15685372 elements is 8.62148243552241e-06 seconds
> Time to create connectivity array for 15685372 elements is
> 0.6341917319162356 seconds
> Time to populate vtkUnstructuredGrid with 15685372 cells is
> 1.5787703795422896 seconds
> Extracted 4739 regions in 10.1558246735332 seconds
> 0 1
> Processed region 0 in 7.074993349142119 seconds
> 1 1
> Processed region 1 in 7.067869541369976 seconds
> 2 1
> Processed region 2 in 6.956592478126069 seconds
> 3 1
> Processed region 3 in 6.968502850836856 seconds
>
>
> I have to believe that there's a more efficient way to do this than the one
> I'm using, but I haven't been able to find one.
>
> Any suggestions, hints or solutions will be gratefully appreciated.
>
> Doug
>
>
>
>
>
> --
> Sent from: http://vtk.1045678.n5.nabble.com/VTK-Users-f1224199.html
> _______________________________________________
> Powered by www.kitware.com
>
> Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html
>
> Please keep messages on-topic and check the VTK FAQ at: http://www.vtk.org/Wiki/VTK_FAQ
>
> Search the list archives at: http://markmail.org/search/?q=vtkusers
>
> Follow this link to subscribe/unsubscribe:
> https://public.kitware.com/mailman/listinfo/vtkusers
_______________________________________________
Powered by www.kitware.com

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Please keep messages on-topic and check the VTK FAQ at: http://www.vtk.org/Wiki/VTK_FAQ

Search the list archives at: http://markmail.org/search/?q=vtkusers

Follow this link to subscribe/unsubscribe:
https://public.kitware.com/mailman/listinfo/vtkusers
Reply | Threaded
Open this post in threaded view
|

Re: Horrible performance partitioning an UnstructuredGrid into connected regions with vtkConnectivityFilter

scotsman60
kenichiro yoshimi wrote
> Hi Doug,
>
> It may be faster to use a SetExtractionModeToAllRegions method in
> vtkConnectivityFilter and then extract each region by vtkThreshold one
> by one.
>
> -

Kenichiro,

Since my original post I had actually tried what you suggested and the
extraction time came down to 2.5 seconds per region. I got even better
performance using the selection method -  vtkSelection, vtkSelectionNode and
vtkExtactSelection brought it down to 0.6 seconds per extracted region.

But with almost 5,000 regions to extract that's still too slow. So my latest
strategy is to use the vtkConnectivityFilter to find the regions - if you
use SetExtractionModeToAllRegionsusing this causes a Scalar array of
RegionIds to be added to the UG. I'm planning to use that array to process
the original cell data that I have in a series of numpy arrays (one per cell
type) so that I can simply create each region as a new UG and the throw away
the original UG. Basically I'm betting that numpy manipulation is faster
than extracting and copying using UG methods...... We'll see how that works
out.....

For reference, here's the code for the selection method. It took a little
while to understand how the classes work together, but in the end it works
quite well - jut too slow for my needs.....


    def extractConnectedRegions2(self):
       
        num_cell_arrays = self.mesh.GetCellData().GetNumberOfArrays()
        start = timer()
        connect = vtk.vtkConnectivityFilter()
        connect.SetInputData(self.mesh)
        connect.SetExtractionModeToAllRegions()
        connect.ColorRegionsOn();
        connect.Update()
        num_regions = connect.GetNumberOfExtractedRegions()
        end = timer()
        print("Extracted " + str(num_regions) + " regions in " +
str(end-start) + " seconds")

        start = timer()

        self.extractGrid = vtk.vtkUnstructuredGrid()
        self.extractGrid.DeepCopy(connect.GetOutput())      

        end = timer()
        print("Deep copied in " + str(end-start) + " seconds")
               
        selectionNode =vtk.vtkSelectionNode()
        selectionNode.SetFieldType(vtk.vtkSelectionNode.CELL)
        selectionNode.SetContentType(vtk.vtkSelectionNode.VALUES)
 
        selection = vtk.vtkSelection()
        selection.AddNode(selectionNode)
 
        extract_selection = vtk.vtkExtractSelection()
        extract_selection.SetInputData(0, self.extractGrid)        
        extract_selection.SetInputData(1, selection)        

        self.mb = vtk.vtkMultiBlockDataSet()

        for i in range(num_regions):
            start = timer()
            ids = vtk.vtkIntArray()
            ids.InsertNextValue(i)
            ids.SetName("RegionId")
            selectionNode.SetSelectionList(ids)
            val = ids.GetValue(0)
           
            extract_selection.Update()  
            leaf = vtk.vtkUnstructuredGrid()
            leaf.DeepCopy(extract_selection.GetOutput())    
            self.mb.SetBlock(i, leaf)
            end = timer()
           
            print("Processed region " + str(i) + " in " + str(end-start) + "
seconds")
           
       
        return num_regions



Here's the timing info from the above

Deep copied in 0.46751967130063576 seconds
Processed region 0 in 0.7530222326956988 seconds
Processed region 1 in 0.6797586976174443 seconds
Processed region 2 in 0.6042322342475757 seconds
Processed region 3 in 0.6011810580749852 seconds
Processed region 4 in 0.5983285860557288 seconds
Processed region 5 in 0.604301616482914 seconds
Processed region 6 in 0.6362392036725879 seconds
Processed region 7 in 0.6126373378241681 seconds






--
Sent from: http://vtk.1045678.n5.nabble.com/VTK-Users-f1224199.html
_______________________________________________
Powered by www.kitware.com

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Please keep messages on-topic and check the VTK FAQ at: http://www.vtk.org/Wiki/VTK_FAQ

Search the list archives at: http://markmail.org/search/?q=vtkusers

Follow this link to subscribe/unsubscribe:
https://public.kitware.com/mailman/listinfo/vtkusers