How to display vtkMappedUnstructuredGrid with my Implementation class

classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|

How to display vtkMappedUnstructuredGrid with my Implementation class

Dmitriy Babkov
Hello, everyone!
I'm new to VTK and would like to use it in my project instead of my own
OpenGL rendering system;
I think, I need to keep data structures as it is and it seems I have to use
vtkMappedUnstructuredGrid class.
I found an example UGrid, which demonstrates, how to work with
vtkUnstructuredGrid.
To examine vtkMappedUnstructuredGrid I decided to modify this example. I
have implemented required by manual Implementation class methods and
replaced usage of vtkUnstructuredGrid to vtkMappedUnstructuredGrid with my
Implementation. But nothing is displayed... I was trying to set breakpoints
to vtkMappedUnstructuredGrid template methods to understand, what I have
missed, but it seems no methods called to request data information. Only
constructor, destructor and GetMTime, which seems not required to be
implemented in Implementation class.  
Here is my c++ code:  main.cpp
<http://vtk.1045678.n5.nabble.com/file/t342524/main.cpp>  
And CMake project file:  CMakeLists.txt
<http://vtk.1045678.n5.nabble.com/file/t342524/CMakeLists.txt>  
Please, help me to understand this template class and find my fault.
My OS is arch linux and VTK vtk-8.1.0 installed.



--
Sent from: http://vtk.1045678.n5.nabble.com/VTK-Users-f1224199.html
_______________________________________________
Powered by www.kitware.com

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Please keep messages on-topic and check the VTK FAQ at: http://www.vtk.org/Wiki/VTK_FAQ

Search the list archives at: http://markmail.org/search/?q=vtkusers

Follow this link to subscribe/unsubscribe:
https://public.kitware.com/mailman/listinfo/vtkusers
Best regards,
Dmitriy
Reply | Threaded
Open this post in threaded view
|

Re: How to display vtkMappedUnstructuredGrid with my Implementation class

Allie Vacanti
The rendering engine is shortcircuiting because the dataset has no coordinate information (the `x` array is never used by the mapped grid implementation -- `mesh->GetPoints()` is returning nullptr). Adding `mesh->SetPoints(points);` to the test script will complete the dataset and let the renderer progress.

Be warned, however, that the mapped unstructured grid stuff is not widely used and there may be some further issues lurking as you go farther with it. They may be a need to debug/patch VTK itself if you want to use it.

I should also point out that the mapped unstructured grid is not used directly in the rendering -- it is first converted to a `vtkPolyData` by a `vtkDataSetSurfaceFilter` inside of the `vtkDataSetMapper`. If you encounter a lot of issues with the mapped grid, you may find it easier to just build a `vtkPolyData` directly from your application's data. The mapped grids were intended to be used on extremely large simulation datasets that could not be physically copied without running out of memory on an HPC node -- if that's not your usecase, directly building the `vtkPolyData` will likely save you a lot of hassle.

HTH,
Allie

On Tue, Aug 7, 2018 at 1:45 AM, dsbabkov <[hidden email]> wrote:
Hello, everyone!
I'm new to VTK and would like to use it in my project instead of my own
OpenGL rendering system;
I think, I need to keep data structures as it is and it seems I have to use
vtkMappedUnstructuredGrid class.
I found an example UGrid, which demonstrates, how to work with
vtkUnstructuredGrid.
To examine vtkMappedUnstructuredGrid I decided to modify this example. I
have implemented required by manual Implementation class methods and
replaced usage of vtkUnstructuredGrid to vtkMappedUnstructuredGrid with my
Implementation. But nothing is displayed... I was trying to set breakpoints
to vtkMappedUnstructuredGrid template methods to understand, what I have
missed, but it seems no methods called to request data information. Only
constructor, destructor and GetMTime, which seems not required to be
implemented in Implementation class. 
Here is my c++ code:  main.cpp
<http://vtk.1045678.n5.nabble.com/file/t342524/main.cpp
And CMake project file:  CMakeLists.txt
<http://vtk.1045678.n5.nabble.com/file/t342524/CMakeLists.txt
Please, help me to understand this template class and find my fault.
My OS is arch linux and VTK vtk-8.1.0 installed.



--
Sent from: http://vtk.1045678.n5.nabble.com/VTK-Users-f1224199.html
_______________________________________________
Powered by www.kitware.com

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Please keep messages on-topic and check the VTK FAQ at: http://www.vtk.org/Wiki/VTK_FAQ

Search the list archives at: http://markmail.org/search/?q=vtkusers

Follow this link to subscribe/unsubscribe:
https://public.kitware.com/mailman/listinfo/vtkusers


_______________________________________________
Powered by www.kitware.com

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Please keep messages on-topic and check the VTK FAQ at: http://www.vtk.org/Wiki/VTK_FAQ

Search the list archives at: http://markmail.org/search/?q=vtkusers

Follow this link to subscribe/unsubscribe:
https://public.kitware.com/mailman/listinfo/vtkusers
Reply | Threaded
Open this post in threaded view
|

Re: How to display vtkMappedUnstructuredGrid with my Implementation class

Dmitriy Babkov
Allie, thank you very much!
I don't believe, that I missed points data. I was sure, that it was used in Implementation class.
Now I am able to run UGrid example on mapped grid. There was one more issue: I had to call Allocate for vtkIdList *ptIds in GetCellPoints method. Otherwise I can see corrupted elements. I don't know is it normal to VTK preallocate list or it's just mapped grid class issue.

I have no idea which classes VTK should I use for better performance and RAM consumption in my project, because I have no experience with VTK
Let me tell you some features of project:
 - Goal of project is to display loaded from custom or known format FEM tetra mesh, modify it and save to custom format
 - There are element/face/node selectors, sections constructed with user planes, display feature angles (edges).
 - Each face of element has it color from global list of ~100 colors.
 - Elements grouped in collectors and visible of collectors can be toggled.
 - It will not be run on HPC node, I think

Memory consumption of application now is about 2-3Gb if i load ~10M elements.
The reason of taking mapped grid was usage of VTK rendering and, maybe, VTK algorithms without significant additional RAM consumption.
Will it be reasonable to build my own vtkPolyData for each type of objects (mesh, sections, selected elements/faces/nodes)? It seems VTK algorithms can work better than mine and with less issues. Also it seems It will take less time.


вт, 7 авг. 2018 г. в 20:19, Allie Vacanti <[hidden email]>:
The rendering engine is shortcircuiting because the dataset has no coordinate information (the `x` array is never used by the mapped grid implementation -- `mesh->GetPoints()` is returning nullptr). Adding `mesh->SetPoints(points);` to the test script will complete the dataset and let the renderer progress.

Be warned, however, that the mapped unstructured grid stuff is not widely used and there may be some further issues lurking as you go farther with it. They may be a need to debug/patch VTK itself if you want to use it.

I should also point out that the mapped unstructured grid is not used directly in the rendering -- it is first converted to a `vtkPolyData` by a `vtkDataSetSurfaceFilter` inside of the `vtkDataSetMapper`. If you encounter a lot of issues with the mapped grid, you may find it easier to just build a `vtkPolyData` directly from your application's data. The mapped grids were intended to be used on extremely large simulation datasets that could not be physically copied without running out of memory on an HPC node -- if that's not your usecase, directly building the `vtkPolyData` will likely save you a lot of hassle.

HTH,
Allie

On Tue, Aug 7, 2018 at 1:45 AM, dsbabkov <[hidden email]> wrote:
Hello, everyone!
I'm new to VTK and would like to use it in my project instead of my own
OpenGL rendering system;
I think, I need to keep data structures as it is and it seems I have to use
vtkMappedUnstructuredGrid class.
I found an example UGrid, which demonstrates, how to work with
vtkUnstructuredGrid.
To examine vtkMappedUnstructuredGrid I decided to modify this example. I
have implemented required by manual Implementation class methods and
replaced usage of vtkUnstructuredGrid to vtkMappedUnstructuredGrid with my
Implementation. But nothing is displayed... I was trying to set breakpoints
to vtkMappedUnstructuredGrid template methods to understand, what I have
missed, but it seems no methods called to request data information. Only
constructor, destructor and GetMTime, which seems not required to be
implemented in Implementation class. 
Here is my c++ code:  main.cpp
<http://vtk.1045678.n5.nabble.com/file/t342524/main.cpp
And CMake project file:  CMakeLists.txt
<http://vtk.1045678.n5.nabble.com/file/t342524/CMakeLists.txt
Please, help me to understand this template class and find my fault.
My OS is arch linux and VTK vtk-8.1.0 installed.



--
Sent from: http://vtk.1045678.n5.nabble.com/VTK-Users-f1224199.html
_______________________________________________
Powered by www.kitware.com

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Please keep messages on-topic and check the VTK FAQ at: http://www.vtk.org/Wiki/VTK_FAQ

Search the list archives at: http://markmail.org/search/?q=vtkusers

Follow this link to subscribe/unsubscribe:
https://public.kitware.com/mailman/listinfo/vtkusers


_______________________________________________
Powered by www.kitware.com

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Please keep messages on-topic and check the VTK FAQ at: http://www.vtk.org/Wiki/VTK_FAQ

Search the list archives at: http://markmail.org/search/?q=vtkusers

Follow this link to subscribe/unsubscribe:
https://public.kitware.com/mailman/listinfo/vtkusers
Best regards,
Dmitriy
Reply | Threaded
Open this post in threaded view
|

Re: How to display vtkMappedUnstructuredGrid with my Implementation class

Allie Vacanti
On Wed, Aug 8, 2018 at 4:39 AM, Дмитрий Бабков <[hidden email]> wrote:
Allie, thank you very much!
I don't believe, that I missed points data. I was sure, that it was used in Implementation class.
Now I am able to run UGrid example on mapped grid. There was one more issue: I had to call Allocate for vtkIdList *ptIds in GetCellPoints method. Otherwise I can see corrupted elements. I don't know is it normal to VTK preallocate list or it's just mapped grid class issue.

In the original implementation of GetCellPoints, you had:

        const Element &element = elements[cellId];
        for (vtkIdType i = 0; i < element.pointsCount; ++i) {
            ptIds->InsertNextId(element.points[i]);
        }

The issue here is that calling InsertNextId always appends to the ptIds, without clear old data first. So the list will always keep growing. Many algorithms reuse the same vtkIdList object in repeated calls to GetPointIds, since the memory can often be reused without reallocating. In this case, you'll definitely see incorrect results.

As you mentioned, resizing the list and setting elements explicitly by index would fix this. Class methods in VTK are expected to manually set the size of these containers prior to filling them.

I have no idea which classes VTK should I use for better performance and RAM consumption in my project, because I have no experience with VTK
Let me tell you some features of project:
 - Goal of project is to display loaded from custom or known format FEM tetra mesh, modify it and save to custom format
 - There are element/face/node selectors, sections constructed with user planes, display feature angles (edges).
 - Each face of element has it color from global list of ~100 colors.
 - Elements grouped in collectors and visible of collectors can be toggled.
 - It will not be run on HPC node, I think

Memory consumption of application now is about 2-3Gb if i load ~10M elements.
The reason of taking mapped grid was usage of VTK rendering and, maybe, VTK algorithms without significant additional RAM consumption.
Will it be reasonable to build my own vtkPolyData for each type of objects (mesh, sections, selected elements/faces/nodes)? It seems VTK algorithms can work better than mine and with less issues. Also it seems It will take less time.

The mapped unstructured grid was originally intended to be used in a workflow such as:

1) Generate large dataset in a native format
2) Create MappedUnstructuredGrid wrapper around native data
3) Use a VTK algorithm to massively reduce the data (extract faces, threshold, clip, etc)
4) Create a renderable vtkPolyData of the reduced dataset that will fit in memory

as opposed to creating a large in-memory copy of the complete dataset in step two.

If you plan to use a simple pipeline, or a filter that converts to a different dataset type early in the pipeline, you should be fine with the mapped unstructured grid. The only caution is that not all filters that work on vtkUnstructuredGrid will work with vtkMappedUnstructuredGrid, though many will.

It makes sense that you want to let VTK handle the conversion of the dataset to a renderable form. In this case, I'd say try using the mapped grid and see if it works for you on the filters you wish to use. But if you run into to trouble, you may need to update a troublesome filter to work on a vtkUnstructuredGridBase instead of vtkUnstructuredGrid, or switch to using vtkUnstructuredGrid directly instead of a vtkMappedUnstructuredGrid.

Allie

вт, 7 авг. 2018 г. в 20:19, Allie Vacanti <[hidden email]>:
The rendering engine is shortcircuiting because the dataset has no coordinate information (the `x` array is never used by the mapped grid implementation -- `mesh->GetPoints()` is returning nullptr). Adding `mesh->SetPoints(points);` to the test script will complete the dataset and let the renderer progress.

Be warned, however, that the mapped unstructured grid stuff is not widely used and there may be some further issues lurking as you go farther with it. They may be a need to debug/patch VTK itself if you want to use it.

I should also point out that the mapped unstructured grid is not used directly in the rendering -- it is first converted to a `vtkPolyData` by a `vtkDataSetSurfaceFilter` inside of the `vtkDataSetMapper`. If you encounter a lot of issues with the mapped grid, you may find it easier to just build a `vtkPolyData` directly from your application's data. The mapped grids were intended to be used on extremely large simulation datasets that could not be physically copied without running out of memory on an HPC node -- if that's not your usecase, directly building the `vtkPolyData` will likely save you a lot of hassle.

HTH,
Allie

On Tue, Aug 7, 2018 at 1:45 AM, dsbabkov <[hidden email]> wrote:
Hello, everyone!
I'm new to VTK and would like to use it in my project instead of my own
OpenGL rendering system;
I think, I need to keep data structures as it is and it seems I have to use
vtkMappedUnstructuredGrid class.
I found an example UGrid, which demonstrates, how to work with
vtkUnstructuredGrid.
To examine vtkMappedUnstructuredGrid I decided to modify this example. I
have implemented required by manual Implementation class methods and
replaced usage of vtkUnstructuredGrid to vtkMappedUnstructuredGrid with my
Implementation. But nothing is displayed... I was trying to set breakpoints
to vtkMappedUnstructuredGrid template methods to understand, what I have
missed, but it seems no methods called to request data information. Only
constructor, destructor and GetMTime, which seems not required to be
implemented in Implementation class. 
Here is my c++ code:  main.cpp
<http://vtk.1045678.n5.nabble.com/file/t342524/main.cpp
And CMake project file:  CMakeLists.txt
<http://vtk.1045678.n5.nabble.com/file/t342524/CMakeLists.txt
Please, help me to understand this template class and find my fault.
My OS is arch linux and VTK vtk-8.1.0 installed.



--
Sent from: http://vtk.1045678.n5.nabble.com/VTK-Users-f1224199.html
_______________________________________________
Powered by www.kitware.com

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Please keep messages on-topic and check the VTK FAQ at: http://www.vtk.org/Wiki/VTK_FAQ

Search the list archives at: http://markmail.org/search/?q=vtkusers

Follow this link to subscribe/unsubscribe:
https://public.kitware.com/mailman/listinfo/vtkusers



_______________________________________________
Powered by www.kitware.com

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Please keep messages on-topic and check the VTK FAQ at: http://www.vtk.org/Wiki/VTK_FAQ

Search the list archives at: http://markmail.org/search/?q=vtkusers

Follow this link to subscribe/unsubscribe:
https://public.kitware.com/mailman/listinfo/vtkusers
Reply | Threaded
Open this post in threaded view
|

Re: How to display vtkMappedUnstructuredGrid with my Implementation class

Dmitriy Babkov
Apologize for the timing.
Many algorithms reuse the same vtkIdList object in repeated calls to GetPointIds, since the memory can often be reused without reallocating.
Thank you for this. Best for me was using Reset method in example.

I think I'll try to use vtkMappedUnstructuredGrid for my project to get fast result. Then I'll try to measure RAM and algorithm time consumption of vtk classes and if it satisfy my project requirements, I'll replace my classes with vtkUnstructuredGrid.

Thank's a lot for the help!

Best regards,
Dmitriy.

2018-08-08 17:10 GMT+03:00 Allie Vacanti <[hidden email]>:
On Wed, Aug 8, 2018 at 4:39 AM, Дмитрий Бабков <[hidden email]> wrote:
Allie, thank you very much!
I don't believe, that I missed points data. I was sure, that it was used in Implementation class.
Now I am able to run UGrid example on mapped grid. There was one more issue: I had to call Allocate for vtkIdList *ptIds in GetCellPoints method. Otherwise I can see corrupted elements. I don't know is it normal to VTK preallocate list or it's just mapped grid class issue.

In the original implementation of GetCellPoints, you had:

        const Element &element = elements[cellId];
        for (vtkIdType i = 0; i < element.pointsCount; ++i) {
            ptIds->InsertNextId(element.points[i]);
        }

The issue here is that calling InsertNextId always appends to the ptIds, without clear old data first. So the list will always keep growing. Many algorithms reuse the same vtkIdList object in repeated calls to GetPointIds, since the memory can often be reused without reallocating. In this case, you'll definitely see incorrect results.

As you mentioned, resizing the list and setting elements explicitly by index would fix this. Class methods in VTK are expected to manually set the size of these containers prior to filling them.

I have no idea which classes VTK should I use for better performance and RAM consumption in my project, because I have no experience with VTK
Let me tell you some features of project:
 - Goal of project is to display loaded from custom or known format FEM tetra mesh, modify it and save to custom format
 - There are element/face/node selectors, sections constructed with user planes, display feature angles (edges).
 - Each face of element has it color from global list of ~100 colors.
 - Elements grouped in collectors and visible of collectors can be toggled.
 - It will not be run on HPC node, I think

Memory consumption of application now is about 2-3Gb if i load ~10M elements.
The reason of taking mapped grid was usage of VTK rendering and, maybe, VTK algorithms without significant additional RAM consumption.
Will it be reasonable to build my own vtkPolyData for each type of objects (mesh, sections, selected elements/faces/nodes)? It seems VTK algorithms can work better than mine and with less issues. Also it seems It will take less time.

The mapped unstructured grid was originally intended to be used in a workflow such as:

1) Generate large dataset in a native format
2) Create MappedUnstructuredGrid wrapper around native data
3) Use a VTK algorithm to massively reduce the data (extract faces, threshold, clip, etc)
4) Create a renderable vtkPolyData of the reduced dataset that will fit in memory

as opposed to creating a large in-memory copy of the complete dataset in step two.

If you plan to use a simple pipeline, or a filter that converts to a different dataset type early in the pipeline, you should be fine with the mapped unstructured grid. The only caution is that not all filters that work on vtkUnstructuredGrid will work with vtkMappedUnstructuredGrid, though many will.

It makes sense that you want to let VTK handle the conversion of the dataset to a renderable form. In this case, I'd say try using the mapped grid and see if it works for you on the filters you wish to use. But if you run into to trouble, you may need to update a troublesome filter to work on a vtkUnstructuredGridBase instead of vtkUnstructuredGrid, or switch to using vtkUnstructuredGrid directly instead of a vtkMappedUnstructuredGrid.

Allie

вт, 7 авг. 2018 г. в 20:19, Allie Vacanti <[hidden email]>:
The rendering engine is shortcircuiting because the dataset has no coordinate information (the `x` array is never used by the mapped grid implementation -- `mesh->GetPoints()` is returning nullptr). Adding `mesh->SetPoints(points);` to the test script will complete the dataset and let the renderer progress.

Be warned, however, that the mapped unstructured grid stuff is not widely used and there may be some further issues lurking as you go farther with it. They may be a need to debug/patch VTK itself if you want to use it.

I should also point out that the mapped unstructured grid is not used directly in the rendering -- it is first converted to a `vtkPolyData` by a `vtkDataSetSurfaceFilter` inside of the `vtkDataSetMapper`. If you encounter a lot of issues with the mapped grid, you may find it easier to just build a `vtkPolyData` directly from your application's data. The mapped grids were intended to be used on extremely large simulation datasets that could not be physically copied without running out of memory on an HPC node -- if that's not your usecase, directly building the `vtkPolyData` will likely save you a lot of hassle.

HTH,
Allie

On Tue, Aug 7, 2018 at 1:45 AM, dsbabkov <[hidden email]> wrote:
Hello, everyone!
I'm new to VTK and would like to use it in my project instead of my own
OpenGL rendering system;
I think, I need to keep data structures as it is and it seems I have to use
vtkMappedUnstructuredGrid class.
I found an example UGrid, which demonstrates, how to work with
vtkUnstructuredGrid.
To examine vtkMappedUnstructuredGrid I decided to modify this example. I
have implemented required by manual Implementation class methods and
replaced usage of vtkUnstructuredGrid to vtkMappedUnstructuredGrid with my
Implementation. But nothing is displayed... I was trying to set breakpoints
to vtkMappedUnstructuredGrid template methods to understand, what I have
missed, but it seems no methods called to request data information. Only
constructor, destructor and GetMTime, which seems not required to be
implemented in Implementation class. 
Here is my c++ code:  main.cpp
<http://vtk.1045678.n5.nabble.com/file/t342524/main.cpp
And CMake project file:  CMakeLists.txt
<http://vtk.1045678.n5.nabble.com/file/t342524/CMakeLists.txt
Please, help me to understand this template class and find my fault.
My OS is arch linux and VTK vtk-8.1.0 installed.



--
Sent from: http://vtk.1045678.n5.nabble.com/VTK-Users-f1224199.html
_______________________________________________
Powered by www.kitware.com

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Please keep messages on-topic and check the VTK FAQ at: http://www.vtk.org/Wiki/VTK_FAQ

Search the list archives at: http://markmail.org/search/?q=vtkusers

Follow this link to subscribe/unsubscribe:
https://public.kitware.com/mailman/listinfo/vtkusers




_______________________________________________
Powered by www.kitware.com

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Please keep messages on-topic and check the VTK FAQ at: http://www.vtk.org/Wiki/VTK_FAQ

Search the list archives at: http://markmail.org/search/?q=vtkusers

Follow this link to subscribe/unsubscribe:
https://public.kitware.com/mailman/listinfo/vtkusers
Best regards,
Dmitriy