How's the Projection Transform Matrix computed in VTK? and Generate Cloud Point From Depth Map Image

classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

How's the Projection Transform Matrix computed in VTK? and Generate Cloud Point From Depth Map Image

maskjp
This post was updated on .
depthmaptopointcloud.zip
<http://vtk.1045678.n5.nabble.com/file/t342591/depthmaptopointcloud.zip

Hi, I wrote a code in python to generate a depth map of a 3D model and save it
as an image.
But when I wrote code to generate the point cloud from the image, only part
of the point cloud was generated. And I found the projection transform matrix is different, even every other camera parameter are the same. The results are different from using the following matrix. So I also what to know how does vtk calculate projection transform matrix.



Thanks!

I attached the code and 3D model in this post.

#####Generate Depth map Images#####
import vtk
import os
import numpy as np
from vtk.util import numpy_support


reader = vtk.vtkXMLPolyDataReader()
data_path = 'C:/Users/jiang/Repository/ModelNet10/chair_train_scaled'
path = os.path.join(data_path, "chair_0001.vtp") #Archive path
reader.SetFileName(path)
reader.Update()

ren = vtk.vtkRenderer()
renWin = vtk.vtkRenderWindow()
renWin.AddRenderer(ren)
iren = vtk.vtkRenderWindowInteractor()
iren.SetRenderWindow(renWin)

bounds = reader.GetOutput().GetBounds()

voxelModeller = vtk.vtkVoxelModeller ()
voxelModeller.SetSampleDimensions(128,128,128)
voxelModeller.SetModelBounds(bounds)
voxelModeller.SetMaximumDistance(0.1)
voxelModeller.SetScalarTypeToFloat()

voxelModeller.SetInputConnection(reader.GetOutputPort())
voxelModeller.Update()
 
# Create transfer mapping scalar value to opacity
opacityTransferFunction = vtk.vtkPiecewiseFunction()
opacityTransferFunction.AddPoint(0, 0.0)
opacityTransferFunction.AddPoint(1, 1)

# Create transfer mapping scalar value to color
colorTransferFunction = vtk.vtkColorTransferFunction()
colorTransferFunction.AddRGBPoint(0.0, 1.0, 1.0, 1.0)
colorTransferFunction.AddRGBPoint(1.0, 1.0, 0.0, 0.0)

# The property describes how the data will look
volumeProperty = vtk.vtkVolumeProperty()
volumeProperty.SetColor(colorTransferFunction)
volumeProperty.SetScalarOpacity(opacityTransferFunction)
volumeProperty.ShadeOn()
volumeProperty.SetInterpolationTypeToLinear()

# The mapper / ray cast function know how to render the data
volumeMapper = vtk.vtkGPUVolumeRayCastMapper()
volumeMapper.SetBlendModeToComposite()
volumeMapper.SetInputConnection(voxelModeller.GetOutputPort())
volumeMapper.RenderToImageOn()
# The volume holds the mapper and the property and
# can be used to position/orient the volume
volume = vtk.vtkVolume()
volume.SetMapper(volumeMapper)
volume.SetProperty(volumeProperty)


ren.AddVolume(volume)
ren.SetBackground(1, 1, 1)
renWin.SetSize(128, 128)


ren.GetActiveCamera().SetPosition(32, 0,0)
ren.GetActiveCamera().SetFocalPoint(0, 0, 0)
ren.GetActiveCamera().SetViewUp(0, 0, 1)
ren.GetActiveCamera().SetClippingRange(20,50)
renWin.Render()

dpimg = vtk.vtkImageData()
volumeMapper.GetDepthImage(dpimg)

scale = vtk.vtkImageShiftScale()
scale.SetOutputScalarTypeToUnsignedChar()

scale.SetInputData(dpimg)
scale.SetShift(0)
scale.SetScale(255)


imageWriter = vtk.vtkBMPWriter()
imageWriter.SetFileName("depthmap1.bmp")
imageWriter.SetInputConnection(scale.GetOutputPort())
imageWriter.Write()


####Generate point cloud from depth map image

import vtk
import os
import numpy as np
from vtk.util import numpy_support

reader = vtk.vtkBMPReader()
reader.SetFileName('depthmap3.bmp')
reader.Update()
dpscalar = reader.GetOutput().GetPointData().GetScalars()
dpnp1d = numpy_support.vtk_to_numpy(dpscalar)
print(np.max(dpnp1d),np.min(dpnp1d))

scale = vtk.vtkImageShiftScale()
scale.SetOutputScalarTypeToFloat()
#scale.SetInputConnection(filter.GetOutputPort())
scale.SetInputConnection(reader.GetOutputPort())
scale.SetShift(0)
#scale.SetScale(-1/255.0)
scale.SetScale(1/255.0)
scale.Update()

dpscalar = scale.GetOutput().GetPointData().GetScalars()
dpnp1d = numpy_support.vtk_to_numpy(dpscalar)
print(np.max(dpnp1d),np.min(dpnp1d))


ren = vtk.vtkRenderer()
renWin = vtk.vtkRenderWindow()
ren.SetBackground(1, 1, 1)
renWin.SetSize(128,128)
renWin.AddRenderer(ren)
iren = vtk.vtkRenderWindowInteractor()
iren.SetRenderWindow(renWin)


camera =ren.GetActiveCamera()
camera.SetPosition(32, 0,0)
camera.SetFocalPoint(0, 0, 0)
camera.SetViewUp(0, 0, 1)
camera.SetClippingRange(20,50)
axial = vtk.vtkMatrix4x4()
axial.DeepCopy((3.73205, 0, 0, 0,
                 0, 3.73205, 0, 0,
                 0, 0, -2.33333, -66.6667,
                 0, 0, -1, 0))
#camera.SetUseExplicitProjectionTransformMatrix(True)
#camera.SetExplicitProjectionTransformMatrix(axial)
print(camera)

print(camera.GetWindowCenter ())

pc = vtk.vtkDepthImageToPointCloud()
pc.SetInputConnection(scale.GetOutputPort())
pc.SetCamera(camera)
pc.CullNearPointsOn()
pc.CullFarPointsOn()
#pc.ProduceVertexCellArrayOff()
pc.Update()
print(pc)
#print(ren.GetActiveCamera())

pcMapper = vtk.vtkPolyDataMapper()
pcMapper.SetInputConnection(pc.GetOutputPort())

# pcMapper = vtk.vtkPointGaussianMapper()
# pcMapper.SetInputConnection(pc.GetOutputPort())
# pcMapper.EmissiveOff()
# pcMapper.SetScaleFactor(0.0)

pcActor = vtk.vtkActor()
pcActor.SetMapper(pcMapper)
iren.Initialize()
ren.AddActor(pcActor)
ren.SetBackground(0,0,0)
# ren.GetActiveCamera().SetPosition(32, 0,0)
# ren.GetActiveCamera().SetFocalPoint(0, 0, 0)
# ren.GetActiveCamera().SetViewUp(0, 0, 1)
ren.ResetCamera()
renWin.Render()
iren.Start()



Anyone have any idea? very appreciative.




Thanks!



--
Sent from: http://vtk.1045678.n5.nabble.com/VTK-Users-f1224199.html
_______________________________________________
Powered by www.kitware.com

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Please keep messages on-topic and check the VTK FAQ at: http://www.vtk.org/Wiki/VTK_FAQ

Search the list archives at: http://markmail.org/search/?q=vtkusers

Follow this link to subscribe/unsubscribe:
https://public.kitware.com/mailman/listinfo/vtkusers
Reply | Threaded
Open this post in threaded view
|

Re: How's the Projection Transform Matrix computed in VTK? and Generate Cloud Point From Depth Map Image

maskjp
I figure out the problem. When I use the vtkBMPReader, the output of this
class is a 3 channel vtkImageData, but the vtkDepthImageToPointCloud only
receive a 1 channel depth image. So We need to use vtkImageExtractComponents
to get the first channel of the vtkBMPReader output.  

But I still don't understand how vtk compute projection transform matrix.



--
Sent from: http://vtk.1045678.n5.nabble.com/VTK-Users-f1224199.html
_______________________________________________
Powered by www.kitware.com

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Please keep messages on-topic and check the VTK FAQ at: http://www.vtk.org/Wiki/VTK_FAQ

Search the list archives at: http://markmail.org/search/?q=vtkusers

Follow this link to subscribe/unsubscribe:
https://public.kitware.com/mailman/listinfo/vtkusers
Reply | Threaded
Open this post in threaded view
|

Re: How's the Projection Transform Matrix computed in VTK? and Generate Cloud Point From Depth Map Image

Allie Vacanti
Depending on settings, there are a few ways it can be computed. vtkCamera::ComputeProjectionTransform is the starting point for tracking down the specifics:

https://github.com/Kitware/VTK/blob/master/Rendering/Core/vtkCamera.cxx#L1035-L1127

HTH,
Allie

On Fri, Oct 19, 2018 at 12:05 PM maskjp <[hidden email]> wrote:
I figure out the problem. When I use the vtkBMPReader, the output of this
class is a 3 channel vtkImageData, but the vtkDepthImageToPointCloud only
receive a 1 channel depth image. So We need to use vtkImageExtractComponents
to get the first channel of the vtkBMPReader output. 

But I still don't understand how vtk compute projection transform matrix.



--
Sent from: http://vtk.1045678.n5.nabble.com/VTK-Users-f1224199.html
_______________________________________________
Powered by www.kitware.com

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Please keep messages on-topic and check the VTK FAQ at: http://www.vtk.org/Wiki/VTK_FAQ

Search the list archives at: http://markmail.org/search/?q=vtkusers

Follow this link to subscribe/unsubscribe:
https://public.kitware.com/mailman/listinfo/vtkusers

_______________________________________________
Powered by www.kitware.com

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Please keep messages on-topic and check the VTK FAQ at: http://www.vtk.org/Wiki/VTK_FAQ

Search the list archives at: http://markmail.org/search/?q=vtkusers

Follow this link to subscribe/unsubscribe:
https://public.kitware.com/mailman/listinfo/vtkusers