Strange problem with vtkPLYReader

classic Classic list List threaded Threaded
10 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Strange problem with vtkPLYReader

Prabhu Ramachandran-3

Hi all,

We've been running into a strange issue with the vtkPLYReader ever since VTK 5.6 all the way up to 8.0.0.  I am attaching a simple VTK Python script.  If you run it a few times you'll see inconsistent results for the bounds of the data like so:

(0.0, 1.0, 0.0, 1.0, 0.0, 1.600000023841858)
(0.0, 1.5845632502852868e+29, 0.0, 1.0, 0.0, 1.600000023841858)

...


I've also attached a standard dataset for this.  I could find the bug but I figured someone more familiar with the code may be quicker fixing it. Thanks! :)


cheers,

Prabhu


_______________________________________________
Powered by www.kitware.com

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Search the list archives at: http://markmail.org/search/?q=vtk-developers

Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/vtk-developers


test_ply_vtk.py (288 bytes) Download Attachment
pyramid.ply (530 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Strange problem with vtkPLYReader

David E DeMarle
Please submit a bug report via gitlab to help us keep track of this.

On Aug 5, 2017 2:04 PM, "Prabhu Ramachandran" <[hidden email]> wrote:

Hi all,

We've been running into a strange issue with the vtkPLYReader ever since VTK 5.6 all the way up to 8.0.0.  I am attaching a simple VTK Python script.  If you run it a few times you'll see inconsistent results for the bounds of the data like so:

(0.0, 1.0, 0.0, 1.0, 0.0, 1.600000023841858)
(0.0, 1.5845632502852868e+29, 0.0, 1.0, 0.0, 1.600000023841858)

...


I've also attached a standard dataset for this.  I could find the bug but I figured someone more familiar with the code may be quicker fixing it. Thanks! :)


cheers,

Prabhu


_______________________________________________
Powered by www.kitware.com

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Search the list archives at: http://markmail.org/search/?q=vtk-developers

Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/vtk-developers



_______________________________________________
Powered by www.kitware.com

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Search the list archives at: http://markmail.org/search/?q=vtk-developers

Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/vtk-developers

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Strange problem with vtkPLYReader

Bill Lorensen
The PLYReader is fine. The file is not a proper PLY file. The face
indices should be zero-based, but in this file they are one-based. So,
the face
3 1 2 5
is referencing an point 5 which exceeds the points array which has
tuples from 0 - 4 (5 points).

On Thu, Aug 10, 2017 at 6:48 AM, David E DeMarle
<[hidden email]> wrote:

> Please submit a bug report via gitlab to help us keep track of this.
>
> On Aug 5, 2017 2:04 PM, "Prabhu Ramachandran" <[hidden email]>
> wrote:
>>
>> Hi all,
>>
>> We've been running into a strange issue with the vtkPLYReader ever since
>> VTK 5.6 all the way up to 8.0.0.  I am attaching a simple VTK Python script.
>> If you run it a few times you'll see inconsistent results for the bounds of
>> the data like so:
>>
>> (0.0, 1.0, 0.0, 1.0, 0.0, 1.600000023841858)
>> (0.0, 1.5845632502852868e+29, 0.0, 1.0, 0.0, 1.600000023841858)
>>
>> ...
>>
>>
>> I've also attached a standard dataset for this.  I could find the bug but
>> I figured someone more familiar with the code may be quicker fixing it.
>> Thanks! :)
>>
>>
>> cheers,
>>
>> Prabhu
>>
>>
>> _______________________________________________
>> Powered by www.kitware.com
>>
>> Visit other Kitware open-source projects at
>> http://www.kitware.com/opensource/opensource.html
>>
>> Search the list archives at: http://markmail.org/search/?q=vtk-developers
>>
>> Follow this link to subscribe/unsubscribe:
>> http://public.kitware.com/mailman/listinfo/vtk-developers
>>
>>
>
> _______________________________________________
> Powered by www.kitware.com
>
> Visit other Kitware open-source projects at
> http://www.kitware.com/opensource/opensource.html
>
> Search the list archives at: http://markmail.org/search/?q=vtk-developers
>
> Follow this link to subscribe/unsubscribe:
> http://public.kitware.com/mailman/listinfo/vtk-developers
>
>



--
Unpaid intern in BillsBasement at noware dot com
_______________________________________________
Powered by www.kitware.com

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Search the list archives at: http://markmail.org/search/?q=vtk-developers

Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/vtk-developers

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Strange problem with vtkPLYReader

Sean McBride
On Thu, 10 Aug 2017 13:06:18 -0400, Bill Lorensen said:

>The PLYReader is fine. The file is not a proper PLY file. The face
>indices should be zero-based, but in this file they are one-based. So,
>the face
>3 1 2 5
>is referencing an point 5 which exceeds the points array which has
>tuples from 0 - 4 (5 points).

In which case one may reasonably argue that the PLYReader is indeed broken, for failing to reject invalid files.  Under ASan/valgrind, would the PLYReader crash with this file?

Sean


_______________________________________________
Powered by www.kitware.com

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Search the list archives at: http://markmail.org/search/?q=vtk-developers

Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/vtk-developers

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Strange problem with vtkPLYReader

Bill Lorensen
I found the error with valgrind...


On Thu, Aug 10, 2017 at 1:29 PM, Sean McBride <[hidden email]> wrote:

> On Thu, 10 Aug 2017 13:06:18 -0400, Bill Lorensen said:
>
>>The PLYReader is fine. The file is not a proper PLY file. The face
>>indices should be zero-based, but in this file they are one-based. So,
>>the face
>>3 1 2 5
>>is referencing an point 5 which exceeds the points array which has
>>tuples from 0 - 4 (5 points).
>
> In which case one may reasonably argue that the PLYReader is indeed broken, for failing to reject invalid files.  Under ASan/valgrind, would the PLYReader crash with this file?
>
> Sean
>
>



--
Unpaid intern in BillsBasement at noware dot com
_______________________________________________
Powered by www.kitware.com

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Search the list archives at: http://markmail.org/search/?q=vtk-developers

Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/vtk-developers

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Strange problem with vtkPLYReader

Will Schroeder-2
In reply to this post by Sean McBride
Sean-

In which case one may reasonably argue that the PLYReader is indeed broken, for failing to reject invalid files.

You are opening a can of worms here. I would humbly suggest that in most all file formats in VTK if the data is invalid then bad things will happen. If you want to make a sanity check as part of the read process then do it as an optional step because fully vetting data can take forever.

Best,
W


On Thu, Aug 10, 2017 at 1:29 PM, Sean McBride <[hidden email]> wrote:
On Thu, 10 Aug 2017 13:06:18 -0400, Bill Lorensen said:

>The PLYReader is fine. The file is not a proper PLY file. The face
>indices should be zero-based, but in this file they are one-based. So,
>the face
>3 1 2 5
>is referencing an point 5 which exceeds the points array which has
>tuples from 0 - 4 (5 points).

In which case one may reasonably argue that the PLYReader is indeed broken, for failing to reject invalid files.  Under ASan/valgrind, would the PLYReader crash with this file?

Sean


_______________________________________________
Powered by www.kitware.com

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Search the list archives at: http://markmail.org/search/?q=vtk-developers

Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/vtk-developers




--
William J. Schroeder, PhD
Kitware, Inc. - Building the World's Technical Computing Software
28 Corporate Drive
Clifton Park, NY 12065
[hidden email]
http://www.kitware.com
(518) 881-4902

_______________________________________________
Powered by www.kitware.com

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Search the list archives at: http://markmail.org/search/?q=vtk-developers

Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/vtk-developers

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Strange problem with vtkPLYReader

Sean McBride
On Thu, 10 Aug 2017 13:38:28 -0400, Will Schroeder said:

>>In which case one may reasonably argue that the PLYReader is indeed broken,
>>for failing to reject invalid files.
>
>You are opening a can of worms here. I would humbly suggest that in most
>all file formats in VTK if the data is invalid then bad things will happen.

Yup, which is a huge weakness of VTK, IMNSHO.

>If you want to make a sanity check as part of the read process then do it
>as an optional step because fully vetting data can take forever.

Vetting the data does take time, yes.  Sometimes the code can be factored such that the validation happens at one level, and some lower level function can assume it's valid.

But the alternative is crashing upon invalid data.  That's bad, especially when dealing with data from untrusted sources like files or the network.  This is how we're in a world where opening maliciously crafted jpeg/pdf/font/etc files can be used to run arbitrary code and do all kinds of nastiness.

If I had infinite time, fuzzing the VTK readers would be a fun project...

Sean


_______________________________________________
Powered by www.kitware.com

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Search the list archives at: http://markmail.org/search/?q=vtk-developers

Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/vtk-developers

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Strange problem with vtkPLYReader

Elvis Stansvik
2017-08-10 19:59 GMT+02:00 Sean McBride <[hidden email]>:

> On Thu, 10 Aug 2017 13:38:28 -0400, Will Schroeder said:
>
>>>In which case one may reasonably argue that the PLYReader is indeed broken,
>>>for failing to reject invalid files.
>>
>>You are opening a can of worms here. I would humbly suggest that in most
>>all file formats in VTK if the data is invalid then bad things will happen.
>
> Yup, which is a huge weakness of VTK, IMNSHO.
>
>>If you want to make a sanity check as part of the read process then do it
>>as an optional step because fully vetting data can take forever.
>
> Vetting the data does take time, yes.  Sometimes the code can be factored such that the validation happens at one level, and some lower level function can assume it's valid.
>
> But the alternative is crashing upon invalid data.  That's bad, especially when dealing with data from untrusted sources like files or the network.  This is how we're in a world where opening maliciously crafted jpeg/pdf/font/etc files can be used to run arbitrary code and do all kinds of nastiness.

+1

I would think that for most applications, crashing hard upon opening a
faulty file is not an option, and not having the library fail
gracefully just forces applications to vet the data themselves, which
in some (many?) cases may be even more costly than had the library
done it during loading. I think the Robustness principle very much
applies here [1].

But yes, time...

Elvis

[1] https://en.wikipedia.org/wiki/Robustness_principle

>
> If I had infinite time, fuzzing the VTK readers would be a fun project...
>
> Sean
>
>
> _______________________________________________
> Powered by www.kitware.com
>
> Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html
>
> Search the list archives at: http://markmail.org/search/?q=vtk-developers
>
> Follow this link to subscribe/unsubscribe:
> http://public.kitware.com/mailman/listinfo/vtk-developers
>
_______________________________________________
Powered by www.kitware.com

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Search the list archives at: http://markmail.org/search/?q=vtk-developers

Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/vtk-developers

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Strange problem with vtkPLYReader

Will Schroeder-2
In reply to this post by Sean McBride
Yup, which is a huge weakness of VTK, IMNSHO.

Actually this is true of almost every scientific computing program that I have ever used, not just VTK.

But the alternative is crashing upon invalid data.

Another alternative is to assume that users are intelligent and know what they are doing and can figure out problems (very naive I know) :-) Vetting data is simple in concept but way worse than we think because if you carry it to extreme you've got to do things like check whether polygons are flat/non-self intersecting, cells are not turned inside out, data values are valid and indeed within the range specified, data time series are mutually consistent, etc. Like everything there are tradeoffs between getting stuff done and perfection, whatever the heck that is anyway :-) There are reasonable things to do but all in balance. I am going to continue to advocate that we need to keep thinking about new algorithms and designs (high-level stuff) and avoid getting stuck in the morass of technical perfection. What makes systems successful in the end are impactful capabilities--solving important problems--not the perfection of implementation. So if you've got some cycles to fuzz, I would go in the directions of added capabilities :-)

Best,
W
 

On Thu, Aug 10, 2017 at 1:59 PM, Sean McBride <[hidden email]> wrote:
On Thu, 10 Aug 2017 13:38:28 -0400, Will Schroeder said:

>>In which case one may reasonably argue that the PLYReader is indeed broken,
>>for failing to reject invalid files.
>
>You are opening a can of worms here. I would humbly suggest that in most
>all file formats in VTK if the data is invalid then bad things will happen.

Yup, which is a huge weakness of VTK, IMNSHO.

>If you want to make a sanity check as part of the read process then do it
>as an optional step because fully vetting data can take forever.

Vetting the data does take time, yes.  Sometimes the code can be factored such that the validation happens at one level, and some lower level function can assume it's valid.

But the alternative is crashing upon invalid data.  That's bad, especially when dealing with data from untrusted sources like files or the network.  This is how we're in a world where opening maliciously crafted jpeg/pdf/font/etc files can be used to run arbitrary code and do all kinds of nastiness.

If I had infinite time, fuzzing the VTK readers would be a fun project...

Sean





--
William J. Schroeder, PhD
Kitware, Inc. - Building the World's Technical Computing Software
28 Corporate Drive
Clifton Park, NY 12065
[hidden email]
http://www.kitware.com
(518) 881-4902

_______________________________________________
Powered by www.kitware.com

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Search the list archives at: http://markmail.org/search/?q=vtk-developers

Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/vtk-developers

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Strange problem with vtkPLYReader

Prabhu Ramachandran-3
In reply to this post by Bill Lorensen
Hi Bill,

This is very embarrassing!  Thank you for finding the source of the problem.  I had assumed that the file was a standard one but apparently a buggy writer had generated it in the past.

While it would be nice to have readers detect these I understand that this is going to be hard.  Thanks again and sorry for the noise!

cheers,
Prabhu


On 8/10/17 10:36 PM, Bill Lorensen wrote:
The PLYReader is fine. The file is not a proper PLY file. The face
indices should be zero-based, but in this file they are one-based. So,
the face
3 1 2 5
is referencing an point 5 which exceeds the points array which has
tuples from 0 - 4 (5 points).

On Thu, Aug 10, 2017 at 6:48 AM, David E DeMarle
[hidden email] wrote:
Please submit a bug report via gitlab to help us keep track of this.

On Aug 5, 2017 2:04 PM, "Prabhu Ramachandran" [hidden email]
wrote:
Hi all,

We've been running into a strange issue with the vtkPLYReader ever since
VTK 5.6 all the way up to 8.0.0.  I am attaching a simple VTK Python script.
If you run it a few times you'll see inconsistent results for the bounds of
the data like so:

(0.0, 1.0, 0.0, 1.0, 0.0, 1.600000023841858)
(0.0, 1.5845632502852868e+29, 0.0, 1.0, 0.0, 1.600000023841858)

...


I've also attached a standard dataset for this.  I could find the bug but
I figured someone more familiar with the code may be quicker fixing it.
Thanks! :)


cheers,

Prabhu


_______________________________________________
Powered by www.kitware.com

Visit other Kitware open-source projects at
http://www.kitware.com/opensource/opensource.html

Search the list archives at: http://markmail.org/search/?q=vtk-developers

Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/vtk-developers


_______________________________________________
Powered by www.kitware.com

Visit other Kitware open-source projects at
http://www.kitware.com/opensource/opensource.html

Search the list archives at: http://markmail.org/search/?q=vtk-developers

Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/vtk-developers






_______________________________________________
Powered by www.kitware.com

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Search the list archives at: http://markmail.org/search/?q=vtk-developers

Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/vtk-developers

Loading...