Quantcast

VTK and Qt5 / QML

classic Classic list List threaded Threaded
14 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

VTK and Qt5 / QML

c.sell
Hello all,

I am looking into the possibility of replacing the 3D rendering engine  
in my Qt5 / QML based mobile tablet-oriented application with VTK.  
What I need is a first class QtQuick integration (not a hack or  
workaround) that is 100% stable in every context (not an unusual  
requirement, I admit). To my amazement, nothing of that kind seems to  
exist (correct me if I'm wrong).

I went on to investigate what had been done so far and to implement my  
first prototypes, using the QQuickFrameBufferObject approach. From the  
very start this felt like an uphill battle, because VTK seems to come  
from a Windowing background and is quite tightly integrated with  
concepts that are not valid in a QML context.

I'll describe my findings together with the 2 prototypical QML items I  
implemented:

1st was an adaptation of the DICOM example which now runs as a QML  
item. All user interaction is handled by QML and forwarded to VTK  
(which is one thing that doesn't come naturally with VTK), and after  
some non-elegant tweaking I was able to have the UI move from slice to  
slice and re-render upon mouse wheel events as expected. The problem  
here was, that QML insists on keeping mouse event handling and OpenGL  
rendering on separate threads, with one "rendering thread" dedicated  
to OpenGL. However, the pre-existing VTK Interactors directly call  
Render() after reconfiguring the UI from the mouse event data, which  
is an absolute QML No-Go. I had to introduce a RenderDelegate that  
works somewhat like this:

QML mouse event:
     tell RenderDelegate about the event
     call update() on the QML item, which triggers rer-endering on the  
dedicated thread
on the QML rendering thread:
     call Render() on the RenderWindow
     inside the RenderDelegate, look at whether a mouse event is  
pending, and call the corresponding VTK mouse handler
     call the renderer's default Render() function while setting the  
delegate's "Used" flag to false

2nd was a QML item that displays a model loaded from a 3ds file. My  
goal here was to move the model around using mouse drag events. I took  
the lessons learned from the first example, threw in a  
vtkInteractorStyleTrackballCamera and hoped for the best. First thing  
I found was that I needed a specialized RenderWindowInteractor, which  
I provided. When I realized that the most important requirement for  
that class was to provide access to timer functionality, I already got  
wary, as throwing around events and doing stuff offline while on the  
QML rendering thread is never a good idea. My fears came true when I  
finished wiring everything together: I was able to move the model  
using the mouse, but after a few moments things got whacky with error  
output written to the console and finally a segemtation fault from  
deep inside VTK.

I still need to investigate the second example, but would like to  
synchronize with the community at this point in order to avoid  
errors/misconceptions on my side, to seek help if possible and to  
offer my contribution to the VTK code base once this becomes  
functional. My first impression is that there are some issues that lie  
on an architectural level and cannot be (elegantly) dealt with on the  
QML side alone. Any comments?

thanks,
Christian Sell

_______________________________________________
Powered by www.kitware.com

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Search the list archives at: http://markmail.org/search/?q=vtk-developers

Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/vtk-developers

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: VTK and Qt5 / QML

c.sell
just for the record (nobody care about this subject?):

I found that contrary to what I said I had NOT adhered to the  
principle established with protoype 1 while implementing protoype 2.  
Rather, I had directly called into the VTK interactor from the QML UI  
thread. And because the VTK interactors immediately go into rendering,  
all hell broke loose.

After cleaning up example 2 according to the rule laid out earlier  
("store event data in RenderDelegate, call update() on the QML item,  
pass the event to VTK inside RenderDelgate.Render()", all works well.

What I would like to do is validate the solution against findings made  
elsewhere, and then establish the "canonical" way of integrating VTK  
with QML. As I said, there are some architectural quirks with the  
solution which I would also like to address.

regards,
Christian


Zitat von [hidden email]:

> Hello all,
>
> I am looking into the possibility of replacing the 3D rendering  
> engine in my Qt5 / QML based mobile tablet-oriented application with  
> VTK. What I need is a first class QtQuick integration (not a hack or  
> workaround) that is 100% stable in every context (not an unusual  
> requirement, I admit). To my amazement, nothing of that kind seems  
> to exist (correct me if I'm wrong).
>
> I went on to investigate what had been done so far and to implement  
> my first prototypes, using the QQuickFrameBufferObject approach.  
> From the very start this felt like an uphill battle, because VTK  
> seems to come from a Windowing background and is quite tightly  
> integrated with concepts that are not valid in a QML context.
>
> I'll describe my findings together with the 2 prototypical QML items  
> I implemented:
>
> 1st was an adaptation of the DICOM example which now runs as a QML  
> item. All user interaction is handled by QML and forwarded to VTK  
> (which is one thing that doesn't come naturally with VTK), and after  
> some non-elegant tweaking I was able to have the UI move from slice  
> to slice and re-render upon mouse wheel events as expected. The  
> problem here was, that QML insists on keeping mouse event handling  
> and OpenGL rendering on separate threads, with one "rendering  
> thread" dedicated to OpenGL. However, the pre-existing VTK  
> Interactors directly call Render() after reconfiguring the UI from  
> the mouse event data, which is an absolute QML No-Go. I had to  
> introduce a RenderDelegate that works somewhat like this:
>
> QML mouse event:
>     tell RenderDelegate about the event
>     call update() on the QML item, which triggers rer-endering on  
> the dedicated thread
> on the QML rendering thread:
>     call Render() on the RenderWindow
>     inside the RenderDelegate, look at whether a mouse event is  
> pending, and call the corresponding VTK mouse handler
>     call the renderer's default Render() function while setting the  
> delegate's "Used" flag to false
>
> 2nd was a QML item that displays a model loaded from a 3ds file. My  
> goal here was to move the model around using mouse drag events. I  
> took the lessons learned from the first example, threw in a  
> vtkInteractorStyleTrackballCamera and hoped for the best. First  
> thing I found was that I needed a specialized  
> RenderWindowInteractor, which I provided. When I realized that the  
> most important requirement for that class was to provide access to  
> timer functionality, I already got wary, as throwing around events  
> and doing stuff offline while on the QML rendering thread is never a  
> good idea. My fears came true when I finished wiring everything  
> together: I was able to move the model using the mouse, but after a  
> few moments things got whacky with error output written to the  
> console and finally a segemtation fault from deep inside VTK.
>
> I still need to investigate the second example, but would like to  
> synchronize with the community at this point in order to avoid  
> errors/misconceptions on my side, to seek help if possible and to  
> offer my contribution to the VTK code base once this becomes  
> functional. My first impression is that there are some issues that  
> lie on an architectural level and cannot be (elegantly) dealt with  
> on the QML side alone. Any comments?
>
> thanks,
> Christian Sell
>
> _______________________________________________
> Powered by www.kitware.com
>
> Visit other Kitware open-source projects at  
> http://www.kitware.com/opensource/opensource.html
>
> Search the list archives at: http://markmail.org/search/?q=vtk-developers
>
> Follow this link to subscribe/unsubscribe:
> http://public.kitware.com/mailman/listinfo/vtk-developers



_______________________________________________
Powered by www.kitware.com

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Search the list archives at: http://markmail.org/search/?q=vtk-developers

Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/vtk-developers

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: VTK and Qt5 / QML

Sankhesh Jhaveri-2

Hi Christian,

Doing all VTK rendering on the QML render thread is the right thing to do. As far as interaction is concerned, make sure that Qt events are queued on the main thread and processed when execution reaches the render thread.

I have done some work on VTK and QtQuick integration that I’m planning to add to a VTK remote module. That way, it will be available as part of VTK.

Thanks,
Sankhesh


On Tue, May 16, 2017 at 4:57 AM <[hidden email]> wrote:
just for the record (nobody care about this subject?):

I found that contrary to what I said I had NOT adhered to the
principle established with protoype 1 while implementing protoype 2.
Rather, I had directly called into the VTK interactor from the QML UI
thread. And because the VTK interactors immediately go into rendering,
all hell broke loose.

After cleaning up example 2 according to the rule laid out earlier
("store event data in RenderDelegate, call update() on the QML item,
pass the event to VTK inside RenderDelgate.Render()", all works well.

What I would like to do is validate the solution against findings made
elsewhere, and then establish the "canonical" way of integrating VTK
with QML. As I said, there are some architectural quirks with the
solution which I would also like to address.

regards,
Christian


Zitat von [hidden email]:

> Hello all,
>
> I am looking into the possibility of replacing the 3D rendering
> engine in my Qt5 / QML based mobile tablet-oriented application with
> VTK. What I need is a first class QtQuick integration (not a hack or
> workaround) that is 100% stable in every context (not an unusual
> requirement, I admit). To my amazement, nothing of that kind seems
> to exist (correct me if I'm wrong).
>
> I went on to investigate what had been done so far and to implement
> my first prototypes, using the QQuickFrameBufferObject approach.
> From the very start this felt like an uphill battle, because VTK
> seems to come from a Windowing background and is quite tightly
> integrated with concepts that are not valid in a QML context.
>
> I'll describe my findings together with the 2 prototypical QML items
> I implemented:
>
> 1st was an adaptation of the DICOM example which now runs as a QML
> item. All user interaction is handled by QML and forwarded to VTK
> (which is one thing that doesn't come naturally with VTK), and after
> some non-elegant tweaking I was able to have the UI move from slice
> to slice and re-render upon mouse wheel events as expected. The
> problem here was, that QML insists on keeping mouse event handling
> and OpenGL rendering on separate threads, with one "rendering
> thread" dedicated to OpenGL. However, the pre-existing VTK
> Interactors directly call Render() after reconfiguring the UI from
> the mouse event data, which is an absolute QML No-Go. I had to
> introduce a RenderDelegate that works somewhat like this:
>
> QML mouse event:
>     tell RenderDelegate about the event
>     call update() on the QML item, which triggers rer-endering on
> the dedicated thread
> on the QML rendering thread:
>     call Render() on the RenderWindow
>     inside the RenderDelegate, look at whether a mouse event is
> pending, and call the corresponding VTK mouse handler
>     call the renderer's default Render() function while setting the
> delegate's "Used" flag to false
>
> 2nd was a QML item that displays a model loaded from a 3ds file. My
> goal here was to move the model around using mouse drag events. I
> took the lessons learned from the first example, threw in a
> vtkInteractorStyleTrackballCamera and hoped for the best. First
> thing I found was that I needed a specialized
> RenderWindowInteractor, which I provided. When I realized that the
> most important requirement for that class was to provide access to
> timer functionality, I already got wary, as throwing around events
> and doing stuff offline while on the QML rendering thread is never a
> good idea. My fears came true when I finished wiring everything
> together: I was able to move the model using the mouse, but after a
> few moments things got whacky with error output written to the
> console and finally a segemtation fault from deep inside VTK.
>
> I still need to investigate the second example, but would like to
> synchronize with the community at this point in order to avoid
> errors/misconceptions on my side, to seek help if possible and to
> offer my contribution to the VTK code base once this becomes
> functional. My first impression is that there are some issues that
> lie on an architectural level and cannot be (elegantly) dealt with
> on the QML side alone. Any comments?
>
> thanks,
> Christian Sell
>
> _______________________________________________
> Powered by www.kitware.com
>
> Visit other Kitware open-source projects at
> http://www.kitware.com/opensource/opensource.html
>
> Search the list archives at: http://markmail.org/search/?q=vtk-developers
>
> Follow this link to subscribe/unsubscribe:
> http://public.kitware.com/mailman/listinfo/vtk-developers



_______________________________________________
Powered by www.kitware.com

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Search the list archives at: http://markmail.org/search/?q=vtk-developers

Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/vtk-developers

--
Sankhesh Jhaveri
Sr. Research & Development Engineer | Kitware | (518) 881-4417

_______________________________________________
Powered by www.kitware.com

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Search the list archives at: http://markmail.org/search/?q=vtk-developers

Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/vtk-developers

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: VTK and Qt5 / QML

gpwright
Hi guys,

I would argue that for qml integration the best approach is for each vtk viewport to render into a texture on a dedicated thread, not the qml rendering thread.  Qml then draws the resulting texture which is more in line with the Qt design intention ( see http://blog.qt.io/blog/2015/05/11/integrating-custom-opengl-rendering-with-qt-quick-via-qquickframebufferobject/ )

I have been using this approach for a couple of years in a high performance real time application.  The code is unfortunately proprietary but very happy to explain more about the approach.  It would be great to add support for this to VTK.

For the interactor part I do what Sankesh said, defer the events and apply them on the render thread of the particular vtk item.

Regards,

G



On Tue, May 16, 2017, 3:06 PM Sankhesh Jhaveri <[hidden email]> wrote:

Hi Christian,

Doing all VTK rendering on the QML render thread is the right thing to do. As far as interaction is concerned, make sure that Qt events are queued on the main thread and processed when execution reaches the render thread.

I have done some work on VTK and QtQuick integration that I’m planning to add to a VTK remote module. That way, it will be available as part of VTK.

Thanks,
Sankhesh


On Tue, May 16, 2017 at 4:57 AM <[hidden email]> wrote:
just for the record (nobody care about this subject?):

I found that contrary to what I said I had NOT adhered to the
principle established with protoype 1 while implementing protoype 2.
Rather, I had directly called into the VTK interactor from the QML UI
thread. And because the VTK interactors immediately go into rendering,
all hell broke loose.

After cleaning up example 2 according to the rule laid out earlier
("store event data in RenderDelegate, call update() on the QML item,
pass the event to VTK inside RenderDelgate.Render()", all works well.

What I would like to do is validate the solution against findings made
elsewhere, and then establish the "canonical" way of integrating VTK
with QML. As I said, there are some architectural quirks with the
solution which I would also like to address.

regards,
Christian


Zitat von [hidden email]:

> Hello all,
>
> I am looking into the possibility of replacing the 3D rendering
> engine in my Qt5 / QML based mobile tablet-oriented application with
> VTK. What I need is a first class QtQuick integration (not a hack or
> workaround) that is 100% stable in every context (not an unusual
> requirement, I admit). To my amazement, nothing of that kind seems
> to exist (correct me if I'm wrong).
>
> I went on to investigate what had been done so far and to implement
> my first prototypes, using the QQuickFrameBufferObject approach.
> From the very start this felt like an uphill battle, because VTK
> seems to come from a Windowing background and is quite tightly
> integrated with concepts that are not valid in a QML context.
>
> I'll describe my findings together with the 2 prototypical QML items
> I implemented:
>
> 1st was an adaptation of the DICOM example which now runs as a QML
> item. All user interaction is handled by QML and forwarded to VTK
> (which is one thing that doesn't come naturally with VTK), and after
> some non-elegant tweaking I was able to have the UI move from slice
> to slice and re-render upon mouse wheel events as expected. The
> problem here was, that QML insists on keeping mouse event handling
> and OpenGL rendering on separate threads, with one "rendering
> thread" dedicated to OpenGL. However, the pre-existing VTK
> Interactors directly call Render() after reconfiguring the UI from
> the mouse event data, which is an absolute QML No-Go. I had to
> introduce a RenderDelegate that works somewhat like this:
>
> QML mouse event:
>     tell RenderDelegate about the event
>     call update() on the QML item, which triggers rer-endering on
> the dedicated thread
> on the QML rendering thread:
>     call Render() on the RenderWindow
>     inside the RenderDelegate, look at whether a mouse event is
> pending, and call the corresponding VTK mouse handler
>     call the renderer's default Render() function while setting the
> delegate's "Used" flag to false
>
> 2nd was a QML item that displays a model loaded from a 3ds file. My
> goal here was to move the model around using mouse drag events. I
> took the lessons learned from the first example, threw in a
> vtkInteractorStyleTrackballCamera and hoped for the best. First
> thing I found was that I needed a specialized
> RenderWindowInteractor, which I provided. When I realized that the
> most important requirement for that class was to provide access to
> timer functionality, I already got wary, as throwing around events
> and doing stuff offline while on the QML rendering thread is never a
> good idea. My fears came true when I finished wiring everything
> together: I was able to move the model using the mouse, but after a
> few moments things got whacky with error output written to the
> console and finally a segemtation fault from deep inside VTK.
>
> I still need to investigate the second example, but would like to
> synchronize with the community at this point in order to avoid
> errors/misconceptions on my side, to seek help if possible and to
> offer my contribution to the VTK code base once this becomes
> functional. My first impression is that there are some issues that
> lie on an architectural level and cannot be (elegantly) dealt with
> on the QML side alone. Any comments?
>
> thanks,
> Christian Sell
>
> _______________________________________________
> Powered by www.kitware.com
>
> Visit other Kitware open-source projects at
> http://www.kitware.com/opensource/opensource.html
>
> Search the list archives at: http://markmail.org/search/?q=vtk-developers
>
> Follow this link to subscribe/unsubscribe:
> http://public.kitware.com/mailman/listinfo/vtk-developers



_______________________________________________
Powered by www.kitware.com

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Search the list archives at: http://markmail.org/search/?q=vtk-developers

Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/vtk-developers

--
Sankhesh Jhaveri
Sr. Research & Development Engineer | Kitware | (518) 881-4417
_______________________________________________
Powered by www.kitware.com

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Search the list archives at: http://markmail.org/search/?q=vtk-developers

Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/vtk-developers


_______________________________________________
Powered by www.kitware.com

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Search the list archives at: http://markmail.org/search/?q=vtk-developers

Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/vtk-developers

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: VTK and Qt5 / QML

c.sell
In reply to this post by Sankhesh Jhaveri-2
Hello Jankesh,

good to hear there's someone out there.

Would you care to elaborate a little more what you mean by "Qt events  
are queued on the main thread"? Moreover, what exact work are you  
doing and when will that be available for public review?

As I said, the only working approach I found for applying UI events  
was to set a RenderDelegate. However, RenderDelegate is quite clumsy  
to use (at least in my context), and there are some aspects that make  
me feel uncomfortable architecture-wise. I would appreciate if you  
could comment on this and/or provide the "correct" solution for  
handling events on the QML render thread.

thanks,
Christian


Zitat von Sankhesh Jhaveri <[hidden email]>:

> Hi Christian,
>
> Doing all VTK rendering on the QML render thread is the right thing to do.
> As far as interaction is concerned, make sure that Qt events are queued on
> the main thread and processed when execution reaches the render thread.
>
> I have done some work on VTK and QtQuick integration that I’m planning to
> add to a VTK remote module. That way, it will be available as part of VTK.
>
> Thanks,
> Sankhesh
> ​
>
> On Tue, May 16, 2017 at 4:57 AM <[hidden email]> wrote:
>
>> just for the record (nobody care about this subject?):
>>
>> I found that contrary to what I said I had NOT adhered to the
>> principle established with protoype 1 while implementing protoype 2.
>> Rather, I had directly called into the VTK interactor from the QML UI
>> thread. And because the VTK interactors immediately go into rendering,
>> all hell broke loose.
>>
>> After cleaning up example 2 according to the rule laid out earlier
>> ("store event data in RenderDelegate, call update() on the QML item,
>> pass the event to VTK inside RenderDelgate.Render()", all works well.
>>
>> What I would like to do is validate the solution against findings made
>> elsewhere, and then establish the "canonical" way of integrating VTK
>> with QML. As I said, there are some architectural quirks with the
>> solution which I would also like to address.
>>
>> regards,
>> Christian
>>
>>
>> Zitat von [hidden email]:
>>
>> > Hello all,
>> >
>> > I am looking into the possibility of replacing the 3D rendering
>> > engine in my Qt5 / QML based mobile tablet-oriented application with
>> > VTK. What I need is a first class QtQuick integration (not a hack or
>> > workaround) that is 100% stable in every context (not an unusual
>> > requirement, I admit). To my amazement, nothing of that kind seems
>> > to exist (correct me if I'm wrong).
>> >
>> > I went on to investigate what had been done so far and to implement
>> > my first prototypes, using the QQuickFrameBufferObject approach.
>> > From the very start this felt like an uphill battle, because VTK
>> > seems to come from a Windowing background and is quite tightly
>> > integrated with concepts that are not valid in a QML context.
>> >
>> > I'll describe my findings together with the 2 prototypical QML items
>> > I implemented:
>> >
>> > 1st was an adaptation of the DICOM example which now runs as a QML
>> > item. All user interaction is handled by QML and forwarded to VTK
>> > (which is one thing that doesn't come naturally with VTK), and after
>> > some non-elegant tweaking I was able to have the UI move from slice
>> > to slice and re-render upon mouse wheel events as expected. The
>> > problem here was, that QML insists on keeping mouse event handling
>> > and OpenGL rendering on separate threads, with one "rendering
>> > thread" dedicated to OpenGL. However, the pre-existing VTK
>> > Interactors directly call Render() after reconfiguring the UI from
>> > the mouse event data, which is an absolute QML No-Go. I had to
>> > introduce a RenderDelegate that works somewhat like this:
>> >
>> > QML mouse event:
>> >     tell RenderDelegate about the event
>> >     call update() on the QML item, which triggers rer-endering on
>> > the dedicated thread
>> > on the QML rendering thread:
>> >     call Render() on the RenderWindow
>> >     inside the RenderDelegate, look at whether a mouse event is
>> > pending, and call the corresponding VTK mouse handler
>> >     call the renderer's default Render() function while setting the
>> > delegate's "Used" flag to false
>> >
>> > 2nd was a QML item that displays a model loaded from a 3ds file. My
>> > goal here was to move the model around using mouse drag events. I
>> > took the lessons learned from the first example, threw in a
>> > vtkInteractorStyleTrackballCamera and hoped for the best. First
>> > thing I found was that I needed a specialized
>> > RenderWindowInteractor, which I provided. When I realized that the
>> > most important requirement for that class was to provide access to
>> > timer functionality, I already got wary, as throwing around events
>> > and doing stuff offline while on the QML rendering thread is never a
>> > good idea. My fears came true when I finished wiring everything
>> > together: I was able to move the model using the mouse, but after a
>> > few moments things got whacky with error output written to the
>> > console and finally a segemtation fault from deep inside VTK.
>> >
>> > I still need to investigate the second example, but would like to
>> > synchronize with the community at this point in order to avoid
>> > errors/misconceptions on my side, to seek help if possible and to
>> > offer my contribution to the VTK code base once this becomes
>> > functional. My first impression is that there are some issues that
>> > lie on an architectural level and cannot be (elegantly) dealt with
>> > on the QML side alone. Any comments?
>> >
>> > thanks,
>> > Christian Sell
>> >
>> > _______________________________________________
>> > Powered by www.kitware.com
>> >
>> > Visit other Kitware open-source projects at
>> > http://www.kitware.com/opensource/opensource.html
>> >
>> > Search the list archives at:
>> http://markmail.org/search/?q=vtk-developers
>> >
>> > Follow this link to subscribe/unsubscribe:
>> > http://public.kitware.com/mailman/listinfo/vtk-developers
>>
>>
>>
>> _______________________________________________
>> Powered by www.kitware.com
>>
>> Visit other Kitware open-source projects at
>> http://www.kitware.com/opensource/opensource.html
>>
>> Search the list archives at: http://markmail.org/search/?q=vtk-developers
>>
>> Follow this link to subscribe/unsubscribe:
>> http://public.kitware.com/mailman/listinfo/vtk-developers
>>
>> --
> Sankhesh Jhaveri *Sr. Research & Development Engineer* | Kitware
> <http://www.kitware.com/> | (518) 881-4417
> ​



_______________________________________________
Powered by www.kitware.com

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Search the list archives at: http://markmail.org/search/?q=vtk-developers

Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/vtk-developers

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: VTK and Qt5 / QML

c.sell
In reply to this post by gpwright
Hi Geoff,

as I understand it, the blog post you linked to doesn't mention  
multiple threads beyond the QML UI thread and the scenegraph render  
thread.

So far, I am using that approach with QQuickFrameBufferObject as  
starting point and vtkExternalOpenGLRenderWindow for the VTK side,  
which causes VTK to use QML's OpenGL context and thus render into the  
FBO maintained by QQuickFrameBufferObject. That seems to work  
flawlessly, as long as ALL rendering is kept on the scenegraph thread,  
which is not always obvious to do at first, because VTK's API's don't  
separate between reading event values and rendering them (e.g.,  
SetSlice(2) will internally set the current slice and then do a  
Render()).

But why maintain more threads?

Anyway, please go ahead and explain your approach as detailed as  
possible, I am all ears.

regards,
Chris

Zitat von Geoff Wright <[hidden email]>:

> Hi guys,
>
> I would argue that for qml integration the best approach is for each vtk
> viewport to render into a texture on a dedicated thread, not the qml
> rendering thread.  Qml then draws the resulting texture which is more in
> line with the Qt design intention ( see
> http://blog.qt.io/blog/2015/05/11/integrating-custom-opengl-rendering-with-qt-quick-via-qquickframebufferobject/
>  )
>
> I have been using this approach for a couple of years in a high performance
> real time application.  The code is unfortunately proprietary but very
> happy to explain more about the approach.  It would be great to add support
> for this to VTK.
>
> For the interactor part I do what Sankesh said, defer the events and apply
> them on the render thread of the particular vtk item.
>
> Regards,
>
> G
>
>
>
> On Tue, May 16, 2017, 3:06 PM Sankhesh Jhaveri <[hidden email]>
> wrote:
>
>> Hi Christian,
>>
>> Doing all VTK rendering on the QML render thread is the right thing to do.
>> As far as interaction is concerned, make sure that Qt events are queued on
>> the main thread and processed when execution reaches the render thread.
>>
>> I have done some work on VTK and QtQuick integration that I’m planning to
>> add to a VTK remote module. That way, it will be available as part of VTK.
>>
>> Thanks,
>> Sankhesh
>> ​
>>
>> On Tue, May 16, 2017 at 4:57 AM <[hidden email]> wrote:
>>
>>> just for the record (nobody care about this subject?):
>>>
>>> I found that contrary to what I said I had NOT adhered to the
>>> principle established with protoype 1 while implementing protoype 2.
>>> Rather, I had directly called into the VTK interactor from the QML UI
>>> thread. And because the VTK interactors immediately go into rendering,
>>> all hell broke loose.
>>>
>>> After cleaning up example 2 according to the rule laid out earlier
>>> ("store event data in RenderDelegate, call update() on the QML item,
>>> pass the event to VTK inside RenderDelgate.Render()", all works well.
>>>
>>> What I would like to do is validate the solution against findings made
>>> elsewhere, and then establish the "canonical" way of integrating VTK
>>> with QML. As I said, there are some architectural quirks with the
>>> solution which I would also like to address.
>>>
>>> regards,
>>> Christian
>>>
>>>
>>> Zitat von [hidden email]:
>>>
>>> > Hello all,
>>> >
>>> > I am looking into the possibility of replacing the 3D rendering
>>> > engine in my Qt5 / QML based mobile tablet-oriented application with
>>> > VTK. What I need is a first class QtQuick integration (not a hack or
>>> > workaround) that is 100% stable in every context (not an unusual
>>> > requirement, I admit). To my amazement, nothing of that kind seems
>>> > to exist (correct me if I'm wrong).
>>> >
>>> > I went on to investigate what had been done so far and to implement
>>> > my first prototypes, using the QQuickFrameBufferObject approach.
>>> > From the very start this felt like an uphill battle, because VTK
>>> > seems to come from a Windowing background and is quite tightly
>>> > integrated with concepts that are not valid in a QML context.
>>> >
>>> > I'll describe my findings together with the 2 prototypical QML items
>>> > I implemented:
>>> >
>>> > 1st was an adaptation of the DICOM example which now runs as a QML
>>> > item. All user interaction is handled by QML and forwarded to VTK
>>> > (which is one thing that doesn't come naturally with VTK), and after
>>> > some non-elegant tweaking I was able to have the UI move from slice
>>> > to slice and re-render upon mouse wheel events as expected. The
>>> > problem here was, that QML insists on keeping mouse event handling
>>> > and OpenGL rendering on separate threads, with one "rendering
>>> > thread" dedicated to OpenGL. However, the pre-existing VTK
>>> > Interactors directly call Render() after reconfiguring the UI from
>>> > the mouse event data, which is an absolute QML No-Go. I had to
>>> > introduce a RenderDelegate that works somewhat like this:
>>> >
>>> > QML mouse event:
>>> >     tell RenderDelegate about the event
>>> >     call update() on the QML item, which triggers rer-endering on
>>> > the dedicated thread
>>> > on the QML rendering thread:
>>> >     call Render() on the RenderWindow
>>> >     inside the RenderDelegate, look at whether a mouse event is
>>> > pending, and call the corresponding VTK mouse handler
>>> >     call the renderer's default Render() function while setting the
>>> > delegate's "Used" flag to false
>>> >
>>> > 2nd was a QML item that displays a model loaded from a 3ds file. My
>>> > goal here was to move the model around using mouse drag events. I
>>> > took the lessons learned from the first example, threw in a
>>> > vtkInteractorStyleTrackballCamera and hoped for the best. First
>>> > thing I found was that I needed a specialized
>>> > RenderWindowInteractor, which I provided. When I realized that the
>>> > most important requirement for that class was to provide access to
>>> > timer functionality, I already got wary, as throwing around events
>>> > and doing stuff offline while on the QML rendering thread is never a
>>> > good idea. My fears came true when I finished wiring everything
>>> > together: I was able to move the model using the mouse, but after a
>>> > few moments things got whacky with error output written to the
>>> > console and finally a segemtation fault from deep inside VTK.
>>> >
>>> > I still need to investigate the second example, but would like to
>>> > synchronize with the community at this point in order to avoid
>>> > errors/misconceptions on my side, to seek help if possible and to
>>> > offer my contribution to the VTK code base once this becomes
>>> > functional. My first impression is that there are some issues that
>>> > lie on an architectural level and cannot be (elegantly) dealt with
>>> > on the QML side alone. Any comments?
>>> >
>>> > thanks,
>>> > Christian Sell
>>> >
>>> > _______________________________________________
>>> > Powered by www.kitware.com
>>> >
>>> > Visit other Kitware open-source projects at
>>> > http://www.kitware.com/opensource/opensource.html
>>> >
>>> > Search the list archives at:
>>> http://markmail.org/search/?q=vtk-developers
>>> >
>>> > Follow this link to subscribe/unsubscribe:
>>> > http://public.kitware.com/mailman/listinfo/vtk-developers
>>>
>>>
>>>
>>> _______________________________________________
>>> Powered by www.kitware.com
>>>
>>> Visit other Kitware open-source projects at
>>> http://www.kitware.com/opensource/opensource.html
>>>
>>> Search the list archives at: http://markmail.org/search/?q=vtk-developers
>>>
>>> Follow this link to subscribe/unsubscribe:
>>> http://public.kitware.com/mailman/listinfo/vtk-developers
>>>
>>> --
>> Sankhesh Jhaveri *Sr. Research & Development Engineer* | Kitware
>> <http://www.kitware.com/> | (518) 881-4417
>> ​
>> _______________________________________________
>> Powered by www.kitware.com
>>
>> Visit other Kitware open-source projects at
>> http://www.kitware.com/opensource/opensource.html
>>
>> Search the list archives at: http://markmail.org/search/?q=vtk-developers
>>
>> Follow this link to subscribe/unsubscribe:
>> http://public.kitware.com/mailman/listinfo/vtk-developers
>>
>>



_______________________________________________
Powered by www.kitware.com

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Search the list archives at: http://markmail.org/search/?q=vtk-developers

Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/vtk-developers

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: VTK and Qt5 / QML

gpwright
Hi Chris,

Sorry for the confusion, yes I am doing what you describe.  The difference may be that perhaps your app has only one viewport.  In my case there are a number of different viewports (aka vtkRenderWindow)  each has its own qml item and is each rendering on it's own thread.  Plus the qml composition thread which is managed by Qt.

I was just making the point that this architecture let's you draw different viewports in parallel provided that each one has its own mappers and actors.

G


On Tue, May 16, 2017, 4:13 PM <[hidden email]> wrote:
Hi Geoff,

as I understand it, the blog post you linked to doesn't mention
multiple threads beyond the QML UI thread and the scenegraph render
thread.

So far, I am using that approach with QQuickFrameBufferObject as
starting point and vtkExternalOpenGLRenderWindow for the VTK side,
which causes VTK to use QML's OpenGL context and thus render into the
FBO maintained by QQuickFrameBufferObject. That seems to work
flawlessly, as long as ALL rendering is kept on the scenegraph thread,
which is not always obvious to do at first, because VTK's API's don't
separate between reading event values and rendering them (e.g.,
SetSlice(2) will internally set the current slice and then do a
Render()).

But why maintain more threads?

Anyway, please go ahead and explain your approach as detailed as
possible, I am all ears.

regards,
Chris

Zitat von Geoff Wright <[hidden email]>:

> Hi guys,
>
> I would argue that for qml integration the best approach is for each vtk
> viewport to render into a texture on a dedicated thread, not the qml
> rendering thread.  Qml then draws the resulting texture which is more in
> line with the Qt design intention ( see
> http://blog.qt.io/blog/2015/05/11/integrating-custom-opengl-rendering-with-qt-quick-via-qquickframebufferobject/
>  )
>
> I have been using this approach for a couple of years in a high performance
> real time application.  The code is unfortunately proprietary but very
> happy to explain more about the approach.  It would be great to add support
> for this to VTK.
>
> For the interactor part I do what Sankesh said, defer the events and apply
> them on the render thread of the particular vtk item.
>
> Regards,
>
> G
>
>
>
> On Tue, May 16, 2017, 3:06 PM Sankhesh Jhaveri <[hidden email]>
> wrote:
>
>> Hi Christian,
>>
>> Doing all VTK rendering on the QML render thread is the right thing to do.
>> As far as interaction is concerned, make sure that Qt events are queued on
>> the main thread and processed when execution reaches the render thread.
>>
>> I have done some work on VTK and QtQuick integration that I’m planning to
>> add to a VTK remote module. That way, it will be available as part of VTK.
>>
>> Thanks,
>> Sankhesh
>> ​
>>
>> On Tue, May 16, 2017 at 4:57 AM <[hidden email]> wrote:
>>
>>> just for the record (nobody care about this subject?):
>>>
>>> I found that contrary to what I said I had NOT adhered to the
>>> principle established with protoype 1 while implementing protoype 2.
>>> Rather, I had directly called into the VTK interactor from the QML UI
>>> thread. And because the VTK interactors immediately go into rendering,
>>> all hell broke loose.
>>>
>>> After cleaning up example 2 according to the rule laid out earlier
>>> ("store event data in RenderDelegate, call update() on the QML item,
>>> pass the event to VTK inside RenderDelgate.Render()", all works well.
>>>
>>> What I would like to do is validate the solution against findings made
>>> elsewhere, and then establish the "canonical" way of integrating VTK
>>> with QML. As I said, there are some architectural quirks with the
>>> solution which I would also like to address.
>>>
>>> regards,
>>> Christian
>>>
>>>
>>> Zitat von [hidden email]:
>>>
>>> > Hello all,
>>> >
>>> > I am looking into the possibility of replacing the 3D rendering
>>> > engine in my Qt5 / QML based mobile tablet-oriented application with
>>> > VTK. What I need is a first class QtQuick integration (not a hack or
>>> > workaround) that is 100% stable in every context (not an unusual
>>> > requirement, I admit). To my amazement, nothing of that kind seems
>>> > to exist (correct me if I'm wrong).
>>> >
>>> > I went on to investigate what had been done so far and to implement
>>> > my first prototypes, using the QQuickFrameBufferObject approach.
>>> > From the very start this felt like an uphill battle, because VTK
>>> > seems to come from a Windowing background and is quite tightly
>>> > integrated with concepts that are not valid in a QML context.
>>> >
>>> > I'll describe my findings together with the 2 prototypical QML items
>>> > I implemented:
>>> >
>>> > 1st was an adaptation of the DICOM example which now runs as a QML
>>> > item. All user interaction is handled by QML and forwarded to VTK
>>> > (which is one thing that doesn't come naturally with VTK), and after
>>> > some non-elegant tweaking I was able to have the UI move from slice
>>> > to slice and re-render upon mouse wheel events as expected. The
>>> > problem here was, that QML insists on keeping mouse event handling
>>> > and OpenGL rendering on separate threads, with one "rendering
>>> > thread" dedicated to OpenGL. However, the pre-existing VTK
>>> > Interactors directly call Render() after reconfiguring the UI from
>>> > the mouse event data, which is an absolute QML No-Go. I had to
>>> > introduce a RenderDelegate that works somewhat like this:
>>> >
>>> > QML mouse event:
>>> >     tell RenderDelegate about the event
>>> >     call update() on the QML item, which triggers rer-endering on
>>> > the dedicated thread
>>> > on the QML rendering thread:
>>> >     call Render() on the RenderWindow
>>> >     inside the RenderDelegate, look at whether a mouse event is
>>> > pending, and call the corresponding VTK mouse handler
>>> >     call the renderer's default Render() function while setting the
>>> > delegate's "Used" flag to false
>>> >
>>> > 2nd was a QML item that displays a model loaded from a 3ds file. My
>>> > goal here was to move the model around using mouse drag events. I
>>> > took the lessons learned from the first example, threw in a
>>> > vtkInteractorStyleTrackballCamera and hoped for the best. First
>>> > thing I found was that I needed a specialized
>>> > RenderWindowInteractor, which I provided. When I realized that the
>>> > most important requirement for that class was to provide access to
>>> > timer functionality, I already got wary, as throwing around events
>>> > and doing stuff offline while on the QML rendering thread is never a
>>> > good idea. My fears came true when I finished wiring everything
>>> > together: I was able to move the model using the mouse, but after a
>>> > few moments things got whacky with error output written to the
>>> > console and finally a segemtation fault from deep inside VTK.
>>> >
>>> > I still need to investigate the second example, but would like to
>>> > synchronize with the community at this point in order to avoid
>>> > errors/misconceptions on my side, to seek help if possible and to
>>> > offer my contribution to the VTK code base once this becomes
>>> > functional. My first impression is that there are some issues that
>>> > lie on an architectural level and cannot be (elegantly) dealt with
>>> > on the QML side alone. Any comments?
>>> >
>>> > thanks,
>>> > Christian Sell
>>> >
>>> > _______________________________________________
>>> > Powered by www.kitware.com
>>> >
>>> > Visit other Kitware open-source projects at
>>> > http://www.kitware.com/opensource/opensource.html
>>> >
>>> > Search the list archives at:
>>> http://markmail.org/search/?q=vtk-developers
>>> >
>>> > Follow this link to subscribe/unsubscribe:
>>> > http://public.kitware.com/mailman/listinfo/vtk-developers
>>>
>>>
>>>
>>> _______________________________________________
>>> Powered by www.kitware.com
>>>
>>> Visit other Kitware open-source projects at
>>> http://www.kitware.com/opensource/opensource.html
>>>
>>> Search the list archives at: http://markmail.org/search/?q=vtk-developers
>>>
>>> Follow this link to subscribe/unsubscribe:
>>> http://public.kitware.com/mailman/listinfo/vtk-developers
>>>
>>> --
>> Sankhesh Jhaveri *Sr. Research & Development Engineer* | Kitware
>> <http://www.kitware.com/> | (518) 881-4417
>> ​
>> _______________________________________________
>> Powered by www.kitware.com
>>
>> Visit other Kitware open-source projects at
>> http://www.kitware.com/opensource/opensource.html
>>
>> Search the list archives at: http://markmail.org/search/?q=vtk-developers
>>
>> Follow this link to subscribe/unsubscribe:
>> http://public.kitware.com/mailman/listinfo/vtk-developers
>>
>>




_______________________________________________
Powered by www.kitware.com

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Search the list archives at: http://markmail.org/search/?q=vtk-developers

Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/vtk-developers

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: VTK and Qt5 / QML

Sankhesh Jhaveri-2
In reply to this post by c.sell

Hi Geoff, Chris,

IMHO, there are two approaches to VTK+QtQuick:

  • Rendering the VTK scene to a QQuickFrameBufferObject.

    • Provides better integration with the QML render stack.
    • Since it is an indirect rendering approach, performance may suffer.
    • May require specializations for some VTK interactor styles and VTK widget interaction.
  • Rendering directly to the QML created OpenGL context using beforeRendering() and afterRendering() calls

    • Better performance as VTK is directly rendering to the FrameBuffer
    • Interactor styles and widget interaction would work out of the box
    • Does not fit within the QML stack as the VTK scene would either be an underlay or an overlay

Note that in both these approaches, VTK rendering would need to be constrained to the QSGRenderThread and Qt events would have to be queued before being processed on the render thread.

Considering that both these approaches have their own merits, demerits and constraints, maybe adding a compile time flag that lets you switch between direct and indirect rendering would help. As such, there is no “correct” solution.

Regarding queueing the events, I’d just sub-class QVTKInteractorAdapter and override ProcessEvent so that the event gets queued. One execution is in the render thread, the queued events should be processed by the interactor.

One more thing as Geoff pointed out, since QML creates an OpenGL context, each VTK item in that context would be an independent vtkViewport. Without this, having multiple QML and VTK items in the same window would not be possible.

Hope this helps.

Sankhesh


On Tue, May 16, 2017 at 10:14 AM <[hidden email]> wrote:
Hi Geoff,

as I understand it, the blog post you linked to doesn't mention
multiple threads beyond the QML UI thread and the scenegraph render
thread.

So far, I am using that approach with QQuickFrameBufferObject as
starting point and vtkExternalOpenGLRenderWindow for the VTK side,
which causes VTK to use QML's OpenGL context and thus render into the
FBO maintained by QQuickFrameBufferObject. That seems to work
flawlessly, as long as ALL rendering is kept on the scenegraph thread,
which is not always obvious to do at first, because VTK's API's don't
separate between reading event values and rendering them (e.g.,
SetSlice(2) will internally set the current slice and then do a
Render()).

But why maintain more threads?

Anyway, please go ahead and explain your approach as detailed as
possible, I am all ears.

regards,
Chris

Zitat von Geoff Wright <[hidden email]>:

> Hi guys,
>
> I would argue that for qml integration the best approach is for each vtk
> viewport to render into a texture on a dedicated thread, not the qml
> rendering thread.  Qml then draws the resulting texture which is more in
> line with the Qt design intention ( see
> http://blog.qt.io/blog/2015/05/11/integrating-custom-opengl-rendering-with-qt-quick-via-qquickframebufferobject/
>  )
>
> I have been using this approach for a couple of years in a high performance
> real time application.  The code is unfortunately proprietary but very
> happy to explain more about the approach.  It would be great to add support
> for this to VTK.
>
> For the interactor part I do what Sankesh said, defer the events and apply
> them on the render thread of the particular vtk item.
>
> Regards,
>
> G
>
>
>
> On Tue, May 16, 2017, 3:06 PM Sankhesh Jhaveri <[hidden email]>
> wrote:
>
>> Hi Christian,
>>
>> Doing all VTK rendering on the QML render thread is the right thing to do.
>> As far as interaction is concerned, make sure that Qt events are queued on
>> the main thread and processed when execution reaches the render thread.
>>
>> I have done some work on VTK and QtQuick integration that I’m planning to
>> add to a VTK remote module. That way, it will be available as part of VTK.
>>
>> Thanks,
>> Sankhesh
>> ​
>>
>> On Tue, May 16, 2017 at 4:57 AM <[hidden email]> wrote:
>>
>>> just for the record (nobody care about this subject?):
>>>
>>> I found that contrary to what I said I had NOT adhered to the
>>> principle established with protoype 1 while implementing protoype 2.
>>> Rather, I had directly called into the VTK interactor from the QML UI
>>> thread. And because the VTK interactors immediately go into rendering,
>>> all hell broke loose.
>>>
>>> After cleaning up example 2 according to the rule laid out earlier
>>> ("store event data in RenderDelegate, call update() on the QML item,
>>> pass the event to VTK inside RenderDelgate.Render()", all works well.
>>>
>>> What I would like to do is validate the solution against findings made
>>> elsewhere, and then establish the "canonical" way of integrating VTK
>>> with QML. As I said, there are some architectural quirks with the
>>> solution which I would also like to address.
>>>
>>> regards,
>>> Christian
>>>
>>>
>>> Zitat von [hidden email]:
>>>
>>> > Hello all,
>>> >
>>> > I am looking into the possibility of replacing the 3D rendering
>>> > engine in my Qt5 / QML based mobile tablet-oriented application with
>>> > VTK. What I need is a first class QtQuick integration (not a hack or
>>> > workaround) that is 100% stable in every context (not an unusual
>>> > requirement, I admit). To my amazement, nothing of that kind seems
>>> > to exist (correct me if I'm wrong).
>>> >
>>> > I went on to investigate what had been done so far and to implement
>>> > my first prototypes, using the QQuickFrameBufferObject approach.
>>> > From the very start this felt like an uphill battle, because VTK
>>> > seems to come from a Windowing background and is quite tightly
>>> > integrated with concepts that are not valid in a QML context.
>>> >
>>> > I'll describe my findings together with the 2 prototypical QML items
>>> > I implemented:
>>> >
>>> > 1st was an adaptation of the DICOM example which now runs as a QML
>>> > item. All user interaction is handled by QML and forwarded to VTK
>>> > (which is one thing that doesn't come naturally with VTK), and after
>>> > some non-elegant tweaking I was able to have the UI move from slice
>>> > to slice and re-render upon mouse wheel events as expected. The
>>> > problem here was, that QML insists on keeping mouse event handling
>>> > and OpenGL rendering on separate threads, with one "rendering
>>> > thread" dedicated to OpenGL. However, the pre-existing VTK
>>> > Interactors directly call Render() after reconfiguring the UI from
>>> > the mouse event data, which is an absolute QML No-Go. I had to
>>> > introduce a RenderDelegate that works somewhat like this:
>>> >
>>> > QML mouse event:
>>> >     tell RenderDelegate about the event
>>> >     call update() on the QML item, which triggers rer-endering on
>>> > the dedicated thread
>>> > on the QML rendering thread:
>>> >     call Render() on the RenderWindow
>>> >     inside the RenderDelegate, look at whether a mouse event is
>>> > pending, and call the corresponding VTK mouse handler
>>> >     call the renderer's default Render() function while setting the
>>> > delegate's "Used" flag to false
>>> >
>>> > 2nd was a QML item that displays a model loaded from a 3ds file. My
>>> > goal here was to move the model around using mouse drag events. I
>>> > took the lessons learned from the first example, threw in a
>>> > vtkInteractorStyleTrackballCamera and hoped for the best. First
>>> > thing I found was that I needed a specialized
>>> > RenderWindowInteractor, which I provided. When I realized that the
>>> > most important requirement for that class was to provide access to
>>> > timer functionality, I already got wary, as throwing around events
>>> > and doing stuff offline while on the QML rendering thread is never a
>>> > good idea. My fears came true when I finished wiring everything
>>> > together: I was able to move the model using the mouse, but after a
>>> > few moments things got whacky with error output written to the
>>> > console and finally a segemtation fault from deep inside VTK.
>>> >
>>> > I still need to investigate the second example, but would like to
>>> > synchronize with the community at this point in order to avoid
>>> > errors/misconceptions on my side, to seek help if possible and to
>>> > offer my contribution to the VTK code base once this becomes
>>> > functional. My first impression is that there are some issues that
>>> > lie on an architectural level and cannot be (elegantly) dealt with
>>> > on the QML side alone. Any comments?
>>> >
>>> > thanks,
>>> > Christian Sell
>>> >
>>> > _______________________________________________
>>> > Powered by www.kitware.com
>>> >
>>> > Visit other Kitware open-source projects at
>>> > http://www.kitware.com/opensource/opensource.html
>>> >
>>> > Search the list archives at:
>>> http://markmail.org/search/?q=vtk-developers
>>> >
>>> > Follow this link to subscribe/unsubscribe:
>>> > http://public.kitware.com/mailman/listinfo/vtk-developers
>>>
>>>
>>>
>>> _______________________________________________
>>> Powered by www.kitware.com
>>>
>>> Visit other Kitware open-source projects at
>>> http://www.kitware.com/opensource/opensource.html
>>>
>>> Search the list archives at: http://markmail.org/search/?q=vtk-developers
>>>
>>> Follow this link to subscribe/unsubscribe:
>>> http://public.kitware.com/mailman/listinfo/vtk-developers
>>>
>>> --
>> Sankhesh Jhaveri *Sr. Research & Development Engineer* | Kitware
>> <http://www.kitware.com/> | <a href="tel:(518)%20881-4417" value="+15188814417" target="_blank">(518) 881-4417
>> ​
>> _______________________________________________
>> Powered by www.kitware.com
>>
>> Visit other Kitware open-source projects at
>> http://www.kitware.com/opensource/opensource.html
>>
>> Search the list archives at: http://markmail.org/search/?q=vtk-developers
>>
>> Follow this link to subscribe/unsubscribe:
>> http://public.kitware.com/mailman/listinfo/vtk-developers
>>
>>



--
Sankhesh Jhaveri
Sr. Research & Development Engineer | Kitware | (518) 881-4417

_______________________________________________
Powered by www.kitware.com

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Search the list archives at: http://markmail.org/search/?q=vtk-developers

Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/vtk-developers

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: VTK and Qt5 / QML

c.sell
In reply to this post by gpwright
Hello all,

@Geoff: are you saying that each pair of QML item / vtkRenderWindow  
has its own render thread? Is that something QML does?

My goal is to have QML/VTK items behave as normal QML items, i.e., I  
could have a dozen items on my screen each rendering something  
different through VTK without influencing each other (apart from  
system performance). I expect that the approach I have taken so far  
will already fulfill this requirement.

@Sankesh: I object to the compile time flag. IMO both approaches  
should be usable at the same time, e.g. by extending different  
superclasses. As you say, the "underlay or overlay" aspect breaks QML  
concepts and makes any item built according to this approach an  
"alien" in the QML crowd that always requires special attention.

I'll look into QVTKInteractorAdapter - that sounds like seomething  
useful. Should have come across it earlier.

@both: is there a chance that usable code will come out of this  
exchange any time soon? I for my part would at least publish my  
prototypical implementations somewhere (GitHub?), unless they become  
obsolete by something "official"

regards,
Chris

Zitat von Geoff Wright <[hidden email]>:

> Hi Chris,
>
> Sorry for the confusion, yes I am doing what you describe.  The difference
> may be that perhaps your app has only one viewport.  In my case there are a
> number of different viewports (aka vtkRenderWindow)  each has its own qml
> item and is each rendering on it's own thread.  Plus the qml composition
> thread which is managed by Qt.
>
> I was just making the point that this architecture let's you draw different
> viewports in parallel provided that each one has its own mappers and actors.
>
> G
>
>
> On Tue, May 16, 2017, 4:13 PM <[hidden email]> wrote:
>
>> Hi Geoff,
>>
>> as I understand it, the blog post you linked to doesn't mention
>> multiple threads beyond the QML UI thread and the scenegraph render
>> thread.
>>
>> So far, I am using that approach with QQuickFrameBufferObject as
>> starting point and vtkExternalOpenGLRenderWindow for the VTK side,
>> which causes VTK to use QML's OpenGL context and thus render into the
>> FBO maintained by QQuickFrameBufferObject. That seems to work
>> flawlessly, as long as ALL rendering is kept on the scenegraph thread,
>> which is not always obvious to do at first, because VTK's API's don't
>> separate between reading event values and rendering them (e.g.,
>> SetSlice(2) will internally set the current slice and then do a
>> Render()).
>>
>> But why maintain more threads?
>>
>> Anyway, please go ahead and explain your approach as detailed as
>> possible, I am all ears.
>>
>> regards,
>> Chris
>>
>> Zitat von Geoff Wright <[hidden email]>:
>>
>> > Hi guys,
>> >
>> > I would argue that for qml integration the best approach is for each vtk
>> > viewport to render into a texture on a dedicated thread, not the qml
>> > rendering thread.  Qml then draws the resulting texture which is more in
>> > line with the Qt design intention ( see
>> >
>> http://blog.qt.io/blog/2015/05/11/integrating-custom-opengl-rendering-with-qt-quick-via-qquickframebufferobject/
>> >  )
>> >
>> > I have been using this approach for a couple of years in a high
>> performance
>> > real time application.  The code is unfortunately proprietary but very
>> > happy to explain more about the approach.  It would be great to add
>> support
>> > for this to VTK.
>> >
>> > For the interactor part I do what Sankesh said, defer the events and
>> apply
>> > them on the render thread of the particular vtk item.
>> >
>> > Regards,
>> >
>> > G
>> >
>> >
>> >
>> > On Tue, May 16, 2017, 3:06 PM Sankhesh Jhaveri <
>> [hidden email]>
>> > wrote:
>> >
>> >> Hi Christian,
>> >>
>> >> Doing all VTK rendering on the QML render thread is the right thing to
>> do.
>> >> As far as interaction is concerned, make sure that Qt events are queued
>> on
>> >> the main thread and processed when execution reaches the render thread.
>> >>
>> >> I have done some work on VTK and QtQuick integration that I’m planning
>> to
>> >> add to a VTK remote module. That way, it will be available as part of
>> VTK.
>> >>
>> >> Thanks,
>> >> Sankhesh
>> >> ​
>> >>
>> >> On Tue, May 16, 2017 at 4:57 AM <[hidden email]> wrote:
>> >>
>> >>> just for the record (nobody care about this subject?):
>> >>>
>> >>> I found that contrary to what I said I had NOT adhered to the
>> >>> principle established with protoype 1 while implementing protoype 2.
>> >>> Rather, I had directly called into the VTK interactor from the QML UI
>> >>> thread. And because the VTK interactors immediately go into rendering,
>> >>> all hell broke loose.
>> >>>
>> >>> After cleaning up example 2 according to the rule laid out earlier
>> >>> ("store event data in RenderDelegate, call update() on the QML item,
>> >>> pass the event to VTK inside RenderDelgate.Render()", all works well.
>> >>>
>> >>> What I would like to do is validate the solution against findings made
>> >>> elsewhere, and then establish the "canonical" way of integrating VTK
>> >>> with QML. As I said, there are some architectural quirks with the
>> >>> solution which I would also like to address.
>> >>>
>> >>> regards,
>> >>> Christian
>> >>>
>> >>>
>> >>> Zitat von [hidden email]:
>> >>>
>> >>> > Hello all,
>> >>> >
>> >>> > I am looking into the possibility of replacing the 3D rendering
>> >>> > engine in my Qt5 / QML based mobile tablet-oriented application with
>> >>> > VTK. What I need is a first class QtQuick integration (not a hack or
>> >>> > workaround) that is 100% stable in every context (not an unusual
>> >>> > requirement, I admit). To my amazement, nothing of that kind seems
>> >>> > to exist (correct me if I'm wrong).
>> >>> >
>> >>> > I went on to investigate what had been done so far and to implement
>> >>> > my first prototypes, using the QQuickFrameBufferObject approach.
>> >>> > From the very start this felt like an uphill battle, because VTK
>> >>> > seems to come from a Windowing background and is quite tightly
>> >>> > integrated with concepts that are not valid in a QML context.
>> >>> >
>> >>> > I'll describe my findings together with the 2 prototypical QML items
>> >>> > I implemented:
>> >>> >
>> >>> > 1st was an adaptation of the DICOM example which now runs as a QML
>> >>> > item. All user interaction is handled by QML and forwarded to VTK
>> >>> > (which is one thing that doesn't come naturally with VTK), and after
>> >>> > some non-elegant tweaking I was able to have the UI move from slice
>> >>> > to slice and re-render upon mouse wheel events as expected. The
>> >>> > problem here was, that QML insists on keeping mouse event handling
>> >>> > and OpenGL rendering on separate threads, with one "rendering
>> >>> > thread" dedicated to OpenGL. However, the pre-existing VTK
>> >>> > Interactors directly call Render() after reconfiguring the UI from
>> >>> > the mouse event data, which is an absolute QML No-Go. I had to
>> >>> > introduce a RenderDelegate that works somewhat like this:
>> >>> >
>> >>> > QML mouse event:
>> >>> >     tell RenderDelegate about the event
>> >>> >     call update() on the QML item, which triggers rer-endering on
>> >>> > the dedicated thread
>> >>> > on the QML rendering thread:
>> >>> >     call Render() on the RenderWindow
>> >>> >     inside the RenderDelegate, look at whether a mouse event is
>> >>> > pending, and call the corresponding VTK mouse handler
>> >>> >     call the renderer's default Render() function while setting the
>> >>> > delegate's "Used" flag to false
>> >>> >
>> >>> > 2nd was a QML item that displays a model loaded from a 3ds file. My
>> >>> > goal here was to move the model around using mouse drag events. I
>> >>> > took the lessons learned from the first example, threw in a
>> >>> > vtkInteractorStyleTrackballCamera and hoped for the best. First
>> >>> > thing I found was that I needed a specialized
>> >>> > RenderWindowInteractor, which I provided. When I realized that the
>> >>> > most important requirement for that class was to provide access to
>> >>> > timer functionality, I already got wary, as throwing around events
>> >>> > and doing stuff offline while on the QML rendering thread is never a
>> >>> > good idea. My fears came true when I finished wiring everything
>> >>> > together: I was able to move the model using the mouse, but after a
>> >>> > few moments things got whacky with error output written to the
>> >>> > console and finally a segemtation fault from deep inside VTK.
>> >>> >
>> >>> > I still need to investigate the second example, but would like to
>> >>> > synchronize with the community at this point in order to avoid
>> >>> > errors/misconceptions on my side, to seek help if possible and to
>> >>> > offer my contribution to the VTK code base once this becomes
>> >>> > functional. My first impression is that there are some issues that
>> >>> > lie on an architectural level and cannot be (elegantly) dealt with
>> >>> > on the QML side alone. Any comments?
>> >>> >
>> >>> > thanks,
>> >>> > Christian Sell
>> >>> >
>> >>> > _______________________________________________
>> >>> > Powered by www.kitware.com
>> >>> >
>> >>> > Visit other Kitware open-source projects at
>> >>> > http://www.kitware.com/opensource/opensource.html
>> >>> >
>> >>> > Search the list archives at:
>> >>> http://markmail.org/search/?q=vtk-developers
>> >>> >
>> >>> > Follow this link to subscribe/unsubscribe:
>> >>> > http://public.kitware.com/mailman/listinfo/vtk-developers
>> >>>
>> >>>
>> >>>
>> >>> _______________________________________________
>> >>> Powered by www.kitware.com
>> >>>
>> >>> Visit other Kitware open-source projects at
>> >>> http://www.kitware.com/opensource/opensource.html
>> >>>
>> >>> Search the list archives at:
>> http://markmail.org/search/?q=vtk-developers
>> >>>
>> >>> Follow this link to subscribe/unsubscribe:
>> >>> http://public.kitware.com/mailman/listinfo/vtk-developers
>> >>>
>> >>> --
>> >> Sankhesh Jhaveri *Sr. Research & Development Engineer* | Kitware
>> >> <http://www.kitware.com/> | (518) 881-4417
>> >> ​
>> >> _______________________________________________
>> >> Powered by www.kitware.com
>> >>
>> >> Visit other Kitware open-source projects at
>> >> http://www.kitware.com/opensource/opensource.html
>> >>
>> >> Search the list archives at:
>> http://markmail.org/search/?q=vtk-developers
>> >>
>> >> Follow this link to subscribe/unsubscribe:
>> >> http://public.kitware.com/mailman/listinfo/vtk-developers
>> >>
>> >>
>>
>>
>>
>>



_______________________________________________
Powered by www.kitware.com

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Search the list archives at: http://markmail.org/search/?q=vtk-developers

Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/vtk-developers

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: VTK and Qt5 / QML

gpwright
Yes each vtk item has its own render thread that is why I would argue this approach is superior to the others since it scales with the number of cores.

I followed the approach described here:

So basically inside each vtk qml item there is a thread drawing a vtk viewport into a texture, then the QSGRenderThread comes along and just does a single texture blit for each item (very cheap). The Qt main thread takes care of other event processing, models etc.

This is sufficient if each item contains an independent scene (vtk pipeline).  If you need to share state between two or more viewports  then you need to be careful to control the timing of when any data objects may mutate since the vtk api is not inherently threadsafe.

Does that make sense?

G



On Tue, May 16, 2017, 6:13 PM <[hidden email]> wrote:
Hello all,

@Geoff: are you saying that each pair of QML item / vtkRenderWindow
has its own render thread? Is that something QML does?

My goal is to have QML/VTK items behave as normal QML items, i.e., I
could have a dozen items on my screen each rendering something
different through VTK without influencing each other (apart from
system performance). I expect that the approach I have taken so far
will already fulfill this requirement.

@Sankesh: I object to the compile time flag. IMO both approaches
should be usable at the same time, e.g. by extending different
superclasses. As you say, the "underlay or overlay" aspect breaks QML
concepts and makes any item built according to this approach an
"alien" in the QML crowd that always requires special attention.

I'll look into QVTKInteractorAdapter - that sounds like seomething
useful. Should have come across it earlier.

@both: is there a chance that usable code will come out of this
exchange any time soon? I for my part would at least publish my
prototypical implementations somewhere (GitHub?), unless they become
obsolete by something "official"

regards,
Chris

Zitat von Geoff Wright <[hidden email]>:

> Hi Chris,
>
> Sorry for the confusion, yes I am doing what you describe.  The difference
> may be that perhaps your app has only one viewport.  In my case there are a
> number of different viewports (aka vtkRenderWindow)  each has its own qml
> item and is each rendering on it's own thread.  Plus the qml composition
> thread which is managed by Qt.
>
> I was just making the point that this architecture let's you draw different
> viewports in parallel provided that each one has its own mappers and actors.
>
> G
>
>
> On Tue, May 16, 2017, 4:13 PM <[hidden email]> wrote:
>
>> Hi Geoff,
>>
>> as I understand it, the blog post you linked to doesn't mention
>> multiple threads beyond the QML UI thread and the scenegraph render
>> thread.
>>
>> So far, I am using that approach with QQuickFrameBufferObject as
>> starting point and vtkExternalOpenGLRenderWindow for the VTK side,
>> which causes VTK to use QML's OpenGL context and thus render into the
>> FBO maintained by QQuickFrameBufferObject. That seems to work
>> flawlessly, as long as ALL rendering is kept on the scenegraph thread,
>> which is not always obvious to do at first, because VTK's API's don't
>> separate between reading event values and rendering them (e.g.,
>> SetSlice(2) will internally set the current slice and then do a
>> Render()).
>>
>> But why maintain more threads?
>>
>> Anyway, please go ahead and explain your approach as detailed as
>> possible, I am all ears.
>>
>> regards,
>> Chris
>>
>> Zitat von Geoff Wright <[hidden email]>:
>>
>> > Hi guys,
>> >
>> > I would argue that for qml integration the best approach is for each vtk
>> > viewport to render into a texture on a dedicated thread, not the qml
>> > rendering thread.  Qml then draws the resulting texture which is more in
>> > line with the Qt design intention ( see
>> >
>> http://blog.qt.io/blog/2015/05/11/integrating-custom-opengl-rendering-with-qt-quick-via-qquickframebufferobject/
>> >  )
>> >
>> > I have been using this approach for a couple of years in a high
>> performance
>> > real time application.  The code is unfortunately proprietary but very
>> > happy to explain more about the approach.  It would be great to add
>> support
>> > for this to VTK.
>> >
>> > For the interactor part I do what Sankesh said, defer the events and
>> apply
>> > them on the render thread of the particular vtk item.
>> >
>> > Regards,
>> >
>> > G
>> >
>> >
>> >
>> > On Tue, May 16, 2017, 3:06 PM Sankhesh Jhaveri <
>> [hidden email]>
>> > wrote:
>> >
>> >> Hi Christian,
>> >>
>> >> Doing all VTK rendering on the QML render thread is the right thing to
>> do.
>> >> As far as interaction is concerned, make sure that Qt events are queued
>> on
>> >> the main thread and processed when execution reaches the render thread.
>> >>
>> >> I have done some work on VTK and QtQuick integration that I’m planning
>> to
>> >> add to a VTK remote module. That way, it will be available as part of
>> VTK.
>> >>
>> >> Thanks,
>> >> Sankhesh
>> >> ​
>> >>
>> >> On Tue, May 16, 2017 at 4:57 AM <[hidden email]> wrote:
>> >>
>> >>> just for the record (nobody care about this subject?):
>> >>>
>> >>> I found that contrary to what I said I had NOT adhered to the
>> >>> principle established with protoype 1 while implementing protoype 2.
>> >>> Rather, I had directly called into the VTK interactor from the QML UI
>> >>> thread. And because the VTK interactors immediately go into rendering,
>> >>> all hell broke loose.
>> >>>
>> >>> After cleaning up example 2 according to the rule laid out earlier
>> >>> ("store event data in RenderDelegate, call update() on the QML item,
>> >>> pass the event to VTK inside RenderDelgate.Render()", all works well.
>> >>>
>> >>> What I would like to do is validate the solution against findings made
>> >>> elsewhere, and then establish the "canonical" way of integrating VTK
>> >>> with QML. As I said, there are some architectural quirks with the
>> >>> solution which I would also like to address.
>> >>>
>> >>> regards,
>> >>> Christian
>> >>>
>> >>>
>> >>> Zitat von [hidden email]:
>> >>>
>> >>> > Hello all,
>> >>> >
>> >>> > I am looking into the possibility of replacing the 3D rendering
>> >>> > engine in my Qt5 / QML based mobile tablet-oriented application with
>> >>> > VTK. What I need is a first class QtQuick integration (not a hack or
>> >>> > workaround) that is 100% stable in every context (not an unusual
>> >>> > requirement, I admit). To my amazement, nothing of that kind seems
>> >>> > to exist (correct me if I'm wrong).
>> >>> >
>> >>> > I went on to investigate what had been done so far and to implement
>> >>> > my first prototypes, using the QQuickFrameBufferObject approach.
>> >>> > From the very start this felt like an uphill battle, because VTK
>> >>> > seems to come from a Windowing background and is quite tightly
>> >>> > integrated with concepts that are not valid in a QML context.
>> >>> >
>> >>> > I'll describe my findings together with the 2 prototypical QML items
>> >>> > I implemented:
>> >>> >
>> >>> > 1st was an adaptation of the DICOM example which now runs as a QML
>> >>> > item. All user interaction is handled by QML and forwarded to VTK
>> >>> > (which is one thing that doesn't come naturally with VTK), and after
>> >>> > some non-elegant tweaking I was able to have the UI move from slice
>> >>> > to slice and re-render upon mouse wheel events as expected. The
>> >>> > problem here was, that QML insists on keeping mouse event handling
>> >>> > and OpenGL rendering on separate threads, with one "rendering
>> >>> > thread" dedicated to OpenGL. However, the pre-existing VTK
>> >>> > Interactors directly call Render() after reconfiguring the UI from
>> >>> > the mouse event data, which is an absolute QML No-Go. I had to
>> >>> > introduce a RenderDelegate that works somewhat like this:
>> >>> >
>> >>> > QML mouse event:
>> >>> >     tell RenderDelegate about the event
>> >>> >     call update() on the QML item, which triggers rer-endering on
>> >>> > the dedicated thread
>> >>> > on the QML rendering thread:
>> >>> >     call Render() on the RenderWindow
>> >>> >     inside the RenderDelegate, look at whether a mouse event is
>> >>> > pending, and call the corresponding VTK mouse handler
>> >>> >     call the renderer's default Render() function while setting the
>> >>> > delegate's "Used" flag to false
>> >>> >
>> >>> > 2nd was a QML item that displays a model loaded from a 3ds file. My
>> >>> > goal here was to move the model around using mouse drag events. I
>> >>> > took the lessons learned from the first example, threw in a
>> >>> > vtkInteractorStyleTrackballCamera and hoped for the best. First
>> >>> > thing I found was that I needed a specialized
>> >>> > RenderWindowInteractor, which I provided. When I realized that the
>> >>> > most important requirement for that class was to provide access to
>> >>> > timer functionality, I already got wary, as throwing around events
>> >>> > and doing stuff offline while on the QML rendering thread is never a
>> >>> > good idea. My fears came true when I finished wiring everything
>> >>> > together: I was able to move the model using the mouse, but after a
>> >>> > few moments things got whacky with error output written to the
>> >>> > console and finally a segemtation fault from deep inside VTK.
>> >>> >
>> >>> > I still need to investigate the second example, but would like to
>> >>> > synchronize with the community at this point in order to avoid
>> >>> > errors/misconceptions on my side, to seek help if possible and to
>> >>> > offer my contribution to the VTK code base once this becomes
>> >>> > functional. My first impression is that there are some issues that
>> >>> > lie on an architectural level and cannot be (elegantly) dealt with
>> >>> > on the QML side alone. Any comments?
>> >>> >
>> >>> > thanks,
>> >>> > Christian Sell
>> >>> >
>> >>> > _______________________________________________
>> >>> > Powered by www.kitware.com
>> >>> >
>> >>> > Visit other Kitware open-source projects at
>> >>> > http://www.kitware.com/opensource/opensource.html
>> >>> >
>> >>> > Search the list archives at:
>> >>> http://markmail.org/search/?q=vtk-developers
>> >>> >
>> >>> > Follow this link to subscribe/unsubscribe:
>> >>> > http://public.kitware.com/mailman/listinfo/vtk-developers
>> >>>
>> >>>
>> >>>
>> >>> _______________________________________________
>> >>> Powered by www.kitware.com
>> >>>
>> >>> Visit other Kitware open-source projects at
>> >>> http://www.kitware.com/opensource/opensource.html
>> >>>
>> >>> Search the list archives at:
>> http://markmail.org/search/?q=vtk-developers
>> >>>
>> >>> Follow this link to subscribe/unsubscribe:
>> >>> http://public.kitware.com/mailman/listinfo/vtk-developers
>> >>>
>> >>> --
>> >> Sankhesh Jhaveri *Sr. Research & Development Engineer* | Kitware
>> >> <http://www.kitware.com/> | (518) 881-4417
>> >> ​
>> >> _______________________________________________
>> >> Powered by www.kitware.com
>> >>
>> >> Visit other Kitware open-source projects at
>> >> http://www.kitware.com/opensource/opensource.html
>> >>
>> >> Search the list archives at:
>> http://markmail.org/search/?q=vtk-developers
>> >>
>> >> Follow this link to subscribe/unsubscribe:
>> >> http://public.kitware.com/mailman/listinfo/vtk-developers
>> >>
>> >>
>>
>>
>>
>>




_______________________________________________
Powered by www.kitware.com

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Search the list archives at: http://markmail.org/search/?q=vtk-developers

Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/vtk-developers

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: VTK and Qt5 / QML

c.sell
ok, things are clearing up a little more now. You ARE creating a  
separate thread for each QML item, which gives us these threads:

- QML GUI thread (where the events come from)
- QML rendering thread (aka scenegraph rendering thread aka  
QSGRenderThread), of which theres is only one
- a dedicated thread for each item (RenderThread class in the example)

the example also uses a QSGSimpleTextureNode subclass that does the  
blit'ing together. All of that decoupled via QueuedConnections and  
events signalled here and there. Not an easy beast, that.

What remains unclear is how you deal with user input. For example, if  
the user were to drag something around in the scene (and your app were  
to support that), where do the MouseMove events go? This is really the  
main crux in this discussion IMO.

To sum up the architectural choices:

a) render into FBO
a1)- do all rendering on the scenegraph thread (exemplified in  
http://doc.qt.io/qt-5/qtquick-scenegraph-textureinsgnode-example.html)

a2) render on a dedicated thread for each item (exemplified in  
http://doc.qt.io/qt-5/qtquick-scenegraph-textureinthread-example.html)

b) render into the QML context directly (not sure about this. Is there  
a Qt example demonstrating the practicability?)
b?) undetermined


The user intaction part is undetermined for me at this point. I am  
trying to get something going using QVTKInteractorAdapter, but havent  
had luck so far.

In terms of ready usable code that demonstrates QML/VTK integration  
(including interaction and all) I see nothing yet on the horizon.

chris



Zitat von Geoff Wright <[hidden email]>:

> Yes each vtk item has its own render thread that is why I would argue this
> approach is superior to the others since it scales with the number of cores.
>
> I followed the approach described here:
>
> http://doc.qt.io/qt-5/qtquick-scenegraph-textureinthread-example.html
>
> So basically inside each vtk qml item there is a thread drawing a vtk
> viewport into a texture, then the QSGRenderThread comes along and just does
> a single texture blit for each item (very cheap). The Qt main thread takes
> care of other event processing, models etc.
>
> This is sufficient if each item contains an independent scene (vtk
> pipeline).  If you need to share state between two or more viewports  then
> you need to be careful to control the timing of when any data objects may
> mutate since the vtk api is not inherently threadsafe.
>
> Does that make sense?
>
> G
>
>
>
> On Tue, May 16, 2017, 6:13 PM <[hidden email]> wrote:
>
>> Hello all,
>>
>> @Geoff: are you saying that each pair of QML item / vtkRenderWindow
>> has its own render thread? Is that something QML does?
>>
>> My goal is to have QML/VTK items behave as normal QML items, i.e., I
>> could have a dozen items on my screen each rendering something
>> different through VTK without influencing each other (apart from
>> system performance). I expect that the approach I have taken so far
>> will already fulfill this requirement.
>>
>> @Sankesh: I object to the compile time flag. IMO both approaches
>> should be usable at the same time, e.g. by extending different
>> superclasses. As you say, the "underlay or overlay" aspect breaks QML
>> concepts and makes any item built according to this approach an
>> "alien" in the QML crowd that always requires special attention.
>>
>> I'll look into QVTKInteractorAdapter - that sounds like seomething
>> useful. Should have come across it earlier.
>>
>> @both: is there a chance that usable code will come out of this
>> exchange any time soon? I for my part would at least publish my
>> prototypical implementations somewhere (GitHub?), unless they become
>> obsolete by something "official"
>>
>> regards,
>> Chris
>>
>> Zitat von Geoff Wright <[hidden email]>:
>>
>> > Hi Chris,
>> >
>> > Sorry for the confusion, yes I am doing what you describe.  The
>> difference
>> > may be that perhaps your app has only one viewport.  In my case there
>> are a
>> > number of different viewports (aka vtkRenderWindow)  each has its own qml
>> > item and is each rendering on it's own thread.  Plus the qml composition
>> > thread which is managed by Qt.
>> >
>> > I was just making the point that this architecture let's you draw
>> different
>> > viewports in parallel provided that each one has its own mappers and
>> actors.
>> >
>> > G
>> >
>> >
>> > On Tue, May 16, 2017, 4:13 PM <[hidden email]> wrote:
>> >
>> >> Hi Geoff,
>> >>
>> >> as I understand it, the blog post you linked to doesn't mention
>> >> multiple threads beyond the QML UI thread and the scenegraph render
>> >> thread.
>> >>
>> >> So far, I am using that approach with QQuickFrameBufferObject as
>> >> starting point and vtkExternalOpenGLRenderWindow for the VTK side,
>> >> which causes VTK to use QML's OpenGL context and thus render into the
>> >> FBO maintained by QQuickFrameBufferObject. That seems to work
>> >> flawlessly, as long as ALL rendering is kept on the scenegraph thread,
>> >> which is not always obvious to do at first, because VTK's API's don't
>> >> separate between reading event values and rendering them (e.g.,
>> >> SetSlice(2) will internally set the current slice and then do a
>> >> Render()).
>> >>
>> >> But why maintain more threads?
>> >>
>> >> Anyway, please go ahead and explain your approach as detailed as
>> >> possible, I am all ears.
>> >>
>> >> regards,
>> >> Chris
>> >>
>> >> Zitat von Geoff Wright <[hidden email]>:
>> >>
>> >> > Hi guys,
>> >> >
>> >> > I would argue that for qml integration the best approach is for each
>> vtk
>> >> > viewport to render into a texture on a dedicated thread, not the qml
>> >> > rendering thread.  Qml then draws the resulting texture which is more
>> in
>> >> > line with the Qt design intention ( see
>> >> >
>> >>
>> http://blog.qt.io/blog/2015/05/11/integrating-custom-opengl-rendering-with-qt-quick-via-qquickframebufferobject/
>> >> >  )
>> >> >
>> >> > I have been using this approach for a couple of years in a high
>> >> performance
>> >> > real time application.  The code is unfortunately proprietary but very
>> >> > happy to explain more about the approach.  It would be great to add
>> >> support
>> >> > for this to VTK.
>> >> >
>> >> > For the interactor part I do what Sankesh said, defer the events and
>> >> apply
>> >> > them on the render thread of the particular vtk item.
>> >> >
>> >> > Regards,
>> >> >
>> >> > G
>> >> >
>> >> >
>> >> >
>> >> > On Tue, May 16, 2017, 3:06 PM Sankhesh Jhaveri <
>> >> [hidden email]>
>> >> > wrote:
>> >> >
>> >> >> Hi Christian,
>> >> >>
>> >> >> Doing all VTK rendering on the QML render thread is the right thing
>> to
>> >> do.
>> >> >> As far as interaction is concerned, make sure that Qt events are
>> queued
>> >> on
>> >> >> the main thread and processed when execution reaches the render
>> thread.
>> >> >>
>> >> >> I have done some work on VTK and QtQuick integration that I’m
>> planning
>> >> to
>> >> >> add to a VTK remote module. That way, it will be available as part of
>> >> VTK.
>> >> >>
>> >> >> Thanks,
>> >> >> Sankhesh
>> >> >> ​
>> >> >>
>> >> >> On Tue, May 16, 2017 at 4:57 AM <[hidden email]> wrote:
>> >> >>
>> >> >>> just for the record (nobody care about this subject?):
>> >> >>>
>> >> >>> I found that contrary to what I said I had NOT adhered to the
>> >> >>> principle established with protoype 1 while implementing protoype 2.
>> >> >>> Rather, I had directly called into the VTK interactor from the QML
>> UI
>> >> >>> thread. And because the VTK interactors immediately go into
>> rendering,
>> >> >>> all hell broke loose.
>> >> >>>
>> >> >>> After cleaning up example 2 according to the rule laid out earlier
>> >> >>> ("store event data in RenderDelegate, call update() on the QML item,
>> >> >>> pass the event to VTK inside RenderDelgate.Render()", all works
>> well.
>> >> >>>
>> >> >>> What I would like to do is validate the solution against findings
>> made
>> >> >>> elsewhere, and then establish the "canonical" way of integrating VTK
>> >> >>> with QML. As I said, there are some architectural quirks with the
>> >> >>> solution which I would also like to address.
>> >> >>>
>> >> >>> regards,
>> >> >>> Christian
>> >> >>>
>> >> >>>
>> >> >>> Zitat von [hidden email]:
>> >> >>>
>> >> >>> > Hello all,
>> >> >>> >
>> >> >>> > I am looking into the possibility of replacing the 3D rendering
>> >> >>> > engine in my Qt5 / QML based mobile tablet-oriented application
>> with
>> >> >>> > VTK. What I need is a first class QtQuick integration (not a hack
>> or
>> >> >>> > workaround) that is 100% stable in every context (not an unusual
>> >> >>> > requirement, I admit). To my amazement, nothing of that kind seems
>> >> >>> > to exist (correct me if I'm wrong).
>> >> >>> >
>> >> >>> > I went on to investigate what had been done so far and to
>> implement
>> >> >>> > my first prototypes, using the QQuickFrameBufferObject approach.
>> >> >>> > From the very start this felt like an uphill battle, because VTK
>> >> >>> > seems to come from a Windowing background and is quite tightly
>> >> >>> > integrated with concepts that are not valid in a QML context.
>> >> >>> >
>> >> >>> > I'll describe my findings together with the 2 prototypical QML
>> items
>> >> >>> > I implemented:
>> >> >>> >
>> >> >>> > 1st was an adaptation of the DICOM example which now runs as a QML
>> >> >>> > item. All user interaction is handled by QML and forwarded to VTK
>> >> >>> > (which is one thing that doesn't come naturally with VTK), and
>> after
>> >> >>> > some non-elegant tweaking I was able to have the UI move from
>> slice
>> >> >>> > to slice and re-render upon mouse wheel events as expected. The
>> >> >>> > problem here was, that QML insists on keeping mouse event handling
>> >> >>> > and OpenGL rendering on separate threads, with one "rendering
>> >> >>> > thread" dedicated to OpenGL. However, the pre-existing VTK
>> >> >>> > Interactors directly call Render() after reconfiguring the UI from
>> >> >>> > the mouse event data, which is an absolute QML No-Go. I had to
>> >> >>> > introduce a RenderDelegate that works somewhat like this:
>> >> >>> >
>> >> >>> > QML mouse event:
>> >> >>> >     tell RenderDelegate about the event
>> >> >>> >     call update() on the QML item, which triggers rer-endering on
>> >> >>> > the dedicated thread
>> >> >>> > on the QML rendering thread:
>> >> >>> >     call Render() on the RenderWindow
>> >> >>> >     inside the RenderDelegate, look at whether a mouse event is
>> >> >>> > pending, and call the corresponding VTK mouse handler
>> >> >>> >     call the renderer's default Render() function while setting
>> the
>> >> >>> > delegate's "Used" flag to false
>> >> >>> >
>> >> >>> > 2nd was a QML item that displays a model loaded from a 3ds file.
>> My
>> >> >>> > goal here was to move the model around using mouse drag events. I
>> >> >>> > took the lessons learned from the first example, threw in a
>> >> >>> > vtkInteractorStyleTrackballCamera and hoped for the best. First
>> >> >>> > thing I found was that I needed a specialized
>> >> >>> > RenderWindowInteractor, which I provided. When I realized that the
>> >> >>> > most important requirement for that class was to provide access to
>> >> >>> > timer functionality, I already got wary, as throwing around events
>> >> >>> > and doing stuff offline while on the QML rendering thread is
>> never a
>> >> >>> > good idea. My fears came true when I finished wiring everything
>> >> >>> > together: I was able to move the model using the mouse, but after
>> a
>> >> >>> > few moments things got whacky with error output written to the
>> >> >>> > console and finally a segemtation fault from deep inside VTK.
>> >> >>> >
>> >> >>> > I still need to investigate the second example, but would like to
>> >> >>> > synchronize with the community at this point in order to avoid
>> >> >>> > errors/misconceptions on my side, to seek help if possible and to
>> >> >>> > offer my contribution to the VTK code base once this becomes
>> >> >>> > functional. My first impression is that there are some issues that
>> >> >>> > lie on an architectural level and cannot be (elegantly) dealt with
>> >> >>> > on the QML side alone. Any comments?
>> >> >>> >
>> >> >>> > thanks,
>> >> >>> > Christian Sell
>> >> >>> >
>> >> >>> > _______________________________________________
>> >> >>> > Powered by www.kitware.com
>> >> >>> >
>> >> >>> > Visit other Kitware open-source projects at
>> >> >>> > http://www.kitware.com/opensource/opensource.html
>> >> >>> >
>> >> >>> > Search the list archives at:
>> >> >>> http://markmail.org/search/?q=vtk-developers
>> >> >>> >
>> >> >>> > Follow this link to subscribe/unsubscribe:
>> >> >>> > http://public.kitware.com/mailman/listinfo/vtk-developers
>> >> >>>
>> >> >>>
>> >> >>>
>> >> >>> _______________________________________________
>> >> >>> Powered by www.kitware.com
>> >> >>>
>> >> >>> Visit other Kitware open-source projects at
>> >> >>> http://www.kitware.com/opensource/opensource.html
>> >> >>>
>> >> >>> Search the list archives at:
>> >> http://markmail.org/search/?q=vtk-developers
>> >> >>>
>> >> >>> Follow this link to subscribe/unsubscribe:
>> >> >>> http://public.kitware.com/mailman/listinfo/vtk-developers
>> >> >>>
>> >> >>> --
>> >> >> Sankhesh Jhaveri *Sr. Research & Development Engineer* | Kitware
>> >> >> <http://www.kitware.com/> | (518) 881-4417
>> >> >> ​
>> >> >> _______________________________________________
>> >> >> Powered by www.kitware.com
>> >> >>
>> >> >> Visit other Kitware open-source projects at
>> >> >> http://www.kitware.com/opensource/opensource.html
>> >> >>
>> >> >> Search the list archives at:
>> >> http://markmail.org/search/?q=vtk-developers
>> >> >>
>> >> >> Follow this link to subscribe/unsubscribe:
>> >> >> http://public.kitware.com/mailman/listinfo/vtk-developers
>> >> >>
>> >> >>
>> >>
>> >>
>> >>
>> >>
>>
>>
>>
>>



_______________________________________________
Powered by www.kitware.com

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Search the list archives at: http://markmail.org/search/?q=vtk-developers

Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/vtk-developers

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: VTK and Qt5 / QML

c.sell
I have to correct myself with regard to the "direct rendering" option.  
There is, of course, an example that demonstrates this  
(http://doc.qt.io/qt-5/qtquick-scenegraph-openglunderqml-example.html). I  
doubt whether this could be implemented in conjunction with  
thread-per-item, so it only adds one architectural option. I still  
think this is the least versatile and generic one, though.

chris

Zitat von [hidden email]:

> ok, things are clearing up a little more now. You ARE creating a  
> separate thread for each QML item, which gives us these threads:
>
> - QML GUI thread (where the events come from)
> - QML rendering thread (aka scenegraph rendering thread aka  
> QSGRenderThread), of which theres is only one
> - a dedicated thread for each item (RenderThread class in the example)
>
> the example also uses a QSGSimpleTextureNode subclass that does the  
> blit'ing together. All of that decoupled via QueuedConnections and  
> events signalled here and there. Not an easy beast, that.
>
> What remains unclear is how you deal with user input. For example,  
> if the user were to drag something around in the scene (and your app  
> were to support that), where do the MouseMove events go? This is  
> really the main crux in this discussion IMO.
>
> To sum up the architectural choices:
>
> a) render into FBO
> a1)- do all rendering on the scenegraph thread (exemplified in  
> http://doc.qt.io/qt-5/qtquick-scenegraph-textureinsgnode-example.html)
>
> a2) render on a dedicated thread for each item (exemplified in  
> http://doc.qt.io/qt-5/qtquick-scenegraph-textureinthread-example.html)
>
> b) render into the QML context directly (not sure about this. Is  
> there a Qt example demonstrating the practicability?)
> b?) undetermined
>
>
> The user intaction part is undetermined for me at this point. I am  
> trying to get something going using QVTKInteractorAdapter, but  
> havent had luck so far.
>
> In terms of ready usable code that demonstrates QML/VTK integration  
> (including interaction and all) I see nothing yet on the horizon.
>
> chris
>
>
>
> Zitat von Geoff Wright <[hidden email]>:
>
>> Yes each vtk item has its own render thread that is why I would argue this
>> approach is superior to the others since it scales with the number of cores.
>>
>> I followed the approach described here:
>>
>> http://doc.qt.io/qt-5/qtquick-scenegraph-textureinthread-example.html
>>
>> So basically inside each vtk qml item there is a thread drawing a vtk
>> viewport into a texture, then the QSGRenderThread comes along and just does
>> a single texture blit for each item (very cheap). The Qt main thread takes
>> care of other event processing, models etc.
>>
>> This is sufficient if each item contains an independent scene (vtk
>> pipeline).  If you need to share state between two or more viewports  then
>> you need to be careful to control the timing of when any data objects may
>> mutate since the vtk api is not inherently threadsafe.
>>
>> Does that make sense?
>>
>> G
>>
>>
>>
>> On Tue, May 16, 2017, 6:13 PM <[hidden email]> wrote:
>>
>>> Hello all,
>>>
>>> @Geoff: are you saying that each pair of QML item / vtkRenderWindow
>>> has its own render thread? Is that something QML does?
>>>
>>> My goal is to have QML/VTK items behave as normal QML items, i.e., I
>>> could have a dozen items on my screen each rendering something
>>> different through VTK without influencing each other (apart from
>>> system performance). I expect that the approach I have taken so far
>>> will already fulfill this requirement.
>>>
>>> @Sankesh: I object to the compile time flag. IMO both approaches
>>> should be usable at the same time, e.g. by extending different
>>> superclasses. As you say, the "underlay or overlay" aspect breaks QML
>>> concepts and makes any item built according to this approach an
>>> "alien" in the QML crowd that always requires special attention.
>>>
>>> I'll look into QVTKInteractorAdapter - that sounds like seomething
>>> useful. Should have come across it earlier.
>>>
>>> @both: is there a chance that usable code will come out of this
>>> exchange any time soon? I for my part would at least publish my
>>> prototypical implementations somewhere (GitHub?), unless they become
>>> obsolete by something "official"
>>>
>>> regards,
>>> Chris
>>>
>>> Zitat von Geoff Wright <[hidden email]>:
>>>
>>>> Hi Chris,
>>>>
>>>> Sorry for the confusion, yes I am doing what you describe.  The
>>> difference
>>>> may be that perhaps your app has only one viewport.  In my case there
>>> are a
>>>> number of different viewports (aka vtkRenderWindow)  each has its own qml
>>>> item and is each rendering on it's own thread.  Plus the qml composition
>>>> thread which is managed by Qt.
>>>>
>>>> I was just making the point that this architecture let's you draw
>>> different
>>>> viewports in parallel provided that each one has its own mappers and
>>> actors.
>>>>
>>>> G
>>>>
>>>>
>>>> On Tue, May 16, 2017, 4:13 PM <[hidden email]> wrote:
>>>>
>>>>> Hi Geoff,
>>>>>
>>>>> as I understand it, the blog post you linked to doesn't mention
>>>>> multiple threads beyond the QML UI thread and the scenegraph render
>>>>> thread.
>>>>>
>>>>> So far, I am using that approach with QQuickFrameBufferObject as
>>>>> starting point and vtkExternalOpenGLRenderWindow for the VTK side,
>>>>> which causes VTK to use QML's OpenGL context and thus render into the
>>>>> FBO maintained by QQuickFrameBufferObject. That seems to work
>>>>> flawlessly, as long as ALL rendering is kept on the scenegraph thread,
>>>>> which is not always obvious to do at first, because VTK's API's don't
>>>>> separate between reading event values and rendering them (e.g.,
>>>>> SetSlice(2) will internally set the current slice and then do a
>>>>> Render()).
>>>>>
>>>>> But why maintain more threads?
>>>>>
>>>>> Anyway, please go ahead and explain your approach as detailed as
>>>>> possible, I am all ears.
>>>>>
>>>>> regards,
>>>>> Chris
>>>>>
>>>>> Zitat von Geoff Wright <[hidden email]>:
>>>>>
>>>>> > Hi guys,
>>>>> >
>>>>> > I would argue that for qml integration the best approach is for each
>>> vtk
>>>>> > viewport to render into a texture on a dedicated thread, not the qml
>>>>> > rendering thread.  Qml then draws the resulting texture which is more
>>> in
>>>>> > line with the Qt design intention ( see
>>>>> >
>>>>>
>>> http://blog.qt.io/blog/2015/05/11/integrating-custom-opengl-rendering-with-qt-quick-via-qquickframebufferobject/
>>>>> >  )
>>>>> >
>>>>> > I have been using this approach for a couple of years in a high
>>>>> performance
>>>>> > real time application.  The code is unfortunately proprietary but very
>>>>> > happy to explain more about the approach.  It would be great to add
>>>>> support
>>>>> > for this to VTK.
>>>>> >
>>>>> > For the interactor part I do what Sankesh said, defer the events and
>>>>> apply
>>>>> > them on the render thread of the particular vtk item.
>>>>> >
>>>>> > Regards,
>>>>> >
>>>>> > G
>>>>> >
>>>>> >
>>>>> >
>>>>> > On Tue, May 16, 2017, 3:06 PM Sankhesh Jhaveri <
>>>>> [hidden email]>
>>>>> > wrote:
>>>>> >
>>>>> >> Hi Christian,
>>>>> >>
>>>>> >> Doing all VTK rendering on the QML render thread is the right thing
>>> to
>>>>> do.
>>>>> >> As far as interaction is concerned, make sure that Qt events are
>>> queued
>>>>> on
>>>>> >> the main thread and processed when execution reaches the render
>>> thread.
>>>>> >>
>>>>> >> I have done some work on VTK and QtQuick integration that I’m
>>> planning
>>>>> to
>>>>> >> add to a VTK remote module. That way, it will be available as part of
>>>>> VTK.
>>>>> >>
>>>>> >> Thanks,
>>>>> >> Sankhesh
>>>>> >> ​
>>>>> >>
>>>>> >> On Tue, May 16, 2017 at 4:57 AM <[hidden email]> wrote:
>>>>> >>
>>>>> >>> just for the record (nobody care about this subject?):
>>>>> >>>
>>>>> >>> I found that contrary to what I said I had NOT adhered to the
>>>>> >>> principle established with protoype 1 while implementing protoype 2.
>>>>> >>> Rather, I had directly called into the VTK interactor from the QML
>>> UI
>>>>> >>> thread. And because the VTK interactors immediately go into
>>> rendering,
>>>>> >>> all hell broke loose.
>>>>> >>>
>>>>> >>> After cleaning up example 2 according to the rule laid out earlier
>>>>> >>> ("store event data in RenderDelegate, call update() on the QML item,
>>>>> >>> pass the event to VTK inside RenderDelgate.Render()", all works
>>> well.
>>>>> >>>
>>>>> >>> What I would like to do is validate the solution against findings
>>> made
>>>>> >>> elsewhere, and then establish the "canonical" way of integrating VTK
>>>>> >>> with QML. As I said, there are some architectural quirks with the
>>>>> >>> solution which I would also like to address.
>>>>> >>>
>>>>> >>> regards,
>>>>> >>> Christian
>>>>> >>>
>>>>> >>>
>>>>> >>> Zitat von [hidden email]:
>>>>> >>>
>>>>> >>> > Hello all,
>>>>> >>> >
>>>>> >>> > I am looking into the possibility of replacing the 3D rendering
>>>>> >>> > engine in my Qt5 / QML based mobile tablet-oriented application
>>> with
>>>>> >>> > VTK. What I need is a first class QtQuick integration (not a hack
>>> or
>>>>> >>> > workaround) that is 100% stable in every context (not an unusual
>>>>> >>> > requirement, I admit). To my amazement, nothing of that kind seems
>>>>> >>> > to exist (correct me if I'm wrong).
>>>>> >>> >
>>>>> >>> > I went on to investigate what had been done so far and to
>>> implement
>>>>> >>> > my first prototypes, using the QQuickFrameBufferObject approach.
>>>>> >>> > From the very start this felt like an uphill battle, because VTK
>>>>> >>> > seems to come from a Windowing background and is quite tightly
>>>>> >>> > integrated with concepts that are not valid in a QML context.
>>>>> >>> >
>>>>> >>> > I'll describe my findings together with the 2 prototypical QML
>>> items
>>>>> >>> > I implemented:
>>>>> >>> >
>>>>> >>> > 1st was an adaptation of the DICOM example which now runs as a QML
>>>>> >>> > item. All user interaction is handled by QML and forwarded to VTK
>>>>> >>> > (which is one thing that doesn't come naturally with VTK), and
>>> after
>>>>> >>> > some non-elegant tweaking I was able to have the UI move from
>>> slice
>>>>> >>> > to slice and re-render upon mouse wheel events as expected. The
>>>>> >>> > problem here was, that QML insists on keeping mouse event handling
>>>>> >>> > and OpenGL rendering on separate threads, with one "rendering
>>>>> >>> > thread" dedicated to OpenGL. However, the pre-existing VTK
>>>>> >>> > Interactors directly call Render() after reconfiguring the UI from
>>>>> >>> > the mouse event data, which is an absolute QML No-Go. I had to
>>>>> >>> > introduce a RenderDelegate that works somewhat like this:
>>>>> >>> >
>>>>> >>> > QML mouse event:
>>>>> >>> >     tell RenderDelegate about the event
>>>>> >>> >     call update() on the QML item, which triggers rer-endering on
>>>>> >>> > the dedicated thread
>>>>> >>> > on the QML rendering thread:
>>>>> >>> >     call Render() on the RenderWindow
>>>>> >>> >     inside the RenderDelegate, look at whether a mouse event is
>>>>> >>> > pending, and call the corresponding VTK mouse handler
>>>>> >>> >     call the renderer's default Render() function while setting
>>> the
>>>>> >>> > delegate's "Used" flag to false
>>>>> >>> >
>>>>> >>> > 2nd was a QML item that displays a model loaded from a 3ds file.
>>> My
>>>>> >>> > goal here was to move the model around using mouse drag events. I
>>>>> >>> > took the lessons learned from the first example, threw in a
>>>>> >>> > vtkInteractorStyleTrackballCamera and hoped for the best. First
>>>>> >>> > thing I found was that I needed a specialized
>>>>> >>> > RenderWindowInteractor, which I provided. When I realized that the
>>>>> >>> > most important requirement for that class was to provide access to
>>>>> >>> > timer functionality, I already got wary, as throwing around events
>>>>> >>> > and doing stuff offline while on the QML rendering thread is
>>> never a
>>>>> >>> > good idea. My fears came true when I finished wiring everything
>>>>> >>> > together: I was able to move the model using the mouse, but after
>>> a
>>>>> >>> > few moments things got whacky with error output written to the
>>>>> >>> > console and finally a segemtation fault from deep inside VTK.
>>>>> >>> >
>>>>> >>> > I still need to investigate the second example, but would like to
>>>>> >>> > synchronize with the community at this point in order to avoid
>>>>> >>> > errors/misconceptions on my side, to seek help if possible and to
>>>>> >>> > offer my contribution to the VTK code base once this becomes
>>>>> >>> > functional. My first impression is that there are some issues that
>>>>> >>> > lie on an architectural level and cannot be (elegantly) dealt with
>>>>> >>> > on the QML side alone. Any comments?
>>>>> >>> >
>>>>> >>> > thanks,
>>>>> >>> > Christian Sell
>>>>> >>> >
>>>>> >>> > _______________________________________________
>>>>> >>> > Powered by www.kitware.com
>>>>> >>> >
>>>>> >>> > Visit other Kitware open-source projects at
>>>>> >>> > http://www.kitware.com/opensource/opensource.html
>>>>> >>> >
>>>>> >>> > Search the list archives at:
>>>>> >>> http://markmail.org/search/?q=vtk-developers
>>>>> >>> >
>>>>> >>> > Follow this link to subscribe/unsubscribe:
>>>>> >>> > http://public.kitware.com/mailman/listinfo/vtk-developers
>>>>> >>>
>>>>> >>>
>>>>> >>>
>>>>> >>> _______________________________________________
>>>>> >>> Powered by www.kitware.com
>>>>> >>>
>>>>> >>> Visit other Kitware open-source projects at
>>>>> >>> http://www.kitware.com/opensource/opensource.html
>>>>> >>>
>>>>> >>> Search the list archives at:
>>>>> http://markmail.org/search/?q=vtk-developers
>>>>> >>>
>>>>> >>> Follow this link to subscribe/unsubscribe:
>>>>> >>> http://public.kitware.com/mailman/listinfo/vtk-developers
>>>>> >>>
>>>>> >>> --
>>>>> >> Sankhesh Jhaveri *Sr. Research & Development Engineer* | Kitware
>>>>> >> <http://www.kitware.com/> | (518) 881-4417
>>>>> >> ​
>>>>> >> _______________________________________________
>>>>> >> Powered by www.kitware.com
>>>>> >>
>>>>> >> Visit other Kitware open-source projects at
>>>>> >> http://www.kitware.com/opensource/opensource.html
>>>>> >>
>>>>> >> Search the list archives at:
>>>>> http://markmail.org/search/?q=vtk-developers
>>>>> >>
>>>>> >> Follow this link to subscribe/unsubscribe:
>>>>> >> http://public.kitware.com/mailman/listinfo/vtk-developers
>>>>> >>
>>>>> >>
>>>>>
>>>>>
>>>>>
>>>>>
>>>
>>>
>>>
>>>
>
>
>
> _______________________________________________
> Powered by www.kitware.com
>
> Visit other Kitware open-source projects at  
> http://www.kitware.com/opensource/opensource.html
>
> Search the list archives at: http://markmail.org/search/?q=vtk-developers
>
> Follow this link to subscribe/unsubscribe:
> http://public.kitware.com/mailman/listinfo/vtk-developers



_______________________________________________
Powered by www.kitware.com

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Search the list archives at: http://markmail.org/search/?q=vtk-developers

Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/vtk-developers

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: VTK and Qt5 / QML

gpwright
Hi Chris,

For the interactor there is a way to basically queue the event information you need e.g. from qmls onPressed and then pass it to an interactor that belongs to the particular viewport that was pressed/clicked so that it will be handled once that thread is being rendered.

However, I found that I quickly wanted a more advanced interaction.  For example if your vtk qml item is inside a qml Flickable you can easily do very nice interactions for touch such as rotational momentum on flick that continues after release.

I am not sure of the latest touch interactor in vtk but when I was working on this a couple of years ago it was easier to handle all of the interactor part in QML (e.g. utilizing Flickable) and then pass info to the rendering thread that would directly mutate the camera, effectively making the vtkCamera a slave to some qml properties.

However if vtk has caught up with momentum based touch interactors it may be worthwhile to use these.  I am not caught up with what's been added to VTK in recent months.  Another advantage of keeping the camera master state in QML is that you can use Qt animation framework easily (e.g. they recently added an orientation animation that can be useful for controlling a camera).

I think it depends how complex are the user interactions that you are planning for.

G





On Wed, May 17, 2017, 10:15 AM <[hidden email]> wrote:
I have to correct myself with regard to the "direct rendering" option.
There is, of course, an example that demonstrates this
(http://doc.qt.io/qt-5/qtquick-scenegraph-openglunderqml-example.html). I
doubt whether this could be implemented in conjunction with
thread-per-item, so it only adds one architectural option. I still
think this is the least versatile and generic one, though.

chris

Zitat von [hidden email]:

> ok, things are clearing up a little more now. You ARE creating a
> separate thread for each QML item, which gives us these threads:
>
> - QML GUI thread (where the events come from)
> - QML rendering thread (aka scenegraph rendering thread aka
> QSGRenderThread), of which theres is only one
> - a dedicated thread for each item (RenderThread class in the example)
>
> the example also uses a QSGSimpleTextureNode subclass that does the
> blit'ing together. All of that decoupled via QueuedConnections and
> events signalled here and there. Not an easy beast, that.
>
> What remains unclear is how you deal with user input. For example,
> if the user were to drag something around in the scene (and your app
> were to support that), where do the MouseMove events go? This is
> really the main crux in this discussion IMO.
>
> To sum up the architectural choices:
>
> a) render into FBO
> a1)- do all rendering on the scenegraph thread (exemplified in
> http://doc.qt.io/qt-5/qtquick-scenegraph-textureinsgnode-example.html)
>
> a2) render on a dedicated thread for each item (exemplified in
> http://doc.qt.io/qt-5/qtquick-scenegraph-textureinthread-example.html)
>
> b) render into the QML context directly (not sure about this. Is
> there a Qt example demonstrating the practicability?)
> b?) undetermined
>
>
> The user intaction part is undetermined for me at this point. I am
> trying to get something going using QVTKInteractorAdapter, but
> havent had luck so far.
>
> In terms of ready usable code that demonstrates QML/VTK integration
> (including interaction and all) I see nothing yet on the horizon.
>
> chris
>
>
>
> Zitat von Geoff Wright <[hidden email]>:
>
>> Yes each vtk item has its own render thread that is why I would argue this
>> approach is superior to the others since it scales with the number of cores.
>>
>> I followed the approach described here:
>>
>> http://doc.qt.io/qt-5/qtquick-scenegraph-textureinthread-example.html
>>
>> So basically inside each vtk qml item there is a thread drawing a vtk
>> viewport into a texture, then the QSGRenderThread comes along and just does
>> a single texture blit for each item (very cheap). The Qt main thread takes
>> care of other event processing, models etc.
>>
>> This is sufficient if each item contains an independent scene (vtk
>> pipeline).  If you need to share state between two or more viewports  then
>> you need to be careful to control the timing of when any data objects may
>> mutate since the vtk api is not inherently threadsafe.
>>
>> Does that make sense?
>>
>> G
>>
>>
>>
>> On Tue, May 16, 2017, 6:13 PM <[hidden email]> wrote:
>>
>>> Hello all,
>>>
>>> @Geoff: are you saying that each pair of QML item / vtkRenderWindow
>>> has its own render thread? Is that something QML does?
>>>
>>> My goal is to have QML/VTK items behave as normal QML items, i.e., I
>>> could have a dozen items on my screen each rendering something
>>> different through VTK without influencing each other (apart from
>>> system performance). I expect that the approach I have taken so far
>>> will already fulfill this requirement.
>>>
>>> @Sankesh: I object to the compile time flag. IMO both approaches
>>> should be usable at the same time, e.g. by extending different
>>> superclasses. As you say, the "underlay or overlay" aspect breaks QML
>>> concepts and makes any item built according to this approach an
>>> "alien" in the QML crowd that always requires special attention.
>>>
>>> I'll look into QVTKInteractorAdapter - that sounds like seomething
>>> useful. Should have come across it earlier.
>>>
>>> @both: is there a chance that usable code will come out of this
>>> exchange any time soon? I for my part would at least publish my
>>> prototypical implementations somewhere (GitHub?), unless they become
>>> obsolete by something "official"
>>>
>>> regards,
>>> Chris
>>>
>>> Zitat von Geoff Wright <[hidden email]>:
>>>
>>>> Hi Chris,
>>>>
>>>> Sorry for the confusion, yes I am doing what you describe.  The
>>> difference
>>>> may be that perhaps your app has only one viewport.  In my case there
>>> are a
>>>> number of different viewports (aka vtkRenderWindow)  each has its own qml
>>>> item and is each rendering on it's own thread.  Plus the qml composition
>>>> thread which is managed by Qt.
>>>>
>>>> I was just making the point that this architecture let's you draw
>>> different
>>>> viewports in parallel provided that each one has its own mappers and
>>> actors.
>>>>
>>>> G
>>>>
>>>>
>>>> On Tue, May 16, 2017, 4:13 PM <[hidden email]> wrote:
>>>>
>>>>> Hi Geoff,
>>>>>
>>>>> as I understand it, the blog post you linked to doesn't mention
>>>>> multiple threads beyond the QML UI thread and the scenegraph render
>>>>> thread.
>>>>>
>>>>> So far, I am using that approach with QQuickFrameBufferObject as
>>>>> starting point and vtkExternalOpenGLRenderWindow for the VTK side,
>>>>> which causes VTK to use QML's OpenGL context and thus render into the
>>>>> FBO maintained by QQuickFrameBufferObject. That seems to work
>>>>> flawlessly, as long as ALL rendering is kept on the scenegraph thread,
>>>>> which is not always obvious to do at first, because VTK's API's don't
>>>>> separate between reading event values and rendering them (e.g.,
>>>>> SetSlice(2) will internally set the current slice and then do a
>>>>> Render()).
>>>>>
>>>>> But why maintain more threads?
>>>>>
>>>>> Anyway, please go ahead and explain your approach as detailed as
>>>>> possible, I am all ears.
>>>>>
>>>>> regards,
>>>>> Chris
>>>>>
>>>>> Zitat von Geoff Wright <[hidden email]>:
>>>>>
>>>>> > Hi guys,
>>>>> >
>>>>> > I would argue that for qml integration the best approach is for each
>>> vtk
>>>>> > viewport to render into a texture on a dedicated thread, not the qml
>>>>> > rendering thread.  Qml then draws the resulting texture which is more
>>> in
>>>>> > line with the Qt design intention ( see
>>>>> >
>>>>>
>>> http://blog.qt.io/blog/2015/05/11/integrating-custom-opengl-rendering-with-qt-quick-via-qquickframebufferobject/
>>>>> >  )
>>>>> >
>>>>> > I have been using this approach for a couple of years in a high
>>>>> performance
>>>>> > real time application.  The code is unfortunately proprietary but very
>>>>> > happy to explain more about the approach.  It would be great to add
>>>>> support
>>>>> > for this to VTK.
>>>>> >
>>>>> > For the interactor part I do what Sankesh said, defer the events and
>>>>> apply
>>>>> > them on the render thread of the particular vtk item.
>>>>> >
>>>>> > Regards,
>>>>> >
>>>>> > G
>>>>> >
>>>>> >
>>>>> >
>>>>> > On Tue, May 16, 2017, 3:06 PM Sankhesh Jhaveri <
>>>>> [hidden email]>
>>>>> > wrote:
>>>>> >
>>>>> >> Hi Christian,
>>>>> >>
>>>>> >> Doing all VTK rendering on the QML render thread is the right thing
>>> to
>>>>> do.
>>>>> >> As far as interaction is concerned, make sure that Qt events are
>>> queued
>>>>> on
>>>>> >> the main thread and processed when execution reaches the render
>>> thread.
>>>>> >>
>>>>> >> I have done some work on VTK and QtQuick integration that I’m
>>> planning
>>>>> to
>>>>> >> add to a VTK remote module. That way, it will be available as part of
>>>>> VTK.
>>>>> >>
>>>>> >> Thanks,
>>>>> >> Sankhesh
>>>>> >> ​
>>>>> >>
>>>>> >> On Tue, May 16, 2017 at 4:57 AM <[hidden email]> wrote:
>>>>> >>
>>>>> >>> just for the record (nobody care about this subject?):
>>>>> >>>
>>>>> >>> I found that contrary to what I said I had NOT adhered to the
>>>>> >>> principle established with protoype 1 while implementing protoype 2.
>>>>> >>> Rather, I had directly called into the VTK interactor from the QML
>>> UI
>>>>> >>> thread. And because the VTK interactors immediately go into
>>> rendering,
>>>>> >>> all hell broke loose.
>>>>> >>>
>>>>> >>> After cleaning up example 2 according to the rule laid out earlier
>>>>> >>> ("store event data in RenderDelegate, call update() on the QML item,
>>>>> >>> pass the event to VTK inside RenderDelgate.Render()", all works
>>> well.
>>>>> >>>
>>>>> >>> What I would like to do is validate the solution against findings
>>> made
>>>>> >>> elsewhere, and then establish the "canonical" way of integrating VTK
>>>>> >>> with QML. As I said, there are some architectural quirks with the
>>>>> >>> solution which I would also like to address.
>>>>> >>>
>>>>> >>> regards,
>>>>> >>> Christian
>>>>> >>>
>>>>> >>>
>>>>> >>> Zitat von [hidden email]:
>>>>> >>>
>>>>> >>> > Hello all,
>>>>> >>> >
>>>>> >>> > I am looking into the possibility of replacing the 3D rendering
>>>>> >>> > engine in my Qt5 / QML based mobile tablet-oriented application
>>> with
>>>>> >>> > VTK. What I need is a first class QtQuick integration (not a hack
>>> or
>>>>> >>> > workaround) that is 100% stable in every context (not an unusual
>>>>> >>> > requirement, I admit). To my amazement, nothing of that kind seems
>>>>> >>> > to exist (correct me if I'm wrong).
>>>>> >>> >
>>>>> >>> > I went on to investigate what had been done so far and to
>>> implement
>>>>> >>> > my first prototypes, using the QQuickFrameBufferObject approach.
>>>>> >>> > From the very start this felt like an uphill battle, because VTK
>>>>> >>> > seems to come from a Windowing background and is quite tightly
>>>>> >>> > integrated with concepts that are not valid in a QML context.
>>>>> >>> >
>>>>> >>> > I'll describe my findings together with the 2 prototypical QML
>>> items
>>>>> >>> > I implemented:
>>>>> >>> >
>>>>> >>> > 1st was an adaptation of the DICOM example which now runs as a QML
>>>>> >>> > item. All user interaction is handled by QML and forwarded to VTK
>>>>> >>> > (which is one thing that doesn't come naturally with VTK), and
>>> after
>>>>> >>> > some non-elegant tweaking I was able to have the UI move from
>>> slice
>>>>> >>> > to slice and re-render upon mouse wheel events as expected. The
>>>>> >>> > problem here was, that QML insists on keeping mouse event handling
>>>>> >>> > and OpenGL rendering on separate threads, with one "rendering
>>>>> >>> > thread" dedicated to OpenGL. However, the pre-existing VTK
>>>>> >>> > Interactors directly call Render() after reconfiguring the UI from
>>>>> >>> > the mouse event data, which is an absolute QML No-Go. I had to
>>>>> >>> > introduce a RenderDelegate that works somewhat like this:
>>>>> >>> >
>>>>> >>> > QML mouse event:
>>>>> >>> >     tell RenderDelegate about the event
>>>>> >>> >     call update() on the QML item, which triggers rer-endering on
>>>>> >>> > the dedicated thread
>>>>> >>> > on the QML rendering thread:
>>>>> >>> >     call Render() on the RenderWindow
>>>>> >>> >     inside the RenderDelegate, look at whether a mouse event is
>>>>> >>> > pending, and call the corresponding VTK mouse handler
>>>>> >>> >     call the renderer's default Render() function while setting
>>> the
>>>>> >>> > delegate's "Used" flag to false
>>>>> >>> >
>>>>> >>> > 2nd was a QML item that displays a model loaded from a 3ds file.
>>> My
>>>>> >>> > goal here was to move the model around using mouse drag events. I
>>>>> >>> > took the lessons learned from the first example, threw in a
>>>>> >>> > vtkInteractorStyleTrackballCamera and hoped for the best. First
>>>>> >>> > thing I found was that I needed a specialized
>>>>> >>> > RenderWindowInteractor, which I provided. When I realized that the
>>>>> >>> > most important requirement for that class was to provide access to
>>>>> >>> > timer functionality, I already got wary, as throwing around events
>>>>> >>> > and doing stuff offline while on the QML rendering thread is
>>> never a
>>>>> >>> > good idea. My fears came true when I finished wiring everything
>>>>> >>> > together: I was able to move the model using the mouse, but after
>>> a
>>>>> >>> > few moments things got whacky with error output written to the
>>>>> >>> > console and finally a segemtation fault from deep inside VTK.
>>>>> >>> >
>>>>> >>> > I still need to investigate the second example, but would like to
>>>>> >>> > synchronize with the community at this point in order to avoid
>>>>> >>> > errors/misconceptions on my side, to seek help if possible and to
>>>>> >>> > offer my contribution to the VTK code base once this becomes
>>>>> >>> > functional. My first impression is that there are some issues that
>>>>> >>> > lie on an architectural level and cannot be (elegantly) dealt with
>>>>> >>> > on the QML side alone. Any comments?
>>>>> >>> >
>>>>> >>> > thanks,
>>>>> >>> > Christian Sell
>>>>> >>> >
>>>>> >>> > _______________________________________________
>>>>> >>> > Powered by www.kitware.com
>>>>> >>> >
>>>>> >>> > Visit other Kitware open-source projects at
>>>>> >>> > http://www.kitware.com/opensource/opensource.html
>>>>> >>> >
>>>>> >>> > Search the list archives at:
>>>>> >>> http://markmail.org/search/?q=vtk-developers
>>>>> >>> >
>>>>> >>> > Follow this link to subscribe/unsubscribe:
>>>>> >>> > http://public.kitware.com/mailman/listinfo/vtk-developers
>>>>> >>>
>>>>> >>>
>>>>> >>>
>>>>> >>> _______________________________________________
>>>>> >>> Powered by www.kitware.com
>>>>> >>>
>>>>> >>> Visit other Kitware open-source projects at
>>>>> >>> http://www.kitware.com/opensource/opensource.html
>>>>> >>>
>>>>> >>> Search the list archives at:
>>>>> http://markmail.org/search/?q=vtk-developers
>>>>> >>>
>>>>> >>> Follow this link to subscribe/unsubscribe:
>>>>> >>> http://public.kitware.com/mailman/listinfo/vtk-developers
>>>>> >>>
>>>>> >>> --
>>>>> >> Sankhesh Jhaveri *Sr. Research & Development Engineer* | Kitware
>>>>> >> <http://www.kitware.com/> | (518) 881-4417
>>>>> >> ​
>>>>> >> _______________________________________________
>>>>> >> Powered by www.kitware.com
>>>>> >>
>>>>> >> Visit other Kitware open-source projects at
>>>>> >> http://www.kitware.com/opensource/opensource.html
>>>>> >>
>>>>> >> Search the list archives at:
>>>>> http://markmail.org/search/?q=vtk-developers
>>>>> >>
>>>>> >> Follow this link to subscribe/unsubscribe:
>>>>> >> http://public.kitware.com/mailman/listinfo/vtk-developers
>>>>> >>
>>>>> >>
>>>>>
>>>>>
>>>>>
>>>>>
>>>
>>>
>>>
>>>
>
>
>
> _______________________________________________
> Powered by www.kitware.com
>
> Visit other Kitware open-source projects at
> http://www.kitware.com/opensource/opensource.html
>
> Search the list archives at: http://markmail.org/search/?q=vtk-developers
>
> Follow this link to subscribe/unsubscribe:
> http://public.kitware.com/mailman/listinfo/vtk-developers



_______________________________________________
Powered by www.kitware.com

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Search the list archives at: http://markmail.org/search/?q=vtk-developers

Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/vtk-developers


_______________________________________________
Powered by www.kitware.com

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Search the list archives at: http://markmail.org/search/?q=vtk-developers

Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/vtk-developers

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: VTK and Qt5 / QML

c.sell
ok, thanks for the exchange, I think I have a better understanding  
now. I have managed to get the code running with QVTKInteractor and  
siblings - it was a matter of instantiating an (Q-)object at the wrong  
time and thus on the wrong thread => no rendering. I may try the  
multi-threaded approach as well, and will publish the sample code on  
github once I am happy with it.

chris

Zitat von Geoff Wright <[hidden email]>:

> Hi Chris,
>
> For the interactor there is a way to basically queue the event information
> you need e.g. from qmls onPressed and then pass it to an interactor that
> belongs to the particular viewport that was pressed/clicked so that it will
> be handled once that thread is being rendered.
>
> However, I found that I quickly wanted a more advanced interaction.  For
> example if your vtk qml item is inside a qml Flickable you can easily do
> very nice interactions for touch such as rotational momentum on flick that
> continues after release.
>
> I am not sure of the latest touch interactor in vtk but when I was working
> on this a couple of years ago it was easier to handle all of the interactor
> part in QML (e.g. utilizing Flickable) and then pass info to the rendering
> thread that would directly mutate the camera, effectively making the
> vtkCamera a slave to some qml properties.
>
> However if vtk has caught up with momentum based touch interactors it may
> be worthwhile to use these.  I am not caught up with what's been added to
> VTK in recent months.  Another advantage of keeping the camera master state
> in QML is that you can use Qt animation framework easily (e.g. they
> recently added an orientation animation that can be useful for controlling
> a camera).
>
> I think it depends how complex are the user interactions that you are
> planning for.
>
> G
>
>
>
>
>
> On Wed, May 17, 2017, 10:15 AM <[hidden email]> wrote:
>
>> I have to correct myself with regard to the "direct rendering" option.
>> There is, of course, an example that demonstrates this
>> (http://doc.qt.io/qt-5/qtquick-scenegraph-openglunderqml-example.html). I
>> doubt whether this could be implemented in conjunction with
>> thread-per-item, so it only adds one architectural option. I still
>> think this is the least versatile and generic one, though.
>>
>> chris
>>
>> Zitat von [hidden email]:
>>
>> > ok, things are clearing up a little more now. You ARE creating a
>> > separate thread for each QML item, which gives us these threads:
>> >
>> > - QML GUI thread (where the events come from)
>> > - QML rendering thread (aka scenegraph rendering thread aka
>> > QSGRenderThread), of which theres is only one
>> > - a dedicated thread for each item (RenderThread class in the example)
>> >
>> > the example also uses a QSGSimpleTextureNode subclass that does the
>> > blit'ing together. All of that decoupled via QueuedConnections and
>> > events signalled here and there. Not an easy beast, that.
>> >
>> > What remains unclear is how you deal with user input. For example,
>> > if the user were to drag something around in the scene (and your app
>> > were to support that), where do the MouseMove events go? This is
>> > really the main crux in this discussion IMO.
>> >
>> > To sum up the architectural choices:
>> >
>> > a) render into FBO
>> > a1)- do all rendering on the scenegraph thread (exemplified in
>> > http://doc.qt.io/qt-5/qtquick-scenegraph-textureinsgnode-example.html)
>> >
>> > a2) render on a dedicated thread for each item (exemplified in
>> > http://doc.qt.io/qt-5/qtquick-scenegraph-textureinthread-example.html)
>> >
>> > b) render into the QML context directly (not sure about this. Is
>> > there a Qt example demonstrating the practicability?)
>> > b?) undetermined
>> >
>> >
>> > The user intaction part is undetermined for me at this point. I am
>> > trying to get something going using QVTKInteractorAdapter, but
>> > havent had luck so far.
>> >
>> > In terms of ready usable code that demonstrates QML/VTK integration
>> > (including interaction and all) I see nothing yet on the horizon.
>> >
>> > chris
>> >
>> >
>> >
>> > Zitat von Geoff Wright <[hidden email]>:
>> >
>> >> Yes each vtk item has its own render thread that is why I would argue
>> this
>> >> approach is superior to the others since it scales with the number of
>> cores.
>> >>
>> >> I followed the approach described here:
>> >>
>> >> http://doc.qt.io/qt-5/qtquick-scenegraph-textureinthread-example.html
>> >>
>> >> So basically inside each vtk qml item there is a thread drawing a vtk
>> >> viewport into a texture, then the QSGRenderThread comes along and just
>> does
>> >> a single texture blit for each item (very cheap). The Qt main thread
>> takes
>> >> care of other event processing, models etc.
>> >>
>> >> This is sufficient if each item contains an independent scene (vtk
>> >> pipeline).  If you need to share state between two or more viewports
>> then
>> >> you need to be careful to control the timing of when any data objects
>> may
>> >> mutate since the vtk api is not inherently threadsafe.
>> >>
>> >> Does that make sense?
>> >>
>> >> G
>> >>
>> >>
>> >>
>> >> On Tue, May 16, 2017, 6:13 PM <[hidden email]> wrote:
>> >>
>> >>> Hello all,
>> >>>
>> >>> @Geoff: are you saying that each pair of QML item / vtkRenderWindow
>> >>> has its own render thread? Is that something QML does?
>> >>>
>> >>> My goal is to have QML/VTK items behave as normal QML items, i.e., I
>> >>> could have a dozen items on my screen each rendering something
>> >>> different through VTK without influencing each other (apart from
>> >>> system performance). I expect that the approach I have taken so far
>> >>> will already fulfill this requirement.
>> >>>
>> >>> @Sankesh: I object to the compile time flag. IMO both approaches
>> >>> should be usable at the same time, e.g. by extending different
>> >>> superclasses. As you say, the "underlay or overlay" aspect breaks QML
>> >>> concepts and makes any item built according to this approach an
>> >>> "alien" in the QML crowd that always requires special attention.
>> >>>
>> >>> I'll look into QVTKInteractorAdapter - that sounds like seomething
>> >>> useful. Should have come across it earlier.
>> >>>
>> >>> @both: is there a chance that usable code will come out of this
>> >>> exchange any time soon? I for my part would at least publish my
>> >>> prototypical implementations somewhere (GitHub?), unless they become
>> >>> obsolete by something "official"
>> >>>
>> >>> regards,
>> >>> Chris
>> >>>
>> >>> Zitat von Geoff Wright <[hidden email]>:
>> >>>
>> >>>> Hi Chris,
>> >>>>
>> >>>> Sorry for the confusion, yes I am doing what you describe.  The
>> >>> difference
>> >>>> may be that perhaps your app has only one viewport.  In my case there
>> >>> are a
>> >>>> number of different viewports (aka vtkRenderWindow)  each has its own
>> qml
>> >>>> item and is each rendering on it's own thread.  Plus the qml
>> composition
>> >>>> thread which is managed by Qt.
>> >>>>
>> >>>> I was just making the point that this architecture let's you draw
>> >>> different
>> >>>> viewports in parallel provided that each one has its own mappers and
>> >>> actors.
>> >>>>
>> >>>> G
>> >>>>
>> >>>>
>> >>>> On Tue, May 16, 2017, 4:13 PM <[hidden email]> wrote:
>> >>>>
>> >>>>> Hi Geoff,
>> >>>>>
>> >>>>> as I understand it, the blog post you linked to doesn't mention
>> >>>>> multiple threads beyond the QML UI thread and the scenegraph render
>> >>>>> thread.
>> >>>>>
>> >>>>> So far, I am using that approach with QQuickFrameBufferObject as
>> >>>>> starting point and vtkExternalOpenGLRenderWindow for the VTK side,
>> >>>>> which causes VTK to use QML's OpenGL context and thus render into the
>> >>>>> FBO maintained by QQuickFrameBufferObject. That seems to work
>> >>>>> flawlessly, as long as ALL rendering is kept on the scenegraph
>> thread,
>> >>>>> which is not always obvious to do at first, because VTK's API's don't
>> >>>>> separate between reading event values and rendering them (e.g.,
>> >>>>> SetSlice(2) will internally set the current slice and then do a
>> >>>>> Render()).
>> >>>>>
>> >>>>> But why maintain more threads?
>> >>>>>
>> >>>>> Anyway, please go ahead and explain your approach as detailed as
>> >>>>> possible, I am all ears.
>> >>>>>
>> >>>>> regards,
>> >>>>> Chris
>> >>>>>
>> >>>>> Zitat von Geoff Wright <[hidden email]>:
>> >>>>>
>> >>>>> > Hi guys,
>> >>>>> >
>> >>>>> > I would argue that for qml integration the best approach is for
>> each
>> >>> vtk
>> >>>>> > viewport to render into a texture on a dedicated thread, not the
>> qml
>> >>>>> > rendering thread.  Qml then draws the resulting texture which is
>> more
>> >>> in
>> >>>>> > line with the Qt design intention ( see
>> >>>>> >
>> >>>>>
>> >>>
>> http://blog.qt.io/blog/2015/05/11/integrating-custom-opengl-rendering-with-qt-quick-via-qquickframebufferobject/
>> >>>>> >  )
>> >>>>> >
>> >>>>> > I have been using this approach for a couple of years in a high
>> >>>>> performance
>> >>>>> > real time application.  The code is unfortunately proprietary but
>> very
>> >>>>> > happy to explain more about the approach.  It would be great to add
>> >>>>> support
>> >>>>> > for this to VTK.
>> >>>>> >
>> >>>>> > For the interactor part I do what Sankesh said, defer the events
>> and
>> >>>>> apply
>> >>>>> > them on the render thread of the particular vtk item.
>> >>>>> >
>> >>>>> > Regards,
>> >>>>> >
>> >>>>> > G
>> >>>>> >
>> >>>>> >
>> >>>>> >
>> >>>>> > On Tue, May 16, 2017, 3:06 PM Sankhesh Jhaveri <
>> >>>>> [hidden email]>
>> >>>>> > wrote:
>> >>>>> >
>> >>>>> >> Hi Christian,
>> >>>>> >>
>> >>>>> >> Doing all VTK rendering on the QML render thread is the right
>> thing
>> >>> to
>> >>>>> do.
>> >>>>> >> As far as interaction is concerned, make sure that Qt events are
>> >>> queued
>> >>>>> on
>> >>>>> >> the main thread and processed when execution reaches the render
>> >>> thread.
>> >>>>> >>
>> >>>>> >> I have done some work on VTK and QtQuick integration that I’m
>> >>> planning
>> >>>>> to
>> >>>>> >> add to a VTK remote module. That way, it will be available as
>> part of
>> >>>>> VTK.
>> >>>>> >>
>> >>>>> >> Thanks,
>> >>>>> >> Sankhesh
>> >>>>> >> ​
>> >>>>> >>
>> >>>>> >> On Tue, May 16, 2017 at 4:57 AM <[hidden email]> wrote:
>> >>>>> >>
>> >>>>> >>> just for the record (nobody care about this subject?):
>> >>>>> >>>
>> >>>>> >>> I found that contrary to what I said I had NOT adhered to the
>> >>>>> >>> principle established with protoype 1 while implementing
>> protoype 2.
>> >>>>> >>> Rather, I had directly called into the VTK interactor from the
>> QML
>> >>> UI
>> >>>>> >>> thread. And because the VTK interactors immediately go into
>> >>> rendering,
>> >>>>> >>> all hell broke loose.
>> >>>>> >>>
>> >>>>> >>> After cleaning up example 2 according to the rule laid out
>> earlier
>> >>>>> >>> ("store event data in RenderDelegate, call update() on the QML
>> item,
>> >>>>> >>> pass the event to VTK inside RenderDelgate.Render()", all works
>> >>> well.
>> >>>>> >>>
>> >>>>> >>> What I would like to do is validate the solution against findings
>> >>> made
>> >>>>> >>> elsewhere, and then establish the "canonical" way of integrating
>> VTK
>> >>>>> >>> with QML. As I said, there are some architectural quirks with the
>> >>>>> >>> solution which I would also like to address.
>> >>>>> >>>
>> >>>>> >>> regards,
>> >>>>> >>> Christian
>> >>>>> >>>
>> >>>>> >>>
>> >>>>> >>> Zitat von [hidden email]:
>> >>>>> >>>
>> >>>>> >>> > Hello all,
>> >>>>> >>> >
>> >>>>> >>> > I am looking into the possibility of replacing the 3D rendering
>> >>>>> >>> > engine in my Qt5 / QML based mobile tablet-oriented application
>> >>> with
>> >>>>> >>> > VTK. What I need is a first class QtQuick integration (not a
>> hack
>> >>> or
>> >>>>> >>> > workaround) that is 100% stable in every context (not an
>> unusual
>> >>>>> >>> > requirement, I admit). To my amazement, nothing of that kind
>> seems
>> >>>>> >>> > to exist (correct me if I'm wrong).
>> >>>>> >>> >
>> >>>>> >>> > I went on to investigate what had been done so far and to
>> >>> implement
>> >>>>> >>> > my first prototypes, using the QQuickFrameBufferObject
>> approach.
>> >>>>> >>> > From the very start this felt like an uphill battle, because
>> VTK
>> >>>>> >>> > seems to come from a Windowing background and is quite tightly
>> >>>>> >>> > integrated with concepts that are not valid in a QML context.
>> >>>>> >>> >
>> >>>>> >>> > I'll describe my findings together with the 2 prototypical QML
>> >>> items
>> >>>>> >>> > I implemented:
>> >>>>> >>> >
>> >>>>> >>> > 1st was an adaptation of the DICOM example which now runs as a
>> QML
>> >>>>> >>> > item. All user interaction is handled by QML and forwarded to
>> VTK
>> >>>>> >>> > (which is one thing that doesn't come naturally with VTK), and
>> >>> after
>> >>>>> >>> > some non-elegant tweaking I was able to have the UI move from
>> >>> slice
>> >>>>> >>> > to slice and re-render upon mouse wheel events as expected. The
>> >>>>> >>> > problem here was, that QML insists on keeping mouse event
>> handling
>> >>>>> >>> > and OpenGL rendering on separate threads, with one "rendering
>> >>>>> >>> > thread" dedicated to OpenGL. However, the pre-existing VTK
>> >>>>> >>> > Interactors directly call Render() after reconfiguring the UI
>> from
>> >>>>> >>> > the mouse event data, which is an absolute QML No-Go. I had to
>> >>>>> >>> > introduce a RenderDelegate that works somewhat like this:
>> >>>>> >>> >
>> >>>>> >>> > QML mouse event:
>> >>>>> >>> >     tell RenderDelegate about the event
>> >>>>> >>> >     call update() on the QML item, which triggers rer-endering
>> on
>> >>>>> >>> > the dedicated thread
>> >>>>> >>> > on the QML rendering thread:
>> >>>>> >>> >     call Render() on the RenderWindow
>> >>>>> >>> >     inside the RenderDelegate, look at whether a mouse event is
>> >>>>> >>> > pending, and call the corresponding VTK mouse handler
>> >>>>> >>> >     call the renderer's default Render() function while setting
>> >>> the
>> >>>>> >>> > delegate's "Used" flag to false
>> >>>>> >>> >
>> >>>>> >>> > 2nd was a QML item that displays a model loaded from a 3ds
>> file.
>> >>> My
>> >>>>> >>> > goal here was to move the model around using mouse drag
>> events. I
>> >>>>> >>> > took the lessons learned from the first example, threw in a
>> >>>>> >>> > vtkInteractorStyleTrackballCamera and hoped for the best. First
>> >>>>> >>> > thing I found was that I needed a specialized
>> >>>>> >>> > RenderWindowInteractor, which I provided. When I realized that
>> the
>> >>>>> >>> > most important requirement for that class was to provide
>> access to
>> >>>>> >>> > timer functionality, I already got wary, as throwing around
>> events
>> >>>>> >>> > and doing stuff offline while on the QML rendering thread is
>> >>> never a
>> >>>>> >>> > good idea. My fears came true when I finished wiring everything
>> >>>>> >>> > together: I was able to move the model using the mouse, but
>> after
>> >>> a
>> >>>>> >>> > few moments things got whacky with error output written to the
>> >>>>> >>> > console and finally a segemtation fault from deep inside VTK.
>> >>>>> >>> >
>> >>>>> >>> > I still need to investigate the second example, but would like
>> to
>> >>>>> >>> > synchronize with the community at this point in order to avoid
>> >>>>> >>> > errors/misconceptions on my side, to seek help if possible and
>> to
>> >>>>> >>> > offer my contribution to the VTK code base once this becomes
>> >>>>> >>> > functional. My first impression is that there are some issues
>> that
>> >>>>> >>> > lie on an architectural level and cannot be (elegantly) dealt
>> with
>> >>>>> >>> > on the QML side alone. Any comments?
>> >>>>> >>> >
>> >>>>> >>> > thanks,
>> >>>>> >>> > Christian Sell
>> >>>>> >>> >
>> >>>>> >>> > _______________________________________________
>> >>>>> >>> > Powered by www.kitware.com
>> >>>>> >>> >
>> >>>>> >>> > Visit other Kitware open-source projects at
>> >>>>> >>> > http://www.kitware.com/opensource/opensource.html
>> >>>>> >>> >
>> >>>>> >>> > Search the list archives at:
>> >>>>> >>> http://markmail.org/search/?q=vtk-developers
>> >>>>> >>> >
>> >>>>> >>> > Follow this link to subscribe/unsubscribe:
>> >>>>> >>> > http://public.kitware.com/mailman/listinfo/vtk-developers
>> >>>>> >>>
>> >>>>> >>>
>> >>>>> >>>
>> >>>>> >>> _______________________________________________
>> >>>>> >>> Powered by www.kitware.com
>> >>>>> >>>
>> >>>>> >>> Visit other Kitware open-source projects at
>> >>>>> >>> http://www.kitware.com/opensource/opensource.html
>> >>>>> >>>
>> >>>>> >>> Search the list archives at:
>> >>>>> http://markmail.org/search/?q=vtk-developers
>> >>>>> >>>
>> >>>>> >>> Follow this link to subscribe/unsubscribe:
>> >>>>> >>> http://public.kitware.com/mailman/listinfo/vtk-developers
>> >>>>> >>>
>> >>>>> >>> --
>> >>>>> >> Sankhesh Jhaveri *Sr. Research & Development Engineer* | Kitware
>> >>>>> >> <http://www.kitware.com/> | (518) 881-4417
>> >>>>> >> ​
>> >>>>> >> _______________________________________________
>> >>>>> >> Powered by www.kitware.com
>> >>>>> >>
>> >>>>> >> Visit other Kitware open-source projects at
>> >>>>> >> http://www.kitware.com/opensource/opensource.html
>> >>>>> >>
>> >>>>> >> Search the list archives at:
>> >>>>> http://markmail.org/search/?q=vtk-developers
>> >>>>> >>
>> >>>>> >> Follow this link to subscribe/unsubscribe:
>> >>>>> >> http://public.kitware.com/mailman/listinfo/vtk-developers
>> >>>>> >>
>> >>>>> >>
>> >>>>>
>> >>>>>
>> >>>>>
>> >>>>>
>> >>>
>> >>>
>> >>>
>> >>>
>> >
>> >
>> >
>> > _______________________________________________
>> > Powered by www.kitware.com
>> >
>> > Visit other Kitware open-source projects at
>> > http://www.kitware.com/opensource/opensource.html
>> >
>> > Search the list archives at:
>> http://markmail.org/search/?q=vtk-developers
>> >
>> > Follow this link to subscribe/unsubscribe:
>> > http://public.kitware.com/mailman/listinfo/vtk-developers
>>
>>
>>
>> _______________________________________________
>> Powered by www.kitware.com
>>
>> Visit other Kitware open-source projects at
>> http://www.kitware.com/opensource/opensource.html
>>
>> Search the list archives at: http://markmail.org/search/?q=vtk-developers
>>
>> Follow this link to subscribe/unsubscribe:
>> http://public.kitware.com/mailman/listinfo/vtk-developers
>>
>>



_______________________________________________
Powered by www.kitware.com

Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html

Search the list archives at: http://markmail.org/search/?q=vtk-developers

Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/vtk-developers

Loading...