python install prefix in setup.py

classic Classic list List threaded Threaded
22 messages Options
12
Reply | Threaded
Open this post in threaded view
|

python install prefix in setup.py

David Gobbi-2
The default install location for the VTK python wrappers in VTKcvs
doesn't make sense to me.
If the VTK install prefix is /usr/local, then the python wrappers will
be installed here:

  /usr/local/python2.3/site-packages/

If python is installed somewhere else, (e.g. /usr/ is a common location)
then installing
the wrappers in /usr/local isn't very useful.

The default behaviour of setup.py is to query the python executable to
find out where it is
installed.  This behaviour is explicitly overridden by
VTK/Wrapping/Python/CMakeList.txt:

  SET(VTK_PYTHON_SETUP_ARGS "--prefix=\"${DOLLAR}{CMAKE_INSTALL_PREFIX}\""
      CACHE STRING "Arguments passed to \"python setup.py install ...\"
during installation.")

I think that having a customizable VTK_PYTHON_SETUP_ARGS is a good idea,
but the default
value of this variable is not going to work for most people.

The best option is to query the PYTHON_EXECUTABLE to see where it was
installed, and use that
as the wrapper install prefix instead of using CMAKE_INSTALL_PREFIX.

The python install prefix can be queried by running 'python -c "import
sys; print sys.prefix"',
where 'python' is whatever PYTHON_EXECUTABLE has been set by cmake.

Does this make sense to anyone else?

 - David
_______________________________________________
vtk-developers mailing list
[hidden email]
http://www.vtk.org/mailman/listinfo/vtk-developers
Reply | Threaded
Open this post in threaded view
|

Re: python install prefix in setup.py

Mathieu Malaterre-2
The whole python support IMHO should be rewritten as mentioned here:

http://vtk.org/Bug/bug.php?op=show&bugid=2257

As for the installation thing, it would be nice if the CMake Modules had
an interface (?mymodule_INSTALL_PREFIX ?) to specify default
installation path. I am thinking that this should be the same issue for
Tcl, Python, Java.

2 cents,
Mathieu

David Gobbi wrote:

> The default install location for the VTK python wrappers in VTKcvs
> doesn't make sense to me.
> If the VTK install prefix is /usr/local, then the python wrappers will
> be installed here:
>
>  /usr/local/python2.3/site-packages/
>
> If python is installed somewhere else, (e.g. /usr/ is a common location)
> then installing
> the wrappers in /usr/local isn't very useful.
>
> The default behaviour of setup.py is to query the python executable to
> find out where it is
> installed.  This behaviour is explicitly overridden by
> VTK/Wrapping/Python/CMakeList.txt:
>
>  SET(VTK_PYTHON_SETUP_ARGS "--prefix=\"${DOLLAR}{CMAKE_INSTALL_PREFIX}\""
>      CACHE STRING "Arguments passed to \"python setup.py install ...\"
> during installation.")
>
> I think that having a customizable VTK_PYTHON_SETUP_ARGS is a good idea,
> but the default
> value of this variable is not going to work for most people.
>
> The best option is to query the PYTHON_EXECUTABLE to see where it was
> installed, and use that
> as the wrapper install prefix instead of using CMAKE_INSTALL_PREFIX.
>
> The python install prefix can be queried by running 'python -c "import
> sys; print sys.prefix"',
> where 'python' is whatever PYTHON_EXECUTABLE has been set by cmake.
>
> Does this make sense to anyone else?
>
> - David
> _______________________________________________
> vtk-developers mailing list
> [hidden email]
> http://www.vtk.org/mailman/listinfo/vtk-developers
>

_______________________________________________
vtk-developers mailing list
[hidden email]
http://www.vtk.org/mailman/listinfo/vtk-developers
Reply | Threaded
Open this post in threaded view
|

Re: python install prefix in setup.py

David Gobbi-2
Thanks for the link, I'll take a lood and add the comments from my email
to the bug report.

Mathieu Malaterre wrote:

> The whole python support IMHO should be rewritten as mentioned here:
>
> http://vtk.org/Bug/bug.php?op=show&bugid=2257
>
> As for the installation thing, it would be nice if the CMake Modules
> had an interface (?mymodule_INSTALL_PREFIX ?) to specify default
> installation path. I am thinking that this should be the same issue
> for Tcl, Python, Java.
>
> 2 cents,
> Mathieu
>
> David Gobbi wrote:
>
>> The default install location for the VTK python wrappers in VTKcvs
>> doesn't make sense to me.
>> If the VTK install prefix is /usr/local, then the python wrappers
>> will be installed here:
>>
>>  /usr/local/python2.3/site-packages/
>>
>> If python is installed somewhere else, (e.g. /usr/ is a common
>> location) then installing
>> the wrappers in /usr/local isn't very useful.
>>
>> The default behaviour of setup.py is to query the python executable
>> to find out where it is
>> installed.  This behaviour is explicitly overridden by
>> VTK/Wrapping/Python/CMakeList.txt:
>>
>>  SET(VTK_PYTHON_SETUP_ARGS
>> "--prefix=\"${DOLLAR}{CMAKE_INSTALL_PREFIX}\""
>>      CACHE STRING "Arguments passed to \"python setup.py install
>> ...\" during installation.")
>>
>> I think that having a customizable VTK_PYTHON_SETUP_ARGS is a good
>> idea, but the default
>> value of this variable is not going to work for most people.
>>
>> The best option is to query the PYTHON_EXECUTABLE to see where it was
>> installed, and use that
>> as the wrapper install prefix instead of using CMAKE_INSTALL_PREFIX.
>>
>> The python install prefix can be queried by running 'python -c
>> "import sys; print sys.prefix"',
>> where 'python' is whatever PYTHON_EXECUTABLE has been set by cmake.
>>
>> Does this make sense to anyone else?
>>
>> - David
>> _______________________________________________
>> vtk-developers mailing list
>> [hidden email]
>> http://www.vtk.org/mailman/listinfo/vtk-developers
>>
>
>

_______________________________________________
vtk-developers mailing list
[hidden email]
http://www.vtk.org/mailman/listinfo/vtk-developers
Reply | Threaded
Open this post in threaded view
|

Re: python install prefix in setup.py

Steve M. Robbins
In reply to this post by David Gobbi-2
Hi,

I realize that discussion has moved to a bug report.  However; one
cannot add a comment without "logging in" and I'm too lazy, so here is
my comment.

On Wed, Feb 01, 2006 at 11:45:46AM -0500, David Gobbi wrote:
> The default install location for the VTK python wrappers in VTKcvs
> doesn't make sense to me.
> If the VTK install prefix is /usr/local, then the python wrappers will
> be installed here:
>
>  /usr/local/python2.3/site-packages/
>
> If python is installed somewhere else, (e.g. /usr/ is a common location)
> then installing the wrappers in /usr/local isn't very useful.

On the other hand: a system (Debian, for example) may come with python
installed in /usr but use /usr/local/lib/python2.3/site-packages as
the preferred location to install non-vendor python packages.

I think this location is the preferred default.  Is there a way to
query python for this?

-Steve

_______________________________________________
vtk-developers mailing list
[hidden email]
http://www.vtk.org/mailman/listinfo/vtk-developers
Reply | Threaded
Open this post in threaded view
|

Re: python install prefix in setup.py

David Gobbi-2
Hi Steve, I though you were a Java guy ;)

You're right, using the python "prefix" might be too simplistic.  I know
that
redhat/suse/mandrake use /usr/lib/python2.3/site-packages and I assumed that
other distros would do the same.

There is probably a way to query the distutils module for the right path.

 - David

Steve M. Robbins wrote:

>Hi,
>
>I realize that discussion has moved to a bug report.  However; one
>cannot add a comment without "logging in" and I'm too lazy, so here is
>my comment.
>
>On Wed, Feb 01, 2006 at 11:45:46AM -0500, David Gobbi wrote:
>  
>
>>The default install location for the VTK python wrappers in VTKcvs
>>doesn't make sense to me.
>>If the VTK install prefix is /usr/local, then the python wrappers will
>>be installed here:
>>
>> /usr/local/python2.3/site-packages/
>>
>>If python is installed somewhere else, (e.g. /usr/ is a common location)
>>then installing the wrappers in /usr/local isn't very useful.
>>    
>>
>
>On the other hand: a system (Debian, for example) may come with python
>installed in /usr but use /usr/local/lib/python2.3/site-packages as
>the preferred location to install non-vendor python packages.
>
>I think this location is the preferred default.  Is there a way to
>query python for this?
>
>-Steve
>
>_______________________________________________
>vtk-developers mailing list
>[hidden email]
>http://www.vtk.org/mailman/listinfo/vtk-developers
>
>  
>

_______________________________________________
vtk-developers mailing list
[hidden email]
http://www.vtk.org/mailman/listinfo/vtk-developers
Reply | Threaded
Open this post in threaded view
|

Re: python install prefix in setup.py

Steve M. Robbins
On Wed, Feb 01, 2006 at 01:15:30PM -0500, David Gobbi wrote:
> Hi Steve, I though you were a Java guy ;)

Sure but I'm multi-lingual ;-)

 
> You're right, using the python "prefix" might be too simplistic.  I know
> that
> redhat/suse/mandrake use /usr/lib/python2.3/site-packages and I assumed that
> other distros would do the same.

Debian's idiom is that /usr/lib/python2.3/site-packages is for
vendor-packaged python extensions, whereas
/usr/local/lib/python2.3/site-packages is for the the sysadmin to
install python packages.  I thought that was standard, but I may be
mistaken -- I don't use python all that often.

> There is probably a way to query the distutils module for the right path.

Ah, yeah.  That sounds promising.

-Steve


_______________________________________________
vtk-developers mailing list
[hidden email]
http://www.vtk.org/mailman/listinfo/vtk-developers
Reply | Threaded
Open this post in threaded view
|

Re: python install prefix in setup.py

Prabhu Ramachandran
>>>>> "Steve" == Steve M Robbins <[hidden email]> writes:

    Steve> Debian's idiom is that /usr/lib/python2.3/site-packages is
    Steve> for vendor-packaged python extensions, whereas
    Steve> /usr/local/lib/python2.3/site-packages is for the the
    Steve> sysadmin to install python packages.  I thought that was
    Steve> standard, but I may be mistaken -- I don't use python all
    Steve> that often.

Right, /usr/local/lib/python2.x/site-packages under Debian is the
canonical place for non-deb Python packages.  While it is extremely
convenient (especially when used with GNU/Stow), it does not seem to
have caught on with other (backward? ;-) distros.  Debian basically
adds the local prefix to site.py.  The default prefixes are:

prefixes = [sys.prefix]

Debian changes it to:

prefixes = [os.path.join(sys.prefix, "local"), sys.prefix]

I am not sure that David's approach is *the* optimal one.  Many VTK
users are unlikely to have root access therefore installing to /usr/
is entirely out of question.  In these cases it is best to do what is
currently done -- install to CMAKE_PREFIX.  Basically the install
should work for most common situations.

cheers,
prabhu

_______________________________________________
vtk-developers mailing list
[hidden email]
http://www.vtk.org/mailman/listinfo/vtk-developers
Reply | Threaded
Open this post in threaded view
|

Re: python install prefix in setup.py

David Gobbi-2
Prabhu Ramachandran wrote:

>Right, /usr/local/lib/python2.x/site-packages under Debian is the
>canonical place for non-deb Python packages.  While it is extremely
>convenient (especially when used with GNU/Stow), it does not seem to
>have caught on with other (backward? ;-) distros.  Debian basically
>adds the local prefix to site.py.  The default prefixes are:
>
>prefixes = [sys.prefix]
>
>Debian changes it to:
>
>prefixes = [os.path.join(sys.prefix, "local"), sys.prefix]
>  
>
Does debian have any way of telling distutils of the new location?  If
we are
using setup.py to install the python modules, then it seems that our default
location should be the same as the one used by setup.py (otherwise, why
would we use setup.py at all wh

>I am not sure that David's approach is *the* optimal one.  Many VTK
>users are unlikely to have root access therefore installing to /usr/
>is entirely out of question.  In these cases it is best to do what is
>currently done -- install to CMAKE_PREFIX.  Basically the install
>should work for most common situations.
>  
>
I disagree that we should install to
  ${CMAKE_PREFIX}/lib/pythonx.x/site-packages,
as is currently done.  It might make sense to install in
 ${CMAKE_PREFIX}/lib/vtkx.x/python

The "make install" should work out-of-the-box for as many
python users as possible.   Right now, it only works "out-of-the-box"
for Redhat users that are installing VTK in /sys or for Debian users
who are installing VTK in /usr/local.  Everyone else needs to set
the PYTHONPATH befor it will work.

We can do much better than that!

I think we need two different Python install options in CMake:
- one that puts the modules in  ${CMAKE_PREFIX}/lib/vtkx.x/python, for
non-administrators
- one that puts it wherever the distutils default is, for sysadmins

 - David

_______________________________________________
vtk-developers mailing list
[hidden email]
http://www.vtk.org/mailman/listinfo/vtk-developers
Reply | Threaded
Open this post in threaded view
|

Re: python install prefix in setup.py

Prabhu Ramachandran
>>>>> "David" == David Gobbi <[hidden email]> writes:

    David> Prabhu Ramachandran wrote:
    >> distros.  Debian basically adds the local prefix to site.py.
[...]
    David> Does debian have any way of telling distutils of the new
    David> location?  If we are using setup.py to install the python
    David> modules, then it seems that our default location should be
    David> the same as the one used by setup.py (otherwise, why would
    David> we use setup.py at all wh

Not that I know of but this could be used:

 import site
 print site.prefixes

 import sys
 print sys.path

    >> I am not sure that David's approach is *the* optimal one.  Many
    >> VTK users are unlikely to have root access therefore installing
    >> to /usr/ is entirely out of question.  In these cases it is
    >> best to do what is currently done -- install to CMAKE_PREFIX.
    >> Basically the install should work for most common situations.
    >>
    >>
    David> I disagree that we should install to
    David> ${CMAKE_PREFIX}/lib/pythonx.x/site-packages, as is
    David> currently done.  It might make sense to install in
    David> ${CMAKE_PREFIX}/lib/vtkx.x/python

${CMAKE_PREFIX}/lib/vtkx.y/python is completely non-standard.  No
Python module ever installs importable Python modules in
lib/libname/python.  Any normal distutils based Python package is
installed like so:

 python setup.py install --prefix=$pth

and everything is installed in

 $pth/lib/site-packages/pythonx.y/package
 $pth/bin/script.py

The alternative is

 python setup.py install --home=$pth

In which case the files are installed in

 $pth/lib/python/package
 $pth/bin

So, that is the default behavior of distutils.  What are you
suggesting?

    David> The "make install" should work out-of-the-box for as many
    David> python users as possible.  Right now, it only works
    David> "out-of-the-box" for Redhat users that are installing VTK
    David> in /sys or for Debian users who are installing VTK in
    David> /usr/local.  Everyone else needs to set the PYTHONPATH
    David> befor it will work.

Everyone else needs to set PYTHONPATH anyway to get any possible
option working unless they are the super user and can install into
sys.prefix or /usr/local (on more fortunate distros).

    David> We can do much better than that!

Yes but I'm not quite clear what you want to do and how you want to do
it.

cheers,
prabhu
_______________________________________________
vtk-developers mailing list
[hidden email]
http://www.vtk.org/mailman/listinfo/vtk-developers
Reply | Threaded
Open this post in threaded view
|

Re: python install prefix in setup.py

David Gobbi-2
Prabhu Ramachandran wrote:

>"David" == David Gobbi <[hidden email]> writes:
>  
>
>    David> Prabhu Ramachandran wrote:
>    >> distros.  Debian basically adds the local prefix to site.py.
>[...]
>    David> Does debian have any way of telling distutils of the new
>    David> location?  If we are using setup.py to install the python
>    David> modules, then it seems that our default location should be
>    David> the same as the one used by setup.py (otherwise, why would
>    David> we use setup.py at all wh
>
>Not that I know of but this could be used:
>
> import site
> print site.prefixes
>  
>
Y

>    >> I am not sure that David's approach is *the* optimal one.  Many
>    >> VTK users are unlikely to have root access therefore installing
>    >> to /usr/ is entirely out of question.  In these cases it is
>    >> best to do what is currently done -- install to CMAKE_PREFIX.
>    >> Basically the install should work for most common situations.
>    >>
>    >>
>    David> I disagree that we should install to
>    David> ${CMAKE_PREFIX}/lib/pythonx.x/site-packages, as is
>    David> currently done.  It might make sense to install in
>    David> ${CMAKE_PREFIX}/lib/vtkx.x/python
>
>${CMAKE_PREFIX}/lib/vtkx.y/python is completely non-standard.  No
>Python module ever installs importable Python modules in
>lib/libname/python.  Any normal distutils based Python package is
>installed like so:
>
> python setup.py install --prefix=$pth
>
>and everything is installed in
>
> $pth/lib/site-packages/pythonx.y/package
>  
>
Installing in $prefix/lib/pythonx.y/site-packages/ only makes sense to me
when $prefix is one of the prefixes in site.prefixes, because those are the
only prefixes that python knows to load modules from.

There is one specific case when it is useful to install in
$prefix/lib/vtkx.y/python, and that is when you want to install multiple
versions
of VTK in the same prefix without collisions.

>The alternative is
>
> python setup.py install --home=$pth
>
>In which case the files are installed in
>
> $pth/lib/python/package
> $pth/bin
>
>So, that is the default behavior of distutils.  What are you
>suggesting?
>  
>
I'm suggesting that we provide the user with options.  Also, the options
should not be marked as "advanced" options in cmake, they should be
clearly visible.

The option would be called VTK_PYTHON_USE_DISTUTILS, it would
be ON by default.

If VTK_PYTHON_USE_DISTUTILS is ON, the option VTK_PYTHON_SETUP_ARGS
will be present and will not be "advanced".  If ${CMAKE_PREFIX} is one
of the prefixes
in site.prefixes, then it will be the default prefix.  This means that if
CMAKE_PREFIX=/usr/local and /usr/local is in site.prefixes, everything works
"out-of-the-box" after installation.  If ${CMAKE_PREFIX} is not in
site.prefix,
then the default prefix will be sys.prefix.  That will make things will
work nicely
for all superuser installs.

If VTK_PYTHON_USE_DISTUTILS is OFF, then the modules will not be installed
by distutils, but will instead be put in ${CMAKE_PREFIX}/lib/vtkx.y/.
There would be a CMake option PYTHON_INSTALL_PATH to allow the user to
specify a different directory for installation.  Ideally VTK would
produce a shell
script that the user could source to set the PYTHONPATH and LD_LIBRARY_PATH.

That should cover all the bases.  What do you think?

 - David
 - David
_______________________________________________
vtk-developers mailing list
[hidden email]
http://www.vtk.org/mailman/listinfo/vtk-developers
Reply | Threaded
Open this post in threaded view
|

Re: python install prefix in setup.py

Prabhu Ramachandran
>>>>> "David" == David Gobbi <[hidden email]> writes:

    David> I disagree that we should install to
    David> ${CMAKE_PREFIX}/lib/pythonx.x/site-packages, as is
    David> currently done.  It might make sense to install in
    David> ${CMAKE_PREFIX}/lib/vtkx.x/python
    >>  ${CMAKE_PREFIX}/lib/vtkx.y/python is completely non-standard.
    >> No Python module ever installs importable Python modules in
    >> lib/libname/python.  Any normal distutils based Python package
    >> is installed like so:
    >>
    >> python setup.py install --prefix=$pth
    >>
    >> and everything is installed in
    >>
    >> $pth/lib/site-packages/pythonx.y/package

    David> Installing in $prefix/lib/pythonx.y/site-packages/ only
    David> makes sense to me when $prefix is one of the prefixes in
    David> site.prefixes, because those are the only prefixes that
    David> python knows to load modules from.

This is how distutils behaves and most Python users have adapted to
deal with this in one way or another via PYTHONPATH or
sitecustomize.py.

    David> There is one specific case when it is useful to install in
    David> $prefix/lib/vtkx.y/python, and that is when you want to
    David> install multiple versions of VTK in the same prefix without
    David> collisions.

For that we should instead use something along the lines of wxversion
or pygtk in order to pick a required version.  That said,
lib/vtk.x.y/python does not avoid any of the issues you raised with
lib/pythonx.y/site-packages/ either, it only breaks the standard.  So
I vote for breaking less standards given that there isn't a specific
advantage here.

    >> So, that is the default behavior of distutils.  What are you
    >> suggesting?
    >>
    David> I'm suggesting that we provide the user with options.
    David> Also, the options should not be marked as "advanced"
    David> options in cmake, they should be clearly visible.

    David> The option would be called VTK_PYTHON_USE_DISTUTILS, it
    David> would be ON by default.

    David> If VTK_PYTHON_USE_DISTUTILS is ON, the option
    David> VTK_PYTHON_SETUP_ARGS will be present and will not be
    David> "advanced".  If ${CMAKE_PREFIX} is one of the prefixes in

+1 for making VTK_PYTHON_SETUP_ARGS advanced.  I am not sure that we
need a VTK_PYTHON_DISTUTILS at all.  Read on for reasons.

    David> site.prefixes, then it will be the default prefix.  This
    David> means that if CMAKE_PREFIX=/usr/local and /usr/local is in
    David> site.prefixes, everything works "out-of-the-box" after
    David> installation.  If ${CMAKE_PREFIX} is not in site.prefix,
    David> then the default prefix will be sys.prefix.  That will make
    David> things will work nicely for all superuser installs.

+1

Although, I've gotten used to Stow and totally disapprove of installs
made into /usr by non-package (i.e. non-deb/rpm) installs.  I guess,
on non-Linux platforms this is OK.

    David> If VTK_PYTHON_USE_DISTUTILS is OFF, then the modules will
    David> not be installed by distutils, but will instead be put in
    David> ${CMAKE_PREFIX}/lib/vtkx.y/.  There would be a CMake option
    David> PYTHON_INSTALL_PATH to allow the user to specify a
    David> different directory for installation.  Ideally VTK would
    David> produce a shell script that the user could source to set
    David> the PYTHONPATH and LD_LIBRARY_PATH.

Not sure about this one because it seems to make the install more
complex.  Right now we have a setup.py, next we'll have two separate
ways to install.  This looks like a maintenance headache.  From what I
can gather of your needs, you'd like a way to install the Python
modules such that you can keep multiple VTK installs handy.  What if
we simple invested in a vtkversion.py script that did this
automatically for us by the following approach:

 $prefix/lib/pythonx.y/site-packages/
                                      vtkversion.py
                                      vtk4.4/vtk
                                      vtk5.0/vtk
                                      vtkcvs/vtk
                                           

Then users simply do:

import vtkversion
vtkversion.require('5.0')

and vtkversion simply adds the right directory to sys.path and we are
set.

This is nice for several reasons.  (0) we still use distutils which is
still the current standard, (1) a few other projects do this so users
are familiar with this, (2) it makes it easy for apps to specify which
VTK they really need without the need for vtk.vtkVersion().GetFoo()
tricks.

Thoughts?

cheers,
prabhu

_______________________________________________
vtk-developers mailing list
[hidden email]
http://www.vtk.org/mailman/listinfo/vtk-developers
Reply | Threaded
Open this post in threaded view
|

Re: python install prefix in setup.py

David Gobbi-2
After taking a break for the weekend, I'm back into the ring on this one.

Prabhu Ramachandran wrote:

>"David" == David Gobbi <[hidden email]> writes:
>
>    David> Installing in $prefix/lib/pythonx.y/site-packages/ only
>    David> makes sense to me when $prefix is one of the prefixes in
>    David> site.prefixes, because those are the only prefixes that
>    David> python knows to load modules from.
>
>This is how distutils behaves and most Python users have adapted to
>deal with this in one way or another via PYTHONPATH or
>sitecustomize.py.
>  
>
You are misunderstanding me on one of my points here.  I'm not saying that
we should change the behaviour of distutils.  I'm just saying that when
we run

  python setup.py install --prefix=$pth

setting --prefix=${CMAKE_PREFIX} is not always the best choice.  Remember
that by default, distutils sets the prefix to python's install prefix.  
You say that's
bad because python's prefix is usually /usr.

My main concern is for people who are either first-time Python users or who
only ocassionally use python.  I think that the default configuration
should be
set up to make life easy for those users.

The largest class of these users is probably Windows-users who are
taking VTK
for a test-drive.  They are likely to have installed Python themselves.
For these users, installing into python's site-packages, as opposed to the
CMAKE_PREFIX, makes a lot of sense.

>    >> So, that is the default behavior of distutils.  What are you
>    >> suggesting?
>    >>
>    David> I'm suggesting that we provide the user with options.
>    David> Also, the options should not be marked as "advanced"
>    David> options in cmake, they should be clearly visible.
>
>    David> The option would be called VTK_PYTHON_USE_DISTUTILS, it
>    David> would be ON by default.
>
>    David> If VTK_PYTHON_USE_DISTUTILS is ON, the option
>    David> VTK_PYTHON_SETUP_ARGS will be present and will not be
>    David> "advanced".  If ${CMAKE_PREFIX} is one of the prefixes in
>
>+1 for making VTK_PYTHON_SETUP_ARGS advanced.  I am not sure that we
>need a VTK_PYTHON_DISTUTILS at all.  Read on for reasons.
>  
>
I said that VTK_PYTHON_SETUP_ARGS should _not_ be advanced.  CMake
should show the user very clearly where these python modules are going to
be installed, so that the user immediately understands that he/she has
an option
to install them somewhere else.

>    David> site.prefixes, then it will be the default prefix.  This
>    David> means that if CMAKE_PREFIX=/usr/local and /usr/local is in
>    David> site.prefixes, everything works "out-of-the-box" after
>    David> installation.  If ${CMAKE_PREFIX} is not in site.prefix,
>    David> then the default prefix will be sys.prefix.  That will make
>    David> things will work nicely for all superuser installs.
>
>+1
>
>Although, I've gotten used to Stow and totally disapprove of installs
>made into /usr by non-package (i.e. non-deb/rpm) installs.  I guess,
>on non-Linux platforms this is OK.
>  
>
It is not a Linux issue, all UNIX variants are the same as far as
intalling in /usr
is concerned.

Really it is an oversight on the part of the part of the python folks
that distutils
will install packages in /usr/ by default and that python does not
search in /usr/local
by default.  Maybe you debian folks can talk some sense into them.

>    David> If VTK_PYTHON_USE_DISTUTILS is OFF, then the modules will
>    David> not be installed by distutils, but will instead be put in
>    David> ${CMAKE_PREFIX}/lib/vtkx.y/.  There would be a CMake option
>    David> PYTHON_INSTALL_PATH to allow the user to specify a
>    David> different directory for installation.  Ideally VTK would
>    David> produce a shell script that the user could source to set
>    David> the PYTHONPATH and LD_LIBRARY_PATH.
>
>Not sure about this one because it seems to make the install more
>complex.  Right now we have a setup.py, next we'll have two separate
>ways to install.  This looks like a maintenance headache.  From what I
>can gather of your needs, you'd like a way to install the Python
>modules such that you can keep multiple VTK installs handy.  What if
>we simple invested in a vtkversion.py script that did this
>automatically for us by the following approach:
>
> $prefix/lib/pythonx.y/site-packages/
>                                      vtkversion.py
>                                      vtk4.4/vtk
>                                      vtk5.0/vtk
>                                      vtkcvs/vtk
>                                          
>
>Then users simply do:
>
>import vtkversion
>vtkversion.require('5.0')
>
>and vtkversion simply adds the right directory to sys.path and we are
>set.
>
>This is nice for several reasons.  (0) we still use distutils which is
>still the current standard, (1) a few other projects do this so users
>are familiar with this, (2) it makes it easy for apps to specify which
>VTK they really need without the need for vtk.vtkVersion().GetFoo()
>tricks.
>  
>
This would work.

We have to move forward here.  I have a proposal:

1) Make VTK_PYTHON_SETUP_ARGS so that it is not advanced.  This will make it
obvious to the user that they have an option about where the python
modules are
installed.  The "Help" tag should indicate that /usr/local and /usr are
other possible options.
We don't want to be dictators.

2) adopt your vtkversion strategy

The third thing that should be done, eventually, is that CMake should
create a shell
script that the user can source to automatically set the PYTHONPATH
variable.  This
will be useful particularly for people that just do "make" and not "make
install".

- David


_______________________________________________
vtk-developers mailing list
[hidden email]
http://www.vtk.org/mailman/listinfo/vtk-developers
Reply | Threaded
Open this post in threaded view
|

Re: python install prefix in setup.py

Prabhu Ramachandran
>>>>> "David" == David Gobbi <[hidden email]> writes:

    >>  This is how distutils behaves and most Python users have
    >> adapted to deal with this in one way or another via PYTHONPATH
    >> or sitecustomize.py.
    David> You are misunderstanding me on one of my points here.  I'm
    David> not saying that we should change the behaviour of
    David> distutils.  I'm just saying that when we run
[...]
    David> The largest class of these users is probably Windows-users
    David> who are taking VTK for a test-drive.  They are likely to
    David> have installed Python themselves.  For these users,
    David> installing into python's site-packages, as opposed to the
    David> CMAKE_PREFIX, makes a lot of sense.

OK, so under win32 and maybe the Mac we can default to using
sys.prefix.  I'm not convinced about the Unix behavior since I think a
good package should do what it is told to do.  If I set
prefix=/usr/local I expect that nothing will be added into /usr/
behind my back.

    David> If VTK_PYTHON_USE_DISTUTILS is ON, the option
    David> VTK_PYTHON_SETUP_ARGS will be present and will not be
    David> "advanced".  If ${CMAKE_PREFIX} is one of the prefixes in
    >>  +1 for making VTK_PYTHON_SETUP_ARGS advanced.  I am not sure
    >> that we need a VTK_PYTHON_DISTUTILS at all.  Read on for
    >> reasons.
    >>
    David> I said that VTK_PYTHON_SETUP_ARGS should _not_ be advanced.

Sorry, I had wanted to say VTK_PYTHON_SETUP_ARGS should *not* be
advanced (after all it is advanced currently!) but I guess there was
too much multitasking going on at my end.

    David> CMake should show the user very clearly where these python
    David> modules are going to be installed, so that the user
    David> immediately understands that he/she has an option to
    David> install them somewhere else.

OK.  Even if it is going to be made clear to the user that cmake will
install things in sys.prefix, it still causes problems for those who
need to build VTK in an RPM or deb.  I think things should be made as
easy as possible for these folks.  At least the special cases should
be documented prominently (README.packagers?).

    >> Then users simply do:
    >>
    >> import vtkversion vtkversion.require('5.0')
[...]
    David> This would work.

    David> We have to move forward here.  I have a proposal:

    David> 1) Make VTK_PYTHON_SETUP_ARGS so that it is not advanced.
    David> This will make it obvious to the user that they have an
    David> option about where the python modules are installed.  The
    David> "Help" tag should indicate that /usr/local and /usr are
    David> other possible options.  We don't want to be dictators.

OK.

    David> 2) adopt your vtkversion strategy

    David> The third thing that should be done, eventually, is that
    David> CMake should create a shell script that the user can source
    David> to automatically set the PYTHONPATH variable.  This will be
    David> useful particularly for people that just do "make" and not
    David> "make install".

I think this sounds fine.  Only thing is I don't have the time for
this during this semester, so count me out of helping with an
implementation.

cheers,
prabhu

_______________________________________________
vtk-developers mailing list
[hidden email]
http://www.vtk.org/mailman/listinfo/vtk-developers
Reply | Threaded
Open this post in threaded view
|

Re: python install prefix in setup.py

David Gobbi-2
Prabhu Ramachandran wrote:

>"David" == David Gobbi <[hidden email]> writes:
>
>    David> The largest class of these users is probably Windows-users
>    David> who are taking VTK for a test-drive.  They are likely to
>    David> have installed Python themselves.  For these users,
>    David> installing into python's site-packages, as opposed to the
>    David> CMAKE_PREFIX, makes a lot of sense.
>
>OK, so under win32 and maybe the Mac we can default to using
>sys.prefix.  I'm not convinced about the Unix behavior since I think a
>good package should do what it is told to do.  If I set
>prefix=/usr/local I expect that nothing will be added into /usr/
>behind my back.
>  
>
I'll only make installing into the Python prefix the default on Win32.
The Mac is more like UNIX.

>    David> CMake should show the user very clearly where these python
>    David> modules are going to be installed, so that the user
>    David> immediately understands that he/she has an option to
>    David> install them somewhere else.
>
>OK.  Even if it is going to be made clear to the user that cmake will
>install things in sys.prefix, it still causes problems for those who
>need to build VTK in an RPM or deb.  I think things should be made as
>easy as possible for these folks.  At least the special cases should
>be documented prominently (README.packagers?).
>  
>
If the VTK_PYTHON_SETUP_ARGS is clearly shown by CMake (i.e. not
advanced) and described in a readme, that should be enough.

>    >> Then users simply do:
>    >>
>    >> import vtkversion vtkversion.require('5.0')
>[...]
>    David> This would work.
>
>    David> We have to move forward here.  I have a proposal:
>
>    David> 1) Make VTK_PYTHON_SETUP_ARGS so that it is not advanced.
>    David> This will make it obvious to the user that they have an
>    David> option about where the python modules are installed.  The
>    David> "Help" tag should indicate that /usr/local and /usr are
>    David> other possible options.  We don't want to be dictators.
>
>OK.
>
>    David> 2) adopt your vtkversion strategy
>
>    David> The third thing that should be done, eventually, is that
>    David> CMake should create a shell script that the user can source
>    David> to automatically set the PYTHONPATH variable.  This will be
>    David> useful particularly for people that just do "make" and not
>    David> "make install".
>
>I think this sounds fine.  Only thing is I don't have the time for
>this during this semester, so count me out of helping with an
>implementation.
>  
>
I can handle most of it myself, so that should not be a problem.

 - David
_______________________________________________
vtk-developers mailing list
[hidden email]
http://www.vtk.org/mailman/listinfo/vtk-developers
Reply | Threaded
Open this post in threaded view
|

Re: python install prefix in setup.py

Sebastien BARRE
In reply to this post by Prabhu Ramachandran
Hello Guys,

Any progress on that python install discussion ?

Many of my dashboards are failing for external projects using
installed VTK's because it's just very hard to find where the Python stuff is.
On Windows for example, I can more or less rely on:
        vtkinstalldir/lib/site-packages
but on Unix, it's worse:
        vtkinstalldir/lib/python2.4/site-packages
there is now a python2.4 in the way to 'site-packages'. My dashboards
are automated, how can I figure out that there is a python2.4
directory there ? Right now, my workaround is to check if we are on
Win32 or not, then check if PYTHON_EXECUTABLE is set, and launch it
to get the python version, etc. Not so practical.

Thanks

--
Sebastien Barre

_______________________________________________
vtk-developers mailing list
[hidden email]
http://www.vtk.org/mailman/listinfo/vtk-developers
Reply | Threaded
Open this post in threaded view
|

Re: python install prefix in setup.py

David Gobbi-2
Hi Sebastien,

We did come to a conclusion on the install discussion, but what we had
decided on is not going
to help you. Our first conclusion was that VTK_PYTHON_SETUP_ARGS should
become a
non-advanced and better documented.  Our second conclusion is that we
need to add some way to allow
people to have multiple installed versions of VTK in site-packages at
once.  Neither of these is relevant.

I'm guessing that you need to know where site-packages is so that you
can set the PYTHONPATH
automatically.  So VTK's CMake will have to figure out how setup.py
calculates the path to site-packages,
and will then create VTK_PYTHON_MODULE_DIRS in UseVTK.cmake so that
other projects
can find it.

Is that the kind of thing you are looking for?

 - David



Sebastien BARRE wrote:

> Hello Guys,
>
> Any progress on that python install discussion ?
>
> Many of my dashboards are failing for external projects using
> installed VTK's because it's just very hard to find where the Python
> stuff is.
> On Windows for example, I can more or less rely on:
>     vtkinstalldir/lib/site-packages
> but on Unix, it's worse:
>     vtkinstalldir/lib/python2.4/site-packages
> there is now a python2.4 in the way to 'site-packages'. My dashboards
> are automated, how can I figure out that there is a python2.4
> directory there ? Right now, my workaround is to check if we are on
> Win32 or not, then check if PYTHON_EXECUTABLE is set, and launch it to
> get the python version, etc. Not so practical.
>
> Thanks
>
> --
> Sebastien Barre
>
> _______________________________________________
> vtk-developers mailing list
> [hidden email]
> http://www.vtk.org/mailman/listinfo/vtk-developers
>

_______________________________________________
vtk-developers mailing list
[hidden email]
http://www.vtk.org/mailman/listinfo/vtk-developers
Reply | Threaded
Open this post in threaded view
|

Re: python install prefix in setup.py

Sebastien BARRE
At 10:50 AM 2/15/2006, you wrote:

>I'm guessing that you need to know where site-packages is so that
>you can set the PYTHONPATH
>automatically.  So VTK's CMake will have to figure out how setup.py
>calculates the path to site-packages,
>and will then create VTK_PYTHON_MODULE_DIRS in UseVTK.cmake so that
>other projects
>can find it.
>
>Is that the kind of thing you are looking for?

Yup, sounds good.


--
Sebastien Barre

_______________________________________________
vtk-developers mailing list
[hidden email]
http://www.vtk.org/mailman/listinfo/vtk-developers
Reply | Threaded
Open this post in threaded view
|

PYTHON_UTIL_LIBRARY

Sebastien BARRE
Hi

Can you guys eventually clarify what PYTHON_UTIL_LIBRARY is for ?
Here is the Wrapping/Python/CMakeLists.txt file:

FIND_LIBRARY(PYTHON_UTIL_LIBRARY
   NAMES util
   PATHS /usr/lib
   DOC "Utility library needed for vtkpython"
)
MARK_AS_ADVANCED(PYTHON_UTIL_LIBRARY)
IF(PYTHON_UTIL_LIBRARY)
   SET(PYTHON_UTIL_LIBRARY_LIB ${PYTHON_UTIL_LIBRARY})
ENDIF(PYTHON_UTIL_LIBRARY)

Apparently if the library is found, it is used to link vtkpython
against it. If it is not found, well, apparently it does not hurt.

My problem is that using Visual C++ 6.0, the Python util library is
found in the wrong place... in the compiler libs themselves (with a
library name like that, that was bound to happen):
PYTHON_UTIL_LIBRARY:FILEPATH=c:/Program Files/Microsoft Visual
Studio/VC98/Lib/UTIL.LIB
Why is it found there, I have no clue.

Then the whole Python framework fails to build:
Linking...
LINK : fatal error LNK1104: cannot open file "c:\Program
Files\Microsoft Visual Studio\VC98\Lib\UTIL.LIB.lib"
Error executing link.exe.
Ooops.

I can set PYTHON_UTIL_LIBRARY manually, but still, I'm curious...

A

--
Sebastien Barre

_______________________________________________
vtk-developers mailing list
[hidden email]
http://www.vtk.org/mailman/listinfo/vtk-developers
Reply | Threaded
Open this post in threaded view
|

Re: PYTHON_UTIL_LIBRARY

Mathieu Malaterre-2
I believe this is a Linux only thing. Maybe you can put it within a
IF/ENDIF UNIX statement.

The real fix is mention in bug #2257 in inspecting distuils var, like SHLIBS
'SHLIBS': '-lpthread -ldl  -lutil',

2 cents,
Mathieu

Sebastien BARRE wrote:

> Hi
>
> Can you guys eventually clarify what PYTHON_UTIL_LIBRARY is for ?
> Here is the Wrapping/Python/CMakeLists.txt file:
>
> FIND_LIBRARY(PYTHON_UTIL_LIBRARY
>   NAMES util
>   PATHS /usr/lib
>   DOC "Utility library needed for vtkpython"
> )
> MARK_AS_ADVANCED(PYTHON_UTIL_LIBRARY)
> IF(PYTHON_UTIL_LIBRARY)
>   SET(PYTHON_UTIL_LIBRARY_LIB ${PYTHON_UTIL_LIBRARY})
> ENDIF(PYTHON_UTIL_LIBRARY)
>
> Apparently if the library is found, it is used to link vtkpython against
> it. If it is not found, well, apparently it does not hurt.
>
> My problem is that using Visual C++ 6.0, the Python util library is
> found in the wrong place... in the compiler libs themselves (with a
> library name like that, that was bound to happen):
> PYTHON_UTIL_LIBRARY:FILEPATH=c:/Program Files/Microsoft Visual
> Studio/VC98/Lib/UTIL.LIB
> Why is it found there, I have no clue.
>
> Then the whole Python framework fails to build:
> Linking...
> LINK : fatal error LNK1104: cannot open file "c:\Program Files\Microsoft
> Visual Studio\VC98\Lib\UTIL.LIB.lib"
> Error executing link.exe.
> Ooops.
>
> I can set PYTHON_UTIL_LIBRARY manually, but still, I'm curious...
>
> A
>
> --
> Sebastien Barre
>
> _______________________________________________
> vtk-developers mailing list
> [hidden email]
> http://www.vtk.org/mailman/listinfo/vtk-developers
>

_______________________________________________
vtk-developers mailing list
[hidden email]
http://www.vtk.org/mailman/listinfo/vtk-developers
Reply | Threaded
Open this post in threaded view
|

Re: PYTHON_UTIL_LIBRARY

David Gobbi-2
I really don't know what libutil is used for, but Mathieu is right.  
It's a part of glibc.

Mathieu Malaterre wrote:

> I believe this is a Linux only thing. Maybe you can put it within a
> IF/ENDIF UNIX statement.
>
> The real fix is mention in bug #2257 in inspecting distuils var, like
> SHLIBS
> 'SHLIBS': '-lpthread -ldl  -lutil',
>
> 2 cents,
> Mathieu
>
> Sebastien BARRE wrote:
>
>> Hi
>>
>> Can you guys eventually clarify what PYTHON_UTIL_LIBRARY is for ?
>> Here is the Wrapping/Python/CMakeLists.txt file:
>>
>> FIND_LIBRARY(PYTHON_UTIL_LIBRARY
>>   NAMES util
>>   PATHS /usr/lib
>>   DOC "Utility library needed for vtkpython"
>> )
>> MARK_AS_ADVANCED(PYTHON_UTIL_LIBRARY)
>> IF(PYTHON_UTIL_LIBRARY)
>>   SET(PYTHON_UTIL_LIBRARY_LIB ${PYTHON_UTIL_LIBRARY})
>> ENDIF(PYTHON_UTIL_LIBRARY)
>>
>> Apparently if the library is found, it is used to link vtkpython
>> against it. If it is not found, well, apparently it does not hurt.
>>
>> My problem is that using Visual C++ 6.0, the Python util library is
>> found in the wrong place... in the compiler libs themselves (with a
>> library name like that, that was bound to happen):
>> PYTHON_UTIL_LIBRARY:FILEPATH=c:/Program Files/Microsoft Visual
>> Studio/VC98/Lib/UTIL.LIB
>> Why is it found there, I have no clue.
>>
>> Then the whole Python framework fails to build:
>> Linking...
>> LINK : fatal error LNK1104: cannot open file "c:\Program
>> Files\Microsoft Visual Studio\VC98\Lib\UTIL.LIB.lib"
>> Error executing link.exe.
>> Ooops.
>>
>> I can set PYTHON_UTIL_LIBRARY manually, but still, I'm curious...
>>
>> A
>>
>> --
>> Sebastien Barre
>>
>> _______________________________________________
>> vtk-developers mailing list
>> [hidden email]
>> http://www.vtk.org/mailman/listinfo/vtk-developers
>>
>
>

_______________________________________________
vtk-developers mailing list
[hidden email]
http://www.vtk.org/mailman/listinfo/vtk-developers
12