You are here

Problems building under different OS

5 posts / 0 new
Last post
Problems building under different OS
#1

I'll do two posts about two different problems I'm having under MacOS and Ubuntu. First:

Trying to build from PyRosetta.develop@54516 on Mac OS X Lion 10.7.5 with (and linking against) python 2.7.3 installed with homebrew.

Main Problem
The build itself completes (with a couple of exceptions - see end of this post) with errors like this:

darwin.compile.c++ bin.v2/libs/python/build/darwin-4.2.1/release/threading-multi/object/function.o
In file included from /usr/local/Cellar/python/2.7.3/Frameworks/Python.framework/Versions/2.7/include/python2.7/Python.h:126,
from ./boost/python/detail/wrap_python.hpp:142,
from ./boost/python/detail/prefix.hpp:13,
from ./boost/python/object/function.hpp:8,
from ./boost/python/docstring_options.hpp:8,
from libs/python/src/object/function.cpp:6:
/usr/local/Cellar/python/2.7.3/Frameworks/Python.framework/Versions/2.7/include/python2.7/modsupport.h:27: warning: 'PyArg_ParseTuple' is an unrecognized format function type

But importing anything from rosetta causes immediate segfault. By strategically commenting out lines in __init__.py files and in _all_at_once.?.cpp files, then recompiling and re-linking (by hand, copying the relevant commands from output of BuildBindings.py) just those all_at_once files, I've convinced myself that segfaults are caused by attempting to import function definitions with default values assigned to any arguments. Example from basic/gpu/__gpu_all_at_once_.0.cpp:

...
Timer_exposer.def( boost::python::init< ::basic::gpu::Timer const & > ( (boost::python::arg("")) , "basic/gpu/Timer.hh:28" ) );
Timer_exposer.def( boost::python::init< ::basic::Tracer::TracerProxy &, boost::python::optional< char const * > > ( (boost::python::arg("t"), boost::python::arg("tag")=(0)) , "basic/gpu/Timer.hh:38" ) );
Timer_exposer.def( boost::python::init< > () );
...

causes segfault on import, but

...
Timer_exposer.def( boost::python::init< ::basic::gpu::Timer const & > ( (boost::python::arg("")) , "basic/gpu/Timer.hh:28" ) );
// Timer_exposer.def( boost::python::init< ::basic::Tracer::TracerProxy &, boost::python::optional< char const * > > ( (boost::python::arg("t"), boost::python::arg("tag")=(0)) , "basic/gpu/Timer.hh:38" ) );
Timer_exposer.def( boost::python::init< > () );
...

doesn't. I think this is related to the 'PyArg_ParseTuple' warnings during compilation, but I can't figure out how to fix it. I've compiled with the native MacOS gcc (LLVM-gcc), CLang, and gcc-4.7 installed with homebrew. Outcome is the same every time.

Inconvenience issues I encountered along the way

1) Running BuildPyRosetta.sh with -jX, where X>1 caused this error:

Traceback (most recent call last):
File "./BuildBindings.py", line 2112, in
if __name__ == "__main__": main(sys.argv)
File "./BuildBindings.py", line 290, in main
buildModule(n, bindings_path, include_paths=options.I, libpaths=options.L, runtime_libpaths=options.L, gccxml_path=options.gccxml)
File "./BuildBindings.py", line 1009, in buildModule
return buildModule_UsingCppParser(path, dest, include_paths, libpaths, runtime_libpaths, gccxml_path)
File "./BuildBindings.py", line 1632, in buildModule_UsingCppParser
mWait(tag=path) # wait for all compilation jobs to finish...
File "./BuildBindings.py", line 484, in mWait
r = os.waitpid(j.pid, os.WNOHANG)
OSError: [Errno 10] No child processes

where the offending PID was the original call to BuildPyRosetta.sh (I think). I encased r = os.waitpid(j.pid, os.WNOHANG) in a try/except block and the build seemed to go fine. Strangely, this did not happen when building from scratch with DeployPyRosetta.py -jX, X>1.

2) Final linking of _all_at_once_.so libraries failed several times because of functions declared in header files but not defined in .cc files. I think I finally got them all:

core::scoring::methods::PBLifetimeCache::set_pbp
core::scoring::methods::PBLifetimeCache::set_pose
protocols::topology_broker::ConstraintClaimer::set_drop_random_rate(double)

3) This is really minor, but setting the --gccxml-compiler flag didn't seem to affect what compiler was used, or at least not the output produced by the script. Every time llvm-g++ was used - the default in MacOS.

Post Situation: 
Wed, 2013-03-27 15:31
sloutsky

After failing to build under MacOS I tried building under Ubuntu:

From PyRosetta.develop@54557 on Ubuntu Linux 12.04 (server version) with x86_64 architecture and python 2.7.3, crashes out during gccxml wrapping stage with lots of following errors:

In file included from /usr/include/c++/4.6/cstring:44,
from ../external/boost_1_46_1/boost/function/function_base.hpp:62,
from ../external/boost_1_46_1/boost/function/detail/prologue.hpp:17,
from ../external/boost_1_46_1/boost/function.hpp:24,
from ./protocols/simple_moves/GenericMonteCarloMover.hh:30,
from /home/sloutsky/code/rosetta/py/rosetta_source/src/python/bindings/rosetta/protocols/simple_moves/__simple_moves_all_at_once_.source.cc:34:
/usr/include/string.h:45: error: reference to 'size_t' is ambiguous
/usr/lib/gcc/x86_64-linux-gnu/4.6/include/stddef.h:212: error: candidates are: typedef long unsigned int size_t
/usr/include/c++/4.6/x86_64-linux-gnu/./bits/c++config.h:155: error: typedef long unsigned int std::size_t
/usr/include/string.h:45: error: reference to 'size_t' is ambiguous
/usr/lib/gcc/x86_64-linux-gnu/4.6/include/stddef.h:212: error: candidates are: typedef long unsigned int size_t
/usr/include/c++/4.6/x86_64-linux-gnu/./bits/c++config.h:155: error: typedef long unsigned int std::size_t
/usr/include/string.h:45: error: 'size_t' has not been declared
/usr/include/string.h:49: error: reference to 'size_t' is ambiguous
/usr/lib/gcc/x86_64-linux-gnu/4.6/include/stddef.h:212: error: candidates are: typedef long unsigned int size_t
/usr/include/c++/4.6/x86_64-linux-gnu/./bits/c++config.h:155: error: typedef long unsigned int std::size_t
/usr/include/string.h:49: error: reference to 'size_t' is ambiguous

Like with the MacOS problem, the fight here seems to be between boost and the compiler.

Wed, 2013-03-27 15:41
sloutsky

I have made Sergey aware of this, he'll take a look when he gets a chance.

Thu, 2013-03-28 06:59
smlewis

Have you tried to build on Mac OS native environment (with using Apple version of GCC and Clang)? - this should work out of the box. And the same should be with Ubuntu 12.04, just make sure to downgrade GCC to 4.4.

The crashes that you encounter after successful build is most likely due to wrong version of Python lib/interpreter was used during the build and run stage.

The GCC 4.7: It is known problem that PyRosetta build fail when GCC 4.7 used. If you still need to use 4.7 then I would recommend trying to use option --gccxml-compiler when invoking BuildPyRosetta script and point it to Apple provided GCC - this should guarantee that parsing/generation phase is ok. And if you want you can use --compiler option to use point script to use GCC 4.7 for actual compilation/linking.

On your attempt to build on Ubuntu 12.04 - which version of GCC did you used?

Thu, 2013-03-28 09:53
Sergey

Thanks for getting back to me!

For MacOS build:

I started out compiling with Apple's native GCC (LLVM-gcc), then tried both CLang and GCC 4.7 from homebrew. I'm happy to use any compiler that works - GCC 4.7 was just my last ditch attempt after the built-in compilers failed.

However, in all cases I'm building with and linking against python 2.7.3 installed with homebrew, not Apple's native python 2.7.1. I can try using the native python, but I would strongly prefer using the homebrew one, since I'd like to use PyRosetta with some other libraries installed there. As far as I can tell, the entire build process uses the homebrew python, so there's no cross-contamination between 2.7.1 and 2.7.3. Is there an inherent problem with using a non-native python? I was under the impression that, as long as you build and run using a single python install, it doesn't matter which install that is. Is that wrong?

Btw - I haven't been able to get the --gccxml-compiler flag to take effect. No matter what compiler I give it, it uses Apple's llvm-gcc. It sounds like that's the one I should use for parsing/generation on MacOS, so I guess it's not a problem.

For Ubuntu build:

I used the version of GCC that came with Ubuntu - 4.6.3. Do I need to downgrade from that?

UPDATE

Running BuildPyRosetta.sh --gccxml-compiler=gcc-4.4 solves the problem on Ubuntu, so I guess gcc-4.6 was the problem.

UPDATE #2

Building in MacOS with native python works as well, so the homebrew install of python is the culprit.

UPDATE #3

Found some information on PyArg_ParseTuple warnings when linking against homebrewed python:

https://github.com/mxcl/homebrew/issues/9215
https://github.com/mxcl/homebrew/issues/12194
https://gist.github.com/aGHz/3758501

Commenting out #define HAVE_ATTRIBUTE_FORMAT_PARSETUPLE 1 in pyconfig.h fixes ‘PyArg_ParseTuple’ is an unrecognized format function type warnings ... but doesn't fix the problem with default argument values and segfaults.

Thu, 2013-03-28 16:57
sloutsky