You are here

Make error during installation of updated ncbi-blast-2.9.0+-src plus error with nr-database

7 posts / 0 new
Last post
Make error during installation of updated ncbi-blast-2.9.0+-src plus error with nr-database
#1

Dear fellows,

my recent situation with setting up the fragment generation for structure prediction ab initio consists in follow:

- It seems as if I solved my problems with version compatibility of gcc-compiler and the environment variable FRAGMENT_PICKER.

- But a problem with installation of updated make_fragments.pl and install_dependencies.pl is still remained unsolved. I carried out next actions with the ncbi-blast-2.9.0+-src package:

1) wget ftp://ftp.ncbi.nlm.nih.gov/blast/executables/blast+/LATEST/ncbi-blast-2.9.0+-src.tar.gz.

2) tar xvzf ncbi-blast-2.9.0+-src.tar.gz.

3) ./configure at ~/ncbi-blast-2.9.0+-src/c++/ directory.

4) /usr/bin/make all_r at ~/ncbi-blast-2.9.0+-src/c++/ReleaseMT/build directory.

I got such errors at the last step:

"make[3]: Leaving directory '~/ncbi-blast-2.9.0+-src/c++/ReleaseMT/build/corelib'

make[3]: *** [~/ncbi-blast-2.9.0+-src/c++/ReleaseMT/build/Makefile.rules:88: test_boost.o] Error 1

FAILED: src/corelib/Makefile.test_boost.lib

make[3]: Entering directory '~/ncbi-blast-2.9.0+-src/c++/ReleaseMT/build/corelib'

/bin/rm -f libtest_boost.a .test_boost.dep .libtest_boost.a.stamp

/bin/rm -f ~/ncbi-blast-2.9.0+-src/c++/ReleaseMT/lib/libtest_boost.a ~/ncbi-blast-2.9.0+-src/c++/ReleaseMT/status/.test_boost.dep \ ~/ncbi-blast-2.9.0+-src/c++/ReleaseMT/lib/libtest_boost-static.a ~/ncbi-blast-2.9.0+-src/c++/ReleaseMT/status/.test_boost-static.dep

make[3]: Leaving directory '~/ncbi-blast-2.9.0+-src/c++/ReleaseMT/build/corelib'

make[2]: *** [~/ncbi-blast-2.9.0+-src/c++/src/build-system/Makefile.meta_l:318: all.nonusr] Error 2

make[2]: Leaving directory '~/ncbi-blast-2.9.0+-src/c++/ReleaseMT/build/corelib'

make[1]: *** [~/ncbi-blast-2.9.0+-src/c++/src/build-system/Makefile.meta_l:276: all_l.real] Error 2

make[1]: Leaving directory '~/ncbi-blast-2.9.0+-src/c++/ReleaseMT/build/corelib'

make: *** [~/ncbi-blast-2.9.0+-src/c++/src/build-system/Makefile.meta_r:41: all_r.real] Error 5."

I attach a file with a full report about make-command running.

On the other hand I got such output with nr-databases errors during fragment generation trial:

"Use of uninitialized value $ENV{"FRAGMENT_PICKER_NUM_CPUS"} in int at ~/rosetta_bin_linux_2019.22.60749_bundle/tools/fragment_tools/make_fragments.pl line 34.

Use of uninitialized value $ENV{"BLAST_NUM_CPUS"} in int at ~/rosetta_bin_linux_2019.22.60749_bundle/tools/fragment_tools/make_fragments.pl line 40.

Use of uninitialized value $ENV{"PSIPRED_USE_weights_dat4"} in int at ~/rosetta_bin_linux_2019.22.60749_bundle/tools/fragment_tools/make_fragments.pl line 50.

Use of uninitialized value $ENV{"SLAVE_LAUNCHER_MAX_JOBS"} in int at ~/rosetta_bin_linux_2019.22.60749_bundle/tools/fragment_tools/make_fragments.pl line 69.

Use of uninitialized value $ENV{"SLAVE_MAX_WAIT"} in int at ~/rosetta_bin_linux_2019.22.60749_bundle/tools/fragment_tools/make_fragments.pl line 71.

Use of uninitialized value $ENV{"SLAVE_MAX_ATTEMPTS"} in int at ~/rosetta_bin_linux_2019.22.60749_bundle/tools/fragment_tools/make_fragments.pl line 72.

Use of uninitialized value $ENV{"PDB2VALL_IGNORE_ERRORS"} in int at ~/rosetta_bin_linux_2019.22.60749_bundle/tools/fragment_tools/make_fragments.pl line 78.

Dependency 'nr' does not exist!

Running install_dependencies.pl

Note: the NCBI non-redundant (nr) sequence database is very large.

Please be patient.....

Fetching NR database from NCBI. Be very patient ......

URL transformed to HTTPS due to an HSTS policy

--2019-10-01 10:57:43-- https://www.ncbi.nlm.nih.gov/blast/docs/update_blastdb.pl

Resolving www.ncbi.nlm.nih.gov (www.ncbi.nlm.nih.gov)... 130.14.29.110, 2607:f220:41e:4290::110

Connecting to www.ncbi.nlm.nih.gov (www.ncbi.nlm.nih.gov)|130.14.29.110|:443... connected.

HTTP request sent, awaiting response... 404 Not Found

2019-10-01 10:57:44 ERROR 404: Not Found.

ERROR! wget http://www.ncbi.nlm.nih.gov/blast/docs/update_blastdb.pl failed.

ERROR! ~/rosetta_bin_linux_2019.22.60749_bundle/tools/fragment_tools/databases/nr does not exist."

I will be really grateful for your help and tips in solving these problems with "make" and nr-databases and I am looking forward to it.

Best regard,

Corvin.

AttachmentSize
Make_error.pdf59.55 KB
Post Situation: 
Tue, 2019-10-01 02:04
Corvin

Hello Corvin

I recently updated install dependencies to not have this error anymore.  could you try the newest available version of 'install_dependencies.pl'? (probably located in the newest available rosetta)

Tue, 2019-10-01 09:55
danpf

Dear danpf,

I have just installed newer version of Rosetta (version 3.11, which was updated on 26st September of this year) and run make_fragments.pl. And I got such error:

"2019-10-02 12:50:02 (1.81 MB/s) - ‘ncbi-blast-2.9.0+-src.tar.gz.2’ saved [25411110]

Can't locate JSON.pm in @INC (you may need to install the JSON module) (@INC contains: /usr/lib/perl5/site_perl/5.28.1/x86_64-linux-thread-multi /usr/lib/perl5/site_perl/5.28.1 /usr/lib/perl5/vendor_perl/5.28.1/x86_64-linux-thread-multi /usr/lib/perl5/vendor_perl/5.28.1 /usr/lib/perl5/5.28.1/x86_64-linux-thread-multi /usr/lib/perl5/5.28.1 /usr/lib/perl5/site_perl) at ~/rosetta_bin_linux_2019.35.60890_bundle/tools/fragment_tools/databases/update_blastdb.pl line 43.

BEGIN failed--compilation aborted at ~/rosetta_bin_linux_2019.35.60890_bundle/tools/fragment_tools/databases/update_blastdb.pl line 43.

Can't locate JSON.pm in @INC (you may need to install the JSON module) (@INC contains: /usr/lib/perl5/site_perl/5.28.1/x86_64-linux-thread-multi /usr/lib/perl5/site_perl/5.28.1 /usr/lib/perl5/vendor_perl/5.28.1/x86_64-linux-thread-multi /usr/lib/perl5/vendor_perl/5.28.1 /usr/lib/perl5/5.28.1/x86_64-linux-thread-multi /usr/lib/perl5/5.28.1 /usr/lib/perl5/site_perl) at ~/rosetta_bin_linux_2019.35.60890_bundle/tools/fragment_tools/databases/update_blastdb.pl line 43.

BEGIN failed--compilation aborted at ~/rosetta_bin_linux_2019.35.60890_bundle/tools/fragment_tools/databases/update_blastdb.pl line 43.

Can't locate JSON.pm in @INC (you may need to install the JSON module) (@INC contains: /usr/lib/perl5/site_perl/5.28.1/x86_64-linux-thread-multi /usr/lib/perl5/site_perl/5.28.1 /usr/lib/perl5/vendor_perl/5.28.1/x86_64-linux-thread-multi /usr/lib/perl5/vendor_perl/5.28.1 /usr/lib/perl5/5.28.1/x86_64-linux-thread-multi /usr/lib/perl5/5.28.1 /usr/lib/perl5/site_perl) at ~/rosetta_bin_linux_2019.35.60890_bundle/tools/fragment_tools/databases/update_blastdb.pl line 43.

BEGIN failed--compilation aborted at ~/rosetta_bin_linux_2019.35.60890_bundle/tools/fragment_tools/databases/update_blastdb.pl line 43.

Can't locate JSON.pm in @INC (you may need to install the JSON module) (@INC contains: /usr/lib/perl5/site_perl/5.28.1/x86_64-linux-thread-multi /usr/lib/perl5/site_perl/5.28.1 /usr/lib/perl5/vendor_perl/5.28.1/x86_64-linux-thread-multi /usr/lib/perl5/vendor_perl/5.28.1 /usr/lib/perl5/5.28.1/x86_64-linux-thread-multi /usr/lib/perl5/5.28.1 /usr/lib/perl5/site_perl) at ~/rosetta_bin_linux_2019.35.60890_bundle/tools/fragment_tools/databases/update_blastdb.pl line 43.

BEGIN failed--compilation aborted at ~/rosetta_bin_linux_2019.35.60890_bundle/tools/fragment_tools/databases/update_blastdb.pl line 43.

Aborting....

rm: cannot remove '~/rosetta_bin_linux_2019.35.60890_bundle/tools/fragment_tools/databases/nr.*tar.gz': No such file or directory

Aborted!

ERROR! ~/rosetta_bin_linux_2019.35.60890_bundle/tools/fragment_tools/databases/nr does not exist."

I should note, that JSON module were installed at the moment of make_fragments.pl script running. I suppose, that a problem consists in nr-databases. How to fix this problem?

Thanks in advance,

Corvin.

Wed, 2019-10-02 03:00
Corvin

hey corvin:

you have to install some perl stuff to get this script to work (I do so VIA conda, but if you know perl you can do so with an alternative method)

 

# This should be somewhat close to what you should run (might be slightly off)
conda create --name make_frags --python=2
conda activate make_frags
conda config --env --add channels bioconda
conda config --env --add channels compbiocore
conda install -y perl perl-if perl-parallel-forkmanager perl-switch perl-text-balanced perl-json

# check for conda env location via conda env list (fill in manually the { } stuff
export PERL5LIB=${put where your conda env is}/lib/site_perl/5.26.2/
export ROSETTA_DATABASE={rosetta location}/database
export VALL={rosetta location}/../tools/fragment_tools/vall.jul19.2011
export FRAGMENT_PICKER={rosetta location}/source/bin/fragment_picker.{extension}
export FRAGMENT_PICKER_NUM_CPUS=1
export BLAST_NUM_CPUS={num_cores_to_use}

# now I run
./install_dependencies.pl standard

if you know how to install perl modules via the system (like apt-get) then you're welcome to do it that way, I found it to be very annoying though.

Wed, 2019-10-02 16:21
danpf

Dear danpf,

I am sincerely thankful for the given algorithm at this response. It works well and installs nr-databases successfully.

But during nr-databases downloading I realized, that their overall size (136 packages, each of them contains several hundred MB) could be too big for my laptop SSD with a memory volume 256 GB, on which my Linux is placed. But my laptop also has the HDD with a memory volume 1 TB, on which the Windows10 is installed. I can make files on HDD accessible for Linux. Is Rosetta software moving to HDD, installation of nr-databases here and call the Rosetta scripts (which are placed on HDD) from Linux for their running on Linux a good solution of this problem? Or creation of separated directory with nr-databases on HDD and export their path for their availability for Linux is better idea?

Kind regards,

Corvin.

Thu, 2019-10-03 08:04
Corvin

Hey Corvin,

The install_dependencies script is unfortunately hardcoded to install the database files wherever 'install_dependencies.pl' is located.

I would go to your 1TB hdd, make a folder, and copy 'install_dependencies.pl' and 'make_fragments.pl' to that location, and run them from there.

I can't remember the total size, but it's approximately 500gb (+-100gb haha) so make sure you have the space

 

~Dan

Thu, 2019-10-03 09:45
danpf

Dear danpf,

my "install_dependencies.pl" script is running correctly. Now it is working on the nr-pfilt data extracting and formatting stage. But it's looked like, that the overall size of nr-databases will be far more than 600 GB. There is some unneccessary files, which I can remove with no harm for this process? For example, can I delete the text file "nr" in ~/NR-databases/databases (size - 138 GB), while "install_dependencies" works with "nr_pfilt" data extracting and formatting?

Many thanks in advance.

Kind regards,

Corvin.

Mon, 2019-10-07 01:20
Corvin