You are here

Is there a setup script to automate the process of installing and testing rosetta on a cloud server ?

6 posts / 0 new
Last post
Is there a setup script to automate the process of installing and testing rosetta on a cloud server ?
#1

Also - is Ubuntu 16 LTS the best choice for running rosetta in the cloud ?

Category: 
Post Situation: 
Wed, 2018-04-04 11:51
covadiuc

If you are interacting with your cloud server as a VM, then it behaves the same as any other machine, so setting up Rosetta is basically the same as doing it locally.

If you are interacting with your cloud server via a local gcloud command line tool (or whatever the AWS equivalent is, or whatever cloud you are using) - Commons does not have official support for it. 

There have been a few half-completed attempts to create Rosetta docker containers for cloud use, but as RosettaCommons runs almost entirely on academic server resources and Rosetta@HOME, neither of which are "cloud" nor use containers, there's not been much interest.  (Based on your location perhaps you already know this?)

From experience, Ubuntu 16 is a perfectly reasonable choice for the purpose.  centos will work well too. 

 

Fri, 2018-04-06 06:45
smlewis

Cool - thanks for the input. I was curious about docker containers and you're answer makes  sense. I ended up writing a bash script - here is my draft ( any feedback would be apreciated)

Yes - using gcloud

#!/bin/bash

# Update system and fetch basic build tools 
sudo apt-get update
sudo apt-get --assume-yes upgrade 
sudo apt-get --assume-yes install build-essential 
sudo apt-get --assume-yes install lib32z1-dev
#sudo apt-get --assume-yes install openmpi openmpi-dev
# better choice 
sudo apt-get --assume-yes install libopenmpi-dev
sudo apt-get --assume-yes install zlib1g-dev


 

#sudo apt-get --assume-yes install openmpi-bin libop
#sudo apt-get --assume-yes install libsqlite3-dev
#sudo apt-get --assume-yes install zlib1g-dev libncurses5-dev
sudo apt-get --assume-yes install htop

# SSH for transfer of files to local mac 
sudo apt-get install ssh
# From Linux to Mac (run from the Linux machine): 
# scp filename.txt user@remote_server:/Users/YOURNAME/
# From Linux to Mac (run from the Mac):
# scp user@remote_server:/Users/YOURNAME/filename.txt .

# Download and extract latest rosetta version (3.9 released 4/2/18)
curl -o a.tar.gz  -u Academic_User:****** https://www.rosettacommons.org/downloads/academic/3.9/rosetta_src_3.9_bundle.tgz
tar -xvzf a.tar.gz
cd rosetta_src_2018.09.60072_bundle/main/source
./scons.py -j 96 mode=release bin
#./scons.py -j 96 mode=release cat=test 

# Run rosetta unit tests 
# Need to pass options to tell rosetta which mode 
## --mode=MODE [ 'mode' (eg. debug, release) to pass to scons to identify the platform path.]
##python test/run.py -j 96 --mode=release

# to do 
# note bechmarks for compilation times and testing time 

 

 

Fri, 2018-04-06 23:22
covadiuc

That is definitely a good script for "install Rosetta on a local machine", which will work for a cloud machine you have logged into.  We never had it as a script per se; we kept the package list on a lab wiki.

 

 

sudo apt-get --assume-yes install lib32z1-dev 

That's unlikely to be necessary.

I guess python was already installed if scons worked.  I guess SSH was already there if you logged in.  You should probably install a text editor of your choice.

Sat, 2018-04-07 06:58
smlewis

I have just spent nearly an hour trying to do exactly this...
The command `curl -o a.tar.gz  -u Academic_User:**** https://www.rosettacommons.org/downloads/academic/3.11/rosetta_bin_linux_3.11_bundle.tgz` features  the same path as the site that gives the Basic auth login page, but is different from that found on the site in the downloads, which is `https://www.rosettacommons.org/download.php?token=xxxxx&file=rosetta_bin_linux_3.11_bundle.tgz ` the latter does not do Basic auth like the /downloads route does but requires its cookies (the token is common to all sessions), but did not like my cookies with wget and Python requests  was not doing well with streamed tar.gzip files .

I only found this thread because I Googled "Academic_User Rosetta" (the common username is not secret?). Therefore, I am commenting here to say a few keywords so hopefully searches work better:

"How to /tutorial / guide / instructions on downloading and installing rosetta on a remote cloud cluster / supercomputer / docker container with SSH by means of headless authentication without a browser using licence / license credentials using the command line / bash / terminal"

Fri, 2019-10-25 03:24
matteoferla

I have just spent nearly an hour trying to do exactly this...
The command `curl -o a.tar.gz  -u Academic_User:**** https://www.rosettacommons.org/downloads/academic/3.11/rosetta_bin_linux_3.11_bundle.tgz` features  the same path as the site that gives the Basic auth login page, but is different from that found on the site in the downloads, which is `https://www.rosettacommons.org/download.php?token=xxxxx&file=rosetta_bin_linux_3.11_bundle.tgz ` the latter does not do Basic auth like the /downloads route does but requires its cookies (the token is common to all sessions), but did not like my cookies with wget and Python requests  was not doing well with streamed tar.gzip files .

I only found this thread because I Googled "Academic_User Rosetta" (the common username is not secret?). Therefore, I am commenting here to say a few keywords so hopefully searches work better:

"How to /tutorial / guide / instructions on downloading and installing rosetta on a remote cloud cluster / supercomputer / docker container with SSH by means of headless authentication without a browser using licence / license credentials using the command line / bash / terminal"

Fri, 2019-10-25 03:27
matteoferla