GBrowse Bio::DB::BigFile


wget http://hgdownload.cse.ucsc.edu/admin/jksrc.zip
unzip jksrc.zip
cd kent/src/lib
echo $MACHTYPE
x86_64
export MACHTYPE=x86_64
make CXXFLAGS=-fPIC CFLAGS=-fPIC CPPFLAGS=-fPIC
cd..
export KENT_SRC=`pwd`
sudo perl -MCPAN -e 'install Bio::DB::BigFile'

Simple awk function to read file line by line and merge columns using unique id

Here is the input:

MA_9270965g0010 PF00010 Helix-loop-helix DNA-binding domain
MA_10437060g0010        PF00082;PF05922 Subtilase family;Peptidase inhibitor I9



Here is the function:
#Start reading file line by line using tab field seperator
awk 'BEGIN{FS="\t"}{
 #split second and third column by ";"
 second_field_array_length=split($2,second_field_array,";");
 third_field_array_length=split($3,third_field_array,";");
 concat_str="";
 #Loop the second column array and merge with 3rd column consecutive element
 for(i=1;i<=second_field_array_length;i++){
  #concat with ";" when count is greater than 0
  if(i>1){
   concat_str=concat_str";"second_field_array[i]"-"third_field_array[i]
  }else{
   concat_str=second_field_array[i]"-"third_field_array[i]
  }
 }
print $1,concat_str
}' Pabies1.0-Pfam-update.txt | head


Here is the output:

MA_9270965g0010 PF00010-Helix-loop-helix DNA-binding domain
MA_10437060g0010 PF00082-Subtilase family;PF05922-Peptidase inhibitor I9


Galaxy Bowttie/BWA/BLAST tools/dataset restricted access per user

Add following line to bowtie_wrapper.xml/bwa_wrapper.xml 
--email=$__user_email__ 

Add following lines to bowtie_wrapper.py/bwa_wrapper.py 
parser.add_option( '', '--email', dest='email', help='email' ) 

Above line should add before the options, args) = parser.parse_args() line After the stdout = '' add following lines.
if options.ref in "[dataset paths]" :
  if options.email in "[email addresses]":
    sys.stdout.write( 'Permission granted! ' )
  else:
    stop_err( 'You dont have permissions!' ) 
else: 
 sys.stdout.write( 'Permission granted! ' ) 

Retart the Galaxy! Now you can restrcit tool datasets per user without having two Galaxy instances.

/Chanaka

Symlink all files from a base directory to a target directory

for f in $(ls -d /base/*); do ln -s $f /target; done && ls -al /target

Install Perl from source

 cd /opt

 wget http://www.cpan.org/src/5.0/perl-5.14.2.tar.gz

 tar -zxf perl-5.14.2.tar.gz

 cd perl-5.14.2

 ./Configure -h

 ./Configure -Dusethreads -Dprefix=/usr

 make

 make test

 make install

 perl -V

 which perl 

To see the status of perl installation 

make install > install.log

To install Perl modules using CPAN:

perl -MCPAN -e shell

Then install lets say if we want to install BioGraphics module.

install Bio::Graphics

Usefull command for test running jobs

fg - run jobs in the foreground

May be you will need to install following dependencies

sudo apt-get install libreadline-dev

Python BeautifulSoup scraper script

Beautiful Soup written in Python.Which is  HTML / XML parser, it can handle non-standard tags and generate inside parse tree. Also provides a simple and commonly used in navigation , search and modify the operation.
Recently i tried BeautifuSoup module inorder to parse html class libraries, Here is the simple scrapy code for extracting informaton from Apple website.

 #! /usr/bin/python  
 print 'Content-type: text/plain\r\n'  
 from BeautifulSoup import BeautifulSoup   
 import urllib   
 webpage = urllib.urlopen(r"http://store.apple.com/us/browse/home/shop_iphone/family/iphone/iphone4s");   
 soup = BeautifulSoup(webpage.read())   
 tags = soup('ul',{'class':'selection-options all-models'})   
 tags = tags[0](lambda tag : len(tag.attrs) == 1 and tag.name in ['span'] and   
            tag['class'] in ['shipping','price','color','title'])   
 for tag in tags :   
   print tag.text   
   print '-' * 30   

Results:

16GB2
------------------------------
black
------------------------------
From$199
------------------------------
In Stock
------------------------------
16GB2
------------------------------
black
------------------------------
From$199
------------------------------
In Stock

Setup Galaxy production server on Ubuntu and Apache environment

1.) Create user called galaxy with password galaxy

 admin@myserver:~# adduser galaxy  
 Adding user `galaxy' ...  
 Adding new group `galaxy' (1007) ...  
 Adding new user `galaxy' (1008) with group `galaxy' ...  
 Creating home directory `/home/galaxy' ...  
 Copying files from `/etc/skel' ...  
 Enter new UNIX password:  
 Retype new UNIX password:  
 passwd: password updated successfully  
 Changing the user information for galaxy  
 Enter the new value, or press ENTER for the default  
     Full Name []:  
     Room Number []:  
     Work Phone []:  
     Home Phone []:  
     Other []:  
 Is the information correct? [Y/n] Y  
 admin@myserver:~#  

2.) Change user to galaxy and clone galaxy production version

 admin@myserver: su galaxy  
 galaxy@myserver:/home/admin$ cd   
 galaxy@myserver: hg clone https://bitbucket.org/galaxy/galaxy-dist  

3.) If you don't have mercurial client for clone galaxy use following step

 admin@myserver:sudo apt-get install mercurial  

4.) Set the $TEMP environment variable to Galaxy's new_files_path directory

 galaxy@myserver:~$ export TEMP=/home/galaxy/galaxy-dist/database/tmp  

5.) We need clean python interpreter with correct python path

 galaxy@myserver:wget http://bitbucket.org/ianb/virtualenv/raw/tip/virtualenv.py  
 galaxy@myserver:/usr/bin/python2.6 virtualenv.py --no-site-packages galaxy_env  

6.) Now we need to setup new database for galaxy.I am going to create PostgreSQL database

 galaxy@myserver:~/galaxy-dist$ psql -h localhost -d postgres -U postgres  
 postgres=#CREATE DATABASE galaxy_prod;  
 postgres=# CREATE USER galaxy_prod_user WITH PASSWORD 'galaxy';  
 postgres=# GRANT ALL PRIVILEGES ON DATABASE galaxy_prod to galaxy_prod_user;  
 postgres=# \q  

7.) Then we need to configure Galaxy default server settings to our server details

 galaxy@myserver:cd galaxy-dist/  
 galaxy@myserver:~/galaxy-dist$ chmod -R 777 universe_wsgi.ini  
 galaxy@myserver:~/galaxy-dist$ vi universe_wsgi.ini  

8.) Here is the basic changes for universe_wsgi.ini file.

 host = xxx.xxx.23.123 [IP ADDRESS]  
 debug = False  
 use_interactive = False  
 database_connection = postgres://galaxy_prod_user:galaxy@localhost:5432/galaxy_prod  

9.) There are many more changes we can do for galaxy by customizing niverse_wsgi.ini for instance adding tracks,user privileges, ftp upload e.t.c.Galaxy has its own server but there are pages with static contents therefore we can setup proxy to enhance efficiency

 admin@myserver:vi /etc/httpd/conf/httpd.conf  

10.) Add following lines to httpd.conf

 <VirtualHost *:80>  
 ServerName xxx.xxx.23.123 [IP ADDRESS]  
 RewriteEngine on  
 #RewriteLog "/etc/httpd/logs/rewrite_log"  
 #RewriteLogLevel 9  
 RewriteRule ^/galaxy$ /galaxy/ [R]  
 #RewriteRule ^/galaxy/static/style/(.*) /home/galaxy/galaxy-dist/static/june_2007_style/blue/$1 [L]  
 #RewriteRule ^/galaxy/static/scripts/(.*) /home/galaxy/galaxy-dist/static/scripts/packed/$1 [L]  
 #RewriteRule ^/galaxy/static/(.*) /home/galaxy/galaxy-dist/static/$1 [L]  
 #RewriteRule ^/galaxy/favicon.ico /home/galaxy/galaxy-dist/static/favicon.ico [L]  
 #RewriteRule ^/galaxy/robots.txt /home/galaxy/galaxy-dist/static/robots.txt [L]  
 RewriteRule ^/galaxy(.*) http://localhost:8080$1 [P]  
 </VirtualHost>  

11.) Now we need to restart proxy server.

 admin@myserver:/etc/init.d/httpd restart  

12.) Finally we can run galaxy

 galaxy@myserver:~/galaxy-dist$ sh ./run.sh --daemon  

13.) We can stop or see the status by using following commands

 galaxy@myserver:~/galaxy-dist$ sh ./run.sh --stop-daemon  
 galaxy@myserver:~/galaxy-dist$ sh ./run.sh --status  

14.) Done!