Setup Galaxy production server on Ubuntu and Apache environment

1.) Create user called galaxy with password galaxy

 admin@myserver:~# adduser galaxy  
 Adding user `galaxy' ...  
 Adding new group `galaxy' (1007) ...  
 Adding new user `galaxy' (1008) with group `galaxy' ...  
 Creating home directory `/home/galaxy' ...  
 Copying files from `/etc/skel' ...  
 Enter new UNIX password:  
 Retype new UNIX password:  
 passwd: password updated successfully  
 Changing the user information for galaxy  
 Enter the new value, or press ENTER for the default  
     Full Name []:  
     Room Number []:  
     Work Phone []:  
     Home Phone []:  
     Other []:  
 Is the information correct? [Y/n] Y  
 admin@myserver:~#  

2.) Change user to galaxy and clone galaxy production version

 admin@myserver: su galaxy  
 galaxy@myserver:/home/admin$ cd   
 galaxy@myserver: hg clone https://bitbucket.org/galaxy/galaxy-dist  

3.) If you don't have mercurial client for clone galaxy use following step

 admin@myserver:sudo apt-get install mercurial  

4.) Set the $TEMP environment variable to Galaxy's new_files_path directory

 galaxy@myserver:~$ export TEMP=/home/galaxy/galaxy-dist/database/tmp  

5.) We need clean python interpreter with correct python path

 galaxy@myserver:wget http://bitbucket.org/ianb/virtualenv/raw/tip/virtualenv.py  
 galaxy@myserver:/usr/bin/python2.6 virtualenv.py --no-site-packages galaxy_env  

6.) Now we need to setup new database for galaxy.I am going to create PostgreSQL database

 galaxy@myserver:~/galaxy-dist$ psql -h localhost -d postgres -U postgres  
 postgres=#CREATE DATABASE galaxy_prod;  
 postgres=# CREATE USER galaxy_prod_user WITH PASSWORD 'galaxy';  
 postgres=# GRANT ALL PRIVILEGES ON DATABASE galaxy_prod to galaxy_prod_user;  
 postgres=# \q  

7.) Then we need to configure Galaxy default server settings to our server details

 galaxy@myserver:cd galaxy-dist/  
 galaxy@myserver:~/galaxy-dist$ chmod -R 777 universe_wsgi.ini  
 galaxy@myserver:~/galaxy-dist$ vi universe_wsgi.ini  

8.) Here is the basic changes for universe_wsgi.ini file.

 host = xxx.xxx.23.123 [IP ADDRESS]  
 debug = False  
 use_interactive = False  
 database_connection = postgres://galaxy_prod_user:galaxy@localhost:5432/galaxy_prod  

9.) There are many more changes we can do for galaxy by customizing niverse_wsgi.ini for instance adding tracks,user privileges, ftp upload e.t.c.Galaxy has its own server but there are pages with static contents therefore we can setup proxy to enhance efficiency

 admin@myserver:vi /etc/httpd/conf/httpd.conf  

10.) Add following lines to httpd.conf

 <VirtualHost *:80>  
 ServerName xxx.xxx.23.123 [IP ADDRESS]  
 RewriteEngine on  
 #RewriteLog "/etc/httpd/logs/rewrite_log"  
 #RewriteLogLevel 9  
 RewriteRule ^/galaxy$ /galaxy/ [R]  
 #RewriteRule ^/galaxy/static/style/(.*) /home/galaxy/galaxy-dist/static/june_2007_style/blue/$1 [L]  
 #RewriteRule ^/galaxy/static/scripts/(.*) /home/galaxy/galaxy-dist/static/scripts/packed/$1 [L]  
 #RewriteRule ^/galaxy/static/(.*) /home/galaxy/galaxy-dist/static/$1 [L]  
 #RewriteRule ^/galaxy/favicon.ico /home/galaxy/galaxy-dist/static/favicon.ico [L]  
 #RewriteRule ^/galaxy/robots.txt /home/galaxy/galaxy-dist/static/robots.txt [L]  
 RewriteRule ^/galaxy(.*) http://localhost:8080$1 [P]  
 </VirtualHost>  

11.) Now we need to restart proxy server.

 admin@myserver:/etc/init.d/httpd restart  

12.) Finally we can run galaxy

 galaxy@myserver:~/galaxy-dist$ sh ./run.sh --daemon  

13.) We can stop or see the status by using following commands

 galaxy@myserver:~/galaxy-dist$ sh ./run.sh --stop-daemon  
 galaxy@myserver:~/galaxy-dist$ sh ./run.sh --status  

14.) Done! 

Ubuntu folder/file permission

chmod -R 754 /www/test

7 – Owner(current user)
5 – Group(set by owner)
4 – anyone else

Basic numbers make persmission settings as follows

0 – no permission, this person cannot read, write or execute
1 – execute only
2 – write only
3 – execute and write only (1 + 2)
4 – read only
5 – execute and read only (1 + 4)
6 – write and read only (2 + 4)
7 – execute, write and read (1 + 2 + 3)

Log calculator

The log base

of

Install ProFTP server for Galaxy

Its not wise to use user sudo apt-get install proftpd or proftpd-basic because there are no postgres modules in default package.but if you have GUI interface udo apt-get install gadmin-proftpd is important
chanaka@localhost:~# wget ftp://ftp.proftpd.org/distrib/source/proftpd-1.3.4a.tar.gz
chanaka@localhost:~# tar xfvz proftpd-1.3.4a.tar.gz
chanaka@localhost:~/proftpd-1.3.4a#
chanaka@localhost:~/proftpd-1.3.4a# ./configure --disable-auth-file --disable-ncurses               \
        --disable-ident --disable-shadow --enable-openssl           \
        --with-modules=mod_sql:mod_sql_postgres:mod_sql_passwd
chanaka@localhost:~/proftpd-1.3.4a# make
chanaka@localhost:~/proftpd-1.3.4a# make install

Then create PostgreSQL user called galaxyftp with least privileges.
chanaka@localhost:~$  createuser -SDR galaxyftp
chanaka@localhost:~$ psql galaxy_prod
psql (8.4.9)
Type "help" for help.

galaxy_prod=# ALTER ROLE galaxyftp PASSWORD '[password]';
ALTER ROLE
galaxy_prod=# GRANT SELECT ON galaxy_user TO galaxyftp;
GRANT
galaxy_prod=# \q

Here is the final proftd.conf (/usr/local/proftpd/proftpd.conf)
# Basics, some site-specific
ServerName                      "myserver.org"
ServerType                      standalone
DefaultServer                   on
Port                            21
Umask                           077
SyslogFacility                  DAEMON
SyslogLevel                     debug
MaxInstances                    30
User                            nobody
Group                           nogroup
DisplayConnect                  /ftp/proftpd_welcome.txt
# Passive port range for the firewall
PassivePorts                    30000 40000
# Cause every FTP user to be "jailed" (chrooted) into their home directory
DefaultRoot                     ~
# Automatically create home directory if it doesn't exist
CreateHome                      on dirmode 700
# Allow users to overwrite their files
AllowOverwrite                  on
# Allow users to resume interrupted uploads
AllowStoreRestart               on
# Bar use of SITE CHMOD
<Limit SITE_CHMOD>
  DenyAll
</Limit>
# Bar use of RETR (download) since this is not a public file drop
<Limit RETR>
  DenyAll
</Limit>
# Do not authenticate against real (system) users
<IfModule mod_auth_pam.c>
AuthPAM                         off
</IfModule>
# Set up mod_sql_password - Galaxy passwords are stored as hex-encoded SHA1
SQLPasswordEngine               on
SQLPasswordEncoding             hex
# Set up mod_sql to authenticate against the Galaxy database
SQLEngine                       on
SQLBackend                      postgres
SQLConnectInfo                  galaxy_prod@myserver.org galaxyftp [password]
SQLAuthTypes                    SHA1
SQLAuthenticate                 users
# An empty directory in case chroot fails
SQLDefaultHomedir               /ftp/proftpd
# Define a custom query for lookup that returns a passwd-like entry.  UID and GID should match your Galaxy user.
SQLUserInfo                     custom:/LookupGalaxyUser
SQLNamedQuery                   LookupGalaxyUser SELECT "email,password,'512','512','/ftp/%U','/bin/bash' FROM galaxy_user WHERE email='%U'"

Note! Don't forget to configure  pg_hba.conf entry for host "XXXXX", Also check the firewall for ftp usually 21,We need to use galaxy user like 512. Done!

We can see the errors by using following
chanaka@localhost:~# cd /usr/local/sbin
chanaka@localhost:~/usr/local/sbin# ./proftpd -nd9
or check PostgreSQL or proftpd log files

If you get "Fatal: unknown configuration directive 'AuthPAM' " error use following solution.

<IfModule mod_auth_pam.c>
AuthPAM                         off
</IfModule>

Also following lines may useful to you:(/usr/local/sbin)
chanaka@localhost:~#ps aux | grep proftpd
chanaka@localhost:~#Kill [pid]
chanaka@localhost:~#Where is proftpd

To start
sudo /etc/init.d/proftpd start

Add multiple columns to Advanced Data Grid

Since three days I was trying to add multiple columns to Flex ADG Finally end up with one line solution amazing!
here is the 72 hours work :)
//tl add the column number
ADG.validateNow();

thanks to http://riaflex.blogspot.com/2008/05/dynamic-advanceddatagrid-columns.html

Install Redmine on Ubuntu server

First Install the following packages.

sudo apt-get install build-essential ruby ruby1.8-dev libopenssl-ruby libmysql-ruby rubygems rake

Install rails.

sudo gem install rails -v=2.3.5

Then create symbolic link to rails.

sudo ln -s /var/lib/gems/1.8/bin/rails /usr/bin/rails

Go to www directory and checkout redmine from SVN.

cd /var/www
svn checkout http://redmine.rubyforge.org/svn/trunk redmine

Copy config file.

sudo cp redmine/config/database.yml.example redmine/config/database.yml

Change database configuration.

sudo nano redmine/config/database.yml
production:
adapter: mysql
database: redmine
host: localhost
username: redmine
password: redmine's password
encoding: utf8

Optional to change email configuration.

sudo cp redmine/config/email.yml.example redmine/config/email.yml
sudo nano redmine/config/email.yml
production:
delivery_method: :sendmail

Then go to redmine directory and generate a session store secret.

cd redmine
RAILS_ENV=production rake config/initializers/session_store.rb

Create database structure.

RAILS_ENV=production rake db:migrate

Add default configuration data to database.

RAILS_ENV=production rake redmine:load_default_data

Now run Redmine for firs time temporally.

ruby script/server webrick -e production

Now we can access Redmine at http://localhost:3000
Use following commands to running redmine on apache and Enable mod proxy and rewrite.

sudo a2enmod proxy
sudo a2enmod rewrite

Restart apache and install mongrel.

sudo /etc/init.d/apache2 restart
sudo apt-get install mongrel

Create folder mongrel inside /var/log.

sudo mkdir /var/log/mongrel

When you are in redmine directory Start mongrel.

sudo mongrel_rails start -d -p 3000 -e production -P /var/log/mongrel/mongrel-1.pid -l /var/log/mongrel/mongrel.log

Access redmine at http://localhost:3000
Create apache’s virtual host.

sudo nano /etc/apache2/sites-available/redmine

Add following proxy configuration.

<virtualhost *:80>
ServerName http://localhost
ProxyPass / http://localhost:3000/
ProxyPassReverse / http://localhost:3000
ProxyPreserveHost on
<proxy *>
Order allow,deny
Allow from all
</Proxy>
</VirtualHost>

Create symbolic link.

sudo ln -s /etc/apache2/sites-available/redmine /etc/apache2/sites-enabled/redmine

Restart apache.

sudo /etc/init.d/apache2 restart

Access redmine at http://localhost:3000
If you need to stop mongrel, run the following command inside redmine directory.

sudo mongrel_rails mongrel::stop -P /var/log/mongrel/mongrel-1.pid

Possible errors and solutions
To see the running rubby sessions

lsof|grep 3000

This will give you a line starting with.

ruby 6205 admin 4u IPv4

If you need destroy the session

kill -9 6205

If you see followwing error

** !!! PID file /var/log/mongrel/mongrel-1.pid already exists. Mongrel could be running already. Check your /var/log/mongrel/mongrel.log for errors.
** !!! Exiting with error. You must stop mongrel and clear the .pid before I'll attempt a start.

Use following command or destroy session using above step.

rm /var/log/mongrel/mongrel-1.pid
rm /var/log/mongrel/mongrel.log

Thank goes to Just Notes

MySQL to PostgreSQL migration

According to common sense mysql to postgre migration should be work with dump mysql script to file (A) and import script(A) to postgre but its not that simple because mysql and postgre has different syntax.so we need to change syntax before import sql file to postgre.There are perl scripts which will help us to change mysql dump file to postgre dump file we can use that wrapper file to change syntax bulk operation.

perl mysql2perl.perl mysqldump.sql(A) postgredump.sql(B)
now we can import compatible postgre script to postgre databse.

therefor abstract of above operations as follows,
Psuedo,

mysqldump --compatible=postgresql -u username -ppassword databse> mysqldump.sql(A);
perl mysql2pg.perl mysqldump.sql(A) postgredump.sql(B);
psql -h localhost -d postgredatabse -U postgres -W < postgredump.sql(B);

here you can find the mysql2pg.perl script,

Backups
Postgres backup

pg_dump -h localhost - drupaldb -U postgres > /file path/file.sql