Saturday, August 25, 2012

NGS talks

Whole-Exome Sequencing to Identify Somatic Variants in Cancer - Yardena Samuels(from NHGRI)

Next Generation Sequencing

Determine genetic map position of SNP':
online tool
Conway Institute Bioinformatics Service

An overview of NGS technology and platforms:
http://www.seqwright.com

Blog on NGS technology (www.yuzuki.org) by Dale Yuzuki:

Next Generation Technologist 

How to remove duplicates reads using picard:


java -Xmx2g -jar /usr/local/picard-tools/MarkDuplicates.jar 
                            MAX_FILE_HANDLES_FOR_READ_ENDS_MAP=1000
                            METRICS_FILE=out.metrics 
                            REMOVE_DUPLICATES=true 
                            ASSUME_SORTED=true  
                            VALIDATION_STRINGENCY=LENIENT 
                            INPUT=input.sorted.bam 
                            OUTPUT=out.dedup.bam 
 
BAM file reheader trick (that works):
 
java -jar /home/zhengyuan/picard-tools-1.105/AddOrReplaceReadGroups.jar \
I=test.srt.bam O=test.srt.rhd.bam LB=wzy PL=illumina PU=wzy SM=wzy
 
samtools index  test.srt.rhd.bam
 
Then you can proceed for others tasks like GATK coverage analysis. 
 

NGS alignment software

http://lh3lh3.users.sourceforge.net/NGSalign.shtml

Software

Currently, this page only includes software I am familiar with. Most of them aim for aligning next-generation sequencing (NGS) data and were developed since 2007. I may extend the list when I have time. Several notes:
  • The programs are listed in the alphabet order in each category.
  • Features shown in brackets are optional and may affect efficiency.
  • The version number shown for each program is the one I have checked, but may not be the latest.

Indexing Reads with Hash Tables

  • CloudBurst [PMID:19357099]. RMAP-like algorithm that works in a cloud.
    • Platform: Illumina
    • Features: support cloud computation
    • Availability: open source
  • Cross_match [1.080730]. The latest cross_match has been substantially improved for short read alignment. Its speed is comparable to other aligners and might be the best choice for local alignment.
    • Platform: Illumina; 454
    • Features: gapped alignment (maximum 2 gaps in the fast mode); local alignment
    • Availability: academic free source codes
  • Eland [1.0]. Probably the first short read aligner. Eland substantially influences many aligners in this category and still outperforms many followers. Although it is not the fastest any more, it is close to the fastest and has the smallest memory footprint. Eland itself works for 32bp single-end reads only. Additional Perl scripts in GAPipeline extend its ability.
    • Platform: Illumina
    • Features: PET mapping; mapping quality; SNP caller; counting suboptimal occurrences.
    • Advantages: fast; light-weighted
    • Availability: free source codes for machine buyers.
  • MAQ [0.7.1, PMID: 18714091]. This is my program to align short reads and to call variants. It has been used in several high-profile papers.
    • Platform: Illumina; SOLiD (partial)
    • Features: PET mapping; quality aware; gapped alignment for PET; mapping quality; adapter trimming; partial occurrences counting; SNP caller
    • Advantages: feature rich; publication proved
    • Limitation: up to 128bp reads; no gapped alignment for single-end reads
    • Availability: GPL
  • mrFast/mrsFAST [0.5.1]. An aligner specifically designed for reporting all hits.
    • Platform: Illumina
    • Features: all hits; up to 3X faster than MAQ (not tested by myself); gapped alignment (mrFAST)
    • Availability: Free binary
  • RazerS [20081029, PMID: 19592482]. q-gram filteration; based on the SeqAn library.
    • Availability: free source codes
  • RMAP [0.41, PMID: 18307793;19736251]. One of the earliest short read aligners.
    • Platform: Illumina
    • Features: quality aware; [gapped alignment]; best unique hits
    • Availability: GPL
  • SeqMap [1.0.8, PMID: 18697769]. An Eland-like program.
    • Platform: Illumina
    • Features: [gapped alignment]
    • Limitation: not counting suboptimal hits
    • Availability: GPL
  • SHRiMP [1.10, PMID: 19461883]. Q-gram based algorithm.
    • Platform: SOLiD; Illumina; 454
    • Features: SOLiD mapping; gapped alignment; potential support for mapping quality
    • Limitations: a little slow
    • Availability: GPL
  • ZOOM [1.2.5, PMID: 18684737]. Eland-like algorithm with the improvement of using spaced seed. ZOOM supports longer reads and faster than Eland, although it uses more memory. ZOOM is feature rich, but some features may come at the cost of speed.
    • Platform: Illumina; SOLiD
    • Features: PET mapping; SOLiD mapping; [gapped alignment]; [mapping quality]; [quality aware]
    • Advantage: fast; feature rich
    • Limitation: up to 224bp reads; gapped alignment comes with cost
    • Availability: commercial

Indexing Genome with Hash Tables

  • BFAST [0.3.1, PMID: 19907642].
    • Platform: Illumina; SOLiD
    • Availability: open source
    • Comment on paper: evaluation for bowtie and bwa may be questionable.
  • gnumap [PMID: 19861355].
    • Platform: Illumina
    • Features: "Assigning a proportion of the read to relevant genomic matches based on the relative likelihood that the read maps to each location". (I do not know how this is compared with randomly distribute repetitive reads)
    • Comment on paper: It is possible to achieve paired-end mapping by indexing reads only. Maq does in this way.
    • Availability: open source (?)
  • MOM [0.1, PMID: 19228804].
    • Platform: Illumina; (?)
    • Features: counting suboptimal occurrences; local alignment
    • Availability: free
  • Mosaik [1.0]. Mosaik has been used in several high-profile publications and delivers good performance.
    • Platform: Illumina; 454; SOLiD
    • Advantages: long reads
    • Availability: open source
  • NovoAlign [2.0]. NovoAlign competes with MAQ on speed and feature set, and may be more accurate than MAQ. It also implements several important features missing in MAQ.
    • Platform: Illumina
    • Features: PET mapping; gapped alignment; mapping quality; quality aware; adapter trimming; MAQ format
    • Advantages: highly accurate; gapped alignment; feature rich
    • Requirements: >8GB RAM for paired-end mapping against the human genome.
    • Availability: proprietary; academic free binary (no multi-threading support)
  • PASS [0.5, PMID: 19218350].
    • Platform: Illumina; SOLiD; 454
    • Features: PET mapping
    • Advantages: long reads
    • Requirement: >15GB RAM against human genome
    • Availability: free source codes to academic users
  • PerM [0.1.0, PMID: 19675096]
    • Platform: Illumina; SOLiD
    • Advantages: fast
    • Availability: GPL
    • Limitation: no paired-end mapping apparently.
    • Requirement: 4.5 bytes per reference base
    • Comment on paper: PerM is very fast. The authors attribute its speed to the use of spaced seeds with higher weight. This is a reason, but to me, not the leading reason. I think PerM is fast mainly because in building index, it aligns the genome against itself under given a specified read length; in alignment, PerM aligns a repetitive read once rather than to each copy. The cost is a user needs to build a huge index for each read length.
  • SOAPv1 [1.11, PMID: 18227114]. The first published short read aligner.
    • Platform: Illumina
    • Features: PET mapping; adapter trimming; gapped alignment; SNP caller; counting occurrences
    • Advantages: feature rich
    • Requirements: >14GB RAM against human genome
    • Availability: GPL

Merge Sorting

  • Slider [0.6, PMID: 18974170]. A very clever short read aligner specifically designed for Illumina reads. It is able to use the second best base call, which potentially improves the accuracy on SNP finding.
    • Platform: Illumina
    • Features: Using second base
    • Advantages: fast; potentially more accurate on SNP discovery
    • Requirements: >160GB disk space
    • Availability: free source codes
  • Slider II [1.1].

Indexing Genome with Suffix Array/BWT

  • Bowtie [0.9.9, PMID: 19261174]. This is probably the fastest short read aligner to date. Although under the default option Bowtie does not guarantee to find the best hit or tell if the hit it finds is unique, it is possible to improve this behaviour at the cost of speed.
    • Platform: Illumina
    • Features: partial PET mapping; quality aware; [mapping quality]
    • Advantages: very fast
    • Availability: GPL
  • BWA [0.5.1, PMID: 19451168]. Another aligner written by me. Given high-quality reads, it is an order of magnitude faster than MAQ while achieving similar alignment accuracy.
    • Platform: Illumina; SOLiD; 454; Sanger
    • Features: PET mapping (short reads only); gapped alignment; mapping quality; counting suboptimal occurrences (short reads only); SAM output
    • Advantages: fast
    • Limitations: short read algorithm is slow for long reads and reads with high error rate
    • Availability: GPL
  • SOAPv2 [2.19, PMID: 19497933]. A marvelous program developed by the group who wrote BWT-SW.
    • Platform: Illumina
    • Features: PET mapping; mapping quality; counting occurrences
    • Advantages: fast
    • Availability: academic free binary
  • segemehl [0.0.7, PMID: 19750212].
    • Platform: Illumina
    • Features: accurate
    • Limitation: large memory requirement; no paired-end mapping
    • Comment on paper: the authors show maq and bwa are not as accurate probably because they were counting ambiguous alignments.
  • vmatch [SpringerLink].
    • Availability: academic free binary

Recommendation

First of all, as I am the key developer of two short read aligners (BWA and MAQ), it is really hard for me to give an unbiased evaluation. Please bear this fact in mind when reading through my comments below.
For Illumina reads, I would recommend my program BWA. BWA implements most of the major features of a practical aligner. It is relatively small in memory and highly efficient with little tradeoff on accuracy. BWA outputs alignment in the SAM format. Users may use SAMtools to sort/merge alignments and to make variants calls. One potential concern about BWA is it has not been widely used at the moment. It may be less robust than those publication-proved aligners such as Eland and MAQ.
[Update: With the help of paired-end reads, MAQ is able to find some SNPs at the edge of highly repetitive regions. However, BWA cannot. Nonetheless, I still prefer BWA given its speed and the fact that SNPs that can be called from repeats are rare and more likely to be false positives.]
Mapping inconsistent read pairs with NovoAlign is recommended for PET-based structural varition detection where alignment accuracy is the leading factor on reducing false positive calls. NovoAlign is the most accurate aligner to date.

Thursday, August 23, 2012

Learning Java

Java Programming: Lecture 1
Java Programming: Lecture 2

JEE programming: Lecture 1
JEE programming: Lecture2
JEE programming: Lecture 3 
JEE Programming: Lecture 4a
JEE programming: Lecture 5 
JEE programming: Lecture 6
JEE programming: Lecture 7
JEE programming: Lecture 8
JEE Programming: Lecture 9
JEE Programming: Lecture 10
JEE programming: Lecture 11


OOP in Java: Lecture 3
OOP in Java: Lecture 4
OOP in Java: Lecture 6
OOP in Java: Lecture 7
OOP in Java: Lecture 8
OOP in Java: Lecture 9
OOP in Java: Lecture 12

Client/Server Programming in Java: Lecture 30

Multi-threading in Java
Servlets tutorial: part 3
Data Structure: Lecture 2

 HTML/CSS programming Lecture
OO Programming in C++ (1)
OO Programming in C++ (2)


Wednesday, August 22, 2012

LoginPanel.java

import javax.swing.*;
import javax.swing.border.EmptyBorder;
import java.awt.*;
import java.awt.event.*;
import java.util.*;


/**
 * Title:        Login Panel
 * Description:  A simple yet complete login/logout panel with user callbacks
 *               for approving login attempts and getting notification of logouts.
 * Copyright:    Copyright (c) 2004
 * Company:      Superliminal Software
 * @author Melinda Green
 * @version 1.0
 */

public class LoginPanel extends JPanel {
    public final static String
        LOG_IN  = "Login",
        LOG_OUT = "Logout";
    protected JButton logButt;
    public JButton getLogButton() { return logButt; }
    private final static int DEFAULT_PSWD_CHARS = 10;
    private JTextField nameField = new JTextField(DEFAULT_PSWD_CHARS);
    public String getUserName() { return nameField.getText(); }

    /**
     * override this method to return true if approved, false otherwise.
     * default is true.
     */
    public boolean approveLogin(String uname, String pswd) {
        return true;
    }

    /**
     * override this method to learn about logout events.
     */
    public void loggedOut(String uname) {
    }

    public LoginPanel() {
        this(false);
    }

    public LoginPanel(final boolean clearPasswords) {
        this(clearPasswords, true, null, null);
    }

    /**
     * @param clearPasswords if true, clears password field on successful login.
     * @param initial_user optional default text to load into the 'user' type-in.
     * @param initial_password optional default text to load into the 'password' type-in.
     */
    public LoginPanel(final boolean clearPasswords, final boolean displayFailures, String initial_user, String initial_password) {
        final JPasswordField pswdField = new JPasswordField(DEFAULT_PSWD_CHARS);
        logButt = new JButton(LOG_IN);
        KeyListener quickLogin = new KeyAdapter() {
            public void keyTyped(KeyEvent ke) {
                if(ke.getKeyChar() == KeyEvent.VK_ENTER) {
                    logButt.doClick();
                    logButt.requestFocus();
                }
            }
        };
        nameField.setText(initial_user);
        pswdField.setText(initial_password);
        logButt.setName(LOG_IN);
        nameField.addKeyListener(quickLogin);
        pswdField.addKeyListener(quickLogin);
        // create the grid
        JPanel grid = new JPanel(new GridLayout(2, 2));
        grid.setBackground(new Color(255,255,255));
        grid.add(new JLabel("User Name"));
        grid.add(nameField);
        grid.add(new JLabel("Password"));
        grid.add(pswdField);

        // create login button row
        JPanel row = new JPanel();
        row.setBorder(new EmptyBorder(5, 0, 5, 0));
        row.setOpaque(false);
        row.setLayout(new BoxLayout(row, BoxLayout.X_AXIS));
        row.add(logButt);
        logButt.setBackground(new Color(220,220,220));

        logButt.addActionListener(new ActionListener() {
            public void actionPerformed(ActionEvent ae) {
                if(logButt.getText().equals(LOG_IN)) {
                    // seek login approval from derived class
                    if(approveLogin(nameField.getText(), new String(pswdField.getPassword()))) {
                        // note: must set logout text *before* clearing password
                        // otherwise component dependancy handler will disable the
                        // login button w/out password text before later setting logout text
                        // this closes bug #2336
                        logButt.setText(LOG_OUT);
                        if(clearPasswords)
                            pswdField.setText(null);
                        nameField.setEnabled(false);
                        pswdField.setEnabled(false);
                        fireLoginEvent(nameField.getText(), true);
                    }
                    else
                        if(displayFailures)
                            JOptionPane.showMessageDialog(LoginPanel.this, "Login Denied", "Login Error", JOptionPane.ERROR_MESSAGE);
                }
                else {
                    logButt.setText(LOG_IN);
                    loggedOut(nameField.getText());
                    nameField.setEnabled(true);
                    pswdField.setEnabled(true);
                    fireLoginEvent(nameField.getText(), false);
                }
            }
        });

        // implement component dependancies
        new ComponentDependencyHandler(nameField, pswdField) {
            public void dependencyNotification() {
                String
                    logtext = logButt.getText(),
                    nametext = nameField.getText(),
                    pswdtext = String.copyValueOf(pswdField.getPassword());
                boolean newstate = logtext.equalsIgnoreCase(LOG_OUT) ||
                    (nameField.getText() != null && nametext.length() > 0 // has login text?
                     && pswdtext.length() > 0);  // has password text?
                logButt.setEnabled(newstate);
            }
        };

        // construct final layout
        setLayout(new BoxLayout(this, BoxLayout.Y_AXIS));
        add(grid);
        add(row);
    }

    public interface LoginListener {
        void loggedIn(String uname);
        void loggedOut(String uname);
    }
    public static class LoginAdapter implements LoginListener {
        public void loggedIn(String uname){}
        public void loggedOut(String uname){}
    }
    private Vector loginListeners = new Vector();
    public void addLoginListener(LoginListener ll) { loginListeners.add(ll); }
    public void removeLoginListener(LoginListener ll) { loginListeners.remove(ll); }
    protected void fireLoginEvent(String uname, boolean in) {
        for(Enumeration e=loginListeners.elements(); e.hasMoreElements(); ) {
            LoginListener ll = (LoginListener)e.nextElement();
            if(in)
                ll.loggedIn(uname);
            else
                ll.loggedOut(uname);
        }
    }

    /**
     * simple example test program for LoginPanel class
     */
    public static void main(String[] args) {
        final String NOT_LOGGED_IN = "LoginPanel Test - Currently Logged Out";
        final JFrame frame = new JFrame(NOT_LOGGED_IN);
        frame.getContentPane().add(new LoginPanel() {
            public boolean approveLogin(String uname, String pswd) {
                // this is where to make the server call to approve or reject login attempt
                frame.setTitle("LoginPanel Test - Currently logged in as " + uname);
                return true;
            }
            public void loggedOut(String uname) {
                frame.setTitle(NOT_LOGGED_IN);
            }
        });
        frame.setDefaultCloseOperation(JFrame.EXIT_ON_CLOSE);
        frame.pack();
        frame.setSize(500, frame.getHeight());
        frame.setVisible(true);
    }
}

Tuesday, August 21, 2012

JAVA: Pop up a dialog box for user input

JTextField firstName = new JTextField();
JTextField lastName = new JTextField();
JPasswordField password = new JPasswordField();
final JComponent[] inputs = new JComponent[] {
                new JLabel("First"),
                firstName,
                new JLabel("Last"),
                lastName,
                new JLabel("Password"),
                password};
 
JOptionPane.showMessageDialog(null, inputs, "My custom dialog", JOptionPane.PLAIN_MESSAGE);
System.out.println("You entered " +
                firstName.getText() + ", " +
                lastName.getText() + ", " +
                password.getText());
 
 

Input Dialog Box Program

Here is example code showing a few ways of how to use the showInputDialog method of the JOptionPane class to make input dialog boxes. The program shows a series of input dialog boxes one after the other. The input method they ask the user to use varies from a text field, combo box and list box. The article relating to this code is Building an Input Dialog Box.
//This program shows a series of input dialog boxes.
//The next dialog is launched on the closing of the current dialog. 
//It provides examples of how to create dialog boxes
//with a text field, combo box and list box.

//Imports are listed in full to show what's being used
//could just import javax.swing.* and java.awt.* etc..
import javax.swing.JFrame;
import javax.swing.JTextArea;
import javax.swing.JOptionPane;
import javax.swing.UIManager;
import javax.swing.Icon;
import java.awt.EventQueue;
import java.awt.event.ActionListener;
import java.awt.event.ActionEvent;
import java.lang.reflect.Field;

public class InputDialogFrame extends JFrame{
    
    private JTextArea tracker;
    
    //Using a standard Java icon
    private Icon optionIcon = UIManager.getIcon("FileView.computerIcon");
    
    //Application start point   
    public static void main(String[] args) {
     
     //Use the event dispatch thread for Swing components
     EventQueue.invokeLater(new Runnable()
     {
         public void run()
         {
             //create GUI frame
             new InputDialogFrame().setVisible(true);          
         }
     });
              
    }
    
    public InputDialogFrame()
    {
        //make sure the program exits when the frame closes
        setDefaultCloseOperation(JFrame.EXIT_ON_CLOSE);
        setTitle("Dialog Box Example");
        setSize(500,300);
        
        //This will center the JFrame in the middle of the screen
        setLocationRelativeTo(null);
        
        
        //Using JTextArea to show clicks and responses
        tracker = new JTextArea("Click tracker:");
        add(tracker);
        setVisible(true);
        
        //Options for the combo box dialog
        String[] choices = {"Monday", "Tuesday"
                ,"Wednesday", "Thursday", "Friday"};
        
        //Options for the list dialog
        //There are more than 20 entries to make the showInputDialog method
        //choose to use a list box
        String[] jumboChoices = {"Abe", "Billy", "Colin", "Dexter"
                , "Edward", "Fred", "Gus", "Harry", "Ira", "Jeff"
                , "Kirk", "Larry", "Monty", "Nigel", "Orville", "Paul"
                , "Quint", "Richard", "Steve", "Tony", "Umberto", "Vinnie"
                , "Wade", "Xavier", "Yogi", "Zigmund"};
        
        //Input dialog with a text field
        String input =  JOptionPane.showInputDialog(this 
                ,"Enter in some text:");
        
        TrackResponse(input);
        
        //Input dialog with default text in the text field
        String defaultText =  JOptionPane.showInputDialog(this 
                ,"Enter in some text:","some text..");
        
        TrackResponse(defaultText);
        
        //Input dialog with a textfield, a message type and title
        String warningText =  JOptionPane.showInputDialog(this 
                ,"Erm, enter in a warning:" ,"Warning Message"
                ,JOptionPane.WARNING_MESSAGE);
        
        TrackResponse(warningText);
        
        //If an icon is used then it overrides the icon from the
        //message type. Likewise if a null is entered for the selection values
        //the dialog box will use a text field
        String entered = (String)JOptionPane.showInputDialog(this
                , "Enter a Day of the week:"
                , "Text Field Dialog", JOptionPane.QUESTION_MESSAGE
                , optionIcon, null, null);
        
        TrackResponse(entered);
        
        //If the String Array has more than 20 entries a JList is used
        //as the method the user gets to select a value
        String boyNames = (String)JOptionPane.showInputDialog(this, "Pick a Name:"
                , "ComboBox Dialog", JOptionPane.QUESTION_MESSAGE
                , optionIcon, jumboChoices, jumboChoices[0]);
        
        TrackResponse(boyNames);
        
        //Input dialog with a combo box created by
        //using a String array for the object message. Note how
        //a null icon for the icon results in the QUESTION_MESSAGE
        //message type being used.
        String picked = (String)JOptionPane.showInputDialog(this, "Pick a Day:"
                , "ComboBox Dialog", JOptionPane.QUESTION_MESSAGE
                , null, choices, choices[0]);
        
        TrackResponse(picked);      
    }
    
    //Append the picked choice to the tracker JTextArea
    public void TrackResponse(String response)
    {
        //showInputDialog method returns null if the dialog is exited
        //without an option being chosen
        if (response == null)
        {
            tracker.append("\nYou closed the dialog without any input..");
        }
        else
        {
            tracker.append("\nYou picked " + response + "..");
        }
    }
}