Monday, March 23, 2015

php-LDAP-admin

Logo.jpg


Web-based LDAP browser to manage your LDAP server 

WIKI Updators: Due to the increased SPAM on this site, I'm slowly locking down pages so that pages cant be updated unless you are a registered user. I am also implementing account creation confirmation. If you want to update the WIKI, please do so - I'm happy to give access to anybody who helps keep this site up to date. Just apply for access, and if I havent given it to you in a timely fashion, just remind me via a mailling list, or send me an email wurley-at-users.sf.net (via sourceforge).
IMPORTANT NOTE: A security vulnerability has been reported in phpLDAPadmin 1.2.x (x<2), whereby a user can inject PHP code and exploit some XSS. Fixes have been released and included in PLA 1.2.2.
I recommend you update to the latest version of PLA 1.2, either download it directly from sourceforge or encourage your OS distribution to make the latest 1.2.x version (if they dont already have it).
IMPORTANT NOTE: A security vulnerability has been reported in phpLDAPadmin 1.1.x, whereby a user can use a null terminated URL to view the contents of files on your server (eg: /etc/passwd). This vulnerability is confirmed in 1.1.0.7 and probably exists in previous releases. It does not appear to affect 1.2.x.

About phpLDAPadmin

phpLDAPadmin (also known as PLA) is a web-based LDAP client. It provides easy, anywhere-accessible, multi-language administration for your LDAP server.
Its hierarchical tree-viewer and advanced search functionality make it intuitive to browse and administer your LDAP directory. Since it is a web application, this LDAP browser works on many platforms, making your LDAP server easily manageable from any location.
phpLDAPadmin is the perfect LDAP browser for the LDAP professional and novice alike. Its user base consists mostly of LDAP administration professionals.
You might also like to see About.

Features

  • LDAP tree browser (see screenshots)
  • Template-based entry editing
  • Copy LDAP entries (even copy between different servers)
  • Recursively copy entire trees
  • Delete LDAP entries
  • Recursively delete entire trees
  • View and edit image attributes (like jpegPhoto)
  • Advanced LDAP schema browser
  • Template-based entry creation
  • LDAP searches (both simple and advanced)
  • LDIF and DSML export
  • LDIF import
  • Rename LDAP entries
  • Manage user password hashes (supports sha, crypt, md5, blowfish, md5crypt)
  • Browse RootDSE LDAP information
  • Automagically determine your LDAP server's root DN
  • Configurable authentication (anonymous, web login, or static)
  • Friendly attribute translation (ie, display "Fax" instead of "facsimileTelephoneNumber")
  • Binary attribute support
  • Configurable support for aliases and referrals
  • Automatically increment UID numbers
  • Hyperlinks for mailto and href attribute values.
  • Configurable read-only and read/write modes.
  • Robust i18n support
  • Available in 10 languages
  • Extendible. (we even provide a function reference)
  • Request a new feature
Manage Group MembershipServer InfoSearch Results

Linux Professional Institute (LPI) exam prep: Overview

Before you take the Linux Professional Institute (LPI) exams, review these developerWorks articles and tutorials, designed as self-study guides for each topic in the exams.

Start here to build your Linux skills

Introduce yourself to Linux and advance your proficiency through a spectrum of self-paced articles and tutorials. Use them to build fundamental skills on Linux system administration at your level of expertise:
See all LPI-oriented articles and tutorials on developerWorks.

And prepare for LPI certification

The Linux Professional Institute (LPI) certifies Linux® system administrators at three levels:
To attain certification level 1, you must pass LPIC-1 exams 101 and 102.
To attain certification level 2, you must pass LPIC-2 exams 201 and 202.
To attain certification level 3, you must pass LPIC-3 exam 301 ("core") and have an active certification level 2. You may also need to pass additional specialty exams at the senior level.
Before you take the exams, review the LPI-oriented articles and tutorials on developerWorks, designed as study guides for each exam topic.

Our readers say...

We're happy to hear our self-study material is helpful, whether your aim is attaining certification or simply polishing your job skills. Here's what our readers are saying about the LPIC-1 series:
"This series by Ian Shields has become one of the major focal points of my exam preparation. Thanks for the opportunity."
"I support Domino on Linux and this is very useful information to me. I am self-taught with Linux, so learning through a self-study resource like this is completely natural."
"Overall, excellent information presented. A great deal of valuable information that was very useful to me that was not presented in the tldp.org HOWTOs - e.g. more emphasis on the theory and reasons for doing it a certain way. Thank you."
"Good, concise intro to basic linux sysadmin skills."
"Thanks a bunch for writing this stuff. It'll help me get certified, for sure."
"I like how this guy does tutorials. They aren't dry and thus I am able to concentrate on them."
"I can't believe this material can be delivered any easier.... I found it easy to understand and it gives good examples. A+"

About the authors

The LPIC-1 articles and tutorials (for exams 101 and 102) are authored by Ian Shields. The LPIC-2 tutorials (for exams 201 and 202) are authored by David Mertz and Brad Huntting, individually or together. The LPIC-3 tutorials (for exam 301) are authored by Sean Walberg.
  • Ian Shields, Ph.D. works on a multitude of Linux projects for the developerWorks Linux zone. He is a Senior Programmer at IBM at the Research Triangle Park, NC. He joined IBM in Canberra, Australia, as a Systems Engineer in 1973, and has since worked on communications systems and pervasive computing in Montreal, Canada, and RTP, NC. He has several patents and has published several papers. His undergraduate degree is in pure mathematics and philosophy from the Australian National University. He has an M.S. and Ph.D. in computer science from North Carolina State University. Browse his published papers and other interests at his personal Web page. You can contact Ian at ishields@us.ibm.com and through his profile on developerWorks.
  • David Mertz, Ph.D. is Turing complete, but probably would not pass the Turing Test. For details on his life, see his personal Web page. He's been writing the developerWorks columns Charming Python and XML Matters since 2000. Check out his book Text Processing in Python. You can contact David at mertz@gnosis.cx.
  • Brad Huntting has been doing UNIX systems administration and network engineering for about 14 years at several companies. He is currently working on a Ph.D. in Applied Mathematics at the University of Colorado in Boulder, and pays the bills by doing UNIX support for the Computer Science department. You can contact Brad at huntting@glarp.com.
  • Sean Walberg has been working with Linux and UNIX since 1994 in academic, corporate, and Internet service provider environments. He has written extensively about systems administration over the past several years. You can contact Sean at sean@ertw.com.

About the LPI certification program

The LPI certification program is designed to certify the competency of system administration using the Linux operating system and its associated tools. It is designed to be distribution neutral, following the Linux Standard Base and other relevant standards and conventions. The Linux Professional Institute does not endorse any third-party exam preparation material or techniques in particular. For details, please contact info@lpi.org.

Linux Professional Institute (LPI) exam prep: LPIC1-exam prep

Introduce yourself to Linux, and advance your proficiency, through a spectrum of self-paced tutorials. With these tutorials, you can build fundamental skills in Linux systems administration at your level of expertise.
Exam 101 is the first of two junior-level system administrator certification exams offered by the Linux Professional Institute (LPI). Both exam 101 and exam 102 are required for junior-level certification, or LPIC-1.
New developerWorks articles corresponding to the April 2009 objectives for exam 101 and exam 102 are in progress. The developerWorks articles below are part of the new series. They help you prepare for the topics in LPI exam 101:
  • LPI exam 101 prep: Hardware and architecture
    Topic 101. Learn to configure your system hardware with Linux. By the end of this tutorial, you will know how Linux configures the hardware found on a modern PC and where to look if you have problems.
  • LPI exam 101 prep: Linux installation and package management
    Topic 102. In the five new articles for this topic, covering the latest (April 2009) LPI exam 101 objectives, learn how to design a hard disk layout, install a boot manager, manage shared libraries, use Debian package management, and use RPM and YUM package management.
  • LPI exam 101 prep: GNU and UNIX commands
    Topic 103. In the eight new articles for this topic, covering the latest (April 2009) LPI exam 101 objectives, learn how to work on the command line; process text streams using filters; perform basic file and directory management; use streams, pipes, and redirects; create, monitor, and kill processes; modify process execution priorities; search text files using regular expressions; and perform basic file editing operations using vi.
  • LPI exam 101 prep: Devices, Linux filesystems, and FHS
    Topic 104. Get acquainted with Linux devices, filesystems, and the Filesystem Hierarchy Standard. By the end of this tutorial, you will know how to create and format partitions with different Linux filesystems and how to manage and maintain those systems.
See all LPI exam-prep tutorials on developerWorks.

Original link:
http://www.ibm.com/developerworks/linux/lpi/101.html

Friday, March 20, 2015

Using cURL for FTP over SSL File Transfers

I recently helped a client work through some errors while trying to transfer a file over a secure FTP connection (FTP over SSL) with cURL. If you haven't used curl, it is a great tool that lends itself to scripted data transfers quite nicely. I'll quote from the curl website:
curl is a command line tool for transferring files with URL syntax, supporting FTP, FTPS, HTTP, HTTPS, SCP, SFTP, TFTP, TELNET, DICT, LDAP, LDAPS and FILE. curl supports SSL certificates, HTTP POST, HTTP PUT, FTP uploading, HTTP form based upload, proxies, cookies, user+password authentication (Basic, Digest, NTLM, Negotiate, kerberos), file transfer resume, proxy tunneling and a busload of other useful tricks.
Anyway, using curl with FTP over SSL is usually done something like this: curl -3 -v --cacert /etc/ssl/certs/cert.pem \ --ftp-ssl -T "/file/to/upload/file.txt" \ ftp://user:pass@ftp.example.com:port Let's go over these options:
  • -3: Force the use of SSL v3.
  • -v: Gives verbose debugging output. Lines starting with '>' mean data sent by curl. Lines starting with '<' show data received by curl. Lines starting with '*' display additional information presented by curl.
  • cacert: Specifies which file contains the SSL certificate(s) used to verify the server. This file must be in PEM format.
  • ftp-ssl: Try to use SSL or TLS for the FTP connection. If the server does not support SSL/TLS, curl will fallback to unencrypted FTP.
  • -T: Specifies a file to upload
The last part of the command line ftp://user:pass@ftp.example.com:port is simply a way to specify the username, password, host and port all in one shot.

How FTP Works

Before I get to the problem, I need to explain a bit about how FTP works. FTP operates in one of two modes - active or passive. In active mode, the client connects to the server on a control port (usually TCP port 21), then starts listening on a random high port and sends this port number back to the server. The server then connects back to the client on the specified port (usually the server's source TCP port is 20). Active mode isn't used much or even recommended anymore, since the reverse connection from the server to the client is frequently blocked, and can be a security risk if not handled properly by intervening firewalls. Contrast this with passive mode, in which the client makes an initial connection to the server on the control port, then waits for the server to send an IP address and port number. The client connects to the specified IP address and port and then sends the data. From a firewall's perspective, this is much nicer, since the control and data connections are in the same direction and the ports are well-defined. Most FTP clients now default to passive mode, curl included.

The Problem

Now, a problem can arise when the server sends back the IP address from a passive mode request. If the server is not configured properly, it will send back it's own host IP address, which is almost always a private IP address and different from the address the client connected to. Usually a firewall or router is doing Network Address Translation (NAT) to map requests from the server's public IP address to the server's internal IP address. When the client gets this IP address from the server, it is trying to connect to a non-routable IP address and the connection times out. How do you know when this problem has manifested itself? Take a look at this partial debug output from curl:
... > PASV < 227 Entering Passive Mode (172,19,2,90,41,20) * Trying 172.19.2.90... Here the client has sent the PASV command, which asks the server for a passive data connection. The server returns a string of six decimal numbers, representing the IP address (first four digits) and port (last two digits). Here the IP address is 172.19.2.90 - a non-routable IP address as per RFC 1918. When the client tries to connect to this address, it will fail.

The Solution...Sort of

In 1998 RFC 2428 was released, which specified 'Extended Passive Mode', specifically meant to address this problem. In extended passive mode, only the port is returned to the client, the client assumes the IP address of the server has not changed. The problem with this solution is that many FTP servers still do not support extended passive mode. If you try, you will see something like this:
> EPSV * Connect data stream passively < 500 'EPSV': command not understood. * disabling EPSV usage > PASV < 227 Entering Passive Mode (172,19,2,90,41,20) * Trying 172.19.2.90... ...and we're back to the same problem again.

The Real Solution

Curl has a neat solution to this problem, requiring two additional options. The first is --disable-epsv, which prevents curl from sending the EPSV command - it will just default to standard passive mode. The second is --ftp-skip-pasv-ip, which tells curl to ignore the IP address returned by the server, and to connect back to the server IP address specified in the command line. Let's put it all together:
curl -3 -v --cacert /etc/ssl/certs/cert.pem \ --disable-epsv --ftp-skip-pasv-ip \ --ftp-ssl -T "/file/to/upload/file.txt" \ ftp://user:pass@ftp.example.com:port If this succeeds, you'll see something like this:
* SSL certificate verify ok. ... < 226- Transfer complete - acknowledgment message is pending. < 226 Transfer complete. > QUIT < 221 Goodbye. The final 226 Transfer complete is the sign that the file was transferred to the server successfully.

Five Minutes to an Even More Secure SSH


lock
This is a followup article to Five Minutes to a More Secure SSH. My impetus for writing the original was seeing too many client's servers left in a default state where they are vulnerable to brute-force attacks. In it, I basically advocate three things:
  • Disabling password authentication
  • Disabling root login
  • Enabling key-based authentication
Those recommendations still hold true and I would encourage you to follow them. However, OpenSSH has many features and there is more you can do to secure your SSH servers, without resorting to external software.
Refresher and Important Notes: The main OpenSSH server configuration file is called sshd_config and will typically be in the /etc/ssh or /etc/sshd directories. Like all of the configuration files used by OpenSSH, it is in plain text and so can be edited with any text editor. After editing your sshd_config file, you will need to reload your SSH server's configuration - restarting the SSH daemon is not necessary. The command typically looks like this (this is on Debian or Ubuntu):
/etc/init.d/ssh reload or (on Red Hat/Fedora):
service sshd reload Also be careful not to lock yourself out of your SSH server when experimenting with these access controls. It's a good idea to always have two SSH sessions into the server, and to always make backup of the relevant configuration files. If you log out of one session and get denied access, you still have one active session to fix things.

Restricting Users and Hosts

OpenSSH allows you to restrict users and groups by host or IP address. There are four different directives you can use in your sshd_config file (they are evaluated in this order):
DenyUsers AllowUsers DenyGroups AllowGroups The format for all of them will be the same - a space-separated list of users or group names, with optional host names. Here is an example:
AllowUsers vader@10.0.0.1 maul@sproing.evillittleman.net sidious tyranus@*.evillitleman.net AllowGroups wheel staff This tells sshd to only allow connections from the user vader and only from the IP address 10.0.0.1. The user maul is also allowed, but only from the host sproing.evillittleman.net. User sidious is allowed from anywhere, and the user tyranus is also allowed, from any host in the evillittleman.net domain (the asterisk matches zero or more characters).
The AllowGroups line allows login only from users whose primary group name or supplementary group list match one of 'wheel' or 'staff'.
Keep in mind that using AllowUsers or AllowGroups means that anyone not matching one of the supplied patterns will be denied access by default. Also, in order for sshd to allow access based on full or partial hostnames, it needs to do a DNS lookup on the incoming IP address. That means the connecting IP address must have a PTR (reverse) entry that maps back to a real hostname. These aren't hard to get if you have a static IP address, usually your ISP or server hosting provider can do this for you on request.
In addition to the asterisk in hostname or group patterns, you can use a question-mark to mean exactly one character, and an exclamation point to negate the sense of a match:
* - Matches zero or more characters
? - Matches exactly one character
! - Negates the host pattern match
Note: In my tests, using ! to negate the sense of the hostname match did not work with the AllowUsers directive. It only seems to work when used with authorized_keys file restrictions (see below).

Restricting Access and Commands

SSH has the concept of authorized keys. If you are using key-based auth, like I suggested in my first article, the user accounts on the SSH server will have an authorized_keys file (which is by default in the ~/.ssh directory of whatever user account you are logging into). This file lists the public keys, one per line, that are authorized for access to that account. Apart from just specifying which public keys are allowed access, there are a some more options that you can use to further restrict SSH sessions. Here are the most useful ones:
from='hostname1,hostname2,'' - Restricts access from the specified IP or hostname patterns
command='command' - Runs the specified command after authentication
no-pty - Does not allocate a pty (does not allow interactive login)
no-port-forwarding - Does not allow port forwarding
Here is an example showing part of an authorized_keys file:
from="deathstar.example.com,!jedi.example.com,10.0.0.?" ssh-rsa AAAAB5...2BQ== vader@evillittleman.net from="pitofdespair.example.com",command="ls",no-pty,no-port-forwarding ssh-dss AAAAZ7...22Q== droidQBX12@evillittleman.net The first line allows login with the specified RSA key from deathstar.example.com, from any host with IP address in 10.0.0.[0-9], but not from the host jedi.example.com. The second line merely runs the 'ls' command whenever the specified DSA key is used - it does not allow any other commands to be run, does not allow interactive login, and does not allow port-forwarding. It also restricts the source of that key to the host pitofdespair.example.com.

Running sshd on a Non-Standard Port

Admittedly this is an attempt at 'security through obscurity', but that doesn't mean it's not useful when combined with other security measures. You may not be able to restrict access by hostname or IP, for example - you may always be sourcing your connections from a dynamic IP address, or you may not be able to get a proper PTR record created. It's quite simple, in your sshd_config file, just change Port=22 to Port=nnnnn (where nnnnn is some high port), then reload the sshd configuration. How do we pick a port number? Some are better than others. First, assume that most port scans are being done with Nmap, and take a look at the nmap-services file. This is a list of ports that Nmap will use by default if you don't specify a port range on the command line. It's probably a fair bet that most script-kiddies are using nmap is this manner. Just pick a high port not on this list, most nmap scans won't notice it. You can also use multiple Port= directives, meaning you can have sshd listen on multiple ports. Connecting to an alternate port is also very easy, use the following options depending on the command used:
ssh -p 65502 vader@deathstar.example.com sftp -oPort=65502 vader@deathstar.example.com scp -P 65502 deathstar_plans.doc vader@deathstar.example.com: You can also edit your client's ~/.ssh/config file, and add the Port= directive to one of your host blocks:
... Host evil Hostname deathstar.example.com User vader Port 65502 ... Then just connecting with the command ssh evil will connect with the specified user and port.

Hashing Known Hosts Files

When you connect to an SSH server, the ssh client stores the server's hostname, IP address and host key in a file named known_hosts. It will by default be in your ~/.ssh directory. Having the IP addresses of the servers you connect to regularly in plaintext can be a security risk if you are on a shared host, or your client gets compromised (stolen laptop, for example). An easy way to avoid this problem is to obscure the information in the known_hosts file by hashing it. Hashing your known_hosts file is easy, you just use the ssh-keygen command, giving it the file path.
ssh-keygen -H -f ~/.ssh/known_hosts While this hashes all existing host keys, any host keys that get added to your known_hosts file after you hash it do not get hashed by default. To make it the default, add the directive HashKnownHosts to your ~/.ssh/config file. Here is an example of hashing a known_hosts file. First, here is what the file looks like beforehand:
dmaxwell@kaylee:~/.ssh$ head known_hosts 10.100.6.151 ssh-rsa AAAAB4NzaC1yc2EAAAABIwAAAIEAuVgRdptT3xsQoGkiNnJb4Zb02p07MaZX02MFs5JhoqmvV5X5Z/LEQH0S7ngSn3b8kQUnocGulJgLchwfThrd/1OkdyOKdpgXxH/rmDXfwh/MZBNBxnMWBa1HpXSc1gxyDfSSxo+VPa1NCP+ob0dWx4sI+JFJ5cVzbQng4rKp3x8= 10.100.6.162 ssh-rsa AAAAB4NzaC1yc2EAAAABIwAAAIEAxpQuMJR4Dq/MmrpUryYlNbP+BIWgJlr0LAfaHTIU64Ho6F58Bb1QzlUeeHQSI9f6qFW9aPsBC3Gd5wgQBUj3byinXXHC/10c3vmb2aEujmyL6en2Pef4AN8bKgaRtJq2G/H4MkPWBzxqZPb/k9c3a26P/DjG4y01TMw9vCld+As= ... Here we run the ssh-keygen command:
dmaxwell@kaylee:~/.ssh$ ssh-keygen -H -f ~/.ssh/known_hosts /home/dmaxwell/.ssh/known_hosts updated. Original contents retained as /home/dmaxwell/.ssh/known_hosts.old WARNING: /home/dmaxwell/.ssh/known_hosts.old contains unhashed entries Delete this file to ensure privacy of hostnames And here is what the file looks like afterward (Note that we deleted the backup file when we were done):
dmaxwell@kaylee:~/.ssh$ head known_hosts |1|PdThGCuhg23t9bcURxyitJTmfKk=|/z+Xvh4xPuDni8PTB5iK7KKnGdA= ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAIEAuVgRdptT3xsQoGkiNnJb4Zb02p07MaZX02MFs5JhoqmvV5X5Z/LEQH0S7ngSn3b8kQUnocGulJgLchwfThrd/1OkdyOKdpgXxH/rmDXfwh/MZBNBxnMWBa1HpXSc1gxyDfSSxo+VPa1NCP+ob0dWx4sI+JFJ5cVzbQng4rKp3x8= |1|vkLZ22nl30gyJ3gIX74FUF7b3eg=|uy5oSZ8avgZQZE+dwMd/mXGoA38= ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAIEAxpQuMJR4Dq/MmrpUryYlNbP+BIWgJlr0LAfaHTIU64Ho6F58Bb1QzlUeeHQSI9f6qFW9aPsBC3Gd5wgQBUj3byinXXHC/10c3vmb2aEujmyL6en2Pef4AN8bKgaRtJq2G/H4MkPWBzxqZPb/k9c3a26P/DjG4y01TMw9vCld+As= ... dmaxwell@kaylee:~/.ssh$ rm known_hosts.old


Five Minutes To A Secure SSH

Here is a quick way to drastically improve the security of your OpenSSH server installations. Apart from past flaws in the OpenSSH daemon itself that have allowed remote compromise (very rare), most break-ins result from successful brute-force attacks. You can see them in your firewall, system or auth logs, they are an extremely common form of attack. Here is an excerpt from the /var/log/messages file on a CentOS Linux box (the attacking hostname has been obfuscated). You can see multiple attempts to login as users root and ftp. Also note the time between repeated attempts - one second or less, much too quick to be a human. This is an automated attack.
Apr 9 09:34:51 server1 sshd(pam_unix)[1511]: authentication failure; logname= uid=0 euid=0 tty=NODEVssh ruser= rhost=host52-182.foobar.pl Apr 9 09:34:52 server1 sshd(pam_unix)[1513]: check pass; user unknown Apr 9 09:34:52 server1 sshd(pam_unix)[1513]: authentication failure; logname= uid=0 euid=0 tty=NODEVssh ruser= rhost=host52-182.foobar.pl Apr 9 09:34:53 server1 sshd(pam_unix)[1515]: authentication failure; logname= uid=0 euid=0 tty=NODEVssh ruser= rhost=host52-182.foobar.pl user=ftp Apr 9 09:34:53 server1 sshd(pam_unix)[1517]: check pass; user unknown Apr 9 09:34:53 server1 sshd(pam_unix)[1517]: authentication failure; logname= uid=0 euid=0 tty=NODEVssh ruser= rhost=host52-182.foobar.pl Apr 9 09:34:54 server1 sshd(pam_unix)[1519]: authentication failure; logname= uid=0 euid=0 tty=NODEVssh ruser= rhost=host52-182.foobar.pl user=root Apr 9 09:34:55 server1 sshd(pam_unix)[1521]: check pass; user unknown Apr 9 09:34:55 server1 sshd(pam_unix)[1521]: authentication failure; logname= uid=0 euid=0 tty=NODEVssh ruser= rhost=host52-182.foobar.pl Apr 9 09:34:56 server1 sshd(pam_unix)[1523]: check pass; user unknown Apr 9 09:34:56 server1 sshd(pam_unix)[1523]: authentication failure; logname= uid=0 euid=0 tty=NODEVssh ruser= rhost=host52-182.foobar.pl
From personal experience with clients over the years, I have found that most administrators tend to install an SSH server and leave it at its default settings, typically to allow password authentication and root logins. Many don't even know that there is an alternative (key-based authentication), or they think the alternative is too hard to use. It is not - it takes all of five minutes to configure key-based authentication and disable root logins, and the security gains are enormous. Below, I'll step you through the process.

Configuring and Testing Key-Based Authentication

This is not really a hard as it seems. By default, all recent OpenSSH configurations allow public key authentication. You just have to generate a pair of SSH keys on the client, and append the generated public key to a file named authorized_keys on the server. This works on any Unix/Linux with OpenSSH, or even under Windows with Cygwin's OpenSSH port (I won't mention it further here, but If you are stuck on a Windows client without Cygwin, use href='http://www.chiark.greenend.org.uk/~sgtatham/putty/download.html'>PuTTY and check out href='http://the.earth.li/~sgtatham/putty/0.58/htmldoc/Chapter8.html#pubkey-puttygen'>Using PuTTYgen).
Let's say you routinely use SSH to login from your home workstation, sshclient, as user obiwan, to a remote SSH server as user vader. For consistency's sake, we'll call the remote server sshserver. I'm assuming you have working password authentication already.
On sshclient, log in under your user account obiwan, and run the following three commands.
obiwan@sshclient:~$ ssh-keygen -t dsa
This first command generates the public/private key pair for you, assigns a passphrase to the private key, and stores both keys in the directory ~/.ssh by default. Just accept the defaults, and enter a passphrase when prompted. obiwan@sshclient:~$ cat ~/.ssh/id_dsa.pub | ssh vader@sshserver 'cat >> .ssh/authorized_keys' or, you can simply use the 'ssh-copy-id' command:
obiwan@sshclient:~$ ssh-copy-id vader@sshserver
This command appends your public key, which is stored in ~/.ssh/id_dsa.pub by default, to the authorized_keys file on the remote host. One caveat, you may not have a '.ssh' directory on the remote server in vader's home directory. Create it if needed.
obiwan@sshclient:~$ ssh vader@sshserver chmod 600 .ssh/authorized_keys This last command changes the permissions on the remote authorized_keys file to be more secure, and to prevent a common problem that prevents key-based authentication.
Now, test SSH login to 'vader@sshserver'. You should be prompted for a passphrase this time, not a password. Enter the passphrase you chose when you created your keys, and you should be logged in.

Disabling Password Authentication

Once key-based logins are working, disable password authentication. Find your SSH server's configuration file. This is usually in /etc/ssh/sshd_config. Find the line that says:
PasswordAuthentication yes and change it to
PasswordAuthentication no You should also disable challenge-response authentication, in case your version of OpenSSH is using PAM to authenticate (see below for an explanation):
ChallengeResponseAuthentication no

Disabling Root-Logins

Finally, disable root logins. Kudos go to NetBSD and FreeBSD, whose default configurations do not allow root logins, so there will be nothing to do for you to do in this step if you are using a recent version of either one. OpenBSD and most Linux distributions allow root logins by default. Again in /etc/ssh/sshd_config, find the line that reads
PermitRootLogin yes and change it to
PermitRootLogin no Reload your SSH server's configuration with /etc/init.d/ssh reload (Debian Linux-based), service sshd reload (Red Hat Linux-based), /etc/rc.d/sshd reload (NetBSD/FreeBSD) or just send sshd a HUP signal, usually with something like kill -HUP pid, where pid is the process ID of the SSH server, you can get it quickly on any Unix platform by running something like ps ax | grep sshd.

Details on PAM Authentication

Disabling PAM-based password authentication is rather un-intuitive. It is needed on pretty much all GNU/Linux distributions (with the notable exception of Slackware), along with FreeBSD. If you're not careful, you can have PasswordAuthentication set to 'no' and still login with just a password through PAM authentication. It turns out that you need to set 'ChallengeResponseAuthentication' to 'no' in order to truly disable PAM authentication. The FreeBSD man pages have this to say, which may help to clarify the situation a bit:
Note that if ChallengeResponseAuthentication is 'yes', and the PAM authentication policy for sshd includes pam_unix(8), password authentication will be allowed through the challenge-response mechanism regardless of the value of PasswordAuthentication.

Common Objections

Many sysadmins advocate leaving password authentication in place, and mandate the use of strong passwords. This is an example of 'shallow defense', and is a bad idea for a few reasons. First, even if you pick a password strong enough to make brute-force attacks next to impossible, you are still relying on a single authentication factor. Depending on your circumstances, you should take into account non-brute force methods of attack that may compromise passwords, like social engineering or exploiting weaknesses in underlying authentication algorithms. Again, this depends on your circumstances. If you are the only SSH user and are willing to accept the risks of password authentication, it may work fine for you. The idea of not relying on a single authentication method, applying defense in depth whenever possible, underlies the idea of using two-factor (keys + passphrases) authentication.
Another common recommendation is to leave OpenSSH in its default state (using password authentication), and use a tool like Denyhosts. I've never been a big fan of methods like this, I think using the security features present in OpenSSH itself makes for a simpler and cleaner solution. I'm also very wary of any tool that automates the blocking of network connections.

Closing Thoughts

There is much more to OpenSSH configuration and security, but these few, simple steps will go a long way towards preventing brute-force attacks and letting you sleep at night. Repeat the above steps on all your SSH servers, copying your SSH public key to each server as needed (it may not be obvious if you've never used key-based authentication before, but don't regenerate your keys for each host).

Filter BAM file for mapped reads only



To get only the mapped reads use the parameter 'F', which works like -v of grep and skips the alignments for a specific flag.
samtools view -b -F 4 file.bam > mapped.bam


To get the unmapped reads from a bam file use :
samtools view -f 4 file.bam > unmapped.sam 
the output will be in sam.
to get the output in bam use : 
samtools view -b -f 4 file.bam > unmapped.bam

From the manual; there are different int codes you can use with the parameter 'f', based on what you want :
-f INT Only output alignments with all bits in INT present in the FLAG field. INT can be in hex in the format of /^0x[0-9A-F]+/ [0]
Each bit in the FLAG field is defined as:
Flag        Chr     Description
0x0001      p       the read is paired in sequencing
0x0002      P       the read is mapped in a proper pair
0x0004      u       the query sequence itself is unmapped
0x0008      U       the mate is unmapped
0x0010      r       strand of the query (1 for reverse)
0x0020      R       strand of the mate
0x0040      1       the read is the first read in a pair
0x0080      2       the read is the second read in a pair
0x0100      s       the alignment is not primary
0x0200      f       the read fails platform/vendor quality checks
0x0400      d       the read is either a PCR or an optical duplicate
Like for getting the unique reads (a single read mapping at one best position); I use :
-q INT Skip alignments with MAPQ smaller than INT [0]
samtools view -bq 1 file.bam > unique.bam

Thursday, March 19, 2015

OWASP Zed Attack Proxy (ZAP)

The OWASP Zed Attack Proxy (ZAP) is an easy to use integrated penetration testing tool for finding vulnerabilities in web applications.
It is designed to be used by people with a wide range of security experience and as such is ideal for developers and functional testers who are new to penetration testing as well as being a useful addition to an experienced pen testers toolbox.
You can show your support for ZAP on Openhub:

Interested in a ZAP talk or training event? See the talks page. Not one near you? Contact a ZAP Evangelist to arrange one!
Want a very quick introduction to ZAP? See the project pamphlet.
For a slightly longer introduction see the project presentation.
For video introductions to ZAP see the links on the wiki videos page.
We recently asked our users for feedback about ZAP, you can see their responses (which are separate for each languages) here: English,French, Spanish, Arabic

ZAP provides automated scanners as well as a set of tools that allow you to find security vulnerabilities manually.
Some of ZAP's functionality:
Some of ZAP's features:
  • Open source
  • Cross platform
  • Easy to install (just requires java 1.7)
  • Completely free (no paid for 'Pro' version)
  • Ease of use a priority
  • Comprehensive help pages
  • Fully internationalized
  • Translated into a dozen languages
  • Community based, with involvement actively encouraged
  • Under active development by an international team of volunteers
It supports the following languages:
  • English
  • Arabic
  • Albanian
  • Bosnian
  • Brazilian Portuguese
  • Chinese
  • Danish
  • Filipino
  • French
  • German
  • Greek
  • Indonesian
  • Italian
  • Japanese
  • Korean
  • Persian
  • Polish
  • Russian
  • Spanish
  • Urdu
To help translate ZAP go to https://crowdin.net/project/owasp-zap
ZAP is a fork of the well regarded Paros Proxy.
Details of the changes made are here: Releases

Linux setup default gateway with route command

Q. How do I setup default gateway with a route command?
A. route command show and/or manipulate the IP routing table under Linux and UNIX oses.
Route manipulates the kernel's IP routing tables. Its primary use is to set up static routes to specific hosts or networks via an interface after it has been configured with the ifconfig program. When the add or del options are used, route modifies the routing tables. Without these options, route displays the current contents of the routing tables.

Display default route

Following three-command display the current routing table:
# route
Output:
Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
192.168.1.0     *               255.255.255.0   U     0      0        0 ra0
default         dsl-router      0.0.0.0         UG    0      0        0 ra0
$ /sbin/route
Output:
Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
191.255.255.0   *               255.255.255.0   U     0      0        0 eth0
169.254.0.0     *               255.255.0.0     U     0      0        0 eth0
default         191.255.255.1   0.0.0.0         UG    0      0        0 eth0
You can use -n option, to display numerical addresses instead of trying to determine symbolic host names (via dns or /etc/hosts file). This is useful if you are trying to determine why the route to your nameserver has vanished.$ /sbin/route -nOutput:
Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
191.255.255.0   0.0.0.0         255.255.255.0   U     0      0        0 venet0
169.254.0.0     0.0.0.0         255.255.0.0     U     0      0        0 venet0
0.0.0.0         191.255.255.1   0.0.0.0         UG    0      0        0 venet0
Please note that a destionation entry 0.0.0.0 (or default) is the default gatway. In above example 191.255.255.1 is a default gatway.

Add / setup a new route

The syntax is as follows:
route add default gw {IP-ADDRESS} {INTERFACE-NAME}
Where,
  • IP-ADDRESS: Specify router IP address
  • INTERFACE-NAME: Specify interface name such as eth0
For example if your router IP address is 192.168.1.254 type the following command as the root user:
# route add default gw 192.168.1.254 eth0OR use hostname such as dsl-router:# route add default gw dsl-router eth0

Setting route using GUI tools/command under Linux

If you find above command hard to use, consider using GUI tools. If your are using Red Hat/CentOS/Fedora core Linux type following command:# redhat-config-networkOR If you are using other Linux distribution use command:# network-admin

Wednesday, March 18, 2015

Submitting Jobs From Within Jobs: qsub

In this section we are going to split the job discussed in section 4.3.4 into four separate jobs. The first job will prepare the GPFS directory and having finished its task, it will submit the second job. The second job will then generate the data file, and having done so it will submit the third job. The third job will process the data file and then it will submit the fourth job, which will will clean up and exit the sequence. The jobs are constructed to be run on the IUPUI cluster, avidd-i.iu.edu.
Here is what the first job script looks like:
[gustav@ih1 PBS]$ cat first.sh
#PBS -S /bin/bash
#PBS -N first
#PBS -o first_out
#PBS -e first_err
#PBS -q bg
#
# first.sh
#
# Prepare a directory on the AVIDD GPFS.
[ -d /N/gpfs/gustav ] || mkdir /N/gpfs/gustav
cd /N/gpfs/gustav
rm -f test
echo "/N/gpfs/gustav prepared and cleaned."
# Now submit second.sh.
ssh ih1 "cd PBS; /usr/pbs/bin/qsub second.sh"
echo "second.sh submitted."
# Exit cleanly.
exit 0
[gustav@ih1 PBS]$
The new element in this job is the line:
ssh ih1 "cd PBS; /usr/pbs/bin/qsub second.sh"
Remember that the job will not run on the head node. It will run on a computational node. But the PBS on the AVIDD cluster is configured so that you cannot submit jobs from computational nodes. So here we have to execute qsub as a remote command on the IUPUI head node ih1 by using the secure shell, since this is the only remote execution shell supported on the cluster.
The first command passed to ssh is ``cd PBS''. On having made the connection the secure shell will land me in my home directory. But I don't want to submit the job from there, because then the job output and error files will be generated in my home directory too. Instead I want all output and error files to be written on my ~/PBS subdirectory. So we go to ~/PBS first.
Then we submit the job. Observe that I use the full path name of the qsub command. The default bash configuration on the AVIDD cluster is such that the remote shell cannot find qsub otherwise. This, of course, I could fix by tweaking my own environment until it does (the PATH should normally be defined on .bashrc, not on .bash_profile), but it is a good practice to specify the full path of the command in this context anyway.
The script second.sh submitted by first.sh looks as follows:
[gustav@ih1 PBS]$ cat second.sh
#PBS -S /bin/bash
#PBS -N second
#PBS -o second_out
#PBS -e second_err
#PBS -q bg
#PBS -j oe
#
# second.sh
#
# The AVIDD GPFS directory should have been prepared by first.sh.
# Generate the data file.
cd /N/gpfs/gustav
time mkrandfile -f test -l 1000
ls -l test
echo "File /N/gpfs/gustav/test generated."
# Now submit third.sh.
ssh ih1 "cd PBS; /usr/pbs/bin/qsub third.sh"
echo "third.sh submitted."
# Exit cleanly.
exit 0
[gustav@ih1 PBS]$

Installing pyodbc on Ubuntu

If like me you have to begrudgingly talk to SQL Server on occasion, but you want to do so from Python on Ubuntu, you're in luck. As I'm finding is true with most things in the Python world there's a module to suit your needs.

Where SQL Server is concerned there are actually two modules available, pyodbc and pymssql, and I'm going to try both of these out to compare and contrast. In theory pymssql should be faster since it doesn't use ODBC, but pyodbc seemed as good a place as any to start.

Installation is simple enough in theory -- download the source zip and run:
python setup.py build install

On Ubuntu, however, you may run into an error or two due to missing libraries. Thankfully this is also quite easy to fix.

The specific error I ran into was:
cc1plus: warning: command line option ‘-Wstrict-prototypes’ is valid for Ada/C/ObjC but not for C++ [enabled by default]

The fix is to install the unixodbc libraries as well as the python-dev libraries:
sudo apt-get install unixodbc-dev unixodbc-bin unixodbc python-dev python2.7-dev

With those installed the build and install for pyodbc works.

Next up is actually using pyodbc to talk to SQL Server, so expect a follow-up post relatively soon.

Monday, March 16, 2015

How to Reset forgotten Root passwords

1.0 Introduction

Suppose you have just taken over as a new system administrator from another person just before they left and they forgot to give you the root password. Now, let's say you have to install the latest version of PHP on the system so that the sales department's website works the way its supposed to. You have to get the website up yesterday, since you are losing money every minute it doesn't work. Or maybe you simply need to add another user to the system.
The above scenarios are just two possible cases when you might need to reset the root password on a system; there are hundreds of other possible cases when you might need to do this, but I am not going to list all of them. Most of us know what to do when something like this happens on a Windows machine, but not a lot of us know how to recover lost passwords from Linux machines. This document attempts to rectify this situation by telling you about the different options available to recover passwords from YOUR machines. (Don't use this to break into other people's systems as that would be stupid and will get you into big trouble if you are caught.)

1.1 Disclaimer

Use the information in this document at your own risk. I disavow any potential liability for the contents of this document. Use of the concepts, examples, and/or other content of this document is entirely at your own risk.
The information in this document should only be used to recover passwords from machines to which you have legal access. If you use this information to break into other people's systems, then I am not responsible for it and you deserve your fate when you are caught. So don't blame me.
You are strongly advised to make a backup of your system before performing any of the actions listed in this document.

1.2 Credits

In this version, I have the pleasure of acknowledging the following people without whose input this would have never seen the light of the day:
  • The community at http://www.justlinux.com, for the really helpful debate about this topic in one of their threads.
  • The creators and maintainers of the various Linux documentation sites using which I gathered this information.
  • Matt Hoskins and Kevin Walsh, for getting me interested in Linux in the first place.
  • My friends and family, for bearing with me when I was compiling this document.

1.3 Before you start

Before you attempt to change or replace the password of any machine, make sure you get permission from the management authorizing it, 'cause otherwise it can be mistaken as an attempt to hack into the machine, which is not good.
Secondly, create a backup of all important data before you do anything else, so if anything goes wrong you'll still have a copy of your data. If you didn't, and something went wrong, don't blame me. I tested most of this stuff on my system and it worked, but you are responsible for your system, not me, so don't blame me if something did go wrong.

2.1 Various Options available

There are various methods available for resetting a root password. In this section, I will list all the major ones, and we will go over each in detail later in the document. I will also go over some steps to prevent some other person from doing this and hacking your machine.
The various methods are:
  • Boot into single-user mode (easiest, least risky)
  • Boot using a boot disk and edit the password file
  • Mount the drive on another computer and edit the password file

2.1.1 Reseting passwords by booting into single-user mode

This is the easiest and the fastest method to reset passwords. The steps are a little different depending on if you are using GRUB or LILO as a bootmanager.
Booting into single-user mode from LILO
Follow these steps to reset the password when using LILO:
  • Reboot the system. When you see the LILO: prompt (see Fig. 1 below), type in linux single and press 'Enter'. This will log you in as root in single-user mode. If your system requires you to enter your root password to log in, then try linux init=/bin/bash instead.
  • Once the system finishes booting, you will be logged in as root in single-user mode. Use passwd and choose a new password for root.
  • Type reboot to reboot the system and then you can login with the new password you just selected.
Lilo Boot Menu 
Figure 1. Lilo Boot Menu
If you have a new version of LILO which gives you a menu selection of the various kernels available press Tab to get the LILO: prompt and then proceed as shown above.
Booting into single user mode from GRUB
Follow these steps to reset the password when using GRUB:
  • Reboot the system, and when you are at the selection prompt (See Fig. 2 below), highlight the line for Linux and press 'e'. You may only have 2 seconds to do this, so be quick.
  • This will take you to another screen where you should select the entry that begins with 'kernel' and press 'e' again.
  • Append ' single' to the end of that line (without the quotes). Make sure that there is a space between what's there and 'single'. If your system requires you to enter your root password to log into single-user mode, then append init=/bin/bash after 'single'. Hit 'Enter' to save the changes.
  • Press 'b' to boot into Single User Mode.
  • Once the system finishes booting, you will be logged in as root. Use passwd and choose a new password for root.
  • Type reboot to reboot the system, and you can login with the new password you just selected.
GRUB boot screen
Fig. 2: GRUB Boot Screen

2.1.2 Reseting passwords by using a boot disk and editing the password file

This method is a little bit more complicated than the previous one and has a very high chance of success (assuming your filesystem is not encrypted and you didn't forget the password to decrypt it if it is). As before, get permission before you do this.
To start, you need a Linux boot disk or a rescue disk. (If you didn't create one when prompted during the installation then let this be a lesson for you.) You can use your installation CD as a rescue disk; most distros have an option to allow you to boot into rescue mode. With my Redhat Linux CD, I have to enter linux rescue to start the rescue mode. But this might be a bit different in each distro. You can also use a live linux CD like Knoppix or Gnoppix for system recovery. (Click here for a list of all the live Linux CD's). In this tutorial I will use Knoppix as my rescue CD but the process is almost the same for any rescue CD you might use.
[ You can also download one of the many single-floppy Linux distributions (e.g., Tom's RootBoot ), and use it to bring up the machine as described. This is, of course, much faster than downloading and burning a rescue CD, especially on a slow connection. -- Ben ]
Follow these steps to reset the password using Knoppix:
  • Reboot the system and configure it to boot from the Knoppix CD (instructions available here)
  • At the Knoppix Boot Prompt (See Fig. 3 below) enter: knoppix lang=us to start boot Knoppix using the english locale. If you understand German, feel free to just hit 'Enter' to boot into Knoppix.
  • Once the system finishes booting, press <Ctrl> + <Alt> + <F1> (The Control, Alt and F1 key together) to switch to a virtual terminal.
  • Type mkdir mountplace to create a directory called 'mountplace'. This is where we will mount the filesystem.
  • Type mount /dev/hdaX mountplace, where /dev/hdaX is your root partition. More information on Linux partitions is available here.
  • Change to the "/etc" directory on your root partition by typing cd mountplace/etc.
  • Use your favorite text editor and open the 'shadow' file for editing. I use 'vi', so I type vi shadow (If you have a really old system, you won't have a shadow file, in which case you need to edit the 'passwd' file.)
  • Scroll down to the line containing the root user's information, which looks something like: 
    root:dsfDSDF!s:12581:0:99999:7:::
  • Delete everything between the first and second colons, so that the line looks like: 
    root::12581:0:99999:7:::
  • Save the file and exit your editor.
  • Type cd to return to your home directory.
  • Type umount mountplace to unmount the partition.
  • Type reboot to reboot your system, and remove the Knoppix CD from the drive.
  • Now you can log into your system as root with no password. Make sure you change the password immediately.
Knoppix boot screen
Fig. 3: Knoppix Boot Screen

2.1.2 Reseting passwords by mounting on another system and editing the password file

This option is a bit more work than any of the earlier options but is almost sure to work (except when the filesystem is encrypted).
Follow these steps to reset the password:
  • Shut down the machine after backing up all important data.
  • Open the casing, unplug the hard drive, and take it to another machine. (This system should be running Linux, since Windows can't read the Linux partition formats.)
  • Connect the hard disk as a slave drive and boot the new system.
  • Once the system finishes booting, mount the slave drive's root partition as shown above and edit the password file.

3.1 How to Prevent someone else from reseting your root password

If you are an even slightly security-consious sysadmin, the previous sections must have set off alarms while you were reading them. Is it really that easy to hack Linux? Yes and No. It all it comes down to the following: Physical Access is Root Access. Meaning, if you give someone physical access to a system, then you are giving them a very good chance of getting root access on your box. This is true for Windows, Linux, or any other OS out there.
But... you say that you need to give some people physical access to the server? There are some precautions you can take to slow down attackers and stop the noob's. In this section I will talk about various ways you can make your computer more secure against these types of attacks. So lets get started.

3.1.1 Password protecting GRUB and LILO

First, edit the /etc/inittab file and insert the following line, right after the "initdefault" line: ~~:S:wait:/sbin/sulogin. This will require a password to boot into single-user mode by making init run 'sulogin' before dropping the machine to a root shell. 'sulogin' requires the user to input the root password before continuing.
Unfortunately, the above step won't protect us against people who know what they are doing and pass init=/bin/bash to the kernel at the LILO prompt. To prevent unauthorized access I would suggest that you password protect LILO/GRUB by following these steps:
How to Protect LILO:
  • Open a shell prompt and log in as root
  • Open /etc/lilo.conf in your favorite text editor
  • Add the following line before the first image stanza: password=<password> , where <password> is your password.
  • Run /sbin/lilo -v to let the changes take effect
  • Type chmod 600 /etc/lilo.conf to give only root access to read and edit the file since all passwords are in plain text
  • Relax a bit, as your system is a little bit more secure
How to password-protect GRUB
  • Open a shell prompt and log in as root
  • Type /sbin/grub-md5-crypt and press enter
  • Enter the password you chose for GRUB when prompted. This will return an MD5 hash of your password
  • Open /boot/grub/grub.conf in your favorite text editor
  • Add password --md5 <password-hash> below the timeout in the main section (Replace <password-hash> with the hash you got in the previous step)
  • Save and exit
  • The next time you reboot, the GRUB menu will not let you access the editor or command interface without first pressing [p] followed by the GRUB password.

3.1.2 Password-protecting the BIOS

There are two primary reasons for password-protecting the BIOS of a computer:
  • Prevent Changes To BIOS Settings: if an intruder has access to the BIOS, they can set it to boot off of a diskette or CD-ROM.
  • Prevent Booting the System: Some BIOSes allow you to password protect the boot process itself. When activated, an attacker would be forced to enter a password for the BIOS to launch the boot loader.
Because the methods for setting a BIOS password vary between computer manufacturers, you should consult the manual for your computer. If you forget the BIOS password, it can often be reset either with jumpers on the motherboard or by disconnecting the CMOS battery. However, you should check the manual for your computer or motherboard before attempting this procedure.

4.1 Conclusion

By now I have hopefully saved you a lot of trouble by telling you how to recover your root password and make it harder for others to get the password. If you think this document helped you, or you have some comments or questions about this please feel free to Contact Me and let me know. However I must warn you that I am a somewhat lazy person who might take a little while before replying to your emails.