Restricting root shell and root user access through sudo

One of the issues I’ve encountered a number of times in assessments of Linux and AIX environments is the provision of excessive permissions using sudo. This article is an attempt to highlight those issues and provide some guidance as to practical resolution.

It is typical in a secured Windows environment that the administrator username is not used for standard business and that those users who require elevated privileges are members of the “Domain Admins” Organisation Unit.  The generic windows administrator account would be renamed, given a randomised long and complex password which would then be physically secured and access restricted.  In Windows, audit trails can be maintained against each user and users do not execute commands as other users.  This is somewhat different in a Linux environment.

In Linux, it has become more standard to use sudo to substitute user and do commands. Sudo in its default implementation is generally in place as (example from CentOS)

wheel ALL = (ALL) ALL

In Red Hat and CentOS, the members of the wheel group are provided full sudo privileges. In Ubuntu or Debian this would be the members of the sudoers group or the admin group.

The above means that a user who is a member of the wheel group can execute ALL commands as ALL users from ALL terminals. In other words, a member of the wheel group can masquerade as other users or can drop to a root shell and no longer have a full audit trail against him.

Users often drop to a root shell to avoid typing sudo before any command. Dropping to a root shell is usually done doing su -, sudo –i, sudo –s, sudo bash etc. In order to prevent sudoers from dropping to a root shell, the shell commands can be removed from the executable files available to the users. This can be done by editing the sudoers file as follows:

Cmnd_Alias SHELLS

SHELLS = /usr/bin/sh, /usr/bin/csh, /usr/bin/ksh, /usr/local/bin/tcsh, usr/bin/rsh, /usr/local/bin/zsh

Cmnd_Alias SU

SU = /usr/bin/su

wheel ALL = ALL, !SHELLS, !SU

In the above members of the wheel group can execute all commands on all systems except those commands listed in SHELLS and SU. Sometimes it is necessary for users to drop to a shell when performing administrative functions. Using the above, you could have a configuration with 1 or 2 users permitted to have root abilities with other domain admins having the above.

admin ALL = (ALL) ALL

wheel ALL = ALL, !SHELLS, !SU

Limit access to the admin group in the same way you might limit access to the administrator account in Windows.

Setup and configure Ganglia-3.6 on CentOS/RHEL 6.3

This article will guide you through the installation and configuration steps for Ganglia-3.6.0 on CentOS/RHEL 6.3
1) About
Ganglia is a scalable distributed monitoring system for high-performance computing systems such as clusters and Grids. It is based on a hierarchical design targeted at federations of clusters. It leverages widely used technologies such as XML for data representation, XDR for compact, portable data transport, and RRDtool for data storage and visualization.
Ganglia has following three main components:
1) Ganglia Monitoring Daemon (gmond)
It is a lightweight service that is installed on every machine you’d like to monitor.
Gmond has four main responsibilities:

• Monitor changes in host state.
• Announce relevant changes.
• Listen to the state of all other ganglia nodes via a unicast or multicast channel.
• Answer requests for an XML description of the cluster state.

Each gmond transmits in information in two different ways:

• Unicasting or Multicasting host state in external data representation (XDR) format using UDP messages.
• Sending XML over a TCP connection.

2) Ganglia Meta Daemon (gmetad)
The ganglia meta daemon (gmetad) is a service that collects data from other gmetad and gmond sources and stores their state to disk in indexed round-robin databases. Gmetad provides a simple query mechanism for collecting specific information about groups of machines.
3) Ganglia PHP Web Front-end
The Ganglia web front-end provides a view of the gathered information via real-time dynamic web pages. Most importantly, it displays Ganglia data in a meaningful way for system administrators and computer users using PHP.
Other component are:
4) Gmetrics
The ganglia metric tool is a commandline application that you can use to inject custom made metrics about hosts that are being monitored by ganglia. It has the ability to spoof messages as coming from a different host in case you want to capture and report metrics from a device where you don’t have gmond running (like a network or other embedded device).
4) Gstat
The ganglia stat tool is a commandline application that you can use to query a gmond for metric information directly.
1) Prerequisite

• Following dependent package needs to be installed first, before installing Ganglia.
  

     yum -y install apr-devel apr-util check-devel cairo-devel pango-devel libxml2-devel rpmbuild glib2-devel \
     dbus-devel freetype-devel fontconfig-devel gcc-c++ expat-devel python-devel libXrender-devel 

• Other dependent packages that need to be Installed are rrdtool and confuse that are covered in installation steps.

2) Installation
2.1) Create user ganglia by which we will be running ganglia services:

   useradd ganglia
   password ganglia

2.2) Create a directory where you would download ganglia source:

   mkdir /usr/local/src
   cd /usr/local/src

2.3) First of all we need to Download/Untar/Compile/Install all necessary files for confuse:

   wget http://savannah.nongnu.org/download/confuse/confuse-2.7.tar.gz
   tar -xzvf confuse-2.7.tar.gz
   cd confuse-2.7
   ./configure
   make
   make install

2.4) Secondly we need to Download/Untar/Compile/Install all necessary files for rrdtool:

   cd /usr/local/src
   wget http://oss.oetiker.ch/rrdtool/pub/rrdtool.tar.gz
   tar -xzvf rrdtool.tar.gz
   cd rrdtool-1.4.8/
   ./configure --prefix=/usr
   make -j8
   make install
   which rrdtool

2.5) Make sure all the new installed package have libraries linked using *ldconfig* command:

   vi /etc/ld.so.conf
      /usr/local/lib
   Then execute the following command:
   ldconfig

2.6) Now all the dependency are installed now is the time to Download/Untar/Compile/Install all necessary files for ganglia-core package:

   cd /usr/local/src
   wget http://sourceforge.net/projects/ganglia/files/ganglia%20monitoring%20core/3.6.0/ganglia-3.6.0.tar.gz
   tar -xzvf ganglia-3.6.0.tar.gz
   ./configure --with-gmetad
   make -j8
   make install

NOTE: You should exit without errors. If you see errors, then you may want to check for missing libraries.
3) Configuring Ganglia
3.1) Create the config directory for Ganglia:

   mkdir /etc/ganglia

3.2) Copy the sample gmetad configuration file:

   cp gmetad/gmetad.conf /etc/ganglia/

3.3) Generate the initial gmond configuration file:

   gmond -t | tee /etc/ganglia/gmond.conf

3.4) Copy the initial startup script, change the binary and config path and enable it on boot:

   cp gmetad/gmetad.init /etc/rc.d/init.d/gmetad
   cp gmond/gmond.init /etc/rc.d/init.d/gmond
   vi /etc/init.d/gmetad

       #GMETAD=/usr/sbin/gmetad
       GMETAD=/usr/local/sbin/gmetad
       #daemon $GMETAD
       daemon $GMETAD -c /etc/ganglia/gmetad.conf

   vi /etc/init.d/gmond

       #GMOND=/usr/sbin/gmond
       GMOND=/usr/local/sbin/gmond
       # daemon $GMOND
       daemon $GMOND -c /etc/ganglia/gmond.conf

   chkconfig --add gmetad
   chkconfig --add gmond

3.4) Now create a storage directory for RRDTool and make sure rrdtool can write to it:

   mkdir -p /var/lib/ganglia/rrds
   chown ganglia:ganglia /var/lib/ganglia/rrds

3.5) Modify the following parameter in gmetad config file:

   vi /etc/ganglia/gmetad.conf
      data_source "Ganglia Test Setup" FQDN Name of Ganglia Server
      setuid_username "ganglia"
      case_sensitive_hostnames 0

Note: Replace “FQDN Name of Ganglia Server” with your DNS server name.
3.6) Lastly modify the following parameter in gmond config file:

   vi /etc/ganglia/gmond.conf
   user = ganglia
   cluster {
      name = "Ganglia Test Setup"
      owner = "Ops"
      latlong = "unspecified"
      url = "unspecified"
    }

   udp_send_channel {
      host = FQDN Name of Ganglia Server
      port = 8649
      ttl = 1
    }

udp_recv_channel {
      port = 8649
    }

tcp_accept_channel {
      port = 8649
}

Note: *) Replace “FQDN Name of Ganglia Server” with your DNS server name.
*) By default gmond will use reverse DNS resolution when displaying hostname, to override this value use “override_hostname” config parameter.
*) Gmond will use IP Address in case DNS resolution and “override_hostname” is not set.
4) Validation and Testing
4.1) First try to run gmetad daemon in debug mode and see everything is fine on one of the terminal window:

   gmond -d 5 -c /etc/ganglia/gmond.conf

   Going to run as user nobody
   Sources are ...
   Source: [Ganglia Test Setup, step 15] has 1 sources
    127.0.0.1
   xml listening on port 8651
   interactive xml listening on port 8652
   Data thread 140442277627648 is monitoring [Ganglia Test Setup] data source
       127.0.0.1
   cleanup thread has been started
   data_thread() for [Ganglia Test Setup] failed to contact node 127.0.0.1
   data_thread() got no answer from any [Ganglia Test Setup] datasource

4.2) First try to run gmond daemon in debug mode and see everything is fine on second terminal window:

   gmond -d 5 -c /etc/ganglia/gmond.conf

   saving metadata for metric: disk_free host: localhost
   Processing a metric value message from localhost
   ***Allocating value packet for host--server001.gauri.com-- and metric --disk_free-- ****

   Processing a metric metadata message from localhost
   ***Allocating metadata packet for host--localhost-- and metric --part_max_used-- ****

4.3) Open another terminal window and see rrd file are created or not:

   ls -lh /var/lib/ganglia/rrds/
   total 8.0K
   drwxr-xr-x. 4 ganglia ganglia 4.0K Aug 16 16:28 Ganglia Test Setup
   drwxr-xr-x. 2 ganglia ganglia 4.0K Aug 16 16:28 __SummaryInfo__

4.4) Once you are convinced that everything is fine, stop these process in daemon mode (by pressing CTRL + C) and start there individual service that we have created earlier:

   service gmetad start
   service gmond start
 

4.5) Verify process are running and respective ports are opened:

   ps -ef | grep -v grep | grep gm
   ganglia   6226     1  0 16:59 ?        00:00:00 /usr/local/sbin/gmetad -c /etc/ganglia/gmetad.conf
   ganglia   6267     1  0 17:01 ?        00:00:00 /usr/local/sbin/gmond -c /etc/ganglia/gmond.conf

   netstat -plane | egrep 'gmon|gme'
   tcp        0      0 0.0.0.0:8649                0.0.0.0:*                   LISTEN      502        1067310    6267/gmond          
   tcp        0      0 0.0.0.0:8651                0.0.0.0:*                   LISTEN      502        1047072    6226/gmetad         
   tcp        0      0 0.0.0.0:8652                0.0.0.0:*                   LISTEN      502        1047073    6226/gmetad         
   udp        0      0 0.0.0.0:8649                0.0.0.0:*                               502        1067309    6267/gmond 
 

5) Deploying Ganglia Web
5.1) Download the package and Untar it:

   cd /usr/local/src/
   wget http://sourceforge.net/projects/ganglia/files/ganglia-web/3.5.10/ganglia-web-3.5.10.tar.gz
   tar -xzvf ganglia-web-3.5.10.tar.gz
   cd ganglia-web-3.5.10

5.2) Modify the Makefile Config that will be used to deploy ganglia web:

   vi Makefile
      # Location where gweb should be installed to (excluding conf, dwoo dirs).
      GDESTDIR = /var/www/html/ganglia

      # Gweb statedir (where conf dir and Dwoo templates dir are stored)
      GWEB_STATEDIR = /var/lib/ganglia-web

      # Gmetad rootdir (parent location of rrd folder)
      GMETAD_ROOTDIR = /var/lib/ganglia

      # User by which your webserver is running
      APACHE_USER =  apache

5.3) Now install Ganglia Web, once we have done the config changes:

   make install

5.4) Try to open Ganglia Web UI in your favourite Web Browser:

http://localhost/ganglia

   OR  

http://Server-IP-Address/ganglia

NOTE : In case of any issue try to disabling iptables and selinux as described in next Section and check.
6) Security Rules
6.1) Firewall Rule for Ganglia
6.1.1) Temporary disabling IPTables rules:

   service iptables stop

6.1.2) Firewall port (8649) that needs to be open for Ganglia daemon:

   iptables -A INPUT -p udp -m udp –dport 8649 -j ACCEPT

6.1.3) Firewall port (80) that needs to be open for Ganglia Web:

   iptables -A INPUT -p tcp -m tcp –dport 80 -j ACCEPT

6.1.4) Save the Iptables rules and restart it:

   service iptables save
   service iptables restart

6.2) SELinux Rule for Ganglia
6.2.1) Temporary disabling SELinux rules:

   echo 0 >/selinux/enforce 

Once we are fully convinced that Ganglia Server is running successfully, now is time to do nodes (i.e. Client) Setup.
7) Preparing Client package for ganglia
Build up the Ganglia Client Package that will be deployed on client machine:

   tar -czvf /tmp/ganglia-client.tar.gz /usr/local/sbin/gmond /etc/ganglia/gmond.conf /etc/init.d/gmond \
   /usr/local/lib64/libganglia-3.6.0.so.0* /lib64/libexpat.so.1* /usr/local/lib/libconfuse.so* \
   /usr/lib64/libapr-1.so* /usr/local/lib64/ganglia

8) Deploying Client package
8.1) SCP the tar package on one of the Client box from Ganglia Server:

   scp /tmp/ganglia-client.tar.gz root@CLIENT-MACHINE-IP-OR-NAME:/tmp/

8.2) Untar the package using following command:

   tar -C / -xzvf /tmp/ganglia-client.tar.gz

8.3) Create user ganglia as well and start the gmond service:

   useradd ganglia
   service gmond start

8.4) Verify the gmond process is running:

   ps -ef | grep -v grep | grep gmond

8.5) Also check the rrd file is created for this machine on the Ganglia Server or checking Ganglia Web UI:

   ls -lh /var/lib/ganglia/rrds/Ganglia\ Test\ Setup/
   total 8.0K
   drwxr-xr-x. 2 ganglia ganglia 4.0K Aug 16 16:28 localhost
   drwxr-xr-x. 2 ganglia ganglia 4.0K Aug 16 16:28 web001
   drwxr-xr-x. 2 ganglia ganglia 4.0K Aug 16 16:28 __SummaryInfo__

8.6) Add the other nodes (Client) to Ganglia Server by repeating the steps mentioned from 8.1 to 8.5:
Congratulations! You have successfully deployed Ganglia Setup, grap a glass of bear and enjoy exploring it.

---

Related Posts:
Setup and configure Ganglia Python Modules on CentOS/RHEL 6.3

Install PBS: the job management software

PBS has serveral subtype: OpenPBS, PBSPro and Torque, here we mainly refer to installation of torque 1. download torque from: http://www.adaptivecomputing.com/resources/downloads/torque/
2. tar zxvf torque-2.5.5.tar.gz
3. cd torque-2.5.5
4. ./configure --prefix=/opt/pbs
5. make
6. make install
7.  ./torque.setup liumh
met problem here:
[root@rocks4 torque-2.5.5]# ./torque.setup liumh
initializing TORQUE (admin: liumh@)
./torque.setup:line 31: pbs_server: command not found
./torque.setup:line 33: qmgr: command not found
ERROR:cannot set TORQUE admins
./torque.setup:line 37: qterm: command not found
7.1  We need to do
[root@rocks4 torque-2.5.5]# PATH=$PATH:/opt/pbs/bin:/opt/pbs/sbin
[root@rocks4 torque-2.5.5]# export PATH
[root@rocks4 torque-2.5.5]# MANPATH=$MANPATH:/opt/pbs/man
[root@rocks4 torque-2.5.5]# export MANPATH
7.2 [root@rocks4 torque-2.5.5]# ./torque.setup liumh
met problem again:
initializing TORQUE (admin: liumh@)
PBS_Server: LOG_ERROR::pbsd_main, unable to determine local server hostname - gethostbyname(rocks4)
failed, h_errno=1
Cannot resolve default server host 'rocks4' - check server_name file.
qmgr: cannot connect to server (errno=15010) Access from host not allowed, or unknown host
ERROR: cannot set TORQUE admins
Cannot resolve default server host 'rocks4' - check server_name file.
qterm: could not connect to server '' (15010) Access from host not allowed, or unknown host
7.3  We need to do:
[root@rocks4 torque-2.5.5]# vi /etc/hosts
! add the following line:
! 210.45.78.9 rocks4.lcg.ustc.edu.cn rocks4
7.4  [root@rocks4 torque-2.5.5]# ./torque.setup liumh
initializing TORQUE (admin: liumh@rocks4SPAMNOT.lcg.ustc.edu.cn)
Max open servers: 4
Max open servers: 4
8. [root@rocks4 torque-2.5.5]# make packages
9. install packages: 在 master 机器上需要安装的是 server 包，在节点上需要安装的是 mom 包。在需要提交 PBS 任务的机器上需要安装 clients 包
./torque-package-server-linux-x86_64.sh --install
./torque-package-mom-linux-x86_64.sh --install
./torque-package-clients-linux-x86_64.sh --install
Start to install worknodes
10. copy torque-package-mom-linux-x86_64.sh and torque-package-clients-linux-x86_64.sh to all work nodes
11. Take bl-3-1.local for an example:
11.1  ./torque-package-clients-linux-x86_64.sh --install
11.2  ./torque-package-mom-linux-x86_64.sh --install
11.3  libtool --finish /opt/pbs/lib
11.4  edit /etc/rc.local, add the following lines:
PATH=$PATH:/opt/pbs/bin:/opt/pbs/sbin
export PATH
MANPATH=$MANPATH:/opt/pbs/man
export MANPATH
11.5  perform the 4 commands in 11.4(PATH...)=
11.6  edit /var/spool/torque/servername, make sure it's "rocks4"
11.7  edit /var/spool/torque/mom_priv/config (new created), add the following lines:
pbsserver rocks4
logevent 255
11.8  run pbs_mom:
pbs_mom -c /var/spool/torque/mom_priv/config
12. configer server node:rocks4
12.1 edit /var/spool/torque/server_priv/nodes, add the following lines
bl-3-1.local np=8
...
12.2 edit /var/spool/torque/mom_priv/config (new created), add the following lines
pbsserver rocks4
logevent 255
12.3 run the following services:
pbs_mom -c /var/spool/torque/mom_priv/config
qterm -t quick
pbs_server
pbs_sched
12.4 add the services in 12.3 to start when the service is started:
vi /etc/rc.local
pbs_mom -c /var/spool/torque/mom_priv/config
pbs_server
pbs_sched
13. since the output of jobs will be returned back to the server with ssh, so we need to config ssh on all the work nodes
13.1 ssh-keygen -P "" -t rsa
13.2 eval `ssh-agent`
13.3 ssh-add /root/.ssh/id_rsa
13.4 put the information in /root/.ssh/id_rsa.pub into [/home/liumh/.ssh/authorized_keys@rocks4]
13.5 try command:
scp file liumh@rocks4SPAMNOT.lcg.ustc.edu.cn:/tmp
if file is copied without passwd, then your setup is successful
13.6 Attention, in 13.5, since bl-3-1.local can't identify rocks4.lcg.ustc.edu.cn, such errors may happen on bl-3-l:
see /var/log/message:
Feb 6 16:22:32 bl-3-1 pbs_mom: LOG_ERROR::req_cpyfile, Unable to copy file /var/sp ool/torque/spool/10.rocks4.lcg.ustc.edu.cn.OU to liumh@rocks4SPAMNOT.lcg.ustc.edu.cn:/home /liumh/test/pbs/pbsjob.o10
Feb 6 16:22:36 bl-3-1 pbs_mom: LOG_ERROR::sys_copy, command '/usr/bin/scp -rpB /va r/spool/torque/spool/10.rocks4.lcg.ustc.edu.cn.ER liumh@rocks4SPAMNOT.lcg.ustc.edu.cn:/hom e/liumh/test/pbs/pbsjob.e10' failed with status=1, giving up after 4 attempts
WHAT WE SHOULD DO IS ADD THE FOLLOWING LINES IN: /etc/hosts AT bl-3-1.local
10.1.1.11 rocks4.lcg.ustc.edu.cn rocks4
14.  Done
Troubleshooting:
Q1.in step 13, after adding content in id_rsa.pub(@bl-3-1.local) to authorized_keys of some users, the scp still need passwd, why?
A1. if the directory: /home/user or /home/user/.ssh has a bad permission, this problem will appear, you just need to perform: chmod 755 /home/user/.ssh
-- MinghuiLiu - 07-Feb-2012

Filter BAM alignments for unmapped pairs

Next up we create three filter categories for:

1. An unmapped read whose mate is mapped.
2. A mapped read who's mate is unmapped
3. Both reads of the pair are unmapped

These categories translate to the following filtering commands:

samtools view -u  -f 4 -F264 alignments.bam  > tmps1.bam
samtools view -u -f 8 -F 260 alignments.bam  > tmps2.bam
samtools view -u -f 12 -F 256 alignments.bam > tmps3.bam

The above step will work on sorted or unsorted BAM files.
We then merge these temporary bam files and sort into read name order. The sort is required to get the mates into the correct order.

samtools merge -u - tmps[123].bam | samtools sort -n - unmapped

Setting up cron jobs in Unix and Solaris

cron is a unix, solaris utility that allows tasks to be automatically run in the background at regular intervals by the cron daemon. These tasks are often termed as cron jobs in unix , solaris.  Crontab (CRON TABle) is a file which contains the schedule of cron entries to be run and at specified times.
This document covers following aspects of Unix cron jobs
1. Crontab Restrictions
2. Crontab Commands
3. Crontab file – syntax
4. Crontab Example
5. Crontab Environment
6. Disable Email
7. Generate log file for crontab activity
1. Crontab Restrictions
You can execute crontab if your name appears in the file /usr/lib/cron/cron.allow. If that file does not exist, you can use
crontab if your name does not appear in the file /usr/lib/cron/cron.deny.
If only cron.deny exists and is empty, all users can use crontab. If neither file exists, only the root user can use crontab. The allow/deny files consist of one user name per line.
2. Crontab Commands
export EDITOR=vi ;to specify a editor to open crontab file.
crontab -e    Edit your crontab file, or create one if it doesn’t already exist.
crontab -l      Display your crontab file.
crontab -r      Remove your crontab file.
crontab -v      Display the last time you edited your crontab file. (This option is only available on a few systems.)
3. Crontab file
Crontab syntax :
A crontab file has five fields for specifying day , date and time followed by the command to be run at that interval.

*     *   *  *   *  command to be executed -     -    -   -  - |     |     |   |    | |     |     |   |    +----- day of week (0 - 6) (Sunday=0) |     |     |   +------- month (1 - 12) |     |     +--------- day of month (1 - 31) |     +----------- hour (0 - 23) +------------- min (0 - 59)

* in the value field above means all legal values as in braces for that column.
The value column can have a * or a list of elements separated by commas. An element is either a number in the ranges shown above or two numbers in the range separated by a hyphen (meaning an inclusive range).
Notes
A. ) Repeat pattern like /2 for every 2 minutes or /10 for every 10 minutes is not supported by all operating systems. If you try to use it and crontab complains it is probably not supported.
B.) The specification of days can be made in two fields: month day and weekday. If both are specified in an entry, they are cumulative meaning both of the entries will get executed .
4. Crontab Example
A line in crontab file like below removes the tmp files from /home/someuser/tmp each day at 6:30 PM.
30     18     *     *     *         rm /home/someuser/tmp/*
Changing the parameter values as below will cause this command to run at different time schedule below :

min	hour	day/month	month	day/week	Execution time
30	0	1	1,6,12	*	– 00:30 Hrs  on 1st of Jan, June & Dec.
0	20	*	10	1-5	–8.00 PM every weekday (Mon-Fri) only in Oct.
0	0	1,10,15	*	*	– midnight on 1st ,10th & 15th of month
5,10	0	10	*	1	– At 12.05,12.10 every Monday & on 10th of every month
:

Note : If you inadvertently enter the crontab command with no argument(s), do not attempt to get out with Control-d. This removes all entries in your crontab file. Instead, exit with Control-c.
5. Crontab Environment
cron invokes the command from the user’s HOME directory with the shell, (/usr/bin/sh).
cron supplies a default environment for every shell, defining:
HOME=user’s-home-directory
LOGNAME=user’s-login-id
PATH=/usr/bin:/usr/sbin:.
SHELL=/usr/bin/sh
Users who desire to have their .profile executed must explicitly do so in the crontab entry or in a script called by the entry.
6. Disable Email
By default cron jobs sends a email to the user account executing the cronjob. If this is not needed put the following command At the end of the cron job line .
>/dev/null 2>&1
7. Generate log file
To collect the cron execution execution log in a file :
30 18 * * * rm /home/someuser/tmp/* > /home/someuser/cronlogs/clean_tmp_dir.log

Centos Linux 5 C / C++ compiler installation

How do I install GNU C / C++ compiler on CentOS Linux server?

To install compiler use yum command:

Code:

yum install gcc gcc-c++ autoconf automake

What are the iptables rules to permit ntp?

Question
My server's clock is wrong because the firewall doesn't permit ntp traffic. What are the iptables rules required to allow the ntp client to get out and back?
Any suggestions how to implement those rules on Ubuntu also appreciated.

 Answer

"out and back" implies you are an NTP client and want to talk to a server i'd imagine by default you can do this; if you haven't set up a firewall to block everything, and have iptables set up at all, you'll have a "allow related/established" rule which means replies to outgoing requests are allowed automatically in any case, NTP is UDP port 123, so, assuming you are a CLIENT and want to access NTP servers you'd do: iptables -A OUTPUT -p udp --dport 123 -j ACCEPT iptables -A INPUT -p udp --sport 123 -j ACCEPT these will append the rules to the end of your OUTPUT and INPUT chains Assuming you want to be a server, you'd do iptables -A INPUT -p udp --dport 123 -j ACCEPT iptables -A OUTPUT -p udp --sport 123 -j ACCEPT I have a script which implements all my firewall rules, and I call it from /etc/rc.local which runs on startup on my machine (ubuntu 8.04 LTS) EDIT: You've clarified that this is because you are a client. In ubuntu's default configuration, you shouldn't have to alter any firewall settings to do this. What firewall configuration have you done? If nothing, I'm inclinced to believe that this isn't a firewall issue.

Linux Iptables block or open DNS / bind service port 53

The domain name service provided by BIND (named) software. It uses both UDP and TCP protocol and listen on port 53. DNS queries less than 512 bytes are transferred using UDP protocol and large queries are handled by TCP protocol such as zone transfer.
i) named/bind server – TCP/UDP port 53
ii)Client (browser, dig etc) – port > 1023

Allow outgoing DNS client request:

Following iptables rules can be added to your shell script.
SERVER_IP is your server ip address
DNS_SERVER stores the nameserver (DNS) IP address provided by ISP or your own name servers.
Following rules are useful when you run single web/smtp server or even DSL/LL/dialup Internet connections:

SERVER_IP="202.54.10.20"
DNS_SERVER="202.54.1.5 202.54.1.6"
for ip in $DNS_SERVER
do
iptables -A OUTPUT -p udp -s $SERVER_IP --sport 1024:65535 -d $ip --dport 53 -m state --state NEW,ESTABLISHED -j ACCEPT
iptables -A INPUT -p udp -s $ip --sport 53 -d $SERVER_IP --dport 1024:65535 -m state --state ESTABLISHED -j ACCEPT
iptables -A OUTPUT-p tcp -s $SERVER_IP --sport 1024:65535 -d $ip --dport 53 -m state --state NEW,ESTABLISHED -j ACCEPT
iptables -A INPUT -p tcp -s $ip --sport 53 -d $SERVER_IP --dport 1024:65535 -m state --state ESTABLISHED -j ACCEPT
done

(B) Allow incoming DNS request at port 53:

Use following rules only if you are protecting dedicated DNS server.
SERVER_IP is IP address where BIND(named) is listing on port 53 for incoming DNS queries.
Please note that here I'm not allowing TCP protocol as I don't have secondary DNS server to do zone transfer.

SERVER_IP="202.54.10.20"
iptables -A INPUT -p udp -s 0/0 --sport 1024:65535 -d $SERVER_IP --dport 53 -m state --state NEW,ESTABLISHED -j ACCEPT
iptables -A OUTPUT -p udp -s $SERVER_IP --sport 53 -d 0/0 --dport 1024:65535 -m state --state ESTABLISHED -j ACCEPT
iptables -A INPUT -p udp -s 0/0 --sport 53 -d $SERVER_IP --dport 53 -m state --state NEW,ESTABLISHED -j ACCEPT
iptables -A OUTPUT -p udp -s $SERVER_IP --sport 53 -d 0/0 --dport 53 -m state --state ESTABLISHED -j ACCEPT

Please note if you have secondary server, add following rules to above rules so that secondary server can do zone transfer from primary DNS server:

DNS2_IP="202.54.10.2"
iptables -A INPUT -p tcp -s $DNS2_IP --sport 1024:65535 -d $SERVER_IP --dport 53 -m state --state NEW,ESTABLISHED -j ACCEPT
iptables -A OUTPUT -p tcp -s $SERVER_IP --sport 53 -d $DNS2_IP --dport 1024:65535

How to log all unix shell commands submitted by all users

Use your favourite text editor to open /etc/bashrc and append the following line at the end:

export PROMPT_COMMAND='RETRN_VAL=$?;logger \

-p local6.debug "$(whoami) [$$]: $(history 1 | \

sed "s/^[ ]*[0-9]\+[ ]*//" ) [$RETRN_VAL]"'

Set the syslogger to trap local6 to a log file by adding this line in the /etc/syslog.conf file:
[for ubuntu it is /etc/rsyslog.conf]

local6.* /var/log/cmdlog.log

root@ubuntu:/var/log# tail -f cmdlog.log
Apr 11 20:39:00 ubuntu root: root [8147]:  [0]
Apr 11 20:39:06 ubuntu root: root [8147]: su – jahmed [1]
Apr 11 20:39:11 ubuntu root: root [8147]: ls [0]
Apr 11 20:39:13 ubuntu root: root [8147]: ls -ltra [0]

Encrypting text/files in unix

Encryption are of two types:

1. Symetric encryption uses same key for encryption and decryption.
2. Asymetric used key pair to encrypt and decrypt, i.e one key used to encrypt and another key used to decrypt; usually it is by public and private key.

In unix system use the following for :
Asymmetric encryption:
Step 1: Generate private key
Option 1:

openssl genrsa -out private.pem -des3 1024 # this would ask for a pass phrase=

Option 2:

openssl genrsa -out private.pem 4096 ##this will not ask for pass phrase

Step 2: Generate public key

openssl rsa -in private.pem -pubout -out public.pem

Step 3: Encrypt text file

openssl rsautl -encrypt -inkey public.pem -pubin -in file.txt -out file.ssl

Step 4: Decrypt text file

openssl rsautl -decrypt -inkey private.pem -in file.ssl

Symmetric encryption:
To encrypt:

openssl aes-256-cbc -a -salt -k <key> -in <input_file> -out <output_file_name>

To decrypt:

openssl aes-256-cbc -a -d -salt -k <key> -in <encrypted_file>

Now some theory, i got it from google search,
Asymmetric is as follows:

Symmetric is as follows:

Using /etc/hosts.allow and /etc/hosts.deny to secure unix

TCP wrapper based access List Rules can be included in the two files
/etc/hosts.allow and
/etc/hosts.deny .
Work precedence:

1. /etc/hosts.allow-
   1. if allow will not check 2
   2. if not found then go to 2
2. /etc/hosts.deny .
   1. if not found allow access.

Points to remember

• You can have only one rule per service in hosts.allow and hosts.deny file.
• Any changes to hosts.allow and hosts.deny file takes immediate effect.
• The last line in the files hosts.allow and hosts.deny must be a new line character. Or else the rule will fail.

Rule Syntax
The syntax for both hosts.allow and hosts.deny file takes the following form:

daemon : client [:option1:option2:...]

Examples
Allow SSH

for xyz.com and

deny access to all the others.

sshd : .xyz.com

… and in the hosts.deny file I include the rule:

sshd : ALL

Denys FTP access to all in abc.com domain  and hosts in the 192.168.1.0 network.

#FILE: /etc/hosts.deny
vsftpd : 192.168.1. , .abc.com

Beautification

#FILE: /etc/hosts.deny
vsftpd : 192.168.1. , .abc.co.in : spawn /bin/echo  `/bin/date` access denied >> /var/log/vsftpd.log : deny

In the above rule, spawn logs a message to the vsftpd log file each time the rule matches. deny is optional if you are including this rule in the hosts.deny file.
For example, you can use spawn option to send mail to the admin when ever a deny rule is matched.
Wildcards
You can use wildcards in the client section of the rule to broadly classify a set of hosts. These are the valid wildcards that can be used.

• ALL – Matches everything
• LOCAL – Matches any host that does not contain a dot (.) like localhost.
• KNOWN – Matches any host where the hostname and host addresses are known or where the user is known.
• UNKNOWN – Matches any host where the hostname or host address are unknown or where the user is unknown.
• PARANOID – Matches any host where the hostname does not match the host address.

Patterns

ALL : 123.12.

Matches all the hosts in the 123.12.0.0 network. Note the dot (.) in the end of the rule.

ALL : 192.168.0.1/255.255.255.0

IP address/Netmask can be used in the rule.

sshd : /etc/sshd.deny

If the client list begins with a slash (/), it is treated as a filename. In the above rule, TCP wrappers looks up the file sshd.deny for all SSH connections.

sshd : ALL EXCEPT 192.168.0.15

will allow ssh connection for only the machine with the IP address 192.168.0.15 and block all other connections.
You can use the options allow or deny to allow or restrict on a per client basis in either of the files hosts.allow and hosts.deny

in.telnetd : 192.168.5.5 : deny
in.telnetd : 192.168.5.6 : allow

Shell Commands
As mentioned above, you can couple the rules to certain shell commands by using the following two options.
spawn – This option launches a shell command as a child process. For example, look at the following rule:

sshd : 192.168.5.5 : spawn /bin/echo `/bin/date` from %h >> /var/log/ssh.log : deny

Each time the rule is satisfied, the current date and the clients hostname %h is appended to the ssh.log file.
twist – This is an option which replaces the request with the specified command. For example, if you want to send to the client trying to connect using ssh to your machine, that they are prohibited from accessing SSH, you can use this option.

sshd : client1.xyz.com : twist /bin/echo “You are prohibited from accessing this service!!” : deny

When using spawn and twist, you can use a set of expressions. They are as follows :
%a — The client’s IP address.
%A — The server’s IP address.
%c — Supplies a variety of client information, such as the username and hostname, or the username and IP address.
%d — The daemon process name.
%h — The client’s hostname (or IP address, if the hostname is unavailable).
%H — The server’s hostname (or IP address, if the hostname is unavailable).
%n — The client’s hostname. If unavailable, unknown is printed. If the client’s hostname and host address do not match, paranoid is printed.
%N — The server’s hostname. If unavailable, unknown is printed. If the server’s hostname and host address do not match, paranoid is printed.
%p — The daemon process ID.
%s — Various types of server information, such as the daemon process and the host or IP address of the server.
%u — The client’s username. If unavailable, unknown is printed.

How To Install Git on Ubuntu 12.04

About Git

Git is a distributed version control system released to the public in 2005. The program allows for non-linear development of projects, and can handle large amounts of data effectively by storing it on the local server. This tutorial will cover two ways to install Git.

How to Install Git with Apt-Get

Installing Git with apt-get is a quick and easy process. The program installs on the virtual private server with one command:

sudo apt-get install git-core

After it finishes downloading, you will have Git installed and ready to use.

How to Install Git from Source

If you are eager to download the most recent version of Git, it is generally a good idea to install it from the source.
Quickly run apt-get update to make sure that you download the most recent packages to your VPS.

sudo apt-get update

Prior to installing Git itself, download all of the required dependancies:

sudo apt-get install libcurl4-gnutls-dev libexpat1-dev gettext libz-dev libssl-dev build-essential

Once they are installed, you can download the latest version of Git from the google code page.

wget https://git-core.googlecode.com/files/git-1.8.1.2.tar.gz

After it downloads, untar the file and switch into that directory:

tar -zxf git-1.8.1.2.tar.gz

cd git-1.8.1.2

If you want to do a global install, install it once as yourself and once as root, using the sudo prefix:

make prefix=/usr/local all
sudo make prefix=/usr/local install

If you need to update Git in the future, you can use Git itself to do it.

git clone git://git.kernel.org/pub/scm/git/git.git

How to Setup Git

After Git is installed, whether from apt-get or from the source, you need to copy your username and email in the gitconfig file. You can access this file at ~/.gitconfig.
Opening it following a fresh Git install would reveal a completely blank page:

sudo nano ~/.gitconfig

You can use the follow commands to add in the required information.

git config --global user.name "NewUser"
git config --global user.email newuser@example.com

You can see all of your settings with this command:

git config --list

If you avoid putting in your username and email, git will later attempt to fill it in for you, and you may end up with a message like this:

[master 0d9d21d] initial project version
 Committer: root 
Your name and email address were configured automatically based
on your username and hostname. Please check that they are accurate.
You can suppress this message by setting them explicitly:

    git config --global user.name "Your Name"
    git config --global user.email you@example.com

After doing this, you may fix the identity used for this commit with:

    git commit --amend --reset-author

See More

This tutorial covered how to install Git on your virtual private server. Stay tuned for a second tutorial on Git Basics.

By Etel Sverdlov

[Rocks-Discuss] ssh root at compute-node requires password

Greg Bruno greg.bruno at gmail.com 
Fri Jan 15 08:54:01 PST 2010

• Previous message: [Rocks-Discuss] ssh root at compute-node requires password
• Next message: [Rocks-Discuss] ssh root at compute-node requires password
• Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]

---

On Fri, Jan 15, 2010 at 7:35 AM, Tent Pig <tentpig at yahoo.com> wrote:
> I just finished a fresh install of Rocks 5.2 on a 500-node cluster, pxebooting all compute-nodes by hand, insert-ethers running, etc. to get a functional cluster.
>
> All of my nodes are up and running.
>
> My problem is if I attempt, as root on the head-end, to ssh root at compute-whatever, I'm being asked for a password... on every single compute node.
>
>
> This is annoyingly aggravating. I have several post-install scripts to run on each of the compute nodes, and I really don't feel like typing the root password several thousand times. It will take me months to complete this rollout, and I was supposed to be finished by the end of business today.
>
> I googled this problem and it seems pretty common, but I can't get a clear resolution. .ssh folder permissions, regening the ssh key, yada yada yada... all of which I've tried and nothing fixes it.
>
> I'm absolutely dead in the water here at the moment.
>
> Is there a simple solution to this problem? Reinstalling the compute nodes (one of the suggested solutions) doesn't seem a reasonable answer, since I just installed the compute nodes, haven't even touched them after they rebooted, and I have this issue. (Nor do I really feel like sitting there to reboot/PXEboot manually all over again.)


here's a little background on how rocks distributes the root user's
public key and on ssh login in general.

in order to login to a host without having to type a password with
ssh, the public key for the user must be in the file
$HOME/.ssh/authorized_keys on the remote host and the user's private
key must be loaded into the user's environment. you can see what
private keys are currently loaded by executing:

    # ssh-add -l

in rocks, when a user logs in and if the file $HOME/.ssh/id_rsa
doesn't exist, then the user is prompted to create their ssh keys.
this is also true for the root user, which is why you create ssh keys
for the root user when you login to the frontend the first time after
the frontend is installed. also, in rocks, after your ssh keys are
created, the public key is copied into $HOME/.ssh/authorized_keys.

on the frontend, the root user's home directory is not NFS mounted on
all the compute nodes, so in rocks, we distribute root's
authorized_keys file inside the kickstart file when a compute node
installs.

so, if you change root's ssh keys on the frontend, then the normal way
authorized_keys is updated on the compute nodes is through
reinstallation.

but, the good news is, i believe there is a workaround.

on the frontend, let's generate new ssh keys for root:

    # rm -rf /root/.ssh/id_rsa*

then logout and log back in. this will prompt you to generate new ssh
keys for root.

now, let's distribute root's authorized_keys file with 411:

   # cd /var/411

edit 'Files.mk' and change the line:

    # FILES += /my/file

to:

    FILES += /root/.ssh/authorized_keys

now tell 411 to push all the files under its control out to the compute nodes:

    # rocks sync users

now try to login to a compute node.

Logwatch How-To for CentOS or Red Hat

By Scott Miller+ | 2013/03/11

Logwatch is the classic log file email utility that emails a daily status of activity from Linux logs. On CentOS, the default install of logwatch does not have many fancy features enabled. I’ll show you how to configure logwatch!

---

First, install logwatch:

$ sudo yum install logwatch

---

Next, navigate to the logwatch services directory which is located as follows:

$ cd /usr/share/logwatch/defaults.conf/services

Here edit the following files:

$ sudo nano zz-disk_space.conf

Uncomment the lines as shown:

#New disk report options
#Uncomment this to show the home directory sizes
$show_home_dir_sizes = 1
$home_dir = "/home"

#Uncomment this to show the mail spool size
$show_mail_dir_sizes = 1
$mail_dir = "/var/spool/mail"

#Uncomment this to show the system directory sizes /opt /usr/ /var/log
$show_disk_usage = 1

---

Next, edit the following file:

$ sudo nano http.conf

Set the following to 1

# Set flag to 1 to enable ignore
# or set to 0 to disable
$HTTP_IGNORE_ERROR_HACKS = 1

---

Next, you may want to edit the email address that logwatch emails the report.

$ cd /usr/share/logwatch/defaults.conf/

$ sudo nano logwatch.conf

Change MailTo = to an email address as desired:

# Default person to mail reports to.  Can be a local account or a
# complete email address.  Variable Print should be set to No to
# enable mail feature.
#MailTo = root
MailTo = linuxadmins@mycompany.com

---

It is common practice to send root mail from all servers to a mailing list that all admins subscribe to.


Once complete, you may run logwatch manually at the command line with no options to test:

$ sudo logwatch

---

Logwatch by default runs with daily cron jobs in /etc/cron.daily.

From the Sys Admin Toolbox: Logwatch

Q: What did one lumberjack say to the other lumberjack? A: Watch those logs.

By Ken Hess

Sunday, June 6th, 2010

Last week we provided an introduction to the Linux system logs. (See Logs: Your Linux System’s Lovable Worker Bees.) Now, what will you, as a system administrator, use to watch logs? Logwatch, of course. As so many others have succinctly put it, messing with log files is a royal pain in the backside. Logwatch makes the experience of keeping track of system activity almost painless for you.

Logwatch is a log file parser program (Perl script) that provides a report to you on any “interesting” activity on your system. It is not, I repeat not, a pre-emptive tool or a tool that’s used to catch anyone “in the act” of breaking into your system. It is an after-the-fact tool that provides you with a daily report of service activity. It reports on yesterday’s log information.

We’ll explore active monitoring tools in the coming weeks to catch a would-be system hacker. Logwatch’s value isn’t in its ability to catch a criminal with his hands on your system, its value is instead, designed to save you the effort of manually scraping logs.

Logwatch

Logwatch is a customizable, pluggable log-monitoring system. It will go through your logs for a given period of time and make a report in the areas that you wish with the detail that you wish. Easy to use – works right out of the package on almost all systems.

The Basics

Before we begin, if you need a bit more background on Linux logs and system information, check out any of the following articles:

• Linux System Information Decoded
• The Linux System Logging Daemon
• Using Logwatch and Watching DVDs
• Writing a Trace System
• Logging with syslog-ng, Part One
• Logging with syslog-ng, Part Two

Now, let’s dive in.

Install logwatch in the usual way for your particular distribution using a package manager or download the source from the Logwatch Project page.

Primary setup is easy. If you installed via package, an automatic cron entry in cron.daily runs logwatch every day for you. Default setup includes all services, default log location /var/logand mail to the local root account. Logwatch installs to the /usr/share/logwatch directory for Debian-based and Red Hat-based systems. The main configuration file is under/usr/share/logwatch/default.conf. Some packagers create a symbolic link from/usr/sbin/logwatch to the perl script under /usr/share/logwatch/scripts/logwatch.pl.

Using Logwatch

Logwatch, by default, runs daily on yesterday’s logs, sends an email to the local root account with a low level of detail. For most of you, this is enough information in a daily summary to satisfy your needs and curiosity about what’s going on with your system. For others with systems that are a bit more security sensitive, you’ll need to slightly tweak the parameters of this “near perfect by default” tool.

You can run logwatch with a specific date range. For example, if you want to see information about today’s SSHD activity, you can run this command:

# logwatch --service sshd --range=Today

Check root’s mail to see the details of the report. Shown is the SSHD section of the report.

 --------------------- SSHD Begin ------------------------

 Illegal users from:
    192.168.1.83: 12 times

 **Unmatched Entries**
 pam_succeed_if(sshd:auth): error retrieving information about user raphael : 3 time(s)
 pam_succeed_if(sshd:auth): error retrieving information about user bob : 6 time(s)
 PAM 2 more authentication failures; logname= uid=0 euid=0 tty=ssh ruser= rhost=192.168.1.83  : 4 time(s)
 pam_succeed_if(sshd:auth): error retrieving information about user george : 3 time(s)

 ---------------------- SSHD End -------------------------

This is your default “Low” detail logwatch output report. For a more detailed report, try using –detail=Medium. Check root’s mail for the report.

# logwatch --service sshd --range=Today --detail=Medium

 --------------------- SSHD Begin ------------------------

 Illegal users from:
    192.168.1.83: 12 times
       bob/password: 6 times
       george/password: 3 times
       raphael/password: 3 times

 **Unmatched Entries**
 pam_succeed_if(sshd:auth): error retrieving information about user raphael : 3time(s)
 pam_succeed_if(sshd:auth): error retrieving information about user bob : 6 time(s)
 PAM 2 more authentication failures; logname= uid=0 euid=0 tty=ssh ruser= rhost=192.168.1.83  : 4 time(s)
 pam_succeed_if(sshd:auth): error retrieving information about user george : 3 time(s)

 ---------------------- SSHD End -------------------------

You can see that there’s more detail in this report. You can try using the High parameter to squeeze more detail from logs.

# logwatch --service sshd --range=Today --detail=High

 --------------------- SSHD Begin ------------------------

 Illegal users from:
    192.168.1.83: 12 times
       bob/password: 6 times
       george/password: 3 times
       raphael/password: 3 times

 **Unmatched Entries**
 pam_succeed_if(sshd:auth): error retrieving information about user raphael : 3time(s)
 pam_succeed_if(sshd:auth): error retrieving information about user bob : 6 time(s)
 PAM 2 more authentication failures; logname= uid=0 euid=0 tty=ssh ruser= rhost=192.168.1.83  : 4 time(s)
 pam_succeed_if(sshd:auth): error retrieving information about user george : 3 time(s)

 ---------------------- SSHD End -------------------------

As you can see, the detail didn’t change from Medium to High for SSHD. Alternatively, you can specify the detail level using numeric values using 0, 5 or 10, where 10 is the highest level of detail. Please note that you won’t receive more detail than what’s supplied in the logs.

For those of you, like me, who don’t love command line mail, you can tell logwatch to save its report to a file.

# logwatch --service sshd --range=Today --detail=High --save=logwatch.today

If you need other command line tweaks, a quick man logwatch (Yes, the developers wrote a manual page for Logwatch) will give you what you need.

Logwatch Paranoid Configuration

As shown in the previous section, you can use logwatch at the command line when needed. Those command line options supercede any configured parameters hard-coded in the configuration file located in /usr/share/logwatch/default.conf.

For those of you who are paranoid, or need to be, there are some settings in the configuration file worth noting.

The defaults settings are

Range = yesterday

Detail = Low

Archives = No.
You should change these to:

Range = All

Detail = High

Archives = Yes.

The Archives setting grabs data from your recent logs as well. If you remember from the first post in this series, they’re located under the /var/log directory and have a .X filename to designate their rotation number.

These settings will process all logs for all services and provide the highest level of reporting. The command line equivalent is shown below.

# logwatch --detail=High --range=All --archives

Customizing Output

Here’s a little bonus for those of you who enjoy reading HTML pages more than catting a text file or reading command line email. You can publish your logwatch reports in HTML. It only takes a minor bit of tweaking to create professional-looking automated HTML reports complete with formatting.

The command line version is shown below.

# logwatch --service sshd --detail=High --range=Today --output=html --save=/var/www/html/logwatch/logwatch.html

The equivalent settings in the configuration file are:

Save = /var/www/html/logwatch.html

Output = html

The SSHD section from the HTML report is shown in Figure 1.

Figure 1: SSHD Section from the Logwatch HTML Report

Logwatch is a useful script for system administrators who don’t have the time or the patience to grep and page through logfiles. The work has been done for you by Kirk Bauer who develops and maintains logwatch. Logwatch should be one of your standard system administrator tools that’s installed on every system you manage.

Next week, you’ll take a look at some real-time log monitoring with swatch.