Monday, January 25, 2016

Reload all sysctl variables without reboot

Terminal - Reload all sysctl variables without reboot
#sysctl -p

Reload all sysctl variables without reboot
Reload all defined kernel variables from /etc/sysctl.conf(if no parameter after -p is given) without the old myth "Ah, you'll need to reboot to apply those variables"...

Alternatives

There are 13 alternatives - vote for the best!
Terminal - Alternatives
#sysctl --system

Use `sysctl -p` without argument will only load /etc/sysctl.conf, but user configs always put in /etc/sysctl.d/*.conf, `sysctl --system` will load all the config files

Know a better way?

If you can do better, submit your command here.

Tuesday, January 19, 2016

BM, NVIDIA and Mellanox Launch New HPC Center in UK

BM, UK STFC Hartree Centre, NVIDIA and Mellanox Launch High Performance Computing POWER Acceleration and Design Center for Business

New Center Designed to Optimize Performance of OpenPOWER Systems for Modelling & Simulation and Big Data Analytics
Daresbury, UK - 22 Oct 2015: IBM (NYSE: IBM), in collaboration with NVIDIA and Mellanox, and the Science and Technology Facilities Council (STFC) Hartree Centre today announced plans for the UK's first POWER Acceleration and Design Center (PADC) designed to help UK businesses exploit high performance computing on OpenPOWER systems for Modelling & Simulation and Big Data Analytics.


Today, IBM and the UK government's Science and Technology Facilities Council deepened their relationship with the opening of a POWER Acceleration and Design Center, an application design center for UK businesses looking gain actionable insight from their data.  Business will have access to IBM POWER-based high performance computing systems infused with innovations developed by members of the OpenPOWER Foundation. (Credit: STFC)
The new facility marks a running start to the Government's recently announced investment and expansion of the Hartree Centre at Daresbury, which is designed to boost Modelling & Simulation, Big Data Analytics and Cognitive Computing research in the UK in order to deliver economic growth on a national scale.

Increasing application performance while minimizing energy consumption are challenges businesses face in exploiting conventional high performance computing technologies and the resulting race towards exascale computing. Optimizing existing applications on POWER-based systems can offer businesses significant competitive advantages.

Dr. Peter Allan, acting Director of the Hartree Centre, said: “The POWER Acceleration and Design Center will help industry and academia take advantage of IBM and NVIDIA’s technological leadership in supercomputing and the Hartree Centre's expertise and experience in delivering solutions to real-world problems. It will also provide industry and academia with access to the Hartree Centre’s research capabilities and network of global partners. The PADC will help the Government's ambition to provide world-leading facilities to enable UK businesses use Modelling & Simulation and Big Data Analytics to develop better products and services that will boost productivity, drive growth and create jobs.”

A new POWER Acceleration and Design Center at the Hartree Centre in Daresbury represents a deeper partnership between IBM and the UK government's Science and Technology Facilities Council. With the help of technology innovations developed by IBM and fellow OpenPOWER Foundation members including NVIDIA and Mellanox, UK businesses will be able to gain competitive differentiation utilizing advanced computing systems to develop applications for modelling and simulation, cognitive computing and Big Data research. (Credit: STFC)
The POWER Acceleration and Design Center will be uniquely placed in the UK to provide both POWER-based infrastructure and consulting. The Hartree Centre, working with IBM, already has a history of collaborating on research and development with industry and academic institutions, and in providing access to its supercomputing facilities. These services are being extended to help organizations define application design and optimization projects and will assist in bringing together an ecosystem of computational scientists and engineers with expertise in specific domains such as chemistry, fluid dynamics, structures, and acoustics.

“This new POWER Acceleration and Design Center demonstrates IBM’s commitment to expanding the software ecosystem around OpenPOWER. The center extends and complements the Acceleration and Design Centers in Jülich, Germany, and Montpellier, France, to a new class of industrial and commercial organizations,” said Dave Turek, IBM’s Vice President of High Performance Computing. “To that end, establishing it at the Hartree Center is in recognition of its successful engagement with industry and its record in commercializing technological developments.”

Staff from IBM Research, NVIDIA and Hartree Centre will provide expertise and consultation to help improve performance for the GPU-accelerated OpenPOWER systems. To achieve the highest levels of performance it is important to exploit all components of a POWER-based system: architecture, memory, storage, interconnects and integration. Consequently, the Center will also be supported by the expertise of other OpenPOWER partners, including Mellanox, and will host a POWER-based system with the Tesla Accelerated Computing Platform, consisting of energy-efficient, high-performance NVIDIA Tesla GPU accelerators and enabling software.

“Building upon NVIDIA’s long-standing collaboration with STFC, we look forward to working with IBM, Mellanox and the Hartree Centre to help UK science and industry achieve major new milestones using Tesla GPU-accelerated OpenPOWER systems,” said Jeremy Purches, Director of High Performance Computing UK&I at NVIDIA. “The Tesla platform was designed from the ground up to deliver unmatched energy-efficient performance for today’s most difficult computational challenges, and will ultimately play an important role in enabling the industry to achieve exascale computing levels.”

“As founding members of the OpenPOWER Foundation, IBM, NVIDIA, and Mellanox share a common vision to bring a new class of systems with advanced interconnect technology to market faster to tackle today’s data challenges,” said Gilad Shainer, vice president of marketing, Mellanox Technologies. “Our joint collaboration with STFC Hartree Centre will enable businesses of all sizes to take advantage of scalable, higher application performance for optimal return-on-investment.”
About the OpenPOWER Foundation
The OpenPOWER Foundation is an open technical community based on the POWER architecture, enabling collaborative development and opportunity for member differentiation and industry growth. The goal of the Foundation is to create an open ecosystem, using the POWER architecture to share expertise, investment, and server-class intellectual property to serve the evolving needs of customers and industry. Founded in December 2013, the organization now has more than 150 members worldwide.

Further information
IBM: www.ibm.com
STFC Hartree Centre: www.stfc.ac.uk/hartree
NVIDIA: www.nvidia.com
Mellanox: www.mellanox.com
OpenPOWER Foundation: http://openpowerfoundation.org

Nagios alerts implemented via VoIP

Nagios Calling


Out of the box, Nagios can only alert you by email when a service fails, but these email messages can be easily overlooked. A call is more sustainable.
It's difficult to ignore a ringing phone, so it makes sense to teach Nagios how to make phone calls. You can do this with a combination of Nagios [1] and Asterisk [2]. Configuring Asterisk is less than trivial, and setting up a complete PBX just for monitoring is slightly over the top; however, you can sign up for a SIP (Session Initiation Protocol) account for not too much money, add a CLI SIP client and a simple shell script, and do the same thing with less overhead (Figure 1).
Figure 1: As you can see, the command that picks up the message – here, an email script – works.
First, the SIP account; you could opt for any VoIP provider. Setting up an account is trivial: Just complete the online form, wait for the confirmation email, and set up your account.

A Matching SIP Client

The process for the CLI SIP client is slightly more complex. Most VoIP clients are designed for the desktop and cannot be scripted at all, or at least not very well. Of the available clients, PJSUA [3] turns out to be the best choice. PJSUA is the PJSIP reference implementation, and it comprises a function library for SIP, RTP, STUN, and some other VoIP-related protocols.
Unfortunately, there are no PJSUA packages, which means building from the source code – although this is done easily enough. You can do the following to pick up the PJSIP/PJSUA source code:
wget http://www.pjsip.org/release/2.0./pjproject-2.0.1.tar.bz2
Next, go to the directory where you unpacked the tarred and zipped file and type:
./configure
make dep
make
You will now have a file starting with pjsua in your pjsip-apps/bin directory. This is the SIP client. You can copy it to anywhere you like on your system and make it executable. Next, you need a configuration file for the client; you define the access credentials for the SIP provider here.
To define the access credentials, create a text file with the content from Listing 1. You need to modify the content to match your own SIP provider, of course; this might be difficult in some cases because providers tend to use their own designations for id and realm. If in doubt, you can call your provider's hotline. --null-audio tells the server not use any audio output and thus avoids outputting calls via the computer's loudspeakers.
Listing 1
Provider Data
--null-audio
--registrar sip:sipgate.de
--realm=sipgate.de
--id sip:username@sipgate.de
--username sipgateusername
--password sipgatepasswort

The Call Script

Next, you need a script that Nagios can call when the need arises (Listing 2). This script also ensures that messages do not interfere with one another. If Nagios needs to make two calls in quick succession, the script ensures that the first call terminates before the second one begins. Otherwise, the client would refuse to cooperate because the line was occupied, and the second message would be lost.
Listing 2
Call Script
01 #! /bin/bash
02
03 EXPECT=/usr/bin/expect
04 PJSUA=/opt/pjsua
05 PJSUACONFIG=/opt/pjsua.conf
06 SOUNDFILE=/tmp/alert.wav
07 TEXT2WAVE=/usr/bin/text2wave
08 DURATION=20
09 NUMBER=01234567890
10 MESSAGE="Monitoring Alert"
11
12 # Setting a lock file
13 # We can't make more than one call
14 # at a time, because pjsua blocks the port
15 # so we have to make sure that nobody else tries to call
16 # If there is already a call we have to wait.
17
18 locked=false
19 while [[ $locked == false ]]; do
20     if [[ ! -f /tmp/caller.lock ]]; then
21         touch /tmp/caller.lock
22         locked=true
23     else
24         sleep 5
25     fi
26 done
27
28 # Generating the message
29 $TEXT2WAVE -o $SOUNDFILE -f 8000 << EOF
30 $MESSAGE
31 EOF
32
33 # Making the call it self.
34 # Expect will start pjsua and work with
35 # it so that it will end it self automatically
36
37 $EXPECT << EOF
38 spawn $PJSUA --config-file $PJSUACONFIG [UCC:z-inhbullet][/UCC]
   --play-file $SOUNDFILE --auto-play --duration $DURATION    --max-calls 1 sip:$NUMBER@sipgate.de
39 expect "VAD re-enabled"
40 sleep $DURATION
41 send "q\n"
42 EOF
43
44 # Cleaning up
45 rm $SOUNDFILE
46
47 # Removing the lock file
48 rm /tmp/caller.lock
The script needs two other tools: expect, to control pjsua, and text2wave, to create a WAV from a line of text. The text2wave tool is part of Festival, a program for voice synthesis [4]. Both packages can be installed via your package manager. Next, you should make the script in Listing 2 executable.
In the listing, you should set the value of NUMBER to the phone number you want Nagios to call. MESSAGE contains what you want the call recipient to hear. If you prefer not to hard wire the phone number into the script, you can pass the number in to Nagios. DURATION lets you configure the maximum duration for the call. If nobody picks up the phone within this time, the script hangs up. If somebody takes the call, the script plays the message in a loop. You may need to modify the last argument to match your own SIP provider in the line starting with spawn.

On My Command

Finally, you need to reconfigure Nagios to call the script in case of an alert. To do so, start by defining a new command. If you are using Ubuntu 12.04 and Nagios3, you will find the matching file in /etc/nagios3/resource.cfg. For other distributions, the path might differ slightly. Next, insert the content from Listing 3.
Listing 3
resource.cfg
01 # Additional commands, so that nagios can call the sysadmins
02 define command {
03     command_name notify-host-by-phone
04     command_line /opt/pjsua_wrapper
05 }
06
07 define command {
08     command_name notify-service-by-phone
09     command_line /opt/pjsua_wrapper
10 }
In Listing 3, /opt/pjsua_wrapper is the path to the script. You may also need to modify this. Then, you simply need to modify the contact definition for the administrator in Nagios. The configuration typically looks like this:
define contact {
    ...
    service_notification_commands notify-service-by-email
    host_notification_commands notify-service-by-email
    ...
}
Simply add the new commands to the two parameters:
define contact {
    ...
    service_notification_commands
    notify-service-by-email,notify-host-
    by-phone host_notification_commands
    notify-service-by-email,
    notify-service-by-phone
    ...
}
Finally, you can restart Nagios and wait for calls to come in.
The Author

Wednesday, January 13, 2016

awk print a line if matching a column

awk '$4 != "something"{print $0} ' input.file

print $0: print the whole line
if column 4 matches the text "something"



ACMG guidelines: http://www.nature.com/gim/journal/v15/n7/pdf/gim201373a.pdf

Friday, January 8, 2016

How to Create ZFS Filesystem with File Compression on Linux


ZFS filesystem is getting a wider recognition on Linux.
In ZFS, you can enable compression at the filesystem level. This will store the data in compressed format, which will save lot of disk space.
In this article, we’ll explain how to create the filesystem from the ZFS storage pool and enable compression on ZFS.

This is the 2nd article in the ZFS filesystem series.
In the first part of this series, we explained the fundamentals of ZFS, and how to install ZFS on linux. We also created a ZFS pool.

Create ZFS Filesystem

First, view all the current ZFS filesystems using zfs list command as shown below. In this example, we currently have one ZFS filesystem.
# zfs list
NAME     USED  AVAIL  REFER  MOUNTPOINT
mypool   296K  5.84G    30K  /mypool
Now, create a new ZFS filesystem using zfs create command.
# zfs create mypool/fs1
As we see below, the new ZFS filesystem is now created successfully.
 
# zfs list
NAME         USED  AVAIL  REFER  MOUNTPOINT
mypool       170K  5.84G    30K  /mypool
mypool/fs1    30K  5.84G    30K  /mypool/fs1

Set ZFS quote and Reservation

When you create a ZFS filesystem, by default it consumes all the space in the pool. So, you must specify a quota and reservation for the filesystem.
To set a quote, use zfs set command as shown below. Here we are specifying the quota as 1GB for this filesystem.
# zfs set quota=1G mypool/fs1
Next, set the reservation for the filesystem. In this example, fs1 is reserved 256M out of 5.59G so that no one can use this space and also it can extend up to 1G based on the quota we set if there is free space available.
# zfs set reservation=256M mypool/fs1

# zfs list
NAME         USED  AVAIL  REFER  MOUNTPOINT
mypool       256M  5.59G  32.5K  /mypool
mypool/fs1    30K  1024M    30K  /mypool/fs1

Create ZFS alternate Mount Point

Instead of mounting it using the “mypool/fs1″ name, you can also set an alternative mount point with any name that you wish for a filesystem.
For example, the following command will set the mount point as “/testmnt”, instead of “mypool/fs1″.
# zfs set mountpoint=/testmnt mypool/fs1
As we see from the following output, the first column NAME indicates the real name of the ZFS filesystem. The last column MOUNTPOINT indicates the alternative mount point that we created above.
# zfs list
NAME         USED  AVAIL  REFER  MOUNTPOINT
mypool       256M  5.59G  32.5K  /mypool
mypool/fs1    30K  1024M    30K  /testmnt
When you execute df command, you’ll see the alternative mount point as shown below.
# df -h
Filesystem                  Size  Used Avail Use% Mounted on
..
mypool                      5.6G  128K  5.6G   1% /mypool
mypool/fs1                  1.0G  128K  1.0G   1% /testmnt

Enable Compression on ZFS Filesystem

To set compression on a ZFS dataset, you can set the compression property as shown below. Once this property is set, any large files stored on this ZFS filesystem will be compressed.
# zfs set compression=lzjb mypool/fs1
The following are the valid compression properties:
  • on
  • off
  • lzjb
  • gzip
  • gzip[1-9]
  • zle
You can enable compression on an existing filesystem as well. In that case, the compression will be applied only to the new and modified data; and any existing data will remain uncompressed.

Verify ZFS Compression

In following example, we have copied the 61M tar file to the ZFS filesystem mypool/fs1 mounted under /testmnt.
# ls -lh /testmnt/test.tar
-rw-r--r--. 1 root root 61M Nov 11 09:44 /testmnt/test.tar
If you look at the total size of USED space from the zfs list command, you will see only 20.9M space is consumed which indicates that compression is turned on and working.
# zfs list
NAME         USED  AVAIL  REFER  MOUNTPOINT
mypool       256M  5.59G  32.5K  /mypool
mypool/fs1  20.9M  1003M  20.9M  /testmnt
You can also get the compression ratio using the following command.
# zfs get compressratio mypool/fs1
NAME        PROPERTY       VALUE  SOURCE
mypool/fs1  compressratio  2.90x  -
Apart from compression, ZFS filesystem has several advanced features. In the next article of the ZFS series, we’ll discuss about how to take ZFS clones and snapshots.

ReactOS: Free Open Sourced Windows?

ReactOS Shop

 ReactOS Project

ReactOS® is a free open source operating system based on the best design principles found in the Windows NT® architecture (Windows versions such as Windows XP, Windows 7, Windows Server 2012 are built on Windows NT architecture). Written completely from scratch, ReactOS is not a Linux based system, and shares none of the UNIX architecture.
The main goal of the ReactOS® project is to provide an operating system which is binary compatible with Windows. This will allow your Windows® applications and drivers to run as they would on your Windows system. Additionally, the look and feel of the Windows operating system is used, such that people accustomed to the familiar user interface of Windows® would find using ReactOS straightforward. The ultimate goal of ReactOS® is to allow you to use it as alternative to Windows® without the need to change software you are used to.
ReactOS 0.3.17 is still in alpha stage, meaning it is not feature-complete and is recommended only for evaluation and testing purposes.

https://reactos.org/

Thursday, January 7, 2016

Will 2015 Be the Year of the FPGA?

Nicole Hemsoth
digital highway abstract 300x
When it comes to HPC, the accelerator market is largely dominated by GPUs, which are known for floating point performance prowess with IDC figures pointing to encroaching competition from Xeon Phi. FPGAs, while still a third-place contender, might have a rather remarkable year ahead in terms of their reach into wider markets, as well as in the “three Ps” arena with increasingly attractive price, performance, and programmability.
With the arrival of “big data” in HPC, hyperscale, and general commercial environments FPGAs have found a comfortable home in a broad range of applications that can be very broadly defined as “search” based—in other words, from actual web search (as is the case with Microsoft’s adoption of FPGAs to power Bing) to search and discovery applications in bioinformatics, security, image recognition and beyond. Last week at Hot Chips, Asian web giant Baidu revealed how they’re using field programmable gate arrays from Xilinx to boost a number of their pattern and image recognition capabilities with great success, and of course, they still have a place in one of the markets that’s always been a sweet spot in financial services.
It’s important to note that at this point, it’s not always a clear either/or question when it comes to GPUs and FPGAs in a number of application areas in this wide “search” classification. There are areas in what can broadly be termed “deep learning” that can benefit from both accelerator options—and to say nothing of the new wave of DSP-powered applications that will emerge. Although GPUs have a hefty head start in expanded markets with advanced programming tools in CUDA and OpenCL, efforts underway with OpenPower, Intel (which recently soft-announced its own FPGA-tuned chips) and ARM in particular could promise a brighter, broader landscape for FPGAs. Others, including Altera and Xilinx and specialized designs from companies like Adapteva are also on the watchlist for big things in 2015.
While a great deal of the more recent adoption of FPGAs has been centered on hyperscale and commercial work, there are still hot areas in HPC that are set to benefit from the expanded ecosystem around the accelerators. According to Convey’s CEO, Bruce Toal, they’re seeing great adoption in both hyperscale and research—particularly in the life sciences. And this is not expected to slow, even if their status as a vendor offering high performance FPGA-driven systems via their time-honed hybrid core integrated FPGA technology may change as new integrated offerings hit the shelves, especially those from close partner Intel.
“There’s never been more activity around FPGAs than we’re seeing now,” Toal said. “We’ve seen Moore’s Law have the expected impact on size and price performance so that the latest 20 and 28-nanometer FPGAs are really delivering a price performance point that hasn’t been possible to date—allowing very small devices to be integrated in a much higher volume sense in computing, rather than just on a specialized basis.”
While their software, tooling, and integration of the current hybrid core technology stands on its own for now, Toal said he’s recognized for a long time that FPGAs were heading toward further integration—a step that he isn’t resisting. While he’s thrilled to see the uptick in growth in capability and options for FPGAs, it does raise questions about how the business model for a company like Convey, which is selling integrated FPGAs that don’t offer quite the same tight integration that Intel and IBM. However, Toal said he is confident that their expertise and ability to adapt to the changing market will continue to place them ahead of the game.
To that end, Convey joined the OpenPower Foundation ranks to add their hybrid-core acceleration to the Power mix. “Our current portfolio of accelerated solutions coupled with the POWER architecture and IBM’s Coherent Accelerator Processor Interface (CAPI) offer data center customers the ability to deploy high performing, cost efficient solutions,” he said.
CAPI is that key dedicated piece of IP on the Power8 that exploits the coherency of the device itself in a very clean simple manner, which means it’s possible interact with the main memory of the processor in a low latency environment versus making unconnected hops. Although they haven’t shipped any systems sporting these new capabilities, Toal said he recognizes the high value of getting away from doing this over PCIe with cache coherency layered on top. He also noted that when IBM pulls through with the promise to keep pushing the memory bandwidth of Power8, performance will be taken to an entirely new level for users in hyperscale, bioinformatics and beyond.
“X86 has been dominant in HPC but the Power architecture has always been recognized as a capable architecture for HPC and other areas,” Toal noted. The OpenPower initiative has been gaining steam with notable sign-ins from Google and companies like Tyan building motherboards to meet demand, particularly in the hyperscale market. As for their HPC roots, Toal said there will be a great deal of interest going forward, especially as users begin to realize the programmability and features that are found in both integrated and co-processor approaches to FPGAs.
When that day comes, the real decisions about which FPGA-based solutions to ride with will come down to sheer price performance—not to mention end results. He said that even before the OpenPower CAPI work that’s been done, his team’s work on an image sizing application they deployed with partner Dell is getting a 48x performance improvement using their integrated hybrid core FPGA solution. When it’s all meshed together and tied with a CAPI string, the FPGA boxes of the future will be quite the present for the big search-based application areas in particular.