Pages

Friday, October 3, 2014

Changing qpidd to rabbitmq for Openstack


Stopping qpidd on all server

service qpidd stop
chkconfig qpidd off

Installing packages
yum install ntp rabbitmq-server

Configuring Cluster
On Controller1
yum install ntp rabbitmq-server
chown rabbitmq:rabbitmq /var/lib/rabbitmq/*
chown rabbitmq:rabbitmq /var/lib/rabbitmq/.erlang.cookie
/etc/init.d/rabbitmq-server restart
scp /var/lib/rabbitmq/.erlang.cookie root@controller2:/var/lib/rabbitmq/


On Controller2
yum install ntp rabbitmq-server
yum install ntp rabbitmq-server
rabbitmqctl stop_app
rabbitmqctl join_cluster rabbit@controller1
rabbitmqctl start_app
rabbitmqctl cluster_status


Edit the configuration Files
Comment out the Qpidd entries In all the configuration and enter the below details. As Rabbitmq is default message broker other entries are not needed.

rabbit_host = controller

In Both the controller set the rabbitmq policy as below


rabbitmqctl set_policy HA '^(?!amq\.).*' '{"ha-mode": "all"}'

Monday, September 29, 2014

Configure HA using Corosync and pacemaker

Opening needed Ports in Iptables if We are using IPtables
/etc/sysconfig/iptables. Towards the end of the file, but before any REJECT statements, we add the following lines:
-A INPUT -p udp -m state --state NEW -m multiport --dports 5404,5405 -j ACCEPT
-A INPUT -m tcp -p tcp --dport 7788 -j ACCEPT
-A INPUT -m tcp -p tcp --dport 3306 -j ACCEPT

Installing modules
yum -y install wget
rpm -Uvh http://elrepo.org/elrepo-release-6-5.el6.elrepo.noarch.rpm
yum -y install drbd84-utils kmod-drbd84 --enablerepo=elrepo
yum -y install pacemaker corosync cluster-glue

wget -P /etc/yum.repos.d/ http://download.opensuse.org/repositories/network:/ha-clustering:/Stable/CentOS_CentOS-6/network:ha-clustering:Stable.repo
yum install crmsh

Configure Corosync 

vi /etc/corosync/corosync.conf
totem {
version: 2
secauth: off
threads: 0
interface {
ringnumber: 0
bindnetaddr: 10.0.0.0
mcastaddr: 226.94.1.1
mcastport: 5405
ttl: 1
}
}

logging {
fileline: off
to_stderr: no
to_logfile: yes
to_syslog: yes
logfile: /var/log/cluster/corosync.log
debug: off
timestamp: on
logger_subsys {
subsys: AMF
debug: off
}
}

amf {
mode: disabled
}

service {
        # Load the Pacemaker Cluster Resource Manager
        ver:       1
        name:      pacemaker
}

aisexec {
        user:   root
        group:  root
}


chkconfig --level 3 corosync on
service corosync start
chkconfig --level 3 pacemaker on
service pacemaker start

Checking the Cluster Connectivity
corosync-objctl runtime.totem.pg.mrp.srp.members

Check the service and cluster status
crm_mon -1


Configuring the cluster
>>crm configure
property no-quorum-policy="ignore" pe-warn-series-max="1000" pe-input-series-max="1000" pe-error-series-max="1000" cluster-recheck-interval="5min"
property stonith-enabled=false
commit

Adding a Cluster server for common IP (VIP)
>>crm configure
primitive p_api-ip ocf:heartbeat:IPaddr2 params ip="10.0.0.199" cidr_netmask="24" op monitor interval="30s"
commit

Now we need to configure the needed services in the CRM.

  

Thursday, September 25, 2014

Checking loadspeed of a Site Using phantomjs

Using  phantomjs to check different parameters of a site.

 Installing the module
sudo yum install fontconfig freetype libfreetype.so.6 libfontconfig.so.1 libstdc++.so.6

wget https://bitbucket.org/ariya/phantomjs/downloads/phantomjs-1.9.7-linux-i686.tar.bz2
tar jxvf phantomjs-1.9.7-linux-i686.tar.bz2
mv phantomjs-1.9.7-linux-x86_64 phantomjs
export PATH=$PATH:/root/phantomjs/bin/

In the  phantomjs directory there are examples for js script. We use a test script provided by them for checking the load speed of site. 

phantomjs loadspeed.js http://www.adminz.in
Page title is Linux Conquering Cloud
Loading time 4559 msec


Basic examples

  • arguments.js shows the arguments passed to the script
  • countdown.js prints a 10 second countdown
  • echoToFile.js writes the command line arguments to a file
  • fibo.js lists the first few numbers in the Fibonacci sequence
  • hello.js displays the famous message
  • module.js and universe.js demonstrate the use of module system
  • outputEncoding.js displays a string in various encodings
  • printenv.js displays the system's environment variables
  • scandir.js lists all files in a directory and its subdirectories
  • sleepsort.js sorts integers and delays display depending on their values
  • version.js prints out PhantomJS version number
  • page_events.js prints out page events firing: useful to better grasp page.on* callbacks

Rendering/rasterization

  • colorwheel.js creates a color wheel using HTML5 canvas
  • rasterize.js rasterizes a web page to image or PDF
  • rendermultiurl.js renders multiple web pages to images
  • technews.js captures Google News as a PNG image

Page automation

  • direction.js uses Google Maps to print driving direction
  • follow.js shows the number of followers of some Twitter accounts
  • imagebin.js uploads an image to imagebin.org
  • injectme.js injects itself into a web page context
  • ipgeocode.js deduces the location via IP geocoding
  • movies.js lists movies from kids-in-mind.com
  • phantomwebintro.js uses jQuery to read #intro element text from phantomjs.org
  • pizza.js uses yelp.com to find pizza places in Mountain View
  • seasonfood.js displays the BBC seasonal food list
  • tweets.js displays the most recent tweets
  • unrandomize.js modifies a global object at page initialization
  • waitfor.js waits until a test condition is true or a timeout occurs

Network

  • detectsniff.js detects if a web page sniffs the user agent
  • loadspeed.js computes the loading speed of a web site
  • netlog.js dumps all network requests and responses
  • netsniff.js captures network traffic in HAR format
  • post.js sends an HTTP POST request to a test server
  • postserver.js starts a web server and sends an HTTP POST request to it
  • server.js starts a web server and sends an HTTP GET request to it
  • serverkeepalive.js starts a web server which answers in plain text
  • simpleserver.js starts a web server which answers in HTML

Script to check the loading time of a Site

Script to check the loading time of a Site

========
#!/bin/bash
CURL="/usr/bin/curl"
GAWK="/usr/bin/gawk"
echo -n "Please pass the url you want to measure: "
read url
URL="$url"
result=`$CURL -o /dev/null -s -w %{time_connect}:%{time_starttransfer}:%{time_total} $URL`
echo " Time_Connect Time_startTransfer Time_total "
echo $result | $GAWK -F: '{ print $1" "$2" "$3}'
========

cat test.sh
#!/bin/bash
CURL="/usr/bin/curl"
GAWK="/usr/bin/gawk"
echo -n "Please pass the url you want to measure: "
read url
URL="$url"
result=`$CURL -o /dev/null -s -w %{time_connect}:%{time_starttransfer}:%{time_total} $URL`
echo " Time_Connect Time_startTransfer Time_total "
echo $result | $GAWK -F: '{ print $1" "$2" "$3}'

Sample Testing
[root@vps examples]# sh test.sh
Please pass the url you want to measure: http://www.adminz.in
 Time_Connect Time_startTransfer Time_total
0.294 0.604 1.255
[root@vps examples]#

BASH Shellshock vulnerability and FIX


It allows the attacker to specify arbitrary commands to execute by changing an environment variable in a specific way. Bash is the default command interpreter for Linux and many other Unix versions and is consequently widespread use. But by itself the vulnerability is not that terrible, after all it is a local vulnerability and BASH is a command interpreter, its only reason to exist is to execute commands, so not such a big deal...

Unfortunately this is not quite true as we need to look at how Bash is used. True in its normal form as command interpreter the attack vectors are quite small. However Bash is very often involved in a networked setup to execute commands and that opens up an interesting attack vector. Imagine a webserver that allows you to ping an IP address (my router at home has that function for example), it will most likely just call the "ping" executable with the argument that you supplied, probably checking whether the argument is formatted correctly as an IP address.


RedHat has an extended list of situations that involve Bash in a remote context and you can see it has the potential be a widespread problem, similar to Heartbleed in April. Some of the security researchers involved at the time, namely @ErrataRob have already started their Internet wide scans looking for vulnerable servers:

  • Apache server using mod_cgi or mod_cgid are affected if CGI scripts are either written in bash, or spawn subshells. Such subshells are implicitly used by system/popen in C, by os.system/os.popen in Python, system/exec in PHP (when run in CGI mode), and open/system in Perl if a shell is used (which depends on the command string)
  • ForceCommand is used in sshd configs to provide limited command execution capabilities for remote users. This flaw can be used to bypass that and provide arbitrary command execution. Some Git and Subversion deployments use such restricted shells. Regular use of OpenSSH is not affected because users already have shell access.
  • DHCP clients invoke shell scripts to configure the system, with values taken from a potentially malicious server. This would allow arbitrary commands to be run, typically as root, on the DHCP client machine.
  • Various daemons and SUID/privileged programs may execute shell scripts with environment variable values set / influenced by the user, which would allow for arbitrary commands to be run.
  • Any other application which is hooked onto a shell or runs a shell script as using bash as the interpreter. Shell scripts which do not export variables are not vulnerable to this issue, even if they process untrusted content and store it in (unexported) shell variables and open subshells.


To check the Vulnerability 

env x='() { :;}; echo vulnerable' bash -c 'echo hello'

If you get an out put like
==
Vulnerable
hello
==
The Bash is said to be Vulnerable 

If you get an output like 
====
bash: warning: x: ignoring function definition attempt
bash: error importing function definition for `x'
hello
====
The Bash is not Vulnerable.

Fixes,
 For redhat ,centos ,debian and ubuntu the patches are already available in the repos 

In redhat/Centos
yum update bash

In debian/Ubuntu 
apt-get update && apt-get install --only-update bash

Or else if you want to Compile and install the latest bash 


apt-get install wget patch gcc make

yum install wget patch gcc make 

mkdir src
cd src
wget http://ftp.gnu.org/gnu/bash/bash-4.3.tar.gz
#download all patches
for i in $(seq -f "%03g" 0 25); do wget     http://ftp.gnu.org/gnu/bash/bash-4.3-patches/bash43-$i; done
tar zxvf bash-4.3.tar.gz 
cd bash-4.3
#apply all patches
for i in $(seq -f "%03g" 0 25);do patch -p0 < ../bash43-$i; done
#build and install
./configure && make && make install
cd .. 
cd ..
rm -r src

Once patches are applied check the vulnerability again and make sure its fine.
More updates are been coming regarding this Will keep you updated. 

Workaround: Using mod_security:

The following mod_security rules can be used to reject HTTP requests containing data that may be interpreted by Bash as function definition if set in its environment. They can be used to block attacks against web services, such as attacks against CGI applications outlined above.
Request Header values:
SecRule REQUEST_HEADERS "^\(\) {" "phase:1,deny,id:1000000,t:urlDecode,status:400,log,msg:'CVE-2014-6271 - Bash Attack'"
SERVER_PROTOCOL values:
SecRule REQUEST_LINE "\(\) {" "phase:1,deny,id:1000001,status:400,log,msg:'CVE-2014-6271 - Bash Attack'"
GET/POST names:
SecRule ARGS_NAMES "^\(\) {" "phase:2,deny,id:1000002,t:urlDecode,t:urlDecodeUni,status:400,log,msg:'CVE-2014-6271 - Bash Attack'"
GET/POST values:
SecRule ARGS "^\(\) {" "phase:2,deny,id:1000003,t:urlDecode,t:urlDecodeUni,status:400,log,msg:'CVE-2014-6271 - Bash Attack'"
File names for uploads:
SecRule FILES_NAMES "^\(\) {" "phase:2,deny,id:1000004,t:urlDecode,t:urlDecodeUni,status:400,log,msg:'CVE-2014-6271 - Bash Attack'"
These may result in false positives but it's unlikely, and they can log them and keep an eye on it. You may also want to avoid logging as this could result in a significant amount of log files.

Workaround: Using IPTables:

A note on using IPTables string matching:
iptables using -m string --hex-string '|28 29 20 7B|'
Is not a good option because the attacker can easily send one or two characters per packet and avoid this signature easily. However, it may provide an overview of automated attempts at exploiting this vulnerability.

Tuesday, September 23, 2014

HAproxy Load Balancing Algorithms

HAproxy Load Balancing Algorithms
The algoritm you define determines how HAproxy balances load across your servers. You can set the algorithm to use with the balance parameter.

Round Robin
Requests are rotated among the servers in the backend.

Servers declared in the backend section also accept a weight parameter which specifies their relative weight. When balancing load, the Round Robin algorithm will respect that weight ratio.
Example:

...
option tcplog
balance roundrobin
maxconn 10000
...
Static Round Robin
Each server is used in turn, according to the defined weight for the server. This algorithm is a static version of the round-robin algoritm, which means that changing the weight ratio for a server on the fly will have no effect. However, you can define as many servers as you like with this algorithm. In addition, when a server comes online, this algoritm ensures that the server is immediately reintroduced into the farm after re-computing the full map. This algoritm also consome slightly less CPU cycles (around -1%).

Example:

...
option tcplog
balance static-rr
maxconn 10000
...
Least Connection
Each server is used in turn, according to the defined weight for the server. This algorithm is a static version of the round-robin algoritm, which means that changing the weight ratio for a server on the fly will have no effect. However, you can define as many servers as you like with this algorithm. In addition, when a server comes online, this algoritm ensures that the server is immediately reintroduced into the farm after re-computing the full map. This algoritm also consome slightly less CPU cycles than the Round Robin algorithm (around -1%).

Example:

...
option tcplog
balance leastconn
maxconn 10000
...
Source
A hash of the source IP is divided by the total weight of the running servers to determine which server will receive the request. This ensures that clients from the same IP address always hit the same server, which is a poor man's session persistence solution.

Example:

...
option tcplog
balance source
maxconn 10000
...
URI
This algorithm hashes either the left part of the URI (before the question mark) or the whole URI (if the whole parameter is present) and divides the hash value by the total weight of the running servers. The result designates which server will receive the request. This ensures that the proxy will always direct the same URI to the same server as long as all servers remain online.

This is used with proxy caches and anti-virus proxies in order to maximize the cache hit rate. This algorithm is static by default, which means that changing a server's weight on the fly will have no effect. However, you can change this using a hash-type parameter.

You can only use this algorithm for a configuration with an HTTP backend.
Exampple:

...
option tcplog
balance uri
maxconn 10000
...
URL Parameter
The URL parameter specified in argument will be looked up in the query string of each HTTP GET request.

You can use this algorithm to check specific parts of the URL, such as values sent through POST requests. For example, you can set this algorithm to direct a request that specifies a user_id with a specific value to the same server using the url_param method. Essentially, this is another way of achieving session persistence in some cases (see the official HAproxy documentation for more information).

Example:

...
option tcplog
balance url_param userid
maxconn 10000
...
or

...
option tcplog
balance url_param session_id check_post 64
maxconn 10000
...

Friday, September 19, 2014

Openstack Heat : Installing Applications along with the heat template

Installing Applications along with the heat template. If needed we can mention the network ID, Image ID etc in the file itself instead of asking it from outside.
================================================
heat_template_version: 2013-05-23

description: Test Template

parameters:
NAME:
type: string
description : Instance Name
ImageID:
type: string
description: Image use to boot a server
NetID:
type: string
description: Network ID for the server

resources:
server1:
type: OS::Nova::Server
properties:
name: { get_param: NAME }
image: { get_param: ImageID }
key_name: Cloud
flavor: "m1.small"
networks:
- network: { get_param: NetID }
user_data_format: RAW
user_data: |
#!/bin/bash -v
echo "nameserver 8.8.8.8" > /etc/resolv.conf
yum update -y
yum install httpd -y

outputs:
server1_private_ip:
description: IP address of the server in the private network
value: { get_attr: [ server1, first_address ] }
================================================

heat stack-create -f heat.yml -P "ImageID=abc9818d-ee5f-4778-ada5-a29105ea9c02;NetID=71ed8a34-a2d5-4d84-9d47-e5e107dd8d7e" Centos-Stack