Neutron + Pacemaker for HA Gives error

I was trying to configure HA for neutron server in icehouse implementation. I was able to set up ha for all other services except neutron. I was trying to use pacemaker for setting up HA  by following http://docs.openstack.org/high-availability-guide/content/_add_neutron_l3_agent_resource_to_pacemaker.html

but still i get following error. dhcp agent and metadata agent is showing no error but l3 agent is not working.

output of crm_mon -1

Last updated:FriJul1814:03:252014Last change:FriJul1813:54:042014 via cibadmin on network1 Stack: classic openais (with plugin)Current DC: network2 - partition with quorum Version:1.1.10-14.el6_5.3-368c7262Nodes configured,2 expected votes 4Resources configured

Online:[ network1 network2 ]

p_api-ip (ocf::heartbeat:IPaddr2):Started network2

p_neutron-dhcp-agent (ocf::openstack:neutron-dhcp-agent):Started network1

p_neutron-metadata-agent (ocf::openstack:neutron-metadata-agent):Started network1

Failed actions: p_neutron-l3-agent_start_0 on network2 'unknown error'(1): call=13, status=TimedOut,last-rc-change='Fri Jul 18 04:32:06 2014', queued=20091ms,exec=0ms p_neutron-l3-agent_start_0 on network1 'unknown error'(1): call=23, status=TimedOut,last-rc-change='Fri Jul 18 14:03:01 2014', queued=20010ms,exec=0ms[root@network1 openstack]#

Solution

The neutron-agent-l3 script to blame as it tries to communicate with neutron server
directly on port 9696, while communication is handled by AMQP service
(Qpid in my case). We need to modify the script to use Qpid port and not neutron server one.

Enable HAProxy logging on Centos

Enable HAProxy logging on CentOS
By default, HAProxy will not log to files unless we make some modifications
1. Create rsyslog configuration file
nano /etc/rsyslog.d/haproxy.conf
Add these lines to the file
# Enable UDP port 514 to listen to incoming log messages from haproxy
$ModLoad imudp
$UDPServerRun 514
$template Haproxy,"%msg%\n"
local0.=info -/var/log/haproxy/haproxy.log;Haproxy
local0.notice -/var/log/haproxy/admin.log;Haproxy
# don't log anywhere else
local0.* ~
Restart rsyslog service
/etc/init.d/rsyslog restart
Ref: http://blog.hintcafe.com/post/33689067443/haproxy-logging-with-rsyslog-on-linux
2. Modify the log rotate config to match the new folder:
nano /etc/logrotate.d/haproxy
Change
/var/log/haproxy.log {
daily
rotate 10
missingok
[...]
to
/var/log/haproxy/*.log {
daily
rotate 10
missingok
[...]
Now we can check if HAProxy logging is working.
tail -f /var/log/haproxy/haproxy.log

===================================================

global to have these messages end up in /var/log/haproxy.log you will need to:

1) configure syslog to accept network log events. This is done by adding the '-r' option to the SYSLOGD_OPTIONS in /etc/sysconfig/syslog

2) configure local2 events to go to the /var/log/haproxy.log file. A line like the following can be added to /etc/sysconfig/syslog
local2.* /var/log/haproxy.log

In haproxy conf file add
log 127.0.0.1 local2 info

Openstack+ Shared Storage(NFS) +Permission denied

While setting up Openstack with shared storage , If we get following error while creating a instance ensure that the server is in permissive selinux mode with

getenfonce

and if that too doesn't work try giving 755 permission to the mounted directory here its /var/lib/nova

2014-06-24 18:58:20.642 5119 TRACE nova.compute.manager [instance: a7996f1f-9af2-4410-8351-139d43f00786] libvirtError: internal error Process exited while reading console log output: qemu-kvm: -chardev file,id=charserial0,path=/var/lib/nova/instances/a7996f1f-9af2-4410-8351-139d43f00786/console.log: Could not open '/var/lib/nova/instances/a7996f1f-9af2-4410-8351-139d43f00786/console.log': Permission denied

 

At last if nothing else worked , tell libvirtd/qemu to use root user to access datas.
[root@compute nova]# cat /etc/libvirt/qemu.conf |grep root
user = "root"
#group = "root"
[root@compute nova]#

Openstack Icehouse Part 6 : Testing the Setup + Horizon

Creating the Key

$ ssh-keygen

Add the public key to your OpenStack environment:
$ nova keypair-add --pub-key ~/.ssh/id_rsa.pub demo-key

Verify addition of the public key:
$ nova keypair-list
+----------+-------------------------------------------------+
| Name | Fingerprint |
+----------+-------------------------------------------------+
| demo-key | 6c:74:ec:3a:08:05:4e:9e:21:22:a6:dd:b2:62:b8:28 |
+----------+-------------------------------------------------+

nova image-list
+--------------------------------------+---------------------+--------+--------+
| ID | Name | Status | Server |
+--------------------------------------+---------------------+--------+--------+
| acafc7c0-40aa-4026-9673-b879898e1fc2 | cirros-0.3.2-x86_64 | ACTIVE | |
+--------------------------------------+---------------------+--------+--------+
Your first instance uses the cirros-0.3.2-x86_64 image.

List available networks:

$ neutron net-list
+--------------------------------------+----------+-------------------------------------------------------+
| id | name | subnets |
+--------------------------------------+----------+-------------------------------------------------------+
| 3c612b5a-d1db-498a-babb-a4c50e344cb1 | demo-net | 20bcd3fd-5785-41fe-ac42-55ff884e3180 192.168.1.0/24 |
| 9bce64a3-a963-4c05-bfcd-161f708042d1 | ext-net | b54a8d85-b434-4e85-a8aa-74873841a90d 203.0.113.0/24 |
+--------------------------------------+----------+-------------------------------------------------------+
Your first instance uses the demo-net tenant network. However, you must reference this network using the ID instead of the name.

List available security groups:

$ nova secgroup-list
+--------------------------------------+---------+-------------+
| Id | Name | Description |
+--------------------------------------+---------+-------------+
| ad8d4ea5-3cad-4f7d-b164-ada67ec59473 | default | default |
+--------------------------------------+---------+-------------+

Creating a New flavor

nova boot --flavor m1.tiny --image cirros-0.3.2-x86_64 --nic net-id=8cc217b0-96a6-4e98-a901-a694ebff173f --security-group default --key-name demo-key demo-instance1

Creating the instance from back end

nova boot --poll --flavor m1.tiny --image cirros-0.3.2-x86_64 --nic net-id=69c6ca95-2f5d-4173-8973-164c5129cb27 --security-group default --key-name Chumma demo-instance

 

Install Horizone on Controller Node

yum install memcached python-memcached mod_wsgi openstack-dashboard

Edit /etc/openstack-dashboard/local_settings:

ALLOWED_HOSTS = ['localhost', 'my-desktop']
service httpd start
service memcached start
chkconfig httpd on
chkconfig memcached on

 

Openstack Icehouse Part 5 : Configuring EXTERNAL NETWORK

To create the external network on controller

source /root/admin-openrc.sh

neutron net-create ext-net --shared --router:external=True

To create a subnet on the external network

neutron subnet-create ext-net --name ext-subnet --allocation-pool start=192.168.255.160,end=192.168.255.180 --disable-dhcp --gateway 192.168.255.2 ext_net 192.168.255.0/24
#To create the tenant network

source /root/demo-openrc.sh

#Create the network:

neutron net-create demo-net

#To create a subnet on the tenant network

neutron subnet-create demo-net --name demo-subnet --gateway 10.0.0.1 10.0.0.0/24

#Create the router:

neutron router-create demo-router

#Attach the router to the demo tenant subnet:

neutron router-interface-add demo-router demo-subnet

#Added interface b1a894fd-aee8-475c-9262-4342afdc1b58 to router demo-router.

neutron router-gateway-set demo-router ext-net

#Set gateway for router demo-router

 

Now check whether the gateway of the external-network here it will be 192.168.255.160 which is first Ip of the range. Try pinging to the IP and if its not working Stop there and remove all the routers and gateway and redo it using the ID.  If the gate way Don't get pinged the instance won't be able to access outside the network.

 

Set the neutron router-interface-add  and neutron router-gateway-set  BY ID

neutron router-list

+--------------------------------------+-------------+--------------------------------------------------------+

| id                                   | name        | external_gateway_info                                  |

+--------------------------------------+-------------+--------------------------------------------------------+

| 020f48d9-182e-4e33-a73f-813333533092 | router-demo | {"network_id": "9a457578-8f85-486b-9cd0-f7f04922ba0c"} |

+--------------------------------------+-------------+--------------------------------------------------------+

# neutron net-list

+--------------------------------------+----------+----------------------------------------------------+

| id                                   | name     | subnets                                            |

+--------------------------------------+----------+----------------------------------------------------+

| 07e10f48-0637-46bb-a444-695646e6bd15 | net-demo | c042e65e-3892-45bc-aeb0-625ce5f4aaaf 50.50.1.0/24  |

| 9a457578-8f85-486b-9cd0-f7f04922ba0c | ext_net  | 0bcccf59-be17-48c7-8032-e00fd4f15b46 1.2.3.0/24 |

+--------------------------------------+----------+----------------------------------------------------+

#neutron router-gateway-set 020f48d9-182e-4e33-a73f-813333533092 9a457578-8f85-486b-9cd0-f7f04922ba0c

Openstack Icehouse Part 4 Neutron

OpenStack Networking (neutron) Configure controller node

$ mysql -u root -p
CREATE DATABASE neutron;
GRANT ALL PRIVILEGES ON neutron.* TO 'neutron'@'localhost' IDENTIFIED BY 'neutron4mar';
GRANT ALL PRIVILEGES ON neutron.* TO 'neutron'@'%' IDENTIFIED BY 'neutron4mar';
exit

keystone user-create --name neutron --pass neutron4mar --email neutron@example.com
keystone user-role-add --user neutron --tenant service --role admin
keystone service-create --name neutron --type network --description "OpenStack Networking"
keystone endpoint-create --service-id $(keystone service-list | awk '/ network / {print $2}') --publicurl http://controller:9696 --adminurl http://controller:9696 --internalurl http://controller:9696

To install the Networking components

# yum -y install openstack-neutron openstack-neutron-ml2 python-neutronclient

openstack-config --set /etc/neutron/neutron.conf database connection mysql://neutron:neutron4mar@controller/neutron

openstack-config --set /etc/neutron/neutron.conf DEFAULT auth_strategy keystone
openstack-config --set /etc/neutron/neutron.conf keystone_authtoken auth_uri http://controller:5000
openstack-config --set /etc/neutron/neutron.conf keystone_authtoken auth_host controller
openstack-config --set /etc/neutron/neutron.conf keystone_authtoken auth_protocol http
openstack-config --set /etc/neutron/neutron.conf keystone_authtoken auth_port 35357
openstack-config --set /etc/neutron/neutron.conf keystone_authtoken admin_tenant_name service
openstack-config --set /etc/neutron/neutron.conf keystone_authtoken admin_user neutron
openstack-config --set /etc/neutron/neutron.conf keystone_authtoken admin_password neutron4mar

openstack-config --set /etc/neutron/neutron.conf DEFAULT rpc_backend neutron.openstack.common.rpc.impl_qpid
openstack-config --set /etc/neutron/neutron.conf DEFAULT qpid_hostname controller
openstack-config --set /etc/neutron/neutron.conf DEFAULT notify_nova_on_port_status_changes True
openstack-config --set /etc/neutron/neutron.conf DEFAULT notify_nova_on_port_data_changes True
openstack-config --set /etc/neutron/neutron.conf DEFAULT nova_url http://controller:8774/v2
openstack-config --set /etc/neutron/neutron.conf DEFAULT nova_admin_username nova
openstack-config --set /etc/neutron/neutron.conf DEFAULT nova_admin_tenant_id $(keystone tenant-list | awk '/ service / { print $2 }')
openstack-config --set /etc/neutron/neutron.conf DEFAULT nova_admin_password nova4mar
openstack-config --set /etc/neutron/neutron.conf DEFAULT nova_admin_auth_url http://controller:35357/v2.0

openstack-config --set /etc/neutron/neutron.conf DEFAULT core_plugin ml2
openstack-config --set /etc/neutron/neutron.conf DEFAULT service_plugins router

openstack-config --set /etc/neutron/plugins/ml2/ml2_conf.ini ml2 type_drivers gre
openstack-config --set /etc/neutron/plugins/ml2/ml2_conf.ini ml2 tenant_network_types gre
openstack-config --set /etc/neutron/plugins/ml2/ml2_conf.ini ml2 mechanism_drivers openvswitch
openstack-config --set /etc/neutron/plugins/ml2/ml2_conf.ini ml2_type_gre tunnel_id_ranges 1:1000
openstack-config --set /etc/neutron/plugins/ml2/ml2_conf.ini securitygroup firewall_driver neutron.agent.linux.iptables_firewall.OVSHybridIptablesFirewallDriver
openstack-config --set /etc/neutron/plugins/ml2/ml2_conf.ini securitygroup enable_security_group True

openstack-config --set /etc/nova/nova.conf DEFAULT network_api_class nova.network.neutronv2.api.API
openstack-config --set /etc/nova/nova.conf DEFAULT neutron_url http://controller:9696
openstack-config --set /etc/nova/nova.conf DEFAULT neutron_auth_strategy keystone
openstack-config --set /etc/nova/nova.conf DEFAULT neutron_admin_tenant_name service
openstack-config --set /etc/nova/nova.conf DEFAULT neutron_admin_username neutron
openstack-config --set /etc/nova/nova.conf DEFAULT neutron_admin_password neutron4mar
openstack-config --set /etc/nova/nova.conf DEFAULT neutron_admin_auth_url http://controller:35357/v2.0
openstack-config --set /etc/nova/nova.conf DEFAULT linuxnet_interface_driver nova.network.linux_net.LinuxOVSInterfaceDriver
openstack-config --set /etc/nova/nova.conf DEFAULT firewall_driver nova.virt.firewall.NoopFirewallDriver
openstack-config --set /etc/nova/nova.conf DEFAULT security_group_api neutron

ln -s /etc/neutron/plugins/ml2/ml2_conf.ini /etc/neutron/plugin.ini

service openstack-nova-api restart
service openstack-nova-scheduler restart
service openstack-nova-conductor restart

service neutron-server start
chkconfig neutron-server on

Neutron ON NETWORK NODE

Edit /etc/sysctl.conf to contain the following:
net.ipv4.ip_forward=1
net.ipv4.conf.all.rp_filter=0
net.ipv4.conf.default.rp_filter=0
Implement the changes:
sysctl -p

yum -y install openstack-neutron openstack-neutron-ml2 openstack-neutron-openvswitch

openstack-config --set /etc/neutron/neutron.conf DEFAULT auth_strategy keystone
openstack-config --set /etc/neutron/neutron.conf keystone_authtoken auth_uri http://controller:5000
openstack-config --set /etc/neutron/neutron.conf keystone_authtoken auth_host controller
openstack-config --set /etc/neutron/neutron.conf keystone_authtoken auth_protocol http
openstack-config --set /etc/neutron/neutron.conf keystone_authtoken auth_port 35357
openstack-config --set /etc/neutron/neutron.conf keystone_authtoken admin_tenant_name service
openstack-config --set /etc/neutron/neutron.conf keystone_authtoken admin_user neutron
openstack-config --set /etc/neutron/neutron.conf keystone_authtoken admin_password neutron4mar

openstack-config --set /etc/neutron/neutron.conf DEFAULT rpc_backend neutron.openstack.common.rpc.impl_qpid
openstack-config --set /etc/neutron/neutron.conf DEFAULT qpid_hostname controller

openstack-config --set /etc/neutron/neutron.conf DEFAULT core_plugin ml2
openstack-config --set /etc/neutron/neutron.conf DEFAULT service_plugins router

verbose = True to the [DEFAULT] section in /etc/neutron/neutron.conf to assist with troubleshooting.
Comment out any lines in the [service_providers] section.

openstack-config --set /etc/neutron/l3_agent.ini DEFAULT interface_driver neutron.agent.linux.interface.OVSInterfaceDriver
openstack-config --set /etc/neutron/l3_agent.ini DEFAULT use_namespaces True

We recommend adding verbose = True to the [DEFAULT] section in /etc/neutron/l3_agent.ini to assist with troubleshooting.
openstack-config --set /etc/neutron/dhcp_agent.ini DEFAULT interface_driver neutron.agent.linux.interface.OVSInterfaceDriver
openstack-config --set /etc/neutron/dhcp_agent.ini DEFAULT dhcp_driver neutron.agent.linux.dhcp.Dnsmasq
openstack-config --set /etc/neutron/dhcp_agent.ini DEFAULT use_namespaces True

openstack-config --set /etc/neutron/metadata_agent.ini DEFAULT auth_url http://controller:5000/v2.0
openstack-config --set /etc/neutron/metadata_agent.ini DEFAULT auth_region regionOne
openstack-config --set /etc/neutron/metadata_agent.ini DEFAULT admin_tenant_name service
openstack-config --set /etc/neutron/metadata_agent.ini DEFAULT admin_user neutron
openstack-config --set /etc/neutron/metadata_agent.ini DEFAULT admin_password neutron4mar
openstack-config --set /etc/neutron/metadata_agent.ini DEFAULT nova_metadata_ip controller
openstack-config --set /etc/neutron/metadata_agent.ini DEFAULT metadata_proxy_shared_secret meta4mar

We recommend adding verbose = True to the [DEFAULT] section in /etc/neutron/metadata_agent.ini to assist with troubleshooting.

Perform the next two steps on the controller node.
On the controller node, configure Compute to use the metadata service:
Replace METADATA_SECRET with the secret you chose for the metadata proxy.
openstack-config --set /etc/nova/nova.conf DEFAULT service_neutron_metadata_proxy true
openstack-config --set /etc/nova/nova.conf DEFAULT neutron_metadata_proxy_shared_secret meta4mar
#On the controller node, restart the Compute API service:
service openstack-nova-api restart
To configure the Modular Layer 2 (ML2) plug-in

Replace INSTANCE_TUNNELS_INTERFACE_IP_ADDRESS with the IP address of the instance tunnels network interface on your network node. 

openstack-config --set /etc/neutron/plugins/ml2/ml2_conf.ini ml2 type_drivers gre
openstack-config --set /etc/neutron/plugins/ml2/ml2_conf.ini ml2 tenant_network_types gre
openstack-config --set /etc/neutron/plugins/ml2/ml2_conf.ini ml2 mechanism_drivers openvswitch
openstack-config --set /etc/neutron/plugins/ml2/ml2_conf.ini ml2_type_gre tunnel_id_ranges 1:1000
openstack-config --set /etc/neutron/plugins/ml2/ml2_conf.ini ovs local_ip 192.168.216.151
openstack-config --set /etc/neutron/plugins/ml2/ml2_conf.ini ovs tunnel_type gre
openstack-config --set /etc/neutron/plugins/ml2/ml2_conf.ini ovs enable_tunneling True
openstack-config --set /etc/neutron/plugins/ml2/ml2_conf.ini securitygroup firewall_driver neutron.agent.linux.iptables_firewall.OVSHybridIptablesFirewallDriver
openstack-config --set /etc/neutron/plugins/ml2/ml2_conf.ini securitygroup enable_security_group True

service openvswitch start
chkconfig openvswitch on
#Add the integration bridge:
ovs-vsctl add-br br-int
#Add the external bridge:
ovs-vsctl add-br br-ex
#Add a port to the external bridge that connects to the physical external network interface:
#Replace INTERFACE_NAME with the actual interface name. For example, eth2 or ens256.
ovs-vsctl add-port br-ex eth4

Depending on your network interface driver, you may need to disable Generic Receive Offload (GRO) to achieve suitable throughput between your instances and the external network.
To temporarily disable GRO on the external network interface while testing your environment:
# ethtool -K INTERFACE_NAME gro off

 

ln -s /etc/neutron/plugins/ml2/ml2_conf.ini /etc/neutron/plugin.ini
cp /etc/init.d/neutron-openvswitch-agent /etc/init.d/neutron-openvswitch-agent.orig
sed -i 's,plugins/openvswitch/ovs_neutron_plugin.ini,plugin.ini,g' /etc/init.d/neutron-openvswitch-agent
service neutron-openvswitch-agent start
service neutron-l3-agent start
service neutron-dhcp-agent start
service neutron-metadata-agent start
chkconfig neutron-openvswitch-agent on
chkconfig neutron-l3-agent on
chkconfig neutron-dhcp-agent on
chkconfig neutron-metadata-agent on

Neutron Configure compute node

Edit /etc/sysctl.conf to contain the following:
net.ipv4.conf.all.rp_filter=0
net.ipv4.conf.default.rp_filter=0
Implement the changes:
# sysctl -p

To install the Networking components

yum -y install openstack-neutron-ml2 openstack-neutron-openvswitch

Replace NEUTRON_PASS with the password you chose for the neutron user in the Identity service.

openstack-config --set /etc/neutron/neutron.conf DEFAULT auth_strategy keystone
openstack-config --set /etc/neutron/neutron.conf keystone_authtoken auth_uri http://controller:5000
openstack-config --set /etc/neutron/neutron.conf keystone_authtoken auth_host controller
openstack-config --set /etc/neutron/neutron.conf keystone_authtoken auth_protocol http
openstack-config --set /etc/neutron/neutron.conf keystone_authtoken auth_port 35357
openstack-config --set /etc/neutron/neutron.conf keystone_authtoken admin_tenant_name service
openstack-config --set /etc/neutron/neutron.conf keystone_authtoken admin_user neutron
openstack-config --set /etc/neutron/neutron.conf keystone_authtoken admin_password neutron4mar

openstack-config --set /etc/neutron/neutron.conf DEFAULT rpc_backend neutron.openstack.common.rpc.impl_qpid
openstack-config --set /etc/neutron/neutron.conf DEFAULT qpid_hostname controller

openstack-config --set /etc/neutron/neutron.conf DEFAULT core_plugin ml2
openstack-config --set /etc/neutron/neutron.conf DEFAULT service_plugins router

Replace INSTANCE_TUNNELS_INTERFACE_IP_ADDRESS with the IP address of the instance tunnels network interface on your compute node. This guide uses 10.0.1.31 for the IP address of the instance tunnels network interface on the first compute node.

openstack-config --set /etc/neutron/plugins/ml2/ml2_conf.ini ml2 type_drivers gre
openstack-config --set /etc/neutron/plugins/ml2/ml2_conf.ini ml2 tenant_network_types gre
openstack-config --set /etc/neutron/plugins/ml2/ml2_conf.ini ml2 mechanism_drivers openvswitch
openstack-config --set /etc/neutron/plugins/ml2/ml2_conf.ini ml2_type_gre tunnel_id_ranges 1:1000
openstack-config --set /etc/neutron/plugins/ml2/ml2_conf.ini ovs local_ip 192.168.216.141
openstack-config --set /etc/neutron/plugins/ml2/ml2_conf.ini ovs tunnel_type gre
openstack-config --set /etc/neutron/plugins/ml2/ml2_conf.ini ovs enable_tunneling True
openstack-config --set /etc/neutron/plugins/ml2/ml2_conf.ini securitygroup firewall_driver neutron.agent.linux.iptables_firewall.OVSHybridIptablesFirewallDriver
openstack-config --set /etc/neutron/plugins/ml2/ml2_conf.ini securitygroup enable_security_group True
service openvswitch start
chkconfig openvswitch on

#Add the integration bridge:

ovs-vsctl add-br br-int

Replace NEUTRON_PASS with the password you chose for the neutron user in the Identity service.

openstack-config --set /etc/nova/nova.conf DEFAULT network_api_class nova.network.neutronv2.api.API
openstack-config --set /etc/nova/nova.conf DEFAULT neutron_url http://controller:9696
openstack-config --set /etc/nova/nova.conf DEFAULT neutron_auth_strategy keystone
openstack-config --set /etc/nova/nova.conf DEFAULT neutron_admin_tenant_name service
openstack-config --set /etc/nova/nova.conf DEFAULT neutron_admin_username neutron
openstack-config --set /etc/nova/nova.conf DEFAULT neutron_admin_password neutron4mar
openstack-config --set /etc/nova/nova.conf DEFAULT neutron_admin_auth_url http://controller:35357/v2.0
openstack-config --set /etc/nova/nova.conf DEFAULT linuxnet_interface_driver nova.network.linux_net.LinuxOVSInterfaceDriver
openstack-config --set /etc/nova/nova.conf DEFAULT firewall_driver nova.virt.firewall.NoopFirewallDriver
openstack-config --set /etc/nova/nova.conf DEFAULT security_group_api neutron

ln -s /etc/neutron/plugins/ml2/ml2_conf.ini /etc/neutron/plugin.ini
#Due to a packaging bug, the Open vSwitch agent initialization script explicitly looks for the Open vSwitch plug-in configuration file rather than a symbolic link /etc/neutron/plugin.ini pointing to the ML2 plug-in configuration file. Run the following commands to resolve this issue:

cp /etc/init.d/neutron-openvswitch-agent /etc/init.d/neutron-openvswitch-agent.orig
sed -i 's,plugins/openvswitch/ovs_neutron_plugin.ini,plugin.ini,g' /etc/init.d/neutron-openvswitch-agent
#Restart the Compute service:

service openstack-nova-compute restart
#Start the Open vSwitch (OVS) agent and configure it to start when the system boots:
service neutron-openvswitch-agent start
chkconfig neutron-openvswitch-agent on

OpenStack – Icehouse –Part 2 Glance

Configure the Image Service On controller Server

yum install openstack-glance python-glanceclient -y

openstack-config --set /etc/glance/glance-api.conf database connection mysql://glance:glance4mar@controller/glance
openstack-config --set /etc/glance/glance-registry.conf database connection mysql://glance:glance4mar@controller/glance

openstack-config --set /etc/glance/glance-api.conf DEFAULT rpc_backend qpid
openstack-config --set /etc/glance/glance-api.conf DEFAULT qpid_hostname controller

mysql -u root -p
CREATE DATABASE glance;
GRANT ALL PRIVILEGES ON glance.* TO 'glance'@'localhost' IDENTIFIED BY 'glance4mar';
GRANT ALL PRIVILEGES ON glance.* TO 'glance'@'%' IDENTIFIED BY 'glance4mar';
exit

su -s /bin/sh -c "glance-manage db_sync" glance
keystone user-create --name=glance --pass=glance4mar --email=glance@example.com
keystone user-role-add --user=glance --tenant=service --role=admin

openstack-config --set /etc/glance/glance-api.conf keystone_authtoken auth_uri http://controller:5000
openstack-config --set /etc/glance/glance-api.conf keystone_authtoken auth_host controller
openstack-config --set /etc/glance/glance-api.conf keystone_authtoken auth_port 35357
openstack-config --set /etc/glance/glance-api.conf keystone_authtoken auth_protocol http
openstack-config --set /etc/glance/glance-api.conf keystone_authtoken admin_tenant_name service
openstack-config --set /etc/glance/glance-api.conf keystone_authtoken admin_user glance
openstack-config --set /etc/glance/glance-api.conf keystone_authtoken admin_password glance4mar
openstack-config --set /etc/glance/glance-api.conf paste_deploy flavor keystone
openstack-config --set /etc/glance/glance-registry.conf keystone_authtoken auth_uri http://controller:5000
openstack-config --set /etc/glance/glance-registry.conf keystone_authtoken auth_host controller
openstack-config --set /etc/glance/glance-registry.conf keystone_authtoken auth_port 35357
openstack-config --set /etc/glance/glance-registry.conf keystone_authtoken auth_protocol http
openstack-config --set /etc/glance/glance-registry.conf keystone_authtoken admin_tenant_name service
openstack-config --set /etc/glance/glance-registry.conf keystone_authtoken admin_user glance
openstack-config --set /etc/glance/glance-registry.conf keystone_authtoken admin_password glance4mar
openstack-config --set /etc/glance/glance-registry.conf paste_deploy flavor keystone

keystone service-create --name=glance --type=image --description="OpenStack Image Service"
keystone endpoint-create --service-id=$(keystone service-list | awk '/ image / {print $2}') --publicurl=http://controller:9292 --internalurl=http://controller:9292 --adminurl=http://controller:9292

service openstack-glance-api start
service openstack-glance-registry start
chkconfig openstack-glance-api on
chkconfig openstack-glance-registry on

#Verify the Image Service installation

mkdir /tmp/images
cd /tmp/images/
wget http://cdn.download.cirros-cloud.net/0.3.2/cirros-0.3.2-x86_64-disk.img

source /root/admin-openrc.sh
glance image-create --name "cirros-0.3.2-x86_64" --disk-format qcow2 --container-format bare --is-public True --progress < cirros-0.3.2-x86_64-disk.img

cd /
rm -rf /tmp/images

glance image-list

 

Importing Images into Glance

You can load an image from the command line with glance, eg:

glance image-create --name 'Fedora 19 x86_64' --disk-format qcow2 --container-format bare --is-public true \
--copy-from http://cloud.fedoraproject.org/fedora-19.x86_64.qcow2

Building Your Own Images

Alternatively, one can use diskimage-builder, which is available in the RDO repository:

yum install diskimage-builder

$ disk-image-create -a amd64 fedora vm -o fedora-image.qcow2

More Images In Following URL

http://openstack.redhat.com/Image_resources

OpenStack - Icehouse --Part 1 Keystone.

The OpenStack project is an open source cloud computing platform that supports all types of cloud environments. The project aims for simple implementation, massive scalability, and a rich set of features. Cloud computing experts from around the world contribute to the project.

OpenStack provides an Infrastructure-as-a-Service (IaaS) solution through a variety of complemental services. Each service offers an application programming interface (API) that facilitates this integration. The following table provides a list of OpenStack services:

Dashboard	Horizon	Provides a web-based self-service portal to interact with underlying OpenStack services, such as launching an instance, assigning IP addresses and configuring access controls.
Compute	Nova	Manages the lifecycle of compute instances in an OpenStack environment. Responsibilities include spawning, scheduling and decommissioning of virtual machines on demand.
Networking	Neutron	Enables network connectivity as a service for other OpenStack services, such as OpenStack Compute. Provides an API for users to define networks and the attachments into them. Has a pluggable architecture that supports many popular networking vendors and technologies.
Storage
Object Storage	Swift	Stores and retrieves arbitrary unstructured data objects via a RESTful, HTTP based API. It is highly fault tolerant with its data replication and scale out architecture. Its implementation is not like a file server with mountable directories.
Block Storage	Cinder	Provides persistent block storage to running instances. Its pluggable driver architecture facilitates the creation and management of block storage devices.
Shared services
Identity service	Keystone	Provides an authentication and authorization service for other OpenStack services. Provides a catalog of endpoints for all OpenStack services.
Image Service	Glance	Stores and retrieves virtual machine disk images. OpenStack Compute makes use of this during instance provisioning.
Telemetry	Ceilometer	Monitors and meters the OpenStack cloud for billing, benchmarking, scalability, and statistical purposes.
Higher-level services
Orchestration	Heat	Orchestrates multiple composite cloud applications by using either the native HOT template format or the AWS CloudFormation template format, through both an OpenStack-native REST API and a CloudFormation-compatible Query API.
Database Service	Trove	Provides scalable and reliable Cloud Database-as-a-Service functionality for both relational and non-relational database engines.

Sample Architecture We are trying to Set up. The Ip's will Vary , Please do check and clear ..



ON ALL THE NODE

#Making Selinux to Permissive
sed -i "s/SELINUX=.*/SELINUX=permissive/g" /etc/sysconfig/selinux

yum -y install policycoreutils setroubleshoot
setenforce 0
yum install -y euca2ools
yum install -y yum-plugin-priorities gedit curl wget nc

yum -y install ntp

service ntpd start
chkconfig ntpd on

yum -y install http://repos.fedorapeople.org/repos/openstack/openstack-icehouse/rdo-release-icehouse-3.noarch.rpm
yum -y install http://dl.fedoraproject.org/pub/epel/6/x86_64/epel-release-6-8.noarch.rpm

yum -y install openstack-utils
yum -y install openstack-selinux
yum -y upgrade



On All node add the following Rules in Iptables

-A INPUT -p tcp -m state --state NEW -m tcp --dport 80 -j ACCEPT
-A INPUT -p tcp -m state --state NEW -m tcp --dport 443 -j ACCEPT
-A INPUT -p tcp -m state --state NEW -m tcp --dport 22 -j ACCEPT
-A INPUT -p tcp -m state --state NEW -m tcp --dport 5000 -j ACCEPT
-A INPUT -p tcp -m state --state NEW -m tcp --dport 5672 -j ACCEPT
-A INPUT -p tcp -m state --state NEW -m tcp --dport 6080 -j ACCEPT
-A INPUT -p tcp -m state --state NEW -m tcp --dport 8774 -j ACCEPT
-A INPUT -p tcp -m state --state NEW -m tcp --dport 9292 -j ACCEPT
-A INPUT -p tcp -m state --state NEW -m tcp --dport 9696 -j ACCEPT
-A INPUT -p tcp -m state --state NEW -m tcp --dport 35357 -j ACCEPT

ON OTHER NODE

yum install MySQL-python -y

ON CONTROLLER NODE

yum -y install qpid-cpp-server memcached
sed -i "s/auth=yes/auth=no/g" /etc/qpidd.conf
service qpidd start
chkconfig qpidd on

yum install mysql mysql-server MySQL-python -y
service mysqld start
chkconfig mysqld on
mysql_secure_installation

 Over ALL Network

192.168.255.130 controller

192.168.216.130 controller
192.168.216.140 compute
192.168.216.141 compute
192.168.255.150 network
eth4 netwrok << Public Connection
192.168.216.150 netwrok
192.168.216.151 network

On NETWORK NODE

One of the external interface uses a special configuration without an IP address assigned to it. Configure the third interface as the external interface:
Replace INTERFACE_NAME with the actual interface name. For example, eth2 or ens256.
Edit the /etc/sysconfig/network-scripts/ifcfg-INTERFACE_NAME file to contain the following:
Do not change the HWADDR and UUID keys.

DEVICE=INTERFACE_NAME
TYPE=Ethernet
ONBOOT="yes"
BOOTPROTO="none"

Restart networking:
service network restart
=========================================
KEYSTONE_DBPASS keystone4mar Database password of Identity service
DEMO_PASS demo4mar Password of user demo
ADMIN_PASS admin4mar Password of user admin
GLANCE_DBPASS glance4mar Database password for Image Service
GLANCE_PASS glance4mar Password of Image Service user glance
NOVA_DBPASS nova4mar Database password for Compute service
NOVA_PASS nova4mar Password of Compute service user nova
DASH_DBPASS dash4mar Database password for the dashboard
CINDER_DBPASS cinder4mar Database password for the Block Storage service
CINDER_PASS cinder4mar Password of Block Storage service user cinder
NEUTRON_DBPASS neutron4mar Database password for the Networking service
NEUTRON_PASS neutron4mar Password of Networking service user neutron
HEAT_DBPASS heat4mar Database password for the Orchestration service
HEAT_PASS heat4mar Password of Orchestration service user heat
CEILOMETER_DBPASS ceil4mar Database password for the Telemetry service
CEILOMETER_PASS ceil4mar Password of Telemetry service user ceilometer
TROVE_DBPASS trove4mar Database password of Database service
TROVE_PASS trove4mar Password of Database Service user trove
=========================================

On Controller Node

In my.cnf configure for INnode DB

bind-address = ***.***.***.***
default-storage-engine = innodb
innodb_file_per_table
collation-server = utf8_general_ci
init-connect = 'SET NAMES utf8'
character-set-server = utf8

Installing Identity Service On Controller Node

yum install openstack-keystone python-keystoneclient -y
openstack-config --set /etc/keystone/keystone.conf database connection mysql://keystone:keystone4mar@controller/keystone
$ mysql -u root -p
CREATE DATABASE keystone;
GRANT ALL PRIVILEGES ON keystone.* TO 'keystone'@'localhost' IDENTIFIED BY 'keystone4mar';
GRANT ALL PRIVILEGES ON keystone.* TO 'keystone'@'%' IDENTIFIED BY 'keystone4mar';
exit

Create the database tables for the Identity Service:
su -s /bin/sh -c "keystone-manage db_sync" keystone
ADMIN_TOKEN=$(openssl rand -hex 10)
echo $ADMIN_TOKEN
openstack-config --set /etc/keystone/keystone.conf DEFAULT admin_token $ADMIN_TOKEN
keystone-manage pki_setup --keystone-user keystone --keystone-group keystone
chown -R keystone:keystone /etc/keystone/ssl
chmod -R o-rwx /etc/keystone/ssl

service openstack-keystone start
chkconfig openstack-keystone on
#Define users, tenants, and roles
*********Replace ADMIN_TOKEN with your authorization token
#export OS_SERVICE_TOKEN=$ADMIN_TOKEN
export OS_SERVICE_TOKEN=$(echo $ADMIN_TOKEN)
export OS_SERVICE_ENDPOINT=http://controller:35357/v2.0

#Create an administrative user

keystone user-create --name=admin --pass=admin4mar --email=ADMIN_EMAIL
keystone role-create --name=admin
keystone tenant-create --name=admin --description="Admin Tenant"
keystone user-role-add --user=admin --tenant=admin --role=admin
keystone user-role-add --user=admin --role=_member_ --tenant=admin

#Create a normal user

keystone user-create --name=demo --pass=demo4mar --email=DEMO_EMAIL
keystone tenant-create --name=demo --description="Demo Tenant"
keystone user-role-add --user=demo --role=_member_ --tenant=demo

#Create a service tenant

keystone tenant-create --name=service --description="Service Tenant"
#Define services and API endpoints

keystone service-create --name=keystone --type=identity --description="OpenStack Identity"
keystone endpoint-create --service-id=$(keystone service-list | awk '/ identity / {print $2}') --publicurl=http://controller:5000/v2.0 --internalurl=http://controller:5000/v2.0 --adminurl=http://controller:35357/v2.0

#Verify the Identity Service installation

unset OS_SERVICE_TOKEN OS_SERVICE_ENDPOINT
keystone --os-username=admin --os-password=admin4mar --os-auth-url=http://controller:35357/v2.0 token-get
keystone --os-username=admin --os-password=admin4mar --os-tenant-name=admin --os-auth-url=http://controller:35357/v2.0 token-get

echo "export OS_USERNAME=admin
export OS_PASSWORD=admin4mar
export OS_TENANT_NAME=admin
export OS_AUTH_URL=http://controller:35357/v2.0" >> /root/admin-openrc.sh

cat /root/admin-openrc.sh

source /root/admin-openrc.sh
keystone token-get
keystone user-list
keystone user-role-list --user admin --tenant admin

IF WE WANT TO INSTALL THE CLIENTS

yum install python-pip
pip install python-PROJECTclient

ceilometer - Telemetry API
cinder - Block Storage API and extensions
glance - Image Service API
heat - Orchestration API
keystone - Identity service API and extensions
neutron - Networking API
nova - Compute API and extensions
swift - Object Storage API
trove - Database Service API

#On Red Hat Enterprise Linux, CentOS, or Fedora, use yum to install the clients from the packaged versions available in RDO:

yum install python-PROJECTclient

Creating a client Profile file

=====================
echo "export OS_USERNAME=demo
export OS_PASSWORD=demo4mar
export OS_TENANT_NAME=demo
export OS_AUTH_URL=http://controller:35357/v2.0" >> /root/demo-openrc.sh
cat /root/demo-openrc.sh

Mail Server-Postfix+Centos+Dovecot

Setup mail server in centos 6

» Installing and configuring postfix

» Installing and configuring dovecot

» Creating users and testing

» Installing and configuring squirrelmail

» Installing and configuring postfix

 

Step 1 »Before installation assign a static ip and add a host entry for your domain to that IP in the /etc/hosts file like below.
echo "23.236.147.74 keralainindia.asia" >> /etc/hosts
Step 2 » install postfix

[root@kerala ~]# yum -y install postfix

Step 3 » install SMTP AUTH packages .

[root@kerala ~]# yum -y install cyrus-sasl cyrus-sasl-devel cyrus-sasl-gssapi cyrus-sasl-md5 cyrus-sasl-plain

Postfix package installation is completed .

Step 4 »creating SSL Cert.

[root@kerala ~]# mkdir /etc/postfix/ssl
[root@kerala ~]# cd /etc/postfix/ssl/
[root@kerala ssl]# openssl genrsa -des3 -rand /etc/hosts -out smtpd.key 1024
[root@kerala ssl]# chmod 600 smtpd.key
[root@kerala ssl]# openssl req -new -key smtpd.key -out smtpd.csr
[root@kerala ssl]# openssl x509 -req -days 365 -in smtpd.csr -signkey smtpd.key -out smtpd.crt
[root@kerala ssl]# openssl rsa -in smtpd.key -out smtpd.key.unencrypted
[root@kerala ssl]# mv -f smtpd.key.unencrypted smtpd.key
[root@kerala ssl]# openssl req -new -x509 -extensions v3_ca -keyout cakey.pem -out cacert.pem -days 365

Step 4 » Now open /etc/postfix/main.cf file .

Find and comment the below lines .
#inet_interfaces = localhost #---> line no 116
#mydestination = $myhostname, localhost.$mydomain, localhost #--> line no 164

and add these lines at the bottom of the file.

myhostname = mail.keralainindia.asia
mydomain = keralainindia.asia
myorigin = $mydomain
home_mailbox = mail/
mynetworks = 127.0.0.0/8
inet_interfaces = all
mydestination = $myhostname, localhost.$mydomain, localhost, $mydomain
smtpd_sasl_auth_enable = yes
smtpd_sasl_type = cyrus
smtpd_sasl_security_options = noanonymous
broken_sasl_auth_clients = yes
smtpd_sasl_authenticated_header = yes
smtpd_recipient_restrictions = permit_sasl_authenticated,permit_mynetworks,reject_unauth_destination
smtpd_tls_auth_only = no
smtp_use_tls = yes
smtpd_use_tls = yes
smtp_tls_note_starttls_offer = yes
smtpd_tls_key_file = /etc/postfix/ssl/smtpd.key
smtpd_tls_cert_file = /etc/postfix/ssl/smtpd.crt
smtpd_tls_CAfile = /etc/postfix/ssl/cacert.pem
smtpd_tls_received_header = yes
smtpd_tls_session_cache_timeout = 3600s
tls_random_source = dev:/dev/urandom

 

Step 5 » Now open /etc/postfix/master.cf file and add the below line after smtp

smtps inet n - n - - smtpd
-o smtpd_sasl_auth_enable=yes
-o smtpd_reject_unlisted_sender=yes
-o smtpd_recipient_restrictions=permit_sasl_authenticated,reject
-o broken_sasl_auth_clients=yes

Step 6 » Now start postfix and saslauthd service

[root@kerala ~]# service postfix start

[root@kerala ~]# service saslauthd start

» Issue the below commands to start the postfix and saslauthd at startup
[root@kerala ~]# chkconfig --level 235 postfix on

[root@kerala ~]# chkconfig --level 235 saslauthd on

Step 7 » Now check your smtp connectivity . just telnet localhost on port 25 and type this command ehlo localhost

[root@kerala ~]# telnet localhost 25
Trying ::1...
Connected to localhost.
Escape character is '^]'.
220 mail.keralainindia.asia ESMTP Postfix
ehlo localhost <---- type this command
250-mail.keralainindia.asia
250-PIPELINING
250-SIZE 10240000
250-VRFY
250-ETRN
250-STARTTLS
250-AUTH PLAIN LOGIN
250-AUTH=PLAIN LOGIN
250-ENHANCEDSTATUSCODES
250-8BITMIME
250 DSN
quit
221 2.0.0 Bye
Connection closed by foreign host.

If you get this output .. Great .. everything is fine till now.

» Installing and configuring dovecot

Step 8 » Issue this command to install dovecot

[root@kerala ~]# yum -y install dovecot

 

Step 9 » After installation open /etc/dovecot/dovecot.conf file and add the below line at the end of the file. please make sure mail_location and home_mailbox in postfix configuration are using the same name.

protocols = imap pop3
mail_location = maildir:~/mail
pop3_uidl_format = %08Xu%08Xv

 

Step 10 » Now start dovecot service
[root@kerala ~]# service dovecot start

» Issue the below command to start the dovecot at startup
[root@kerala ~]# chkconfig --level 235 dovecot on

 

Step 11 » Now test your pop3 connectivity .
[root@kerala ~]# telnet localhost 110
Trying ::1...
Connected to localhost.
Escape character is '^]'.
+OK Dovecot ready.
quit
+OK Logging out
Connection closed by foreign host.

Yes .. your server is ready to receive mails .

» Creating users and testing
Step 11 » Now create users to test your configuration.
[root@kerala ~]# useradd -m ramraj -s /sbin/nologin
and create passwords for the users created
[root@kerala ~]# passwd ramraj
Step 12 » Now you can send and receive mails using this server. In case of any issues please check the log file /var/log/maillog )

your mail server is ready …

» Installing and configuring squirrelmail

Step 13 » you need to add EPEL repository to install squirrelmail package. you can find latest EPEL repository rpm here ( http://ftp.jaist.ac.jp/pub/Linux/Fedora/epel/6/i386/repoview/epel-release.html )
[root@kerala ~]# rpm -ivh http://ftp.jaist.ac.jp/pub/Linux/Fedora/epel/6/i386/epel-release-6-8.noarch.rpm

Step 14» Issue the below command to install squirrelmail.
[root@kerala ~]# yum install squirrelmail

This command will install squirrelmail along with apache and php.

Step 15 » Now run the below command to configure squirrelmail .
[root@kerala ~]# perl /usr/share/squirrelmail/config/conf.pl

»»» 1 »»» 1 »»» krizna (type Organization name ) »»» R ( return )
»»» 2 »»» 1 »»» ( hit space for empty Domain name ) and choose 3 »»» SMTP ( choose SMTP ) »»» R ( return )
»»» D »»» dovecot ( type ) »»» press enter with default
»»» s ( save and quit)
Step 16 » Open /etc/httpd/conf.d/squirrelmail.conf file and uncomment below lines if you are using ssl. If you are not using ssl Don't Do it.
# RewriteCond %{HTTPS} !=on
# RewriteRule (.*) https://%{HTTP_HOST}%{REQUEST_URI}

Step 17» start apache service
[root@kerala ~]# service httpd start

» Issue the below commands to start the httpd at startup
[root@kerala ~]# chkconfig --level 235 httpd on

Step 18» Now open http://serverip/webmail path in your browser .

Git Part 2

http://superuser.com/questions/261060/git-how-can-i-config-git-to-ignore-file-permissions-changes
turn off the filemode so that permissions of files are not considered.

For Mac Machines
http://stackoverflow.com/questions/8402281/github-push-error-permission-denied
cd ~
ssh-keygen
cat .ssh/id_rsa.pub > .ssh/authorized_keys

Internalize a project in server
cd /opt/git/
mkdir <Project-name>
cd <Project-name>
git inti --bare

In client
git clone xxxx@xxx.xxx.xxx.xxx:/opt/git/<Project-name>
cd <Project-name>
git add *
git commit -m "Test Files"
>>git remote add <remote-name> <git-repo-URL>
git remote add orgin xxxx@xxx.xxx.xxx.xxx:/opt/git/<Project-name>
git push orgin master

Branching
git checkout -b <Branch-name>
git push <remote-name> <branch-name>
git push <remote-name> <local-branch-name>:<remote-branch-name>

List ALL Branching
git branch -a
List Remote Branching
git branch -r

Merge two branch
git checkout a (you will switch to branch a)
git merge b (this will merge all changes from branch b into branch a)
git commit -a (this will commit your changes)

List Merged Branches
git branch --merged lists the branches that have been merged into the current branch
git branch --no-merged lists the branches that have not been merged into the current branch

 

IFS Internal Field Separator in Bash Scripting

IFS stands for  Internal Field Separator - it's a character that separate fields. In the example you posted it is set to new line character (\n), so after setting it for will process text line by line. In that example you could change value of $IFS (to some letter that you have in your input file) and check how text will be splitted.

 

[root@ip-192-168-1-36 tmp]# for i in `cat sample.txt`; do echo $i; done
Mar 10
Mar 11
Mar 7
Mar 8
Mar 9
[root@ip-192-168-1-36 tmp]# IFS=$' '
[root@ip-192-168-1-36 tmp]# for i in `cat sample.txt`; do echo $i; done

Mar
10
Mar
11
Mar
7
Mar
8
Mar
9
[root@ip-192-168-1-36 tmp]# IFS=$'\n'
[root@ip-192-168-1-36 tmp]# for i in `cat sample.txt`; do echo $i; done
Mar 10
Mar 11
Mar 7
Mar 8
Mar 9
[root@ip-192-168-1-36 tmp]#

Installing rar in linux

for 64 bit

install unrar centos x64
#wget http://pkgs.repoforge.org/unrar/unrar-4.0.7-1.el6.rf.x86_64.rpm
#rpm -Uvh unrar-4.0.7-1.el6.rf.x86_64.rpm

install rar centos x64
#wget http://pkgs.repoforge.org/rar/rar-3.8.0-1.el6.rf.x86_64.rpm
#rpm -Uvh rar-3.8.0-1.el6.rf.x86_64.rpm

for 32 bit

install unrar centos 32
#wget http://pkgs.repoforge.org/unrar/unrar-4.0.7-1.el6.rf.i686.rpm
#rpm -Uvh unrar-4.0.7-1.el6.rf.i686.rpm

install rar centos 32

#wget http://pkgs.repoforge.org/rar/rar-3.8.0-1.el6.rf.i686.rpm

#rpm -Uvh rar-3.8.0-1.el6.rf.i686.rpm

you can also use matching OS rpm from

http://pkgs.repoforge.org/rpmforge-release/

 

Installing Amazon Command Line using PIP

Installing the repo needed for pip

cd /tmp
wget http://mirror-fpt-telecom.fpt.net/fedora/epel/6/i386/epel-release-6-8.noarch.rpm
rpm -ivh epel-release-6-8.noarch.rpm

Installing C-compiler for Pip

yum install gcc

Installing amazon cli

pip install awscli

Configure Amazon Cli

aws configrue

you need aws access key ,secret key, default region and output format.

Installing and configuring Amazon EC2 command line

Now download the Amazon API CLI tools using following command and extract them at a proper place. For this example, we are using /opt directory.

# mkdir /opt/ec2
# wget http://s3.amazonaws.com/ec2-downloads/ec2-api-tools.zip
# unzip ec2-api-tools.zip -d /tmp
# mv /tmp/ec2-api-tools-* /opt/ec2/tools
Step 3- Download Private Key and Certificate Files

Now create and download X.509 certificate (private key file and certificate file) files from your account from Security Credentials page and copy to /opt/ec2/certs/ directory.

# ls -l /opt/ec2/certs/

-rw-r--r--. 1 root root 1281 May 15 12:57 my-ec2-cert.pem
-rw-r--r--. 1 root root 1704 May 15 12:56 my-ec2-pk.pem
Step 4- Configure Environment

Install JAVA
The Amazon EC2 command line tools required Java 1.6 or later version. Make sure you have proper java installed on your system. You can install JRE or JDK , both are ok to use.
# java -version
java version "1.8.0_05"
Java(TM) SE Runtime Environment (build 1.8.0_05-b13)
Java HotSpot(TM) Client VM (build 25.5-b02, mixed mode)
If you don’t have Java installed your system, Use below links to install Java on your system first
Installing JAVA/JDK 8 on CentOS, RHEL and Fedora
Installing JAVA/JDK 8 on Ubuntu

Now edit ~/.bashrc file and add the following values at end of file

export EC2_BASE=/opt/ec2
export EC2_HOME=$EC2_BASE/tools
export EC2_PRIVATE_KEY=$EC2_BASE/certs/my-ec2-pk.pem
export EC2_CERT=$EC2_BASE/certs/my-ec2-cert.pem
export EC2_URL=https://ec2.xxxxxxx.amazonaws.com
export AWS_ACCOUNT_NUMBER=
export PATH=$PATH:$EC2_HOME/bin
export JAVA_HOME=/opt/jdk1.8.0_05
Now execute the following command to set environment variables

$ source ~/.bashrc

After completing all configuration, let’s run following command to quickly verify setup.

# ec2-describe-regions

REGION eu-west-1 ec2.eu-west-1.amazonaws.com
REGION sa-east-1 ec2.sa-east-1.amazonaws.com
REGION us-east-1 ec2.us-east-1.amazonaws.com
REGION ap-northeast-1 ec2.ap-northeast-1.amazonaws.com
REGION us-west-2 ec2.us-west-2.amazonaws.com
REGION us-west-1 ec2.us-west-1.amazonaws.com
REGION ap-southeast-1 ec2.ap-southeast-1.amazonaws.com
REGION ap-southeast-2 ec2.ap-southeast-2.amazonaws.com

MySql Server Cluster (Maria+galera)

Add MariaDB Repositories
========================
Create a mariadb repository /etc/yum.repos.d/mariadb.repo using following content in your system. Below repository will work on CentOS 6.x systems, For other system use repository generation tool and add to your system.

Disable Selinux in redhat sever's.

For CentOS 6 – 64bit

[mariadb]
name = MariaDB
baseurl = http://yum.mariadb.org/5.5/centos6-amd64
gpgkey=https://yum.mariadb.org/RPM-GPG-KEY-MariaDB
gpgcheck=1
For CentOS 6 – 32bit

[mariadb]
name = MariaDB
baseurl = http://yum.mariadb.org/5.5/centos6-x86
gpgkey=https://yum.mariadb.org/RPM-GPG-KEY-MariaDB
gpgcheck=1
Install MariaDB and Galera
==========================
Before installing MariaDB Galera cluster packages, remove any existing MySQL or MariaDB packages installed on system. After that use following command to install on all nodes.

# yum install MariaDB-Galera-server MariaDB-client galera

Initial MariaDB Configuration
=============================
After successfully installing packages in above steps do the some initial MariaDB configurations. Use following command and follow the instructions on all nodes of cluster. If will prompt to set root account password also.

# mysql_secure_installation
# service mysql start
After that create a user in MariaDB on all nodes, which can access database from your network in cluster.

# mysql -u root -p

MariaDB [(none)]> GRANT ALL PRIVILEGES ON *.* TO 'root'@'%' IDENTIFIED BY 'password' WITH GRANT OPTION;
MariaDB [(none)]> FLUSH PRIVILEGES;
MariaDB [(none)]> exit
and stop MariaDB service before starting cluster configuration

# service mysql stop
Setup Cluster Configuration on database1
========================================
Lets start setup MariaDB Galera cluster from database1 server. Edit MariaDB server configuration file and add following values under [mariadb] section.

[root@database1 ~]# vim /etc/my.cnf.d/server.cnf
query_cache_size=0
binlog_format=ROW
default_storage_engine=innodb
innodb_autoinc_lock_mode=2
wsrep_provider=/usr/lib/galera/libgalera_smm.so
wsrep_cluster_address=gcomm://10.0.0.4,10.0.0.5
wsrep_cluster_name='cluster1'
wsrep_node_address='10.0.0.2'
wsrep_node_name='database1'
wsrep_sst_method=rsync
wsrep_sst_auth=root:password
Start cluster using following command.

[root@database1 ~]# /etc/init.d/mysql bootstrap
Bootstrapping the clusterStarting MySQL.... SUCCESS!
If you get any problem during startup check MariaDB error log file /var/lib/mysql/<hostname>.err

Add database2 in MariaDB Cluster
================================
After successfully starting cluster on database1. Start configuration on database2. Edit MariaDB server configuration file and add following values under [mariadb] section. All the settings are similar to database1 except wsrep_node_address, wsrep_cluster_address and wsrep_node_name.

[root@database2 ~]# vim /etc/my.cnf.d/server.cnf

query_cache_size=0
binlog_format=ROW
default_storage_engine=innodb
innodb_autoinc_lock_mode=2
wsrep_provider=/usr/lib/galera/libgalera_smm.so
wsrep_cluster_address=gcomm://10.0.0.2,10.0.0.5
wsrep_cluster_name='cluster1'
wsrep_node_address='10.0.0.4'
wsrep_node_name='database2'
wsrep_sst_method=rsync
wsrep_sst_auth=root:password

Start cluster using following command.

[root@database2 ~]# /etc/init.d/mysql start
Starting MySQL..... SUCCESS!

Add database3 in MariaDB Cluster
================================
This server is optional, If you want only two server in cluster, you can ignore this step, but you need to remove third server ip from database1/database2 configuration files. To add this server make changes same as database2.

[root@database3 ~]# vim /etc/my.cnf.d/server.cnf
query_cache_size=0
binlog_format=ROW
default_storage_engine=innodb
innodb_autoinc_lock_mode=2
wsrep_provider=/usr/lib/galera/libgalera_smm.so
wsrep_cluster_address=gcomm://10.0.0.2,10.0.0.4
wsrep_cluster_name='cluster1'
wsrep_node_address='10.0.0.5'
wsrep_node_name='database3'
wsrep_sst_method=rsync
wsrep_sst_auth=root:password
Start cluster using following command.

[root@db3 ~]# /etc/init.d/mysql start
Starting MySQL..... SUCCESS!

Enable up/Down arrow in powershell

For this, you need PSReadline. First, install PsGet if you don’t have it:To install it just run the following URL in powershell.

(new-object Net.WebClient).DownloadString("http://psget.net/GetPsGet.ps1 ' ') | iex
Then, install PSReadline:

install-module PSReadline
Import PSReadline after loading the persistent history:

Import-Module PSReadLine
And you will be able to recall previous commands with up arrow key. Add the following to have partial history search with up/down arrow key:

Set-PSReadlineKeyHandler -Key UpArrow -Function HistorySearchBackward
Set-PSReadlineKeyHandler -Key DownArrow -Function HistorySearchForward
Lastly, to enable bash style completion:

Set-PSReadlineKeyHandler -Key Tab -Function Complete

History Command in PowerShell

To add the a command to a new or existing profile
On the computer that hosts the data warehouse management server,

click Start, point to Programs, point to Windows PowerShell 1.0, right-click Windows PowerShell, and then click Run as administrator.
At the Windows PowerShell prompt, type the following command, and then press ENTER:
>>Test-path $profile
If the results of the previous command are false, go to step 4. If the results are true, go to step 5.
Type the following command, and then press ENTER.
New-item –type file –force $profile
Type the following command, and then press ENTER.
Notepad $profile

Add the following Content into profile file.

===================

Set-Location C:\
$a = Get-Date
“Date: ” + $a.ToShortDateString()
“Time: ” + $a.ToShortTimeString()
$MaximumHistoryCount = 1KB

if (!(Test-Path ~\PowerShell -PathType Container))
{ New-Item ~\PowerShell -ItemType Directory
}

function bye
{ Get-History -Count 1KB |Export-CSV ~\PowerShell\history.csv
exit
}

if (Test-path ~\PowerShell\History.csv)
{ Import-CSV ~\PowerShell\History.csv |Add-History
}

====================

 

PS> Get-History |Get-Member -MemberType Property
TypeName: Microsoft.PowerShell.Commands.HistoryInfo

Name MemberType Definition
---- ---------- ----------
CommandLine Property System.String CommandLine {get;}
EndExecutionTime Property System.DateTime EndExecutionTime {get;}
ExecutionStatus Property System.Management.Automation.Runspaces.Pip...
Id Property System.Int64 Id {get;}
StartExecutionTime Property System.DateTime StartExecutionTime {get;}

This means that you can find out when something was executed (e.g. which session it happened in) using the following command:

PS> ghy |ft id,endexecutiontime,commandline -auto

Id EndExecutionTime CommandLine
-- ---------------- -----------
612 6/29/2006 5:39:34 AM gcm export-csv |fl *
613 6/30/2006 6:51:16 PM ipconfig
614 6/30/2006 8:51:38 PM cd C:\kits

 

Create a Windows PowerShell Profile

To add the a command to a new or existing profile
On the computer that hosts the data warehouse management server, click Start, point to Programs, point to Windows PowerShell 1.0, right-click Windows PowerShell, and then click Run as administrator.
At the Windows PowerShell prompt, type the following command, and then press ENTER:
Test-path $profile
If the results of the previous command are false, go to step 4. If the results are true, go to step 5.
Type the following command, and then press ENTER.
New-item –type file –force $profile
Type the following command, and then press ENTER.
Notepad $profile
In the profile, type the command you need ,If you are adding this command to an existing profile, add it on a new line at the end of the profile.
On the menu bar, click File, and then click Exit.
In Notepad, click Save.
Type the following commands, and then press ENTER after each command.
Set-ExecutionPolicy RemoteSigned
. $profile

Cgroups--Part 2 sample examples and commands

 

https://www.kernel.org/doc/Documentation/cgroups/

Mounting all the subsystems in the system
for i in `lssubsys -am`; do mkdir -p /cgroup/$i ; mount -t cgroup -o $i $i /cgroup/$i ;done
blkio — this subsystem sets limits on input/output access to and from block devices such as physical drives (disk, solid state, USB, etc.).
cpu — this subsystem uses the scheduler to provide cgroup tasks access to the CPU.
cpuacct — this subsystem generates automatic reports on CPU resources used by tasks in a cgroup
memory — this subsystem sets limits on memory use by tasks in a cgroup, and generates automatic reports on memory resources used by those tasks.
Define two cgroups that can be used to assign tasks to run on different sets of CPU cores.
mount -t cgroup -o cpu cpu /cgroup/cpu
mount -t cgroup -o memory memory /cgroup/memory
mount -t cgroup -o blkio blkio /cgroup/blkio

cgcreate -g blkio:high
cgcreate -g blkio:low
cgcreate -g cpu:high
cgcreate -g cpu:low
cgcreate -g memory:high
cgcreate -g memory:low

cgset -r blkio.weight=1000 high
cgset -r blkio.weight=500 low
cgset -r memory.swappiness=30 high
cgset -r memory.swappiness=60 low
cgset -r cpu.shares=4096 high
cgset -r cpu.shares=2048 low
Smaple configuration's ..we can edit the /etc/cgconfig.conf to make our own groups

Controlling CPU Cores on which process need to run

mount {
cpuset = /cgroup/coregrp;
}

group locores {
cpuset {
cpuset.mems="0";
# Run tasks on cores 0 through 3
cpuset.cpus="0-3";
}
}

group hicores {
cpuset {
cpuset.mems="0";
# Run tasks on cores 4 through 7
cpuset.cpus="4-7";
}
}
Controlling CPU and Memory Usage

# High priority group
group hipri {
cpu {
# Set the relative share of CPU resources equal to 75%
cpu.shares="750";
}
cpuset {
# No alternate memory nodes if the system is not NUMA
cpuset.mems="0";
# Make all CPU cores available to tasks
cpuset.cpus="0-7";
}
memory {
# Allocate at most 2 GB of memory to tasks
memory.limit_in_bytes="2G";
# Allocate at most 4 GB of memory+swap to tasks
memory.memsw.limit_in_bytes="4G";
# Apply a soft limit of 1 GB to tasks
memory.soft_limit_in_bytes="1G";
}
}

# Low priority group
group lopri {
cpu {
# Set the relative share of CPU resources equal to 25%
cpu.shares="250";
}
cpuset {
# No alternate memory nodes if the system is not NUMA
cpuset.mems="0";
# Make only cores 0 and 1 available to tasks
cpuset.cpus="0,1";
}
memory {
# Allocate at most 1 GB of memory to tasks
memory.limit_in_bytes="1G";
# Allocate at most 2 GB of memory+swap to tasks
memory.memsw.limit_in_bytes="2G";
# Apply a soft limit of 512 MB to tasks
memory.soft_limit_in_bytes="512M";
}
}

Throttling I/O Bandwidth

Define a cgroup that limits the I/O bandwidth to 50MB/s when reading from /dev/sda1.

mount {
blkio = /cgroup/iolimit;
}

group iocap1 {
blkio {
# Limit reads from /dev/sda1 to 50 MB/s
blkio.throttle.read_bps_device="8:1 52428800";
}
}
Define a cgroup that limits the number of read transactions to 100 per second when reading from /dev/sdd.

mount {
blkio = /cgroup/iolimit;
}

group iocap2 {
blkio {
# Limit read tps from /dev/sdd to 100 per second
blkio.throttle.read_iops_device="8:48 100";
}
}

Define two cgroups with different shares of I/O access to /dev/sdb .
mount {
blkio = /cgroup/iolimit;
}

# Low access share group
group iolo {
blkio {
# Set the share of I/O access by /dev/sdb to 25%
blkio.weight_device="8:16 250";
}
}

# High access share group
group iohi {
blkio {
# Set the share of I/O access by /dev/sdb to 75%
blkio.weight_device="8:16 750";
}
}
ruining the application in the corresponding group

cgexec -g cpu,memory:group2 httpd
cgexec -g blkio:iohi httpd

 

CpuLimit -- Limiting User's resource Usage

sudo yum install cpulimit
If it’s not possible in your distro then it’s extremely easy to compile it:

cd /usr/src/
wget --no-check-certificate https://github.com/opsengine/cpulimit/tarball/master -O cpulimit.tar
tar -xvf cpulimit.tar
cd opsengine-cpulimit-9df7758
make
ln -s src/cpulimit /usr/sbin/cpulimit

/usr/sbin/cpulimit --limit=10 /bin/gzip test.tar
Using cpulimit you can also allocate CPU limit to already running processes, e.g. below command will allocate 20% CPU limit to process with PID 2342:

/usr/sbin/cpulimit -p 2342 -l 20
It’s possible to specify process by its executable file instead of PID:

/usr/sbin/cpulimit -P /usr/sbin/nginx -l 30