Docker + Nova Time Out Error

http://paste.openstack.org/show/131728/

Sample Error
==========
    out = f(*args, **kwds)
  File "/usr/lib/python2.7/site-packages/requests/sessions.py", line 468, in get
    return self.request('GET', url, **kwargs)
  File "/usr/lib/python2.7/site-packages/novadocker/virt/docker/client.py", line 36, in wrapper
    out = f(*args, **kwds)
  File "/usr/lib/python2.7/site-packages/requests/sessions.py", line 456, in request
    resp = self.send(prep, **send_kwargs)
 File "/usr/lib/python2.7/site-packages/novadocker/virt/docker/client.py", line 36, in wrapper
    out = f(*args, **kwds)
 File "/usr/lib/python2.7/site-packages/requests/sessions.py", line 559, in send
    r = adapter.send(request, **kwargs)
File "/usr/lib/python2.7/site-packages/requests/adapters.py", line 327, in send
    timeout=timeout
  File "/usr/lib/python2.7/site-packages/requests/packages/urllib3/connectionpool.py", line 516, in urlopen
    body=body, headers=headers)
  File "/usr/lib/python2.7/site-packages/requests/packages/urllib3/connectionpool.py", line 299, in _make_request
    timeout_obj = self._get_timeout(timeout)
 File "/usr/lib/python2.7/site-packages/requests/packages/urllib3/connectionpool.py", line 279, in _get_timeout
    return Timeout.from_float(timeout)
  File "/usr/lib/python2.7/site-packages/requests/packages/urllib3/util/timeout.py", line 152, in from_float
    return Timeout(read=timeout, connect=timeout)
  File "/usr/lib/python2.7/site-packages/requests/packages/urllib3/util/timeout.py", line 95, in __init__
    self._connect = self._validate_timeout(connect, 'connect')
  File "/usr/lib/python2.7/site-packages/requests/packages/urllib3/util/timeout.py", line 125, in _validate_timeout
    "int or float." % (name, value))
ValueError: Timeout value connect was Timeout(connect=10, read=10, total=None), but it must be an int or float.

To fix the problem i have to modify directly "/usr/lib/python2.7/site-packages/requests/packages/urllib3/connectionpool.py"

def _get_timeout(self, timeout):                                            

    """ Helper that always returns a :class:`urllib3.util.Timeout` """      

    if timeout is _Default:                                                           

        return self.timeout.clone()                                         

    if isinstance(timeout, Timeout): <========================== Timeout is not a urllib3 timeout

        return timeout.clone()                                              

    else:                                                                   

        # User passed us an int/float. This is for backwards compatibility, 

        # can be removed later                                                                                                 

        return Timeout.from_float(timeout._connect ) <======================= manually entered _connect

I

Removing Nova and Neutron Services from Mysql

Some times we need to remove the services listed in the Nova or neutron as they are duplicated or they are removed from the entire system. So we can do it in the following way.

Removing Nova Service from Mysql Database. 

>>nova service-list
>>nova hypervisor-list

mysql> use nova;
mysql> SELECT id, created_at, updated_at, hypervisor_hostname FROM compute_nodes;

mysql> DELETE FROM compute_node_stats WHERE compute_node_id='1';
mysql> DELETE FROM compute_nodes WHERE hypervisor_hostname='compute1';
mysql> DELETE FROM services WHERE host='compute1';



Removing Nneutron  Service from Mysql Database. 

>>neutron agent-list

mysql> use neutorn
mysql> DELETE FROM agents WHERE host='compute1';

Parse Error Caused Due to Blank Space Before the entries.

   I noticed that in Openstack Juno if there are white spaces on the beginning of lines containing 'key' = 'value' we get parse error in the logs. 

Sample Error. 

Nov 06 13:29:42 controller.novalocal neutron-server[14563]: File "/usr/lib64/python2.7/argparse.py", line 1794...ion
Nov 06 13:29:42 controller.novalocal neutron-server[14563]: action(self, namespace, argument_values, option_string)
Nov 06 13:29:42 controller.novalocal neutron-server[14563]: File "/usr/lib/python2.7/site-packages/oslo/config...l__
Nov 06 13:29:42 controller.novalocal neutron-server[14563]: ConfigParser._parse_file(values, namespace)
Nov 06 13:29:42 controller.novalocal neutron-server[14563]: File "/usr/lib/python2.7/site-packages/oslo/config...ile
Nov 06 13:29:42 controller.novalocal neutron-server[14563]: raise ConfigFileParseError(pe.filename, str(pe))
Nov 06 13:29:42 controller.novalocal neutron-server[14563]: oslo.config.cfg.ConfigFileParseError: Failed to pa...ue'

Nov 06 13:29:42 controller.novalocal systemd[1]: neutron-server.service: main process exited, code=exited, st...LURE

solution is to find out the line and remove the blank Space. 

Docker with Openstack Giving Error "ova.openstack.common.threadgroup ValueError: Timeout value connect was Timeout"

When I try to integrate Docker to Openstack Juno, I am not able to start the nova service in the compute node. I followed https://wiki.openstack.org/wiki/Docker .

When I remove or comment out #compute_driver = novadocker.virt.docker.DockerDriver from nova configuration, the service is able to start but the pid gets killed soon.

I am getting following error while trying to start the nova service.

Complete Error.
http://paste.openstack.org/show/128805/

Sample Error
****2014-11-03 14:14:08.138 5264 TRACE nova.openstack.common.threadgroup   File "/usr/lib/python2.7/site-packages/requests/packages/urllib3/util/timeout.py", line 125, in _validate_timeout
2014-11-03 14:14:08.138 5264 TRACE nova.openstack.common.threadgroup     "int or float." % (name, value))
2014-11-03 14:14:08.138 5264 TRACE nova.openstack.common.threadgroup ValueError: Timeout value connect was Timeout(connect=10, read=10, total=None), but it must be an int or float.
2014-11-03 14:14:08.138 5264 TRACE nova.openstack.common.threadgroup****


The issue has been fixed , I didn't installed the docker requirement . Once i installed it and rebooted the server its working fine now .

For testing I have used * for installation ,we just need to install the correct packages. https://github.com/stackforge/nova-do...

yum install *pbr*
yum install *six*
yum install *babel*
yum install *oslo*
yum install docker-py

Squid Proxy Server

Squid is a proxy server and web cache daemon. It has a wide variety of uses, from speeding up a web server by caching repeated requests; to caching web, DNS and other computer network lookups for a group of people sharing network resources; to aiding security by filtering traffic. Although primarily used for HTTP and FTP, Squid includes limited support for several other protocols including TLS, SSL, Internet Gopher and HTTPS


yum -y install squid
chkconfig squid on

IMPORTANT: First write all the ACLS and Later the http_access order. The Order in which the rules are written in having effect on the working of Proxy.
#Port to which squid listens
http_port 3128


Allowing the Know network/IP
============================
Declare all the known network and allow those network/IP

acl our_networks src 192.168.25.0/24 192.168.2.0/24 10.1.0.1
http_access allow our_networks

The same way we can deny the access using

http_access deny our_networks


Blocking Sites using proxy.
==========================
acl blocksite1 dstdomain www.yahoo.com .facebook.com
http_access deny blocksite1

Blocking List of Sites.
======================
acl blocksitelist dstdomain "/etc/squid/restricted_sites"
http_access deny blocksitelist


Blocking Sites with Specific Words using proxy.
==============================================
acl blockwords url_regex gmail
http_access deny blockwords

Blocking List of Words.
======================
acl blockwordlist url_regex "/etc/squid/restricted_words"
http_access deny blockwordlist


Display Custom message For Blocked Site.
========================================
deny_info <Error-Page-Name> <acl-name>

You can get the error page name from  /usr/share/squid/errors/templates/ some of the error pages are as follow's.
ERR_ACCESS_DENIED            ERR_FTP_FAILURE       ERR_INVALID_URL          ERR_SOCKET_FAILURE
ERR_CACHE_ACCESS_DENIED      ERR_FTP_FORBIDDEN     ERR_LIFETIME_EXP         ERR_TOO_BIG
ERR_CACHE_MGR_ACCESS_DENIED  ERR_FTP_NOT_FOUND     ERR_NEW                  ERR_UNSUP_HTTPVERSION
ERR_CANNOT_FORWARD           ERR_FTP_PUT_CREATED   ERR_NO_RELAY             ERR_UNSUP_REQ
ERR_CONNECT_FAIL             ERR_FTP_PUT_ERROR     ERR_ONLY_IF_CACHED_MISS  ERR_URN_RESOLVE
ERR_DIR_LISTING              ERR_FTP_PUT_MODIFIED  ERR_PRECONDITION_FAILED  ERR_WRITE_ERROR
ERR_DNS_FAIL                 ERR_FTP_UNAVAILABLE   ERR_READ_ERROR           ERR_ZERO_SIZE_OBJECT
ERR_ESI                      ERR_ICAP_FAILURE      ERR_READ_TIMEOUT
ERR_FORWARDING_DENIED        ERR_INVALID_REQ       ERR_SECURE_CONNECT_FAIL
ERR_FTP_DISABLED             ERR_INVALID_RESP      ERR_SHUTTING_DOWN

If we need to input custom pages we need to create the page here and mention it in deny_info part. Theis can be mentioned just above corresponding http_access.
For example if we make a Error page as ERR_NEW the rules will be like.

acl blockwordlist url_regex "/etc/squid/restricted_words"
deny_info ERR_NEW blockwordlist
http_access deny blockwordlist

FOR HTTPS WE WILL GET A PROXY REFUSING MESSAGE DUE TO https://bugzilla.mozilla.org/show_bug.cgi?id=493699 .


Blocking and Allowing By Time
=============================
In second acl the time MTWHFA means the Monday to Saturday
Time 16:00-19:00 is the time frame in 24hr time frame

acl myip src 192.168.25.31
acl worktime time MTWHFA 16:00-19:00
http_access allow myip worktime



Setting up maxconn ACL
======================
acl ACCOUNTSDEPT 192.168.5.0/24
acl limitusercon maxconn 3
http_access deny ACCOUNTSDEPT limitusercon

acl ACCOUNTSDEPT 192.168.3.0/24 : Our accounts department IP range
acl limitusercon maxconn 3 : Set 3 simultaneous web access from the same client IP
http_access deny ACCOUNTSDEPT limitusercon : Apply ACL

Mentioning Allowed Ports
========================
acl SSL_ports port 443
acl Safe_ports port 80          # http
acl Safe_ports port 21          # ftp
acl Safe_ports port 443         # https
acl Safe_ports port 70          # gopher

# Deny requests to certain unsafe ports
http_access deny !Safe_ports

# Deny CONNECT to other than secure SSL ports
http_access deny CONNECT !SSL_ports



Adding User Autnetication to Squid
==================================
Check the ncsa_auth file under squid and enter the following line in squid.conf. The ncsa_auth can be in either lib or lib64 directory as per your OS architecture.

#Add Following Line in squid.conf#
auth_param basic program /usr/lib64/squid/ncsa_auth /etc/squid/squid_user

#Creating the User file and adding the user in to the List.#
touch /etc/squid/squid_user
htpasswd /etc/squid/squid_user <username>

#To enable the authentication in the current proxy add the following Line in squid.conf along another acl and http_access rules #

acl class proxy_auth REQUIRED
http_access allow clas

And finally deny all other access to this proxy
==============================================
http_access deny all

Installing Swish Module for php

Swish package does not comes with current repo's of centos or redhat so we need to compile and install it before installing the swish package through the pecl. Else we may end up in error while installing Swish package with pecl

Downloading and installing the swish packages.
wget http://swish-e.org/distribution/swish-e-2.4.7.tar.gz
tar zxvf swish-e-2.4.7.tar.gz
cd swish-e-2.4.7
./configure
make
make check
make install

cd ~

Installing swish php module using pecl
pecl install swish-beta
chmod 755 /usr/lib64/php/modules/swish.so
echo "extension=swish.so" >> /etc/php.ini

Installing PHP modules using pecl command.

Once you have installed the php you need to install needed modules to support the development process. we can use the pecl function to install the modules.

To install pecl function.

yum install php-pear

Now to install needed modules just use pecl

pecl install <Module Name>

To install a beta version
pecl install <Module Name>-beta

To list all modules in pecl database

pecl list-all

To check whether the module is installed or not

php -m