Pages

Wednesday, December 16, 2015

AWS EBS SnapShot Scipt for Linux


The script itself is fairly straightforward, but took several dry-runs in order to fine tune it so that it would reliably create the snapshots, but more importantly would clear out old snapshots older than 7 days.

You can see the for loop for deleting older snapshots. This is done by parsing snapshot dates, converting the dates to a pure numeric value and comparing said numeric value to a “7 days ago” date variable.

Take a look at the script below, hopefully it will be useful to you! There could be more error checking, but that should be fairly easy to do.



1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
#!/bin/sh
# EBS Snapshot volume script
# Written by Star Dot Hosting
# www.stardothosting.com
 
# Constants
ec2_bin="/opt/aws/bin"
my_cert="/opt/aws/cert.txt"
my_key="/opt/aws/key.txt"
instance_id=`wget -q -O- http://169.254.169.254/latest/meta-data/instance-id`
 
# Dates
datecheck_7d=`date +%Y-%m-%d --date '7 days ago'`
datecheck_s_7d=`date --date="$datecheck_7d" +%s`
 
# Get all volume info and copy to temp file
$ec2_bin/ec2-describe-volumes -C $my_cert -K $my_key  --filter "attachment.instance-id=$instance_id" > /tmp/volume_info.txt 2>&1
 
 
# Get all snapshot info
$ec2_bin/ec2-describe-snapshots -C $my_cert -K $my_key | grep "$instance_id" > /tmp/snap_info.txt 2>&1
 
# Loop to remove any snapshots older than 7 days
for obj0 in $(cat /tmp/snap_info.txt)
do
 
        snapshot_name=`cat /tmp/snap_info.txt | grep "$obj0" | awk '{print $2}'`
        datecheck_old=`cat /tmp/snap_info.txt | grep "$snapshot_name" | awk '{print $5}' | awk -F "T" '{printf "%sn", $1}'`
        datecheck_s_old=`date "--date=$datecheck_old" +%s`
 
#       echo "snapshot name: $snapshot_name"
#       echo "datecheck 7d : $datecheck_7d"
#       echo "datecheck 7d s : $datecheck_s_7d"
#       echo "datecheck old : $datecheck_old"
#       echo "datecheck old s: $datecheck_s_old"
 
        if (( $datecheck_s_old <= $datecheck_s_7d ));
        then
                echo "deleting snapshot $snapshot_name ..."
                $ec2_bin/ec2-delete-snapshot -C $my_cert -K $my_key $snapshot_name
        else
                echo "not deleting snapshot $snapshot_name ..."
 
        fi
 
done
 
 
# Create snapshot
for volume in $(cat /tmp/volume_info.txt | grep "VOLUME" | awk '{print $2}')
do
        description="`hostname`_backup-`date +%Y-%m-%d`"
        echo "Creating Snapshot for the volume: $volume with description: $description"
        $ec2_bin/ec2-create-snapshot -C $my_cert -K $my_key -d $description $volume
done

Wednesday, December 2, 2015

"No such file or directory” when adding application with alternatives in Linux


Issue 

# alternatives --install /usr/bin/javaws javaws /usr/java/jdk1.7.0_07/jre/bin/javaws  20000
failed to read link /usr/bin/javaws: No such file or directory

or 

sudo update-alternatives --install /usr/bin/jar jar /usr/java/latest/bin/jar 20000
failed to read link /usr/bin/jar: No such file or directory

Fix 

sudo rm /usr/bin/jar
sudo rm /var/lib/alternatives/jar
sudo alternatives --install /usr/bin/jar jar /usr/java/latest/bin/jar 20000

rm /var/lib/alternatives/javaws
rm /var/lib/alternatives/javawsrm: remove regular file `/var/lib/alternatives/javaws'? y alternatives --install /usr/bin/javaws javaws /usr/java/jdk1.7.0_07/jre/bin/javaws  20000

Saturday, November 21, 2015

How To Grant Access To Only One S3 Bucket Using AWS IAM Policy

Click on “My Account/Console” and select “Security Credentials”.

Select “Continue to Security Credentials”.

Select “Policies” on the left menu, then click “Create Policy”.

Select “Create Your Own Policy”.

Fill out the “Policy Name”, “Description” and “Policy Document” fields.
Replace “YOUR-BUCKET” in the example below with your bucket name.
Please note that we set “ListAllMyBuckets” to list all buckets owned by you, so that tools that lists buckets will work.

NOTE: If you explicitly list out the actions for your bucket, please also include
"s3:GetBucketLocation" so that ObjectiveFS can select the right S3 endpoint to talk with.
Example policy:
{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                        "s3:GetBucketLocation",
                        "s3:ListAllMyBuckets"
                      ],
            "Resource": "arn:aws:s3:::*"
        },
        {
            "Effect": "Allow",
            "Action": "s3:*",
            "Resource": [
                "arn:aws:s3:::YOUR-BUCKET",
                "arn:aws:s3:::YOUR_BUCKET/*"
            ]
        }
    ]
}


Wednesday, November 18, 2015

Create SFTP server in Windows Server.



Normally we need to install ftp role and get the site configured to use the FTP/SFTP in the windows server. Now we can Create SFTP server in windows server by enabling freeftpd from following URL

http://www.freesshd.com/?ctt=download

Tuesday, November 10, 2015

Oracle Database backup to AWS S3 : Error occured when installing OSB(Oracle Security Backup) on Amazon S



I tried to set up rman backup using amazon cloud module and I faced up following error.
Internet connections are positively working.

#> java -jar osbws_install.jar -AWSID MyAWSID -AWSKey MYAWSKEY -otnUser MYOTNID -otnPass MYOTNPASS -walletDir $ORACLE_HOME/dbs/osbws_wallet -libDir $ORACLE_HOME/lib -debug










Fix:  The OSB module works only with Java version 1.5 and 1.6. The new Machines are running with java version 1.7. try with Java version 1.6.


Tuesday, October 27, 2015

Monday, October 12, 2015

AWS S3 : Limiting Access from Certain IP's

We need to our nodes to connect to S3 from a single IP address, an elastic IP attached to our gateway node.

So - based on this design, we needed a way to only allow access to a set of buckets from this single IP address. For example purposes we are using the IP of 72.309.38.2.

IF you were to follow the initial example laid out by the AWS documentation - you’ll end up with a policy that probably looks similar to this.

{
    "Id": "S3PolicyId1",
    "Statement": [
        {
            "Sid": "IPAllow",
            "Effect": "Allow",
            "Principal": {
                "AWS": "*"
            },
            "Action": "s3:*",
            "Resource": "arn:aws:s3:::my-wicked-awesome-bucket/*",
            "Condition": {
                "IpAddress": {
                    "aws:SourceIp": "72.309.38.2/32"
                }
            }
        }
    ]
}

What you’re going to find, after banging your head on the table a few times, is that this policy does not work. There does not appear to be an implied deny rule with S3 buckets (similar to how IAM access policies are setup). I did my testing with the s3-curl command - When you have that tool setup - you can make a query like the following:


./s3curl.pl --head --id=AKIAYOURACCESSKEY --key=YourSecretKey -- https://s3.amazonaws.com/my-wicked-awesome-bucket/even-more-awesome.file
Using the policy above - returned this:

HTTP/1.1 200 OK
x-amz-id-2: BLPqeibX8nZGnSDNi9zRhb+6a8fDiOW6Ij1OXhadWknJKCX9WAb7x1xNETvdXAEv
x-amz-request-id: F6AF36D912E6003B
Date: Fri, 27 Apr 2012 00:25:52 GMT
Last-Modified: Tue, 17 Apr 2012 17:50:11 GMT
ETag: "8553074962ba71d8b2b600b971ba80a8"
Accept-Ranges: bytes
Content-Type: application/pdf
Content-Length: 1451065
Server: AmazonS3
After troubleshooting this with the AWS support team - they let me know of the following:

By default accounts are restricted from accessing S3 unless they have been given access via policy. However, S3 is designed by default to allow any IP address access. So to block IP's you would have to specify denies explicitly in the policy instead of allows.

Once I learned this - the policy was easy to adjust. I flipped around the policy from allowing access from only my IP address to denying access from everywhere that was NOT my IP address.

My new policy looked like this:

{
    "Id": "S3PolicyId1",
    "Statement": [
        {
            "Sid": "IPDeny",
            "Effect": "Deny",
            "Principal": {
                "AWS": "*"
            },
            "Action": "s3:*",
            "Resource": "arn:aws:s3:::my-wicked-awesome-bucket/*",
            "Condition": {
                "NotIpAddress": {
                    "aws:SourceIp": "72.309.38.2/32"
                }
            }
        }
    ]
}

And when I ran my s3-curl command again - I now received the following response.

HTTP/1.1 403 Forbidden
x-amz-request-id: 3B90483F655AA692
x-amz-id-2: 0XEO2miEzZEwbU2N5zPwzLtX4J7BJx/LLXpvSLPpQ7e2wuocx49katsD+ZcqwYA7
Content-Type: application/xml
Transfer-Encoding: chunked
Date: Fri, 27 Apr 2012 00:27:14 GMT
Server: AmazonS3