Managing EC2 Security Groups with SaltStack and Python Part 1

This is the first in a series of posts related to automating the management and auditing of AWS EC2 Security Groups.  This first post will cover how to use SaltStack state files to maintain your security group rules.  In the second post, we will use python to populate a database that contains instances. groups, rules and their associations.  In the final post, we will create a CLI that will allow you to add/remove rules to security groups and it will push all changes to AWS.

Assumptions

I assume you already have a salt master server setup and understand the basics of using Saltstack.  It is beyond the scope of this series of posts to cover installing and/or configuring a salt master.  If you have not done this yet or need some additional info on using SaltStack, please visit their official Walk-through.

Goals

My goals when starting this were pretty simple.  I needed a central way to manage all 135 of my organizations security groups.  I needed a way to ensure that any ports that might have been opened for testing or temporary reasons are automatically closed on a regular interval.  I needed all of this to be auditable for HIPPA and SOX.

Getting Started

In this post, I am going to discuss using salt states to manage EC2 security groups.

Before we get going, you need to make sure to have the boto python library installed on your salt master.  This library is used by SaltStack for communicating with AWS.

check if boto is installed:

sudo pip freeze | grep boto

If no boto version is outputted, you will need to install the library.

sudo pip install boto

Once boto has completed installing, we are ready to get started.

Authentication

There ate a couple different ways that you can authenticate SaltStack with AWS.  One of the ways is to place your secret id and access key in your minion config file.  This is ok if you only have a singe set of credentials for your whole AWS environment.

Add the following to your minion config file:

secgroup.keyid: GKTADJGHEIQSXMKKRBJ08H
secgroup.key: askdjghsdfjkghWupUjasdflkdfklgjsdfjajkghs

Again, while this method does work, unless you only have very few state files, this will be very inefficient. An alternative method, though still not ideal, would be to put the access id and access key in the state files directly.

Ensure mysecgroup exists:
    boto_secgroup.present:
        - name: mysecgroup
        - description: My security group
        - region: us-east-1
        - keyid: GKTADJGHEIQSXMKKRBJ08H
        - key: askdjghsdfjkghWupUjasdflkdfklgjsdfjajkghs

this method has the same drawbacks as the previous methos, with one additional added drawback of having to change every state file in order to change your AWS access keys. Fortunately, SaltStack has Pillars, a mechanism to store static data that can be referenced by state files. THis is what using a pillar to store the credentials would look like:
pillar file:

myprofile:
    keyid: GKTADJGHEIQSXMKKRBJ08H
    key: askdjghsdfjkghWupUjasdflkdfklgjsdfjajkghs
    region: us-east-1

state file:

Ensure mysecgroup exists:
    boto_secgroup.present:
        - name: mysecgroup
        - description: My security group
        - profile: myprofile

By referencing the profile from the pillar, we dynamically pull in the keyid, key and region into our state file. This now alows us to change the credentials in a single location and have all the states referencing the pillar to use the updated credentials.

State file

Now, let’s take a look at a state file that reflects the security group we want to manage.

State file jenkins.sls:

Ensure jenkins exists:
  boto_secgroup.present:   # ensures the group exists, it will be created otherwise
    - name: jenkins      # the name of the group
    - description: My Jenkins sec group   # a description of the group
    - vpc_id: vpc-41614324      # create group in specified vpc
    - profile: myprofile     # pull profile settings from pillar
    - rules:                   # a list of rules
      - ip_protocol: tcp         # tcp or udp
        from_port: 9090          # from port
        to_port: 9090            # destination port
        source_group_group_id:   # authorize another security groups
          - sg-eccaf68a          # the group id to authorize
      - ip_protocol: tcp        
        from_port: 22
        to_port: 22
        cidr_ip:                 # here, we want to authorize and IP Range
          - 0.0.0.0/0            # open to the world
      - ip_protocol: tcp
        from_port: 443
        to_port: 443
        cidr_ip:
          - 0.0.0.0/0

This state file, when executed will check to see if a security group in the specified vpc with the name jenkins already exists. If this group does not already exist, it will be created. The list of rules will be added to the list and any rules that may exist in the group that are not defined in the state file will be removed.

Execution

Executing states that target AWS vs a minion are a slight bit different to execute.  We now have the above state file created in our base state folder.  Now, to run our state use:

salt "salt-master" state.sls jenkins

Now you will see the output of the running state.

Next steps

Unless I want to manage 135 state files for my infrastrucute, I need to make this a more dynamic process.

In the next post, I will discuss using python and jinja templates with Salt States to allow for a more dynamic way to manage the security groups.  Finally, in the third post, I will cover adding a cli to our python app to allow full security group management to be fully handled by our python app.

Leave a Reply

Your email address will not be published. Required fields are marked *