bunch of changes leading up to the merge to develop.

@@ -31,7 +31,8 @@ deploy
 
 
 The ``deploy`` command does whatever is required to deploy the
 The ``deploy`` command does whatever is required to deploy the
 current version of your Lambda function such as creating/updating policies and
 current version of your Lambda function such as creating/updating policies and
-roles and creating or updating the function itself.
+roles, creating or updating the function itself, and adding any event sources
+specified in your config file.
 
 
 When the command is run the first time, it creates all of the relevant
 When the command is run the first time, it creates all of the relevant
 resources required.  On subsequent invocations, it will attempt to determine
 resources required.  On subsequent invocations, it will attempt to determine
@@ -49,6 +50,10 @@ invoke
 The ``invoke`` command makes a synchronous call to your Lambda function,
 The ``invoke`` command makes a synchronous call to your Lambda function,
 passing test data and display the resulting log data and any response returned
 passing test data and display the resulting log data and any response returned
 from your Lambda function.
 from your Lambda function.
+
+The ``invoke`` command takes one positional argument, the ``data_file``.  This
+should be the path to a JSON data file that will be sent to the function as
+data.
   
   
 tag
 tag
 ---
 ---
@@ -84,9 +89,10 @@ event_sources
 -------------
 -------------
 
 
 The ``event_sources`` command provides access the commands available for
 The ``event_sources`` command provides access the commands available for
-dealing with event sources.  This command has an additional option:
+dealing with event sources.  This command takes an additional positional
+argument, ``command``.
 
 
-* --command - the command to run (add|update|enable|disable)
+* command - the command to run (list|enable|disable)
 
 
 status
 status
 ------
 ------

+The Config File
+===============
+
+The config file is at the heart of kappa.  It is what describes your functions
+and drives your deployments.  This section provides a reference for all of the
+elements of the kappa config file.
+
+
+Example
+-------
+
+Here is an example config file showing all possible sections.
+
+.. sourcecode:: yaml
+    :linenos:
+
+    ---
+    name: kappa-python-sample
+    environments:
+      env1:
+        profile: profile1
+        region: us-west-2
+        policy:
+          resources:
+            - arn: arn:aws:dynamodb:us-west-2:123456789012:table/foo
+              actions:
+              - "*"
+            - arn: arn:aws:logs:*:*:*
+              actions:
+              - "*"
+       event_sources:
+         -
+           arn: arn:aws:kinesis:us-west-2:123456789012:stream/foo
+           starting_position: LATEST
+           batch_size: 100
+      env2:
+        profile: profile2
+        region: us-west-2
+        policy_resources:
+          - arn: arn:aws:dynamodb:us-west-2:234567890123:table/foo
+            actions:
+            - "*"
+          - arn: arn:aws:logs:*:*:*
+            actions:
+            - "*"
+       event_sources:
+         -
+           arn: arn:aws:kinesis:us-west-2:234567890123:stream/foo
+           starting_position: LATEST
+           batch_size: 100
+    lambda:
+      description: A simple Python sample
+      handler: simple.handler
+      runtime: python2.7
+      memory_size: 256
+      timeout: 3
+      vpc_config:
+        security_group_ids:
+          - sg-12345678
+          - sg-23456789
+        subnet_ids:
+          - subnet-12345678
+          - subnet-23456789
+          
+
+Explanations:
+
+===========    =============================================================
+Line Number    Description
+===========    =============================================================
+2              This name will be used to name the function itself as well as
+               any policies and roles created for use by the function.
+3              A map of environments.  Each environment represents one
+               possible deployment target.  For example, you might have a
+               dev and a prod.  The names can be whatever you want but the
+               environment names are specified using the --env option when
+               you deploy.
+5              The profile name associated with this environment.  This
+               refers to a profile in your AWS credential file.
+6              The AWS region associated with this environment.
+7              This section defines the elements of the IAM policy that will
+               be created for this function in this environment.
+9              Each resource your function needs access to needs to be
+               listed here.  Provide the ARN of the resource as well as
+               a list of actions.  This could be wildcarded to allow all
+               actions but preferably should list the specific actions you
+               want to allow.
+15             If your Lambda function has any event sources, this would be
+               where you list them.  Here, the example shows a Kinesis
+               stream but this could also be a DynamoDB stream, an SNS
+               topic, or an S3 bucket.
+18             For Kinesis streams and DynamoDB streams, you can specify
+               the starting position (one of LATEST or TRIM_HORIZON) and
+               the batch size.
+35             This section contains settings specify to your Lambda
+               function.  See the Lambda docs for details on these.
+===========    =============================================================

@@ -3,8 +3,8 @@
    You can adapt this file completely to your liking, but it should at least
    You can adapt this file completely to your liking, but it should at least
    contain the root `toctree` directive.
    contain the root `toctree` directive.
 
 
-Welcome to kappa's documentation!
-=================================
+Welcome to kappa's documentation
+================================
 
 
 Contents:
 Contents:
 
 
@@ -12,7 +12,8 @@ Contents:
    :maxdepth: 2
    :maxdepth: 2
 
 
    why
    why
-   config_file
+   how
+   config_file_example
    commands
    commands
 
 
 
 

@@ -187,6 +187,15 @@ class Context(object):
         for event_source in self.event_sources:
         for event_source in self.event_sources:
             event_source.update(self.function)
             event_source.update(self.function)
 
 
+    def list_event_sources(self):
+        event_sources = []
+        for event_source in self.event_sources:
+            event_sources.append({'arn': event_source.arn,
+                                  'starting_position': event_source.starting_position,
+                                  'batch_size': event_source.batch_size,
+                                  'enabled': event_source.enabled})
+        return event_sources
+
     def enable_event_sources(self):
     def enable_event_sources(self):
         for event_source in self.event_sources:
         for event_source in self.event_sources:
             event_source.enable(self.function)
             event_source.enable(self.function)
@@ -206,6 +215,7 @@ class Context(object):
         LOG.debug('Waiting for policy/role propogation')
         LOG.debug('Waiting for policy/role propogation')
         time.sleep(5)
         time.sleep(5)
         self.function.create()
         self.function.create()
+        self.add_event_sources()
 
 
     def deploy(self):
     def deploy(self):
         if self.policy:
         if self.policy:

@@ -33,7 +33,7 @@ class EventSource(object):
 
 
     @property
     @property
     def starting_position(self):
     def starting_position(self):
-        return self._config.get('starting_position', 'TRIM_HORIZON')
+        return self._config.get('starting_position', 'LATEST')
 
 
     @property
     @property
     def batch_size(self):
     def batch_size(self):
@@ -41,7 +41,7 @@ class EventSource(object):
 
 
     @property
     @property
     def enabled(self):
     def enabled(self):
-        return self._config.get('enabled', True)
+        return self._config.get('enabled', False)
 
 
 
 
 class KinesisEventSource(EventSource):
 class KinesisEventSource(EventSource):
@@ -161,7 +161,7 @@ class S3EventSource(EventSource):
 
 
     def add(self, function):
     def add(self, function):
         notification_spec = {
         notification_spec = {
-            'LambdaFunctionConfigurations':[
+            'LambdaFunctionConfigurations': [
                 {
                 {
                     'Id': self._make_notification_id(function.name),
                     'Id': self._make_notification_id(function.name),
                     'Events': [e for e in self._config['events']],
                     'Events': [e for e in self._config['events']],

@@ -57,7 +57,7 @@ class Function(object):
     @property
     @property
     def dependencies(self):
     def dependencies(self):
         return self._config.get('dependencies', list())
         return self._config.get('dependencies', list())
-    
+
     @property
     @property
     def description(self):
     def description(self):
         return self._config['description']
         return self._config['description']
@@ -71,6 +71,18 @@ class Function(object):
         return self._config['memory_size']
         return self._config['memory_size']
 
 
     @property
     @property
+    def vpc_config(self):
+        vpc_config = {}
+        if 'vpc_config' in self._config:
+            if 'security_group_ids' in self._config['vpc_config']:
+                sgids = self._config['vpc_config']['security_group_ids']
+                vpc_config['SecurityGroupIds'] = ','.join(sgids)
+            if 'subnet_ids' in self._config['vpc_config']:
+                snids = self._config['vpc_config']['subnet_ids']
+                vpc_config['SubnetIds'] = ','.join(snids)
+        return vpc_config
+
+    @property
     def zipfile_name(self):
     def zipfile_name(self):
         return '{}.zip'.format(self._context.name)
         return '{}.zip'.format(self._context.name)
 
 
@@ -372,6 +384,7 @@ class Function(object):
                         Description=self.description,
                         Description=self.description,
                         Timeout=self.timeout,
                         Timeout=self.timeout,
                         MemorySize=self.memory_size,
                         MemorySize=self.memory_size,
+                        VpcConfig=self.vpc_config,
                         Publish=True)
                         Publish=True)
                     LOG.debug(response)
                     LOG.debug(response)
                     description = 'For stage {}'.format(
                     description = 'For stage {}'.format(
@@ -418,6 +431,7 @@ class Function(object):
                 response = self._lambda_client.call(
                 response = self._lambda_client.call(
                     'update_function_configuration',
                     'update_function_configuration',
                     FunctionName=self.name,
                     FunctionName=self.name,
+                    VpcConfig=self.vpc_config,
                     Role=exec_role,
                     Role=exec_role,
                     Handler=self.handler,
                     Handler=self.handler,
                     Description=self.description,
                     Description=self.description,

@@ -23,34 +23,32 @@ LOG = logging.getLogger(__name__)
 
 
 class Policy(object):
 class Policy(object):
 
 
+    _path_prefix = '/kappa/'
+
     def __init__(self, context, config):
     def __init__(self, context, config):
-        self._context = context
-        self._config = config
+        self.context = context
+        self.config = config
         self._iam_client = kappa.awsclient.create_client(
         self._iam_client = kappa.awsclient.create_client(
-            'iam', self._context.session)
-        self._arn = self._config['policy'].get('arn', None)
-
-    @property
-    def environment(self):
-        return self._context.environment
+            'iam', self.context.session)
+        self._arn = self.config['policy'].get('arn', None)
 
 
     @property
     @property
     def name(self):
     def name(self):
-        return '{}_{}'.format(self._context.name, self.environment)
+        return '{}_{}'.format(self.context.name, self.context.environment)
 
 
     @property
     @property
     def description(self):
     def description(self):
         return 'A kappa policy to control access to {} resources'.format(
         return 'A kappa policy to control access to {} resources'.format(
-            self.environment)
+            self.context.environment)
 
 
     def document(self):
     def document(self):
-        if ('resources' not in self._config['policy'] and
-                'statements' not in self._config['policy']):
+        if ('resources' not in self.config['policy'] and
+                'statements' not in self.config['policy']):
             return None
             return None
-        document = {"Version": "2012-10-17"}
+        document = {'Version': '2012-10-17'}
         statements = []
         statements = []
         document['Statement'] = statements
         document['Statement'] = statements
-        for resource in self._config['policy']['resources']:
+        for resource in self.config['policy']['resources']:
             arn = resource['arn']
             arn = resource['arn']
             _, _, service, _ = arn.split(':', 3)
             _, _, service, _ = arn.split(':', 3)
             statement = {"Effect": "Allow",
             statement = {"Effect": "Allow",
@@ -60,15 +58,11 @@ class Policy(object):
                 actions.append("{}:{}".format(service, action))
                 actions.append("{}:{}".format(service, action))
             statement['Action'] = actions
             statement['Action'] = actions
             statements.append(statement)
             statements.append(statement)
-        for statement in self._config['policy'].get('statements', []):
+        for statement in self.config['policy'].get('statements', []):
             statements.append(statement)
             statements.append(statement)
         return json.dumps(document, indent=2, sort_keys=True)
         return json.dumps(document, indent=2, sort_keys=True)
 
 
     @property
     @property
-    def path(self):
-        return self._config.get('path', '/kappa/')
-
-    @property
     def arn(self):
     def arn(self):
         if self._arn is None:
         if self._arn is None:
             policy = self.exists()
             policy = self.exists()
@@ -79,7 +73,7 @@ class Policy(object):
     def _find_all_policies(self):
     def _find_all_policies(self):
         try:
         try:
             response = self._iam_client.call(
             response = self._iam_client.call(
-                'list_policies', PathPrefix=self.path)
+                'list_policies', PathPrefix=self._path_prefix)
         except Exception:
         except Exception:
             LOG.exception('Error listing policies')
             LOG.exception('Error listing policies')
             response = {}
             response = {}
@@ -130,11 +124,11 @@ class Policy(object):
         m = hashlib.md5()
         m = hashlib.md5()
         m.update(document.encode('utf-8'))
         m.update(document.encode('utf-8'))
         policy_md5 = m.hexdigest()
         policy_md5 = m.hexdigest()
-        cached_md5 = self._context.get_cache_value('policy_md5')
+        cached_md5 = self.context.get_cache_value('policy_md5')
         LOG.debug('policy_md5: %s', policy_md5)
         LOG.debug('policy_md5: %s', policy_md5)
         LOG.debug('cached md5: %s', cached_md5)
         LOG.debug('cached md5: %s', cached_md5)
         if policy_md5 != cached_md5:
         if policy_md5 != cached_md5:
-            self._context.set_cache_value('policy_md5', policy_md5)
+            self.context.set_cache_value('policy_md5', policy_md5)
             return True
             return True
         return False
         return False
 
 
@@ -156,7 +150,7 @@ class Policy(object):
             try:
             try:
                 response = self._iam_client.call(
                 response = self._iam_client.call(
                     'create_policy',
                     'create_policy',
-                    Path=self.path, PolicyName=self.name,
+                    Path=self._path_prefix, PolicyName=self.name,
                     PolicyDocument=document,
                     PolicyDocument=document,
                     Description=self.description)
                     Description=self.description)
                 LOG.debug(response)
                 LOG.debug(response)

@@ -139,20 +139,19 @@ def delete(ctx):
 
 
 
 
 @cli.command()
 @cli.command()
-@click.option(
-    '--command',
-    type=click.Choice(['add', 'update', 'enable', 'disable']),
-    help='Operation to perform on event sources')
+@click.argument('command',
+                type=click.Choice(['list', 'enable', 'disable']))
 @pass_ctx
 @pass_ctx
 def event_sources(ctx, command):
 def event_sources(ctx, command):
-    """Add any event sources specified in the config file"""
-    if command == 'add':
-        click.echo('adding event sources')
-        ctx.add_event_sources()
-        click.echo('done')
-    elif command == 'update':
-        click.echo('updating event sources')
-        ctx.update_event_sources()
+    """List, enable, and disable event sources specified in the config file"""
+    if command == 'list':
+        click.echo('listing event sources')
+        event_sources = ctx.list_event_sources()
+        for es in event_sources:
+            click.echo('arn: {}'.format(es['arn']))
+            click.echo('starting position: {}'.format(es['starting_position']))
+            click.echo('batch size: {}'.format(es['batch_size']))
+            click.echo('enabled: {}'.format(es['enabled']))
         click.echo('done')
         click.echo('done')
     elif command == 'enable':
     elif command == 'enable':
         click.echo('enabling event sources')
         click.echo('enabling event sources')

-boto3>=1.2.2
-placebo>=0.4.1
+boto3>=1.2.3
+placebo>=0.7.0
 click==5.1
 click==5.1
 PyYAML>=3.11
 PyYAML>=3.11
 mock>=1.0.1
 mock>=1.0.1