cfnpp: cloudformation plus plus

Goal

Make it easier to write, re-use and template CloudFormation templates

Why did you do this?

Writing and maintaining CloudFormation templates is tedious, but using tools that take you too far away from the CloudFormation templates gives more opportunity for bugs to creep in and leaves you operating several months behind updates to AWS services and CloudFormation.

But mainly I needed something to fill up some free time.

What it does

• Works with existing CloudFormation templates. (Some possible issues with the Symfony YAML parser notwithstanding.)
• Adds support for a cfnpp: meta block which provides some extended functionality detailed below.
• Adds several additional !functions detailed below.
• More planned, detailed at the end of the README.

Meta Block

At the top level, a cfnpp block can be specified to make use of some extended functionality:

• Includes
• Variables

For example:

cfnpp:
  stack:
    - parent.yaml
  variables:
    A: 1
    B: 2
    foo: bar

Stack

Any files listed will be processed in the order given before the current document. If those documents specify any documents in their stack, those will be processed before those documents in the order given and so on.

There is nothing preventing you from including a file more than once, nor from creating a loop. It's undecided at the time being whether this is desirable or whether we need some more complicated dependency graph or something to process these.

The first document is loaded, then each subsequent document is merged with it, and then any functions present processed.

Merging follows some simple rules:

• Maps are merged, with duplicate keys being overwritten.
• Arrays are appended.
• Scalars overwrite.
• Any type mismatch, and the value in the template being merged in simply overwrites whatever's in the original.

For example, starting with:

Resources:
  Resource1:
    Name: My Resource Name
    VpcId: vpc-1234
    AvailabilityZones:
      - us-east-1a
      - us-east-1b
    ValidUsers:
      Alice: 'aws:arn:1234:alice'
      Bob: 'aws:arn:1234:bob'
    AllowPorts:
      - 80
      - 443

And merging in:

Resources:
  Resource1:
    Name: New Name
    AvailabilityZones:
      - us-east-1c
    ValidUsers:
      Charlie: 'aws:arn:1234:charlie'
    AllowPorts: False

Will result in:

Resources:
  Resource1:
    Name: New Name
    VpcId: vpc-1234
    AvailabilityZones:
      - us-east-1a
      - us-east-1b
      - us-east-1c
    ValidUsers:
      Alice: 'aws:arn:1234:alice'
      Bob: 'aws:arn:1234:bob'
      Charlie: 'aws:arn:1234:charlie'
    AllowPorts: False

Variables

Variables listed are set from all templates before the first one is processed. Variables are processed in the same order that the documents will be compiled, and later values will overwrite earlier ones.

That is, a base template can define some default values, and a dependent one can specify new values and they will be visible to the base template.

Functions

!replace

Tells the merging process to not attempt to perform any merging, but simply replace whatever's in the target document with these values. For example, given a base template:

Resources;
  Resource1:
    AvailabilityZones:
      - us-east-1a
      - us-east-1b
      - us-east-1c

Merging in:

Resources:
  Resource1:
    AvailabilityZones: !replace
      - us-east-1c

Will result in:

Resources:
  Resource1:
    AvailabilityZones:
      - us-east-1c

!unset

Removes a property completely, rather than simply overriding its value or something. The YAML parser requires that tags have a value, so simply pass '~' (null) or some other garbage.

Given a base template:

Resources:
  Resource1:
    Name: Test
    SuperSecurityLevel: 5

Merging in:

Resources:
  Resource1:
    SuperSecurityLevel: !unset ~

Will result in:

Resources:
  Resource1:
    Name: Test

!var

Replaced with the value of a variable.

Given:

cfnpp:
  variables:
    A: 1
    B:
    	- 1
    	- 2
    	- 3

MyFavouriteNumber: !var A
SomeOtherNumbers: !var B

Will result in:

MyFavouriteNumber: 1
SomeOtherNumbers:
	- 1
	- 2
	- 3

!if

Conditionally include portions of the document.

Expects to receive an array where the first element is a condition, the second is the value if the condition is true, and the third is the value if the condition is false (this one is optional).

Expressions use postfix notation because this is a play project right now and it's both fun and easy to implement and my HP calculator demanded it. Bright side, you never have to worry about operator precedence!

Example:

cnfpp:
	variables:
		LoadBalancerType: internal

Resources:
	MyLoadBalancer:
		Subnet: !if
			- LoadBalancerType "internal" eq
			- subnet-1234
			- subnet-6789

The true/false values can include maps/arrays/any other valid YAML.

Expressions

Expressions are composed of space separated values and operators. Value components are:

• Numeric Literals: Any integer or float value.
• String Literals: Any value enclosed in double quotes ("). A backslash can be used to quote/escape literal quotes within the string.
• Variables: Any non-quoted string consisting of letters and numbers (no leading digit) is interpreted as a reference to one of the variables.
• Operator: Currently supported comparison/logical operators are:
  • eq: equal
  • neq: not equal
  • gt: greater than
  • gte: greater than or equal
  • lt: less than
  • lte: less than or equal
  • and: boolean and
  • or: boolean or

Usage

Running

• Install dependencies with composer install
• Execute cfnpp with PHP 8, it will either complain about any missing PHP extensions or present a list of commands.
• Run ./cfnpp <infile.yaml> <outfile.yaml>

Building

Note: Building is not required to run.

• Install dependencies with composer install
• Run ./cfnpp app:build; outputs phar at builds/

TODO

• Provision rendered templates directly to CloudFormation
• Add flow control: !if, !foreach, etc
• Some sort of macro system would be good.