Puppet ENC With MongoDB Backend

So I decided to release this bit of code I wrote which allows you to leverage mongodb as an ENC for puppet. I am not yet sure if this is a premature move or not but I plan to be using it in production in the next few weeks. I have a list of features I would like to add and will probably be adding over the next few months but I wanted to get some eyes on this and maybe some suggestions as far as the direction this project should go. Hopefully someone will actually take the time to use it.

Why A MongoDB As An ENC?

The reason I decided to write another ENC for puppet is that I didn’t feel there where many good ways to automate defining nodes in puppet when working with EC2. Currently I am running a few hundred nodes in Amazon’s EC2 environment and have been using LDAP as an ENC. LDAP provided a good way to get up and running quickly but LDAP has too many limitations when it comes to being used as an overall strategy for the cloud.

For instance not only do I want to use my mongodb replica set as a way to store puppet related info (ENC) but I also want to add EC2 related metadata to the same database where I was storing puppet info. For instance I want to be able to query a node and get info about what AMI is being used, when the node was deployed, when the node was destroyed, what IP address was assigned to that node, etc.. I also want to store puppet facts in the same database under the same node, and using LDAP to store that information becomes difficult to say the least.

For this project I will be focusing on puppet related information but writing your own scripts to update the same mongodb database with “other” information will be a simply task for anyone to accomplish as long as they don’t update the embedded document where the ENC data is

Features

Here is a list of the current features as of 8/5/2012.

Basic ENC Functionality

According to the puppet documentation a basic ENC must be able to output a YAML hash or nothing when supplied with a FQDN. This hash MAY contain classes, parameters, and environment keys, and MUST contain at least either classes or parameters. ENCs SHOULD exit with an exit code of 0 when functioning normally, and MAY exit with a non-zero exit code if you wish puppet master to behave as though the requested node was not found.

Here is the example output of the mongodb-enc:

classes:
  base: ''
  mongodb: ''
environment: production
parameters:
  ntpserver: 10.10.10.10

Multi-Node Inheritance

By default an ENC doesn’t support node inheritance (excluding LDAP), for instance in LDAP you can define a parentNode from where classes can be inherited; however I have coded that functionality into the mongodb-enc. When you first setup your environment and run the setup.py script an initial default node will be created, the documentation on how to use setup.py can be found here:

https://github.com/bcarpio/mongodb-enc/wiki/setup.py

Once you setup your fist node, that node by default will inherit the default node. So Lets say for example your default node looks like this:

classes:
  base: ''
  nptclient: ''
  nagiosclient: ''
  backupagent: ''

And you define a new node that only includes the mongodb class, when you query that node you will get the following output:

classes:
  base: ''
  nptclient: ''
  nagiosclient: ''
  backupagent: ''
  mongodb: ''
environment: production
parameters:

That isn’t all, lets say you define your default node, and you also define a generic node called mongodb which contains the class mongodb and inherits default your output for the mongodb node will again look like:

classes:
  base: ''
  nptclient: ''
  nagiosclient: ''
  backupagent: ''
  mongodb: ''
environment: production
parameters:

Next you want to define a real node, lets call it prod-mongodb-s1-04, you don’t need to redefine any of the classes, you can simply just add the node and set the inheritance to mongodb and get the same output as above. But lets say this is your delayed secondary so you also want to include the class mongodb::backup:: all you need to do is the following:

./add_node.py -a new -n prod-mongod-s1-04 -c mongodb::backup -i mongodb

When the node is queried you will get the following output:

classes:
  base: ''
  nptclient: ''
  nagiosclient: ''
  backupagent: ''
  mongodb: ''
  mongodb::backup: ''
environment: production
parameters:

Documentation and Project Code

It would be silly of me to try and attempt to rewrite all of the documentation in this blog post regarding how to use the mongodb-enc. I simply wanted to give people an overview about how it worked. Please remember this project is in its infancy, and I will continue to add features as I need them for my current needs. However I am open to others offering suggestions and code changes to make this project even better.

The wiki can be found here:

https://github.com/bcarpio/mongodb-enc/wiki

And the code can be checked out from github.com, here:

https://github.com/bcarpio/mongodb-enc