Docker: Practice with Jenkins, Salt

Automate generate Dockerfile within Jenkins pipeline job

# generate Dockerfile
/bin/rm -rf ${WORKSPACE}/docker
/bin/mkdir -p ${WORKSPACE}/docker

cd ${WORKSPACE}/docker
touch Dockerfile

cat <<EOF > Dockerfile
# set the base image to Centos
FROM centos:6.7

# File Author / Maintainer
MAINTAINER buildmaster <buildmaster@qxc.com>

RUN rpm -Uvh http://download.fedoraproject.org/pub/epel/6/i386/epel-release-6-8.noarch.rpm

RUN yum install -y salt salt-minion

# udpate salt file_roots, and pillar_roots
RUN echo 'file_roots:' >> /etc/salt/minion
RUN echo '  base:' >> /etc/salt/minion
RUN echo '    - /srv/salt' >> /etc/salt/minion
RUN echo '    - /srv/formulas' >> /etc/salt/minion
RUN echo '  dev:' >> /etc/salt/minion
RUN echo '    - /srv/salt' >> /etc/salt/minion
RUN echo '    - /srv/formulas' >> /etc/salt/minion
RUN echo 'pillar_roots:' >> /etc/salt/minion
RUN echo '  base:' >> /etc/salt/minion
RUN echo '    - /srv/pillar' >> /etc/salt/minion
RUN echo '  dev:' >> /etc/salt/minion
RUN echo '    - /srv/pillar/qa' >> /etc/salt/minion

# use local file_client
RUN echo 'file_client: local' >> /etc/salt/minion

# define Salt role
RUN salt-call grains.setval roles "[${MY_ROLE}]"
EOF

Build docker image, consume with docker container

# build a docker image from Dockerfile
cd ${WORKSPACE}/docker

docker build -t ${DOCKER_IMAGE}.v${PIPELINE_VERSION} .

# add host directory as a data volume (which only works with at least centos6.7)
# invoke salt-call with docker container 
docker run –v salt_scripts:/srv/rw -t ${DOCKER_IMAGE}.v${PIPELINE_VERSION} salt-call --local pillar.items | /usr/bin/tee ${OUTPUT}

# handle return code and clean up successful docker container
CHECK_SH=${WORKSPACE}/${PIPELINE_VERSION}.`date "+%Y%m%d%M%S"`.sh
/bin/echo "#!/bin.sh" > ${CHECK_SH}
/bin/echo "RET_CODE=0" >> ${CHECK_SH}
/bin/echo "/bin/grep -n -i error ${OUTPUT} > ${OUTPUT}.tmp" >> ${CHECK_SH}
/bin/echo "if [ ! -s ${OUTPUT}.tmp ]; then" >> ${CHECK_SH}
/bin/echo "   docker rmi -f ${DOCKER_IMAGE}.v${PIPELINE_VERSION}" >> ${CHECK_SH}
/bin/echo "else" >> ${CHECK_SH}
/bin/echo "   RET_CODE=1" >> ${CHECK_SH}
/bin/echo "fi" >> ${CHECK_SH}
/bin/echo "exit ${RET_CODE}" >> ${CHECK_SH}

/bin/sh ${CHECK_SH}

Enterprise Linux: Automated CI build framework

An automated CI build framework

Based on open source tools (CruiseControl/Jenkins, PERL, Puppet + MCollective + Hiera) for listening commits, compiling source code, generating RPM,  deploying to different clusters from YUM repositories.

Puppet Deployment: MCollective in Puppet 3.x

Puppet master/agent deployment approach is a pull-based mode (by default) fetching the master node controlled catalog to target agent nodes and apply to target agent nodes periodically (30 minutes, by default).

Puppet 2.7 Out-Of-Box puppet kick (Remotely control puppet agent) has been deprecated in Puppet 3.0.

The Marionette Collective (MCollective for short) is a tool for system administration. It can run commands on large numbers of servers in parallel, and uses a broadcast architecture so that you can administer a large network without the need for a central master server or asset database.

Each node runs an MCollective daemon that listens for requests, and can execute commands locally or return information about the server. This can be used to filter the list of target nodes. So, for example, you could use MCollective to execute a given command on all nodes that match certain criteria. You can think of MCollective as a complement to Puppet.

For example, your provisioning process for a new node might require firewall changes on other machines, permissions granted on a database server, and so on, which is not very easy to do with Puppet. Although you could automate specific jobs using shell scripts and SSH, MCollective provides a powerful and flexible way to solve this general problem.

In order to have a Puppet master/agent deployment infrastructure with a push-based mode and deploy the master nodes controlled catalog into target agent nodes on demand. 

Here is the puppet 3.0.2 + mcollective configuration example to achieve a Puppet master/agent deployment infrastructure in a push-mode mode:
1. Within a Puppet master/agent deployment infrastrucutre with cert sign (make sure all of puppet nodes sync with Network Time Protocol otherwise there will be some difficulties to do puppet cert sign)
2. On puppet agent nodes, configured puppet agent service to non-daemonized
3. Installation of RPMs:

1. yum install activemq activemq-info-provider on a middle-ware node
2. yum install mcollective mcollective-client mcollective-puppet-agent mcollective-puppet-client on mcollective client node where the requests come from 
3. yum install mcollective mcollective-puppet-agent on puppet agent nodes

4. Configuration

1. make mcollective daemon is up running on all of nodes
2. example of /etc/mcollective/server.cfg

cat /etc/mcollective/server.cfg
# main config
libdir = /usr/libexec/mcollective
logfile = /var/log/mcollective.log
loglevel = info
daemonize = 1

# connector plugin config
connector = activemq
plugin.activemq.pool.size = 1
plugin.activemq.pool.1.host = <activemq_middle-ware_node>
plugin.activemq.pool.1.port = 61613
plugin.activemq.pool.1.user = mcollective
plugin.activemq.pool.1.password = secret

# facts
factsource = yaml
plugin.yaml = /etc/mcollective/facts.yaml

# Plugins
securityprovider = psk
plugin.psk = unset

# puppet
plugin.puppet.command = puppet agent
plugin.puppet.splay = true
plugin.puppet.splaylimit = 30
plugin.puppet.config = /etc/puppet/puppet.conf
plugin.puppet.resource_allow_managed_resources = true
5. mcollective command with puppet plugin 
[root@linux64-client ~]#  mco puppet runonce -v
Discovering hosts using the mc method for 2 second(s) .... 2

* [ ============================================================> ] 2 / 2

linux64-node2                               : OK
    {:summary=>      "Started a background Puppet run using the 'puppet agent --onetime --daemonize --color=false --splay --splaylimit 30' command"}

linux64-client                               : OK
    {:summary=>      "Started a background Puppet run using the 'puppet agent --onetime --daemonize --color=false --splay --splaylimit 30' command"}



---- rpc stats ----
           Nodes: 2 / 2
     Pass / Fail: 2 / 0
      Start Time: Tue May 14 21:58:51 -0400 2013
  Discovery Time: 2015.99ms
      Agent Time: 1148.63ms
      Total Time: 3164.62ms
[root@linux64-client ~]# mco puppet status linux64-node2 -v
Discovering hosts using the mc method for 2 second(s) .... 2

* [ ============================================================> ] 2 / 2

   linux64-client: Currently stopped; last completed run 12 seconds ago
   linux64-node2: Currently applying a catalog; last completed run 9 minutes 32 seconds ago

Summary of Applying:

    true = 1
   false = 1

Summary of Daemon Running:

   running = 1
   stopped = 1

Summary of Enabled:

   enabled = 2

Summary of Idling:

   false = 2

Summary of Status:

              stopped = 1
   applying a catalog = 1


---- rpc stats ----
           Nodes: 2 / 2
     Pass / Fail: 2 / 0
      Start Time: Tue May 14 21:59:31 -0400 2013
  Discovery Time: 2015.43ms
      Agent Time: 50.95ms

      Total Time: 2066.38ms

Puppet Deployment: puppet kick in Puppet 2.7

Puppet master/agent deployment approach is a pull-based mode (be default) fetching the master node controlled catalog to target agent nodes and apply to target agent nodes periodically (30 minuts, by default).

Puppet 2.7 Out-Of-Box puppet kick (Remotely control puppet agent) needs puppet agent on the target node to be running as a daemon, be configured to listen for incoming network connection, and have an appropriate security configuration.

In order to have a Puppet master/agent deployment infrastructure with a push-based mode and deploy the master nodes controlled catalog into target agent nodes on demand.

Here is my puppet kick configuration example to achieve a Puppet master/agent deployment infrastructure in a push-mode mode:
1. Within a Puppet master/agent deployment infrastrucutre with cert sign (make sure all of puppet nodes sync with Network Time Protocol otherwise there will be some difficulties to do puppet cert sign)
2. On puppet agent nodes, snippets of /etc/puppet/auth.conf and /etc/puppet/puppet.conf

cat /etc/puppet/auth.conf

path /run

method save

auth no

allow <puppet-master-node>

cat /etc/puppet/puppet.conf

...

[agent]

...

   listen = true

   ignorecache = true

...

3. Trigger puppet kick from puppet master catalog to multiple puppet target nodes

puppet kick --host <puppet_agent_node>

Triggering <puppet_agent_node>

Getting status

status is success

<puppet_agent_node> finished with exit code 0

Finished

Puppet Deployment: Parameterized Classes

# Parameterized Classes on puppet master

[root@linux64-puppet-server ~]# puppet module list
/etc/puppet/modules
+- pptdeploy(???)
[root@linux64-puppet-server ~]# puppet config print confdir
/etc/puppet
[root@linux64-puppet-server ~]# find /etc/puppet/manifests/
/etc/puppet/manifests/
/etc/puppet/manifests/site.pp
[root@linux64-puppet-server ~]# find /etc/puppet/modules/
/etc/puppet/modules/
/etc/puppet/modules/pptdeploy
/etc/puppet/modules/pptdeploy/manifests
/etc/puppet/modules/pptdeploy/manifests/init.pp
/etc/puppet/modules/pptdeploy/manifests/install.pp
/etc/puppet/modules/pptdeploy/manifests/lockforinstall.pp
/etc/puppet/modules/pptdeploy/manifests/uninstall.pp
/etc/puppet/modules/pptdeploy/manifests/unlockforinstall.pp

# Define Puppet Parameterized Classes

cat /etc/puppet/modules/pptdeploy/manifests/init.pp

class pptdeploy {
}

cat /etc/puppet/modules/pptdeploy/manifests/install.pp

class pptdeploy::install($pptdeploy_release, $pptdeploy_install_dir, $pptdeploy_product_shortname, $pptdeploy_product_name, $pptdeploy_response_file) {


 notify{ ">>> Starting to install ${pptdeploy_release} ${pptdeploy_product_name} ...":
         before => File [ installer ] }


 file { "installer":
        path => "${pptdeploy_install_dir}/${pptdeploy_product_shortname}/${pptdeploy_product_name}.bin",
        mode => '0755', }


 Exec { path => [ "/bin/", "/sbin/" , "/usr/bin/", "/usr/sbin/" ] }


 exec { "install ${pptdeploy_release} ${pptdeploy_product_name} ...":
        command => "${pptdeploy_install_dir}/${pptdeploy_product_shortname}/${pptdeploy_product_name}.bin -i silent -DMY_RESPONSE_FILE=${pptdeploy_install_dir}/${pptdeploy_product_shortname}/${pptdeploy_response_file}",
        timeout => 0,
        creates => "/etc/install_registry/${pptdeploy_release}_${pptdeploy_product_shortname}_installed",
        require => File [ installer ] }


 exec { "<<< verify ${pptdeploy_release} ${pptdeploy_product_name} installation ...":
        command => "cat /etc/install_registry/${pptdeploy_product_name}.properties",
        logoutput => true,
        creates => "/etc/install_registry/${pptdeploy_release}_${pptdeploy_product_shortname}_installed",
        require => Exec [ "install pptdeploy ${pptdeploy_release} ${pptdeploy_product_name} ..." ] }


 exec { "Lock installation for ${pptdeploy_product_name} ...":
        command => "/bin/touch /etc/install_registry/${pptdeploy_release}_${pptdeploy_product_shortname}_installed",
        creates => "/etc/install_registry/${pptdeploy_release}_${pptdeploy_product_shortname}_installed",
        require => Exec [ "<<< verify ${pptdeploy_release} ${pptdeploy_product_name} installation ..." ] }
}

cat /etc/puppet/modules/pptdeploy/manifests/lockforinstall.pp

class pptdeploy::lockforinstall($pptdeploy_release, $pptdeploy_install_dir, $pptdeploy_product_shortname, $pptdeploy_product_name, $pptdeploy_response_file) {


 Exec { path => [ "/bin/", "/sbin/" , "/usr/bin/", "/usr/sbin/" ] }


 exec { "Lock installation for ${pptdeploy_product_name} ...":
         command => "/bin/touch /etc/install_registry/${pptdeploy_release}_${pptdeploy_product_shortname}_installed",}
}

cat /etc/puppet/modules/pptdeploy/manifests/uninstall.pp

class pptdeploy::uninstall($pptdeploy_release, $pptdeploy_install_dir, $pptdeploy_product_shortname, $pptdeploy_product_name, $pptdeploy_response_file) {


 notify{ ">>> Starting to uninstall ${pptdeploy_product_name} ...": }


 Exec { path => [ "/bin/", "/sbin/" , "/usr/bin/", "/usr/sbin/" ] }


 exec { "uninstall ${pptdeploy_product_name} ...":
        command => "/opt/Uninstall_${pptdeploy_product_name} -i silent -uninstall",
        timeout => 0,
        require => Exec [ "Unlock installation for ${pptdeploy_product_name} ..." ] }


 exec { "Unlock installation for ${pptdeploy_product_name} ...":
        command => "rm -f /etc/install_registry/*_${pptdeploy_product_shortname}_installed",
        timeout => 0, }
}

cat /etc/puppet/modules/pptdeploy/manifests/unlockforinstall.pp

class pptdeploy::unlockforinstall($pptdeploy_release, $pptdeploy_install_dir, $pptdeploy_product_shortname, $pptdeploy_product_name, $pptdeploy_response_file) {


 Exec { path => [ "/bin/", "/sbin/" , "/usr/bin/", "/usr/sbin/" ] }


 exec { "Unlock installation for ${pptdeploy_product_name} ...":
         command => "rm -f /etc/install_registry/*_${pptdeploy_product_shortname}_installed",}
}

# Global namespace - declare a Parameterized Class

cat /etc/puppet/manifests/site.pp
node default {
}


################################################
$pptdeploy_release = "3.0"
$pptdeploy_install_dir = "/mnt/work/COTS_install_sources"
$pptdeploy_product_shortname = "COTS"
$pptdeploy_product_name = "COTS-3.0"
$pptdeploy_response_file = "cots.response"


#-----------------------
# pptdeploy installation
#-----------------------
  node /linux64-puppet.*\.domain\.com/ {
    class { 'pptdeploy::install': 
      pptdeploy_release => $pptdeploy_release, 
      pptdeploy_install_dir => $pptdeploy_install_dir, 
      pptdeploy_product_shortname => $pptdeploy_product_shortname, 
      pptdeploy_product_name => $pptdeploy_product_name, 
      pptdeploy_response_file => $pptdeploy_response_file,
    }
  }


#-----------------------
# pptdeploy uninstallation
#-----------------------
#   node /linux64-puppet.*\.domain\.com/ {
#     class { 'pptdeploy::uninstall': 
#       pptdeploy_release => '3.0', 
#       pptdeploy_install_dir => '/mnt/work/COTS_install_sources', 
#       pptdeploy_product_shortname => 'COTS', 
#       pptdeploy_product_name => 'COTS-3.0', 
#       pptdeploy_response_file => 'cots.response,
#     }
#   }
################################################

# deploy on puppet agent

[root@linux64-puppet-agent ~]# puppet agent -t --noop
info: Caching catalog for linux64-puppet-agent
info: Applying configuration version '1340827332'
notice: >>> Starting to install 3.0 COTS-3.0 ...
notice: /Stage[main]/pptdeploy::install/Notify[>>> Starting to install  3.0 COTS-3.0 ...]/message: defined 'message' as '>>> Starting to install 3.0 COTS-3.0 ...'
notice: Finished catalog run in 0.08 seconds

Puppet Deployment: modules

# Custom Classes on puppet master

find /etc/puppet/modules

/etc/puppet/modules

/etc/puppet/modules/pptdeploy_install

/etc/puppet/modules/pptdeploy_install/manifests

/etc/puppet/modules/pptdeploy_install/manifests/init.pp

/etc/puppet/modules/pptdeploy_uninstall

/etc/puppet/modules/pptdeploy_uninstall/manifests

/etc/puppet/modules/pptdeploy_uninstall/manifests/init.pp

/etc/puppet/modules/pptdeploy_lockforinstall

/etc/puppet/modules/pptdeploy_lockforinstall/manifests

/etc/puppet/modules/pptdeploy_lockforinstall/manifests/init.pp

/etc/puppet/modules/pptdeploy_unlockforinstall

/etc/puppet/modules/pptdeploy_unlockforinstall/manifests

/etc/puppet/modules/pptdeploy_unlockforinstall/manifests/init.pp

cat /etc/puppet/modules/pptdeploy_install/manifests/init.pp

class pptdeploy_install {

 notify{ ">>> Starting to install ${pptdeploy_release} ${pptdeploy_product_name} ...":

         before => File [ installer ] }

 file { "installer":

        path => "${pptdeploy_install_dir}/${pptdeploy_product_shortname}/${pptdeploy_product_name}.bin",

        mode => '0755', }

 Exec { path => [ "/bin/", "/sbin/" , "/usr/bin/", "/usr/sbin/" ] }

 exec { "install ${pptdeploy_release} ${pptdeploy_product_name} ...":

        command => "${pptdeploy_install_dir}/${pptdeploy_product_shortname}/${pptdeploy_product_name}.bin -i silent -DMY_RESPONSE_FILE=${pptdeploy_install_dir}/${pptdeploy_product_shortname}/${pptdeploy_response_file}",

        timeout => 0,

        creates => "/etc/install_registry/${pptdeploy_release}_${pptdeploy_product_shortname}_installed",

        require => File [ installer ] }

 exec { "<<< verify ${pptdeploy_release} ${pptdeploy_product_name} installation ...":

        command => "cat /etc/install_registry/${pptdeploy_product_name}.properties",

        logoutput => true,

        creates => "/etc/install_registry/${pptdeploy_release}_${pptdeploy_product_shortname}_installed",

        require => Exec [ "install ${pptdeploy_release} ${pptdeploy_product_name} ..." ] }

 exec { "Lock installation for ${pptdeploy_product_name} ...":

        command => "/bin/touch /etc/install_registry/${pptdeploy_release}_${pptdeploy_product_shortname}_installed",

        creates => "/etc/install_registry/${pptdeploy_release}_${pptdeploy_product_shortname}_installed",

        require => Exec [ "<<< verify pptdeploy ${pptdeploy_release} ${pptdeploy_product_name} installation ..." ] }

}

cat /etc/puppet/modules/pptdeploy_uninstall/manifests/init.pp

class pptdeploy_uninstall {

 notify{ ">>> Starting to uninstall ${pptdeploy_product_name} ...": }

 Exec { path => [ "/bin/", "/sbin/" , "/usr/bin/", "/usr/sbin/" ] }

 exec { "uninstall ${pptdeploy_product_name} ...":
        command => "/opt/Uninstall_${pptdeploy_product_name} -i silent -uninstall",
        timeout => 0,
        require => Exec [ "Unlock installation for ${pptdeploy_product_name} ..." ] }

 exec { "Unlock installation for ${pptdeploy_product_name} ...":
        command => "rm -f /etc/install_registry/*_${pptdeploy_product_shortname}_installed",
        timeout => 0, }
}

cat /etc/puppet/modules/pptdeploy_lockforinstall/manifests/init.pp

class pptdeploy_lockforinstall {

 Exec { path => [ "/bin/", "/sbin/" , "/usr/bin/", "/usr/sbin/" ] }

 exec { "Lock installation for ${pptdeploy_product_name} ...":
         command => "/bin/touch /etc/install_registry/${pptdeploy_release}_${pptdeploy_product_shortname}_installed",}
}

cat /etc/puppet/modules/pptdeploy_unlockforinstall/manifests/init.pp

class pptdeploy_unlockforinstall {

 Exec { path => [ "/bin/", "/sbin/" , "/usr/bin/", "/usr/sbin/" ] }

 exec { "Unlock installation for ${pptdeploy_product_name} ...":
         command => "rm -f /etc/install_registry/*_${pptdeploy_product_shortname}_installed",}
}

cat /etc/puppet/manifests/site.pp

node default {

}

################################################

$pptdeploy_release = "3.0"

$pptdeploy_install_dir = "/mnt/work/COTS_install_sources"

$pptdeploy_product_shortname = "COTS"

$pptdeploy_product_name = "deploy_COTS"

$pptdeploy_response_file = "cots.response"

#-----------------------

# deployment installation

#-----------------------

node /linux64-puppet.*\.domain\.com/ {

  include pptdeploy_install

}

#-----------------------

# deployment uninstallation

#-----------------------

# node /linux64-puppet.*\.domain\.com/ {

#   include pptdeploy_uninstall

# }

################################################

# deploy on puppet agent

puppet agent -t --noop

warning: peer certificate won't be verified in this SSL session

info: Caching certificate for linux64-puppet-agent

info: Caching certificate_revocation_list for ca

info: Caching catalog for linux64-puppet-agent

info: Applying configuration version '1334886903'

...
notice: Finished catalog run in 0.08 seconds

puppet agent -t

info: Caching catalog for linux64-puppet-agent

info: Applying configuration version '1334886903'

...

notice: Finished catalog run in 328.60 seconds

Puppet Deployment: Puppet Open Source

Puppet Open Source installation: Puppet Open Source 2.7.13 

# puppet-server installation

# enable pupperlabs yum repository

[root@linux64-puppet-server ~]# cat /etc/yum.repos.d/puppetlabs.repo

[puppetlabs]
name=(local yum repo of) Puppet Labs Packages
baseurl=http://linux64-yum-repo-server/puppet/el/5/products/x86_64/
enabled=1
gpgcheck=1
gpgkey=http://linux64-yum-repo-server/puppet/RPM-GPG-KEY-puppetlabs
[puppetlabs_dependencies]
name=(local yum repo of) Puppet Labs Packages
baseurl=http://linux64-yum-repo-server/puppet/el/5/dependencies/x86_64/
enabled=1
gpgcheck=1
gpgkey=http://linux64-yum-repo-server/puppet/RPM-GPG-KEY-puppetlabs

[root@linux64-puppet-server ~]# yum install puppet-server

Loaded plugins: security

puppetlabs                                                                                                                | 1.9 kB     00:00

puppetlabs/primary_db                                                                                                     |  26 kB     00:00

puppetlabs_dependencies                                                                                                   | 1.9 kB     00:00

puppetlabs_dependencies/primary_db                                                                                        |  13 kB     00:00

Setting up Install Process

Resolving Dependencies

--> Running transaction check

---> Package puppet-server.noarch 0:2.7.13-1.el5 set to be updated

--> Processing Dependency: puppet = 2.7.13-1.el5 for package: puppet-server

--> Processing Dependency: /usr/bin/ruby for package: puppet-server

--> Running transaction check

---> Package puppet.noarch 0:2.7.13-1.el5 set to be updated

--> Processing Dependency: ruby(abi) >= 1.8 for package: puppet

--> Processing Dependency: facter >= 1.5 for package: puppet

--> Processing Dependency: ruby-shadow for package: puppet

--> Processing Dependency: ruby-augeas for package: puppet

---> Package ruby.x86_64 0:1.8.5-5.el5_4.8 set to be updated

--> Running transaction check

---> Package facter.x86_64 0:1.6.7-1 set to be updated

---> Package ruby-augeas.x86_64 0:0.4.1-1 set to be updated

--> Processing Dependency: augeas-libs >= 0.8.0 for package: ruby-augeas

--> Processing Dependency: libaugeas.so.0(AUGEAS_0.8.0)(64bit) for package: ruby-augeas

--> Processing Dependency: libaugeas.so.0(AUGEAS_0.11.0)(64bit) for package: ruby-augeas

--> Processing Dependency: libaugeas.so.0(AUGEAS_0.10.0)(64bit) for package: ruby-augeas

--> Processing Dependency: libaugeas.so.0(AUGEAS_0.1.0)(64bit) for package: ruby-augeas

--> Processing Dependency: libaugeas.so.0(AUGEAS_0.12.0)(64bit) for package: ruby-augeas

--> Processing Dependency: libaugeas.so.0()(64bit) for package: ruby-augeas

---> Package ruby-libs.x86_64 0:1.8.5-5.el5_4.8 set to be updated

---> Package ruby-shadow.x86_64 0:1.4.1-7 set to be updated

--> Running transaction check

---> Package augeas-libs.x86_64 0:0.10.0-3 set to be updated

--> Finished Dependency Resolution

Dependencies Resolved

=================================================================================================================================================

Package                          Arch                      Version                             Repository                                  Size

=================================================================================================================================================

Installing:

puppet-server                    noarch                    2.7.13-1.el5                        puppetlabs                                  23 k

Installing for dependencies:

augeas-libs                      x86_64                    0.10.0-3                            puppetlabs_dependencies                    376 k

facter                           x86_64                    1.6.7-1                             puppetlabs                                  68 k

puppet                           noarch                    2.7.13-1.el5                        puppetlabs                                 1.0 M

ruby                             x86_64                    1.8.5-5.el5_4.8                     el5_u5_base                                280 k

ruby-augeas                      x86_64                    0.4.1-1                             puppetlabs_dependencies                     22 k

ruby-libs                        x86_64                    1.8.5-5.el5_4.8                     el5_u5_base                                1.6 M

ruby-shadow                      x86_64                    1.4.1-7                             puppetlabs_dependencies                     10 k

Transaction Summary

=================================================================================================================================================

Install       8 Package(s)

Upgrade       0 Package(s)

Total download size: 3.5 M

Is this ok [y/N]:y

Downloading Packages:

(1/8): ruby-shadow-1.4.1-7.x86_64.rpm                                                                                     |  10 kB     00:00

(2/8): ruby-augeas-0.4.1-1.x86_64.rpm                                                                                     |  22 kB     00:00

(3/8): puppet-server-2.7.13-1.el5.noarch.rpm                                                                              |  23 kB     00:00

(4/8): facter-1.6.7-1.el5.x86_64.rpm                                                                                      |  68 kB     00:00

(5/8): ruby-1.8.5-5.el5_4.8.x86_64.rpm                                                                                    | 280 kB     00:00

(6/8): augeas-libs-0.10.0-3.x86_64.rpm                                                                                    | 376 kB     00:00

(7/8): puppet-2.7.13-1.el5.noarch.rpm                                                                                     | 1.0 MB     00:01

(8/8): ruby-libs-1.8.5-5.el5_4.8.x86_64.rpm                                                                               | 1.6 MB     00:01

-------------------------------------------------------------------------------------------------------------------------------------------------

Total                                                                                                            705 kB/s | 3.5 MB     00:05

warning: rpmts_HdrFromFdno: Header V3 RSA/SHA1 signature: NOKEY, key ID 4bd6ec30

puppetlabs_dependencies/gpgkey                                                                                            | 1.7 kB     00:00

Importing GPG key 0x4BD6EC30 "Puppet Labs Release Key (Puppet Labs Release Key) <info@puppetlabs.com>" from http://linux64-yum-repo-server/puppet/RPM-GPG-KEY-puppetlabs

Is this ok [y/N]: y

Running rpm_check_debug

Running Transaction Test

Finished Transaction Test

Transaction Test Succeeded

Running Transaction

  Installing     : ruby-libs                                                                                                                 1/8

  Installing     : ruby                                                                                                                      2/8

  Installing     : ruby-shadow                                                                                                               3/8

  Installing     : augeas-libs                                                                                                               4/8

  Installing     : ruby-augeas                                                                                                               5/8

  Installing     : facter                                                                                                                    6/8

  Installing     : puppet                                                                                                                    7/8

  Installing     : puppet-server                                                                                                             8/8

Installed:

  puppet-server.noarch 0:2.7.13-1.el5

Dependency Installed:

  augeas-libs.x86_64 0:0.10.0-3      facter.x86_64 0:1.6.7-1                 puppet.noarch 0:2.7.13-1.el5      ruby.x86_64 0:1.8.5-5.el5_4.8

  ruby-augeas.x86_64 0:0.4.1-1       ruby-libs.x86_64 0:1.8.5-5.el5_4.8      ruby-shadow.x86_64 0:1.4.1-7

Complete!

# puppet-agent installation

# enable pupperlabs yum repository
[root@linux64-puppet-agent ~]# cat /etc/yum.repos.d/puppetlabs.repo

[puppetlabs]
name=(local yum repo of) Puppet Labs Packages
baseurl=http://linux64-yum-repo-server/puppet/el/5/products/x86_64/
enabled=1
gpgcheck=1
gpgkey=http://linux64-yum-repo-server/puppet/RPM-GPG-KEY-puppetlabs
[puppetlabs_dependencies]
name=(local yum repo of) Puppet Labs Packages
baseurl=http://linux64-yum-repo-server/puppet/el/5/dependencies/x86_64/
enabled=1
gpgcheck=1
gpgkey=http://linux64-yum-repo-server/puppet/RPM-GPG-KEY-puppetlabs

[root@linux64-puppet-agent ~]# yum install puppet

Loaded plugins: security

el5_u5_base                                                                                                               | 1.1 kB     00:00

ol5_u5_base                                                                                                               |  951 B     00:00

puppetlabs                                                                                                                | 1.9 kB     00:00

puppetlabs/primary_db                                                                                                     |  26 kB     00:00

puppetlabs_dependencies                                                                                                   | 1.9 kB     00:00

puppetlabs_dependencies/primary_db                                                                                        |  13 kB     00:00

Setting up Install Process

Resolving Dependencies

--> Running transaction check

---> Package puppet.noarch 0:2.7.13-1.el5 set to be updated

--> Processing Dependency: ruby >= 1.8.5 for package: puppet

--> Processing Dependency: ruby(abi) >= 1.8 for package: puppet

--> Processing Dependency: facter >= 1.5 for package: puppet

--> Processing Dependency: ruby-shadow for package: puppet

--> Processing Dependency: ruby-augeas for package: puppet

--> Processing Dependency: /usr/bin/ruby for package: puppet

--> Running transaction check

---> Package facter.x86_64 0:1.6.7-1 set to be updated

---> Package ruby.x86_64 0:1.8.5-5.el5_4.8 set to be updated

---> Package ruby-augeas.x86_64 0:0.4.1-1 set to be updated

--> Processing Dependency: augeas-libs >= 0.8.0 for package: ruby-augeas

--> Processing Dependency: libaugeas.so.0(AUGEAS_0.8.0)(64bit) for package: ruby-augeas

--> Processing Dependency: libaugeas.so.0(AUGEAS_0.11.0)(64bit) for package: ruby-augeas

--> Processing Dependency: libaugeas.so.0(AUGEAS_0.10.0)(64bit) for package: ruby-augeas

--> Processing Dependency: libaugeas.so.0(AUGEAS_0.1.0)(64bit) for package: ruby-augeas

--> Processing Dependency: libaugeas.so.0(AUGEAS_0.12.0)(64bit) for package: ruby-augeas

--> Processing Dependency: libaugeas.so.0()(64bit) for package: ruby-augeas

---> Package ruby-libs.x86_64 0:1.8.5-5.el5_4.8 set to be updated

---> Package ruby-shadow.x86_64 0:1.4.1-7 set to be updated

--> Running transaction check

---> Package augeas-libs.x86_64 0:0.10.0-3 set to be updated

--> Finished Dependency Resolution

Dependencies Resolved

=================================================================================================================================================

Package                        Arch                      Version                               Repository                                  Size

=================================================================================================================================================

Installing:

puppet                         noarch                    2.7.13-1.el5                          puppetlabs                                 1.0 M

Installing for dependencies:

augeas-libs                    x86_64                    0.10.0-3                              puppetlabs_dependencies                    376 k

facter                         x86_64                    1.6.7-1                               puppetlabs                                  68 k

ruby                           x86_64                    1.8.5-5.el5_4.8                       el5_u5_base                                280 k

ruby-augeas                    x86_64                    0.4.1-1                               puppetlabs_dependencies                     22 k

ruby-libs                      x86_64                    1.8.5-5.el5_4.8                       el5_u5_base                                1.6 M

ruby-shadow                    x86_64                    1.4.1-7                               puppetlabs_dependencies                     10 k

Transaction Summary

=================================================================================================================================================

Install       7 Package(s)

Upgrade       0 Package(s)

Total download size: 3.4 M

Is this ok [y/N]: y

Downloading Packages:

(1/7): ruby-shadow-1.4.1-7.x86_64.rpm                                                                                     |  10 kB     00:00

(2/7): ruby-augeas-0.4.1-1.x86_64.rpm                                                                                     |  22 kB     00:00

(3/7): facter-1.6.7-1.el5.x86_64.rpm                                                                                      |  68 kB     00:00

(4/7): ruby-1.8.5-5.el5_4.8.x86_64.rpm                                                                                    | 280 kB     00:00

(5/7): augeas-libs-0.10.0-3.x86_64.rpm                                                                                    | 376 kB     00:00

(6/7): puppet-2.7.13-1.el5.noarch.rpm                                                                                     | 1.0 MB     00:00

(7/7): ruby-libs-1.8.5-5.el5_4.8.x86_64.rpm                                                                               | 1.6 MB     00:01

-------------------------------------------------------------------------------------------------------------------------------------------------

Total                                                                                                            1.0 MB/s | 3.4 MB     00:03

warning: rpmts_HdrFromFdno: Header V3 RSA/SHA1 signature: NOKEY, key ID 4bd6ec30

puppetlabs_dependencies/gpgkey                                                                                            | 1.7 kB     00:00

Importing GPG key 0x4BD6EC30 "Puppet Labs Release Key (Puppet Labs Release Key) <info@puppetlabs.com>" from http://linux64-yum-repo-server/puppet/RPM-GPG-KEY-puppetlabs

Is this ok [y/N]: y

Running rpm_check_debug

Running Transaction Test

Finished Transaction Test

Transaction Test Succeeded

Running Transaction

  Installing     : ruby-libs                                                                                                                 1/7

  Installing     : ruby                                                                                                                      2/7

  Installing     : ruby-shadow                                                                                                               3/7

  Installing     : augeas-libs                                                                                                               4/7

  Installing     : ruby-augeas                                                                                                               5/7

  Installing     : facter                                                                                                                    6/7

  Installing     : puppet                                                                                                                    7/7

Installed:

  puppet.noarch 0:2.7.13-1.el5

Dependency Installed:

  augeas-libs.x86_64 0:0.10.0-3           facter.x86_64 0:1.6.7-1           ruby.x86_64 0:1.8.5-5.el5_4.8      ruby-augeas.x86_64 0:0.4.1-1

  ruby-libs.x86_64 0:1.8.5-5.el5_4.8      ruby-shadow.x86_64 0:1.4.1-7

Complete!

# configure puppetmaster

[root@linux64-puppet-server ~]# puppet apply -V

2.7.13

[root@linux64-puppet-server ~]# chkconfig puppetmaster on

# insert “modulepath = /etc/puppet/modules:/usr/share/puppet/modules” into /etc/puppet/puppet.conf

[root@linux64-puppet-server ~]# cat /etc/puppet/puppet.conf

[main]

    # The Puppet log directory.

    # The default value is '$vardir/log'.

    logdir = /var/log/puppet

    # Where Puppet PID files are kept.

    # The default value is '$vardir/run'.

    rundir = /var/run/puppet

    modulepath = /etc/puppet/modules:/usr/share/puppet/modules

    # Where SSL certificates are kept.

    # The default value is '$confdir/ssl'.

    ssldir = $vardir/ssl

[master]

    certname = linux64-puppet-server

    dns_alt_names =  linux64-puppet-server ,puppet

[agent]

    certname = linux64-puppet-server

    server = linux64-puppet-server

    # The file in which puppetd stores a list of the classes

    # associated with the retrieved configuratiion.  Can be loaded in

    # the separate ``puppet`` executable using the ``--loadclasses``

    # option.

    # The default value is '$confdir/classes.txt'.

    classfile = $vardir/classes.txt

    # Where puppetd caches the local configuration.  An

    # extension indicating the cache format is added automatically.

    # The default value is '$confdir/localconfig'.

    localconfig = $vardir/localconfig

[root@linux64-puppet-server ~]# service puppetmaster start

[root@linux64-puppet-server ~]# puppet cert list

   linux64-puppet-server  (2E:DC:48:EC:2B:63:F5:28:09:8F:D3:9F:3C:9B:80:2C)

[root@linux64-puppet-server ~]# puppet cert sign linux64-puppet-agent

notice: Signed certificate request for linux64-puppet-server

notice: Removing file Puppet::SSL::CertificateRequest  linux64-puppet-server  at '/var/lib/puppet/ssl/ca/requests/ linux64-puppet-server .pem'

[root@linux64-puppet-server ~]# puppet cert list

[root@linux64-puppet-server ~]# puppet cert -la

+  linux64-puppet-server  (61:68:28:1D:C6:03:27:75:D0:71:07:6B:AD:B0:A8:99) (alt names: DNS: linux64-puppet-server, DNS:puppet)

+  linux64-puppet-agent (23:FE:14:30:F9:68:C7:F4:6F:FE:BF:D9:14:50:49:7C)

# update puppet to latest release (2.7.16) for module list command

[root@linux64-puppet-server ~]# yum install puppet -y

[root@mt-olinux64-b03 ~]# puppet apply -V
2.7.16
[root@mt-olinux64-b03 ~]# puppet module list
/etc/puppet/modules
+-- pptdeploy (???)
/usr/share/puppet/modules (no modules installed)

# configure puppet agent

[root@linux64-puppet-agent ~]#
puppet apply -V

2.7.13

[root@linux64-puppet-agent ~]#
chkconfig puppet on

#insert “server=<puppet server>” into /etc/puppet/puppet.conf

[root@linux64-puppet-agent ~]# cat /etc/puppet/puppet.conf

[main]

    # The Puppet log directory.

    # The default value is '$vardir/log'.

    logdir = /var/log/puppet

    # Where Puppet PID files are kept.

    # The default value is '$vardir/run'.

    rundir = /var/run/puppet

    modulepath = modulepath = /etc/puppet/modules:/usr/share/puppet/modules
    # Where SSL certificates are kept.

    # The default value is '$confdir/ssl'.

    ssldir = $vardir/ssl

[agent]

    certname = linux64-puppet-agent

    server = linux64-puppet-server

    # The file in which puppetd stores a list of the classes

    # associated with the retrieved configuratiion.  Can be loaded in

    # the separate ``puppet`` executable using the ``--loadclasses``

    # option.

    # The default value is '$confdir/classes.txt'.

    classfile = $vardir/classes.txt

    # Where puppetd caches the local configuration.  An

    # extension indicating the cache format is added automatically.

    # The default value is '$confdir/localconfig'.

localconfig = $vardir/localconfig

[root@linux64-puppet-agent ~]# service puppet start

# troubleshooting

[root@linux64-puppet-server~]# puppet config print modulepath

/etc/puppet/modules:/usr/share/puppet/modules

[root@linux64-puppet-server ~]# puppet config print confdir

/etc/puppet

[root@linux64-puppet-server ~]# puppet config print vardir

/var/lib/puppet

[root@linux64-puppet-agent ~]# puppet agent -t --noop

info: Creating a new SSL key for linux64-puppet-agent

warning: peer certificate won't be verified in this SSL session

info: Caching certificate for ca

warning: peer certificate won't be verified in this SSL session

info: Caching certificate for linux64-puppet-agent

err: Could not request certificate: The certificate retrieved from the master does not match the agent's private key.

Certificate fingerprint: 70:14:D9:C3:E1:E5:4D:44:C2:43:21:9B:64:47:F0:B3

To fix this, remove the certificate from both the master and the agent and then start a puppet run, which will automatically regenerate a certficate.

On the master:

  puppet cert clean linux64-puppet-agent

On the agent:

  rm -f /var/lib/puppet/ssl/certs/linux64-puppet-agent.pem

  puppet agent -t

Exiting; failed to retrieve certificate and waitforcert is disabled